Participant Resources
6 Benefits of submitting your hackathon project early
Here’s a look at the numerous benefits of submitting early to a hackathon.
Read more
Take a look inside the inner workings of the judging process from some of our most experienced hackathon judges.
When you enter a hackathon, you might take note of the judging criteria and, perhaps, the profiles of the judges themselves. But have you ever wished you could talk to one of the hackathon judges and find out exactly what they’re looking for?
That’s just what we did here at Devpost. We sat down with five judges from across the tech spectrum to ask them about their judging process, what stands out to them in submissions, and their advice for participants.
Whether you're entering a hackathon as a participant or as a first-time judge, you'll find valuable insights and wisdom from these veteran judges.
Meet the judges:
Check out each of their organization’s hackathons on Devpost here: Square, Databricks, Google, NEAR Foundation, and Atlassian.
Richard: “After [the technical screening] we parse the projects out between different judges. We have some judges who specialize in a particular product, and they view all the submissions involved with that product to see which one is the best reflection of the use of the product.
For me personally, I watch the video to get context and then I use that for testing the submission. From there it's evaluating how well the project matches our criteria."
"When I’ve coached other judges on our hackathons, I encourage them to try their best to evaluate on an individual basis because it's very easy to try to evaluate projects relative to one another,” said Richard Moot, Square.
Karen: “When I'm looking at submissions, the first thing I do is go over the requirements that the hackathon has asked for. As a judge, it was surprising to see how many submissions did not fulfill the basic requirements. The next thing that I look for is creativity or enthusiasm. I feel like that signals to me that somebody has put a lot of time, thoughtfulness, and energy into their submission and the work they were doing. It also can indicate a level of quality within their submission.”
Kelvin: “Ours is a bit specific to Flutter. Flutter is an inherently visual product, so the first thing we look at is how visually appealing the project is. Then we play the video or we use a demo. That's always a really important part for us—using the project and seeing how it performs helps us find out if it’s in line with what we initially set as the problem to solve. So initially, we're not necessarily looking at the best-written code, we're looking to see what the end-user experience is.
We always ensure that each project will be seen by two different judges. They'll be given a score, and then we move on to a second round of judging to determine the winners. What people are looking for during both rounds is that aesthetic, that performance, and how visually exciting and performant the project is.”
Maria: “First, we use Devpost’s help to screen the submissions initially to see if they meet our criteria and make sure that they’re eligible. Second, we distribute these projects among our group of engineers and we ask them to do a technical review (which is still not judging). Finally, the top 30 or so projects make it to the judges, and these judges have criteria and a rubric which they use to score the projects, not just based on technology, but based on whether it's self-sustained, if the team is great, if the idea is great, if it is original, and so on.”
Warren: “For our current hackathon, we’re doing three rounds of judging. There's a preliminary round where we look to see if the application installs, whether the basic premise is solid, and whether it works at all.
Then we're going to have a second round which is a technical screening. The second round will be looking in more detail at the functionality of the software and whether it does cool stuff, and we have AI subject matter experts who are going to take a deeper look at the AI portion.
The last 30 go to the final round of judging. From there, we’ll pick nine. We have three categories, so we'll have the top three places in each of those categories. Then all three first-place winners are going to come to the live stage show in Amsterdam and they're going to present on stage, and the audience will pick the final winner.”
Richard: “The demo video gives the most amount of scope. Sometimes we're judging lots of different submissions, so the video becomes crucial in giving us our first indicator of how much time was invested into the actual submission. But I also think it's a combo of all of the different components, because sometimes you have a really nice video and a really lacking text description, and I think it's important to have all of those things complementing one another.”
Karen: “Presentation and storytelling matters and it can be a huge part of your success."
"I might look at a project and think, “Wow, this looks really interesting.” Then I’d make sure the code looked tight and that everything checked the boxes, but storytelling within the video and the text description definitely helps keep the judges engaged and focused,” said Karen Bajza-Terlouw, Databricks.
Kelvin: “The first thing we're gonna do is just play the game, right?” (Editor’s note: Kelvin’s team at Google held a hackathon that asked participants to build games for their submissions.) “It’s a really fun process, I can’t lie! You get so much from that—a sense of the animations and a sense of the performance. Sometimes the demo videos serve as a secondary input because if you're confused about what to do in the game, oftentimes the videos will help you.
Other content is really helpful for us, too. Whether that's a blog post explaining your process, or maybe a slightly longer video where you explain what you learned along the way.”
Maria: “I am focused on the pitch and description. Technology is just one side of things, but I look to see if there is actually a startup mindset. Do they see the problem behind the solution, or are they just trying to fit their solution to this particular hackathon? We have awarded great projects that were not great from the technical side of things, but we saw potential in the team—a lot of expertise and a lot of background knowledge on this particular problem. Hence we award that because it's a lot of work—it's not just about code.”
Warren: “The first hackathon we did was for a large marketing event, and we asked our participants to build a game that was going to be put on a stage show live. What we were really looking for in this particular case was something that would make a cool stage show. We wanted good graphics, good sound, good animations, a good slide deck, a story, and a cool stage presence. Our goal for this hackathon was really to make that marketing moment happen, so we were evaluating based on that.”
Richard: “A project that really stands out is one that was clearly considering the judging criteria. I say this because I have come across quite a few submissions where it’s almost as if they had an idea that was independent of the actual contest and they're shoehorning it to fit. We look for submissions where they're very clearly trying to balance across all the judging criteria—sometimes people pick only one of the judging criteria but miss everything else.”
Karen: “I think the more open you can be with how you created your project, the better. Within the videos, I saw a couple in which they would demo their code—which I liked—or they would go through doing some kind of voiceover while they showed the different components of the project, which I really liked. Be as thorough as you can, especially when they're technical projects. With highly technical judges, they’re going to want to see all that stuff.”
Kelvin: “We always offer a template to start with, and the projects that stand out the most are the ones that depart the most from that template. We get a lot of submissions that keep the overall theme or overall logic of the template, and then there will be other projects where somebody took the idea and ran with it in a completely different direction.
Adding elements of sound and storytelling makes it really fun to play for us. It's that extra effort that totally deserves to be recognized.”
Maria: “Entrepreneurial mindset, time, commitment, and technical excellence."
"If I see that unusual and fresh approach to a certain problem, if I see the drive, if I see an excellent pitch, if they sell it, if I see the fire in their eyes—that’s what I'm looking for in a winner,” said Maria Yarotska, NEAR Foundation.
Warren: “It's all about that finished product. Is that finished product something that I would want to use? Would I be interested in installing it and actually spending some time with it? The winner needs to be something that catches your eye, has something unique, and has something that differentiates it from the rest of the applications.”
Richard: “I'd say the one that really negatively impacts the score is when they over-indexed on one particular criteria at the detriment of the rest of their submission. The same can be true when you have submissions that are very heavily skewed towards a particular part of the tech stack—when it's extremely back-end heavy and has almost no front end, there's no UI.”
Karen: “Ambiguity is a red flag, and also projects that lack detail and code.
I remember a couple of projects where the video intro looked really cool or they built a home page for their project which looked really slick, but then when you dug into the project or their GitHub, it was a lot lighter on code."
"It depends on what the requirements are for the hackathon. In our case, we were trying to have people produce something a little bit more fully baked,” said Karen Bajza-Terlouw, Databricks.
Kelvin: “If we just got our template back. Maybe the submission just changed the background color or something—it's still great from a product perspective because this person tried, but it isn't great from a judging perspective.”
Maria: “Devpost is very valuable because you can go to a person's profile and see the history of the hackathons they participated in. If I see the same project with a different label or slightly different interface, it looks like they are just looking for a problem for their solution, that could be a red flag for me.
Also, a really big team is not a red flag but… pinkish. I try to avoid bigger teams because I know for a fact that the contribution from each participant was unequal, and there may be a conflict in the bounty distribution stage, which doesn't contribute to the team's stickiness in the ecosystem.”
Warren: “Rehashed ideas that aren't interesting, something that's already been done and already in the market, or is extremely simple. Something that looks like it was a submission just to get a submission out there.”
Richard: “Start early. There are so many things that you don't know as you start to figure out what you’re building and who you're building it for. Do your best to build what you know, and for who you know, or really spend the time to learn who you're building it for.
Focus on what you find extremely interesting and you have fun doing. If you're just doing this because you want to win the prize money but don't really care so much about the thing that you're building, it usually comes through.”
Karen: “The storytelling component is huge, don’t underestimate this. Think of the videos as an opportunity to grab the judges’ attention and really show them why they should sit and watch your project demo. Definitely err on more details, and always double-check you’re hitting every single requirement.”
Kelvin: “Make sure that the basic guidelines are met, and then on top of that, I would be as creative as possible. You really want something that's going to make the judges think.
The second point is taking advantage of whatever resources are provided. If we offer a sound package, make cloud credits free, or partner with another team to provide access to a resource, I would find a way to incorporate that because we’re likely really interested in seeing Flutter used with that product.”
Maria: “Having fun, being approachable, being relatable—this is what a hackathon is about. Keep this spirit going instead of 100% diving into the geekiness of it and the bounty hunting of it.”
Warren: “Definitely make sure that all of the requirements are given significant time. It's not sufficient to just build the app and have it out there. If you're asked for a slide deck and explanations of how stuff works, that needs to be important because it's going to get looked at."
"If the prompt is to build something that does collaboration across multiple products, really show off that use case. Ancillary benefits are cool, but if it doesn't really nail the prompt, it's going to be hard to give it full points," said Warren Marusiak, Atlassian.
Richard: “My favorite part of the judging process is just being absolutely blown away. There's always one to three projects that get submitted that you never would have thought of and you're blown away. I love connecting with those folks after the competition and getting to talk to them more.”
Karen: “I love being a part of the energy around hackathons, the possibilities, and innovation. I get to enable folks to be creative, what’s better than that? The projects and the range of topics and interests that you see are really inspiring.”
Kelvin: “It's that initial moment where you get to see everyone's takes on the challenge that is most exciting to me. And then playing the games, all that kind of stuff is fun."
"It’s that initial moment—we're all sending messages back and forth of, “This one’s so cool! And that one’s so cool!” It's always an eye-opening moment where I'm like, “Really?! I can’t believe people are this creative!” That part is really fun,” said Kelvin Boateng, Google.
Maria: “One of the reasons I really like in-person hackathons is because we get to have pitch competitions. When you see the team on stage, when you coach them on how to present, when you rehearse with them, you see how passionate they are about the project. Pitching is my favorite part.
With online hackathons or longer hackathons, I compare it to watching the flowers grow. As a gardener, you cannot influence all of the factors but you can influence some. When the flowers actually grow, you can own this part of the success and pat yourself on the back and say, ‘Okay, I've done a good job.’”
Warren: “It's cool to see it go from a prompt that we put up a couple of months in the past all the way to the stage show. Picking applications that are going to have a really good stage presence and get a lot of audience reaction is the cool part. Trying to judge whether the audience is going to respond effectively and trying to find the best examples is pretty rewarding. And then when you get them on stage and they do well and the audience is cheering for them—that feels pretty cool.”
Huge thanks to Richard, Karen, Kelvin, Maria, and Warren for sharing their hackathon judging expertise! Check out the current hackathons on Devpost or contact our team if you’d like to know more about running a hackathon.
