One year after — learnings on judging ETHBerlin and beyond

By Christopher Heymann and Lasse Claussen, Judging Committee ETHBerlin — and veteran hackathon judges all around the globe :)

We’re huge hackathon fans. We have attended and judged at many of them, the diversity and energy around these events cannot compare to anything we have seen in the traditional startup space. So naturally, we were very happy when we were tasked at last year’s ETHBerlin to organize the judging process. This was also a great opportunity to experiment a bit and try to improve upon a few things along the process, based on our experience and hacker feedback from similar events.

In the 2018 edition of ETHBerlin, we wanted to make sure that the judging would not be reduced to merely a one-directional presentation of a project from a team that has poured their heart into it over a very intense 36- hour period, and turn the process into a feedback session by experts in the space who can share their experiences about the specific project.

Additionally, to make the process fairer, we decided that each team would be judged twice to a) expose them to more judge feedback, and diverse opinions and b) to reduce any biases from individual judging groups — therefore getting a fairer total scoring.

Our points system also produced great results last year. Instead of having every judging group pick a winner from their corresponding judged projects, they gave points across 5 criteria per project, on a scale of 1 to 3. These were then calculated to create a total score which determined the winners. This system allowed better chances for projects that would have not won with the ordinary one-winner-per-judge-group scenari.

Here’s the long read about last years process: https://medium.com/ethberlin/open-sourcing-our-judging-process-75b77f6ba459

After ETHBerlin 2018, we continued attending and taking an active role in many hackathons as judges and we were glad to see that many of them have introduced similar improvements to the ones we created. Almost all Ethereum hackathons have put a great emphasis on giving valuable feedback to the attending developers as the core outcome of the judging process.

For instance, ETHSingapore also chose to adopt a similar points system which creates a global score to determine the winners. This resulted in many cool winning projects, which would’ve probably gone unnoticed in other judging processes. My judging team, for example, had two projects in our cohort that made to the winners’ stage, which in turn meant another judging group had no winners on the final stage. This means that teams don’t need to rely on “luck” to be assigned a judging group that has reviewed less ambitious or smaller projects.

All in all, we think these hackathons that came after ETHBerlin 2018 affirmed our notion that the direction we took last year was successful, and therefore we will rely on it again.

However not everything went smoothly as in every big event. Since the judging period took too long, some judges had to leave in the middle of the process. The points system had no fallback solution for that case, so we needed to quickly find a replacement judge to not severely penalize the projects from a given group. We have been working to ensure judges understand the process better and in more detail for ETHBerlinZwei, in order to avoid similar situations.

If you participated in ETHBerlin 2018or in any other hackathon and have some ideas on how to improve the judging process and system, we would love to hear from you — email us at joinus@ethberlin.com — we are looking forward to your feedback, there’s always room for ideas and improvement!.