How Distill 2014 Was Built

When it comes to planning a conference, the core is always the speakers. A venue, the parties, swag, even ticket sales… all these things are secondary to content. Without talks people will be interested in, none of the rest matters: people will not buy the tickets, collect the swag, attend the party, or be in the venue.

As we approached Distill 2014, we wanted to take the best of what we learned from our content curation in 2013 and improve upon it. The content leaders of last year assembled and began by putting together a team they felt would best represent the essence of development and hold true to the ideals of the conference as we set forward.

To ensure the content team was balanced, representing Engine Yard, the developer community, and the goals we wanted to achieve, we reached out to a diverse group within our company. Developers, community engineers, product managers, marketing specialists, and database engineers were all represented. Once the volunteers agreed to join the team it was time to decide on how talks would be selected.

Invited Speakers

One of the touchiest things we needed to deal with was the idea of invited speakers vs. pure CFP. Being slightly selfish, the content team made a wishlist of invited speakers who we had heard speak at past conferences and felt gave talks that were both amazing and highly relevant to what we are trying to achieve with Distill 2014. We decided we would allow for 6 invited speakers (not including our keynote speakers) so that the majority could come from the CFP. Everyone was allowed to add speakers to the list and from there we all voted. After tabulating the votes we contacted the speakers with the most votes to fill our 6 slots. This, in comparison to the CFP, was the easy part.

CFP Round One

This year we really wanted to expand the number of talks compared to last year. After given venue information we knew we could add a third track. Expanding breaks between talks slightly, we established we had space for 21 speakers, not including keynotes. All the pieces were in place, it was time to roll-out the CFP.

Initially, there was a trickle, which was not much of a surprise as we expected a majority of submissions, as is the case with most conferences, to come in at the end. With the launch of the Distill site, along with a blog post announcing this, we began to see steady traffic. It was time to start reviewing.

Given the setup of our CFP system, it was possible for us to extract information from the database without including speaker name, email, or any identifying information. The goal here was to ensure there was no bias based on speaker experience, gender, or any other characteristic a speaker might include.

Once identifying information was removed we began reviewing the submissions in waves. This prevented us from having overwhelming amounts of content to review all at the end of the process. While this doesn’t sound like an insurmountable problem, it is tiring and unfair as not all submissions would be given the same level of time and energy.

We scored each submission between 1 and 5 points based on its fit with the theme of our conference, its potential interest to the audience, and its openness (not being specific to any language or product).As this was the first round, any talks rating 3.01 or higher would move to the second round. We reviewed hundreds of submissions to fill 15 slots with 6 slots reserved for invited speakers.

CFP Round Two

After the first round, we found we had 31 talks at or over the 4.0 mark. Looking at standard deviation we could see which talks were universally loved and those that were more contentious. We discussed points on those that not everyone agreed upon and began reviewing full talks with identifying information. Knowing the identity did not change the selections so much as confirm we were building the best content lineup we could.

This brought us to the second round. No longer blind, we were able see the speakers, their past experience, and any other information they wished to include. We then scored the submissions with 15 points for content and 10 points for how well it related to the theme. After this second round we’d found our speakers.

Accepted and Rejected

The last step was contacting the 15 accepted speakers, 6 invited speakers, and confirming everyone could make it. It was also necessary to contact those speakers who were rejected and gently let them know we were not able to accept them for this year’s Distill. We also offered feedback if requested so they would have insight into why they were not accepted.

Conclusion

The content team felt the process was effective. While it was an arduous process, it was a labor of love and we are all satisfied with the outcome: a lineup of speakers we would all be happy to see!

On May 15th we announced the speakers for Distill 2014. The full lineup can be found here and you can get tickets here. We hope to see you on August 7th and 8th at Distill 2014!