Wikimedia Developer Summit/2018/Lessons Learned

The Wikimedia Developer Summit 2018 feedback survey was sent out to participants on 6 February, 2018.

We will close the survey on 20 February, 2018.

Results / summary of the feedback survey are being posted here, currently this page is incomplete.

---

Feedback survey background
rfarrand created the feedback survey. Many members of the Organizing Team made edits or suggested changes.

The feedback survey was sent to participants / wikitech-l, Engineering@ & WMFAll@ on 6 February, 2018.

A reminder about the feedback survey was sent out to all participants who had not filled out the survey (and those who choose to remain anonymous) on 13 February.

The feedback survey closed on 20 February.

The feedback survey received 47 responses

Privacy Statement for this survey

Considerations for next time
This section is based on the fill-in-the-blank/comment sections of the feedback form. Some of these comments will be contradictory. We are mostly trying to include common themes and issues that were felt by groups of people. This section is a bit more subjective than the #Data section below and suggestions may be paraphrased and combined with each other.

In this section comments at the top of each section were mentioned more often while comments at the bottom were mentioned by one person. Some comments are combined and many are shortened for easier summary. The future program committees will review full comments again in the future when they are working on specific areas of the event organization.

Questions for Participants
Please provide ideas for how we can improve the pre-event engagement of participants.

Any suggestions on how to improve session facilitation?

Any suggestions on how to improve topic leader preparation?

Process Questions
Ideas for improvement (related to 2018 participant selection process)

Please suggest topic ideas for 2019 position statements

Documentation Questions
What did you find helpful or confusing about the way we used Phabricator/wikipages/etherpads for organizing the work before and during the summit?

Please provide ideas for how we can improve the event session format

 * Sessions were too large (too many people in each)
 * Sessions should be a max of 2 hours (otherwise, split into smaller sessions)
 * There was not enough time during breaks for informal discussions
 * Social gatherings were too noisy to continue informal discussions
 * If a person’s position statement was accepted, have them lead / talk during the sessions
 * Position papers didn’t always “fit” into the session topics
 * Add lightning talks / unconferences
 * Keep the fireside chat
 * Product managers present their FY goals and their 3-5 year plan at the beginning of each session and then do hour long slots during the day to go over those plans
 * Keep keynotes to less than 30 minutes
 * Keep the wrap-up session at the end of each day for a good recap
 * ‘next steps for languages and cross project collaboration’ and ‘MediaWiki architecture’ programs were too big for one session
 * Some sessions were lacking in a clear purpose or goal and were too broad in subject
 * Base the participants selected on merit, not their position statements
 * Don’t break up the sessions over a break or for keynotes (keep them about 1 1/2 hours each)
 * No more than 2 keynotes a day
 * With fewer tracks and less participation, it felt like it resulted in less getting discussed
 * Create a small event of it’s own for discussions around machine learning
 * Better prepare the topic leaders for their session and to be in sync with other topics
 * Make sure all the AV equipment works
 * Facilitation was good

Do you have a suggestion for a specific keynote speaker for 2019? (Please provide their name, and affiliation / background)
Do you have suggestions for specific topics you’d like the keynote speakers to cover for 2019?
 * Two survey respondents asked not to have keynotes at all
 * Tim Starling, Wikimedia Foundation
 * Don Knuth
 * Someone from GitHub
 * Something related to the events theme
 * Someone from Mozilla
 * Dries Buytaert, founder and lead developer of the Drupal CMS
 * Somebody technical, not politics/social justice
 * John Gilmore (EFF board member)
 * Zeynep Tufecki
 * StackOverflow people like Joel Spolsky or Jeff Atwood, people working in large opensource projects like Drupal, Symfony, PHP, Node, developers or managers from other wikis like Wikia / Wikihow, Clay Shirky or another social computing researcher.

Do you have suggestions for specific topics that you would like to hear about from WMF management at the 2019 summit?

 * some of technical challenges they had in 2018-2019 and how they addressed them
 * thoughts on some of the technical challenges ahead of us, and what we should do to overcome them.
 * how exactly are the funds distributed, and how can the developers influence in the allocation of resources for a small project. i.e. things that a single developer can do by herself
 * Integration of DevSummit outcomes with WMF Annual Plan and resource allocation. How they interact within each other and which one influence and guide the others.
 * A clear direction on third party operated MediaWiki and the appropriate degree of investment as a Wikimedia Foundation.
 * Insight into what and how we should build for interactive agents (voice, chatbots), the direction for A/V+VR and the tie-in to mobile computing and sensors/embedded.
 * A clear direction on our approach for channel expansion while ensuring growth in first party usage across WMF/WMDE-maintained channels.
 * I would like some clarity on our mission. According to T&C's cultural orientation, our technology is a means to an end. However, most people at the dev summit do not agree with this.
 * Specific examples of trade-offs we make, and why. How do we choose to focus on work?
 * How can we be more dynamic and collaborate more across teams?
 * A broad (still concrete) roadmap.
 * Implementation plans for the outcomes of the last summit.
 * How the Stewardship review process will be incorporated into your planning.
 * Less management involvement.
 * WMF management should prepare something, *anything* to the level they prepare a statement to the board.
 * How to manage tech debt
 * I would love to hear/talk about building the ecosystem and tech partnerships within and beyond the movement. But we shouldn't ask that the WMF management alone.
 * Mostly what they hope to get out of the summit.
 * What management did in 2018 to promote MW as a tool outside of WMF and inside WMF. Talk about how MW encapsulates the culture of WM in code.

Are there other things that we should be measuring that we are not asking about?

 * Post-summit impacts / effects / evaluation of outcomes / productivity
 * What changes have you seen since past Dev Summits? / How many sessions from previous years had any of the resolutions carried out, or blocked by lack of resources.
 * Interaction with other developers
 * Who was missing?
 * How can we measure success with such large generic topic areas?
 * Ask people who didn’t apply why they choose not to attend?
 * Ask me what I took away from this event -- why my attendance was important.
 * Did anything useful come out of the event itself or was it just because of the people attending?
 * Did anyone who did not attend gain anything either from reading documentation or following along?
 * Challenges that resulted from dividing staff (not allowing some at this event)

Data
Did you attend the Wikimedia Developer Summit 2018 in Berkeley?

Yes: 61.7%

No: 38.3%

Have you attended any previous Wikimedia Developer Summits and/or Architecture Summits organized by WMF in 2017 or earlier?

Yes: 87.2%

No: 12.8%

Questions for Participants
Omitted: How useful for you were the keynotes? Because these answers might negatively reflect the contribution of one individual they are not included. The answers were helpful to the organizers.

Omitted: How useful for you were WMF management keynotes? Because these answers might negatively reflect the contribution of one individual they are not included. The answers were helpful to the organizers.

How useful was each session for you?

Response options are ranked in order after giving 2 points for every "very useful" response, 1 point for every "useful" response, 0 points for every neutral responses, -1 points for every not useful response and dividing by the number of responses.
 * Growing the MW technical community: 17 responses
 * Supporting 3rd party use of MW: 16 reponses
 * Embracing open source software: 20 responses
 * Knowledge as a service : 16 responses
 * Research Analytics and Machines Learning: 11 responses (tied with next steps for language)
 * Next steps for language and cross project collaboration: 11 responses (tired with research and analytics)
 * Advancing the contributor experience: 15 responses
 * Evolving the MW Architecture: 22 responses

Please tell us how much you agree or disagree with the following statements

Attending this event was worth my time

Strongly Agree: 10

Agree: 10

Neither Agree or Disagree: 5

Disagree: 1

Strongly Disagree: 3

The opportunity to meet fellow developers was valuable to me

Strongly Agree: 17

Agree:

Neither Agree or Disagree: 2

Disagree: 2

Strongly Disagree: 1

I would like to attend this event again next year

Strongly Agree: 14

Agree: 7

Neither Agree or Disagree: 5

Disagree: 2

Strongly Disagree: 1

Process Questions
Based on your understanding of the process, do you think asking participants to write a Position Statement was fair?

Yes: 29.8% (14)

Indifferent: 8.5% (4)

No: 31.9% (15)

I don't know: 6.4% (3)

Other: 11 participants

Based on your understanding of the process, do you think the participant selection process was fair?

Yes: 21.3% (10)

Indifferent: 8.5% (4)

No: 14.9% (7)

I don't know: 46.8% (22)

Other: 4 participants

Documentation Questions
Please rank these tools in order of how helpful they were to you and your topics of interest in following the actives and outcomes of the Wikimedia Developer Summit 2018

Phabricator

Very helpful: 18

Moderately Helpful: 0

Not Helpful : 4

Etherpad

Very helpful: 18

Moderately Helpful: 0

Not Helpful : 4

Wiki

Very helpful: 14

Moderately Helpful: 0

Not Helpful: 5