Wikimedia Developer Summit/2016/Lessons Learned

From mediawiki.org

This page is a combination of feedback from a survey sent to participants and ideas from the organizers.

It focuses on improvements for next time and things that went poorly. However keep in mind that 94.1% of the people that responded to the feedback survey strongly agreed or agreed that the event was worth their time. This is an attempt to improve.

Suggested Changes for the next Developer Summit

  • Research changes of location. Ideally we will find a location with cheaper food and open ports. https://phabricator.wikimedia.org/T132398
  • Keep a hand written (because of unconference) schedule up at the registration desk, also have a large map of the venue and room locations
  • Spend some time rethinking the use of the combination of Phab / Etherpad / Wiki. https://phabricator.wikimedia.org/T132400
  • More professional signage
  • Better food for vegetarians
  • If the event is at Mission Bay, have a shuttle bus to the MBC in the morning as well.
  • Change the wrap-up session to be more interesting and useful
  • Help people find each other and projects during the unscheduled day
  • Don't hold this event near the holidays

Things to keep

  • Roles / role cards
  • Un-conference
  • Hackathon / Unscheduled 3rd day
  • Small group dinners
  • Pre-event discussion of sessions and session topics

Background on Feedback Survey[edit]

  • rfarrand created the feedback survey form during the Developer Summit.
  • The feedback survey began accepting responses during the last day of the Developer Summit, January 6.
  • Participants received two emails asking / reminding them to fill out the survey. The importance of completing the survey was also mentioned during the summit multiple times.
  • The deadline for filling out the feedback survey was January 18th, the feedback survey was closed on January 19th.
  • 84 out of 155 (54%) participants filled out the feedback survey.
  • An in personal retrospective will be scheduled for the week of January 25th, 2016. Notes from that meeting will be posted.

Considerations for Next Year[edit]

This section is based on the fill-in-the-blank/comment sections of the feedback form. Some of these comments will be contradictory. I included common themes and issues that were mentioned by more than one or two people. This section focuses more heavily on areas that have room for improvement instead of things that went well. This section is a bit more subjective than the #Data section below.

Day 1 & 2 Schedule and Content[edit]

Pre-scheduled Sessions[edit]

Common Themes[edit]
  • There was a good range of topics and it was hard to choose between sessions.
  • The session descriptions could have used a lot of work. They were too long or unclear and it was hard to understand what the session was going to be about.
  • It was helpful to assign roles
  • The sessions were not friendly to people without technical backgrounds
  • The process for scheduling these sessions was "unclear," "overwhelming," and "messy." The process changed at the last minute and and confused everything further.
  • The overview of how the sessions were going to work was helpful in the beginning. The language rules of engagement should have been more internationally targeted, different things are offensive or not in different cultures.
  • The larger sessions were less productive and often without resolution. Sometimes the purpose of the session was completely unclear.
  • Presenters were well prepared
Suggestions for next time[edit]
  • All sessions should have a clear "goal" at the top of its description.
  • Session scheduling process should be worked out at least a month in advance and not changed.
  • Pre-event IRC or hangout with information on what to expect in the prescheduled sessions so it is easy to choose.
  • Wrap-up session was not a good use of time, change it and shorten it. Questions from wrap up should have been discussed on a mailing list.
  • Keynote should be improved

Unconference Sessions[edit]

Common Themes[edit]
  • People liked them and found them to be engaging and useful. The smaller unconference sessions were lead by passionate speakers and resulted in great knowledgeable conversations.
  • It was hard to choose between sessions
  • Roles cards were useful but people did not always stick to using them. More facilitation to make sure people don't forget next time.
  • The scheduling was unfair. It was stated that you could not schedule in advance, but some people did and got to keep their spot.
  • Mixing scheduled and unconference is weird. It also makes it hard to pre plan your time at the Summit.
Suggestions for next time[edit]
  • Encourage more community members to run sessions
  • Ensure enforcement of roles during sessions
  • Sessions should have better agendas created in advance
  • Unconference should be better explained in advance for those not familiar.
  • Define unconference. To some people it means needing to vote on sessions in advance. To others it means more fluidity and encouraging people to get up and move between sessions.
  • Build in "you can leave now" breaks in the middle?

Most valuable session[edit]

Most mentioned:

(TODO: add actual session names / associated tasks + send positive feedback to session leaders)

  • Real time collaboration
  • Unconference Day
  • Code Review
  • Community Wishlist
  • Content format
  • Community Relations
  • Shadow Namespaces
  • Mediawiki Stakeholders

Use of the combination of etherpads, Phabricator and wikis to coordinate the sessions[edit]

In general the reception was positive. 20 people liked the combination of the three tools the way we used them, however there were multiple other mentions of feeling disjointed and overwhelmed with the massive amount of information and places to look for it. Next time we should officially designate an IRC channel. Multiple complaints throughout that wikis and tasks were created at the last minute. Wikipages, tasks, and etherpad links should be created in advance.

Suggestions for Wiki use[edit]

Pretty positive overall reaction to the use of wikis, some people felt they were redundant.

  • All relevant links should always be on one wiki page.
  • Keep the overall schedule on wiki
  • Name wiki pages after session topic and not Phabricator ticket number
  • Improve wikitable display on phones
  • If we continue to use wikis, find a way to ensure that they are kept up to date
Suggestions for Phabricator use[edit]

Phabricator seems to divide the people who were able to put time into focus on the summit in advance and those who showed up and wanted to get a good idea of what was going on. People who were able to participate in the discussions in advance were grateful to have all of the information in one place on the specific topic. People who were jumping in and trying to get involved on a particular issue at the summit felt overwhelmed and missed key pieces of information that were buried in the task. The use of phabricator was polarizing but more people liked its use than not. However there is still a lot of room for improvement.

  • All Phab tasks should be updated before a session with a TLDR. Speakers should also give that TLDR. How can newcomers easily understand the discussion.
  • Phabricator tickets are not a substitute for schedules or session description
  • More follow up tasks should have been created during the event.
  • Each session should have one task. It was too confusing to link three tasks for one session.
  • Requiring the pre-event phab activity in order to schedule a session paid off.
  • Phab is best for history and complete documentation about a topic.
  • Format to consider: Brian Wolff's session: devised in phab:T122987, implemented in phab:T114419
Suggestions for Etherpad use[edit]

People really liked the use of Etherpads at the Developer Summit. Newcomers were impressed and found it easy to use and jump in and start helping.

  • Ensure that useful information and action items from etherpads is transfered to phab and wikis later. Assign someone?
  • Etherpads should be set up in advance, even when using unconference style
  • Whenever possible etherpads should be projected
  • People should identify themselves in the etherpads
  • Someone suggested audio recording the sessions and then going back and checking the etherpad against the recording, we could suggest that and leave it up to the session leaders?
  • Etherpads encouraged collaboration
Overall suggestions for next time[edit]
  • Designate one ultimate source of truth, the use of all three tools was too much for a few people
  • More constantly use tools in the same ways and cross link them in the same ways
  • There was too much duplication and redundancy between Phab and Wiki

Day 3: Unscheduled Day[edit]

The majority of comments about the unscheduled day were that it was valuable, should be kept and that no changes are recommended.

Common Themes and Changes for next time[edit]

  • Don't be so strict with rules "door physically open" and "no scheduling at all"
  • Create a better method for people to track what others are doing. Use IRC, whiteboards, small "what am I working on" signs, or etherpads.
  • It made the previous two days more valuable and it was easier to focus during the previous two days knowing that an unschedule / informal day was going to follow.
  • Have a designated quite space
  • Good emphasizes on the importance of extra time for getting to know volunteers and getting some hacking work done
  • Figure out a system for people who want to find projects or want to learn about ongoing projects.
  • Could we have monthly or quarterly designated "hacking days" for people to continue the work and discussions of the Developer Summit?
  • Collect a short list of projects that could use some work for people who are feeling uninspired, have more support for newcomers.

What did people do[edit]

This is going to be a bit of an abstract list because it is compiled from fill-in-the-blank answers putting it together involved some interpretation of responses. Next time we will use the responses bellow in a multiple choice checkbox format to get an ever better idea. The things below are listed in order of number of times it was mentioned. If someone mentioned multiple things, everything they mentioned was counted separately.

  • Meetings with others continuing dev summit discussions (23)
  • Socializing / small discussions (17)
  • Hacking (13)
  • Talked to people fro my team, team meetings (10)
  • Collaborated on projects / discussions that needed face to face time (10)
  • Helped a newcomer / helped someone else with a problem / Taught something (10)
  • Some Regular Work (8)
  • Collaborative decision making (7)
  • Code review / bug fixing / low priority backlog stuff (5)
  • Did not attend / Overlapping team offsite (5)
  • Learned from someone (5)
  • Introduced people to each other (4)
  • All-Hands Prep (3)
  • Worked with others to solve a problem identified at the summit (3)
  • Spoke with others about projects unrelated to my day-to-day job (1)

Logistics[edit]

Food[edit]

  • Food quality should be improved
  • Better options for vegetarians
  • More types of food, more options of hot food vrs snack food
  • More caffeine free and sugar free food

Transportation[edit]

  • Muni was confusing and difficult
  • The Hotel is too far from the venue
  • Have busses to the venue

Wifi[edit]

  • If you can't help having closed ports, let people know in advance

Other[edit]

  • Don't have the event near the holidays
  • Identify organizers with different badges
  • Help newcomers identify people and what they work on
  • Include directions from the venue to the nearest coffee shop / small store

Anything else?[edit]

  • In the consensus-building process, some facilitation training and tools might be helpful.
  • Acknowledge first time attendees
  • Clarify purpose of the event
  • Wrap-up sessions was confusing and disconnected
  • There was a small tension between the two overarching themes of "running a project" and "building software"
  • It would be interesting to have more non-WMF people attending the event. We could benefit a lot from their input
  • Add lighting talks
  • Speakers should publish slides on commons

What did people like most[edit]

Listed in terms of times it was mentioned

  • In person discussions / meeting people in person (13)
  • Talking to people / Social (11)
  • Unconference (6)
  • WMF and volunteers meeting (5)
  • Atmosphere (4)
  • Scheduled sessions (4)
  • Learning (4)
  • Unscheduled day (3)
  • Decisions that were made / Problem Solving (3)
  • Constructive (2)

Everything else was mentioned once

  • Logistics
  • Quiet room
  • Seeing teammates
  • Having the opportunity to present my ideas

What did people liked least[edit]

Listed in terms of times it was mentioned

  • Sessions scheduled at the same time, hat to choose (4)
  • The Start time was too early (3)
  • Arguments, off topic and boring sessions in the main room (3)
  • Public transportation / venue location (2)
  • Food (2)
  • Wes's keynote (2)
  • The wrap-up (2)

Everything else was mentioned once

  • Undefined Event
  • Intro about what is OK to say and what is not
  • Participants should have prepared better
  • People is sessions did not pay attention
  • Cozy place to rest was missing
  • Closed ports
  • Staff attitudes and culture around development
  • The message that you have to be planning in advance to attend or you should not go
  • More community
  • Unconference scheduling

Data[edit]

Day 1 & 2 Schedule and Content Questions[edit]

How satisfied were you with the pre-scheduled sessions?[edit]

# %
Very satisfied 12 14.3%
Satisfied 58 66.7%
Neither satisfied or dissatisfied 12 14.3%
Dissatisfied 4 4.8%
Very dissatisfied 0 0%

How satisfied were you with the keynote?[edit]

# %
Very satisfied 2 2.5%
Satisfied 20 25%
Neither satisfied or dissatisfied 38 47.5%
Dissatisfied 17 21.3%
Very Dissatisfied 3 3.8%

How satisfied were you with the unconference sessions?[edit]

# %
Very satisfied 28 34.1%
Satisfied 42 51.2%
Neither satisfied or dissatisfied 12 14.6%
Dissatisfied 0 0%
Very Dissatisfied 0 0%

Day 3: Unscheduled Day Questions[edit]

How satisfied were you with the unscheduled day?[edit]

# %
Very satisfied 32 40%
Satisfied 34 42.5%
Neither satisfied or dissatisfied 12 15%
Dissatisfied 2 2.5%
Very Dissatisfied 0 0%

Do you think the unscheduled day was a useful addition to the Developer Summit[edit]

# %
Yes 66 81.5%
No 2 2.5%
Unsure 13 16%

Logistics Questions[edit]

This event overall [Ratings of the Summit logistics][edit]

# %
Very satisfied 29 35.4%
Satisfied 49 59.8%
Neither satisfied or dissatisfied 1 1.2%
Dissatisfied 3 3.7%
Very Dissatisfied 0 0%

Overall organization of the summit [Ratings of the Summit logistics][edit]

# %
Very satisfied 33 40.2%
Satisfied 44 53.7%
Neither satisfied or dissatisfied 2 2.4%
Dissatisfied 3 3.7%
Very Dissatisfied 0 0%
N/A 0 0%

Organized social activities [Ratings of the Summit logistics][edit]

# %
Very satisfied 29 35.4%
Satisfied 41 50%
Neither satisfied or dissatisfied 8 9.8%
Dissatisfied 0 0%
Very Dissatisfied 0 0%
N/A 4 4.9%

Availability of space for informal discussions  [Ratings of the Summit logistics][edit]

# %
Very satisfied 43 52.4%
Satisfied 32 39%
Neither satisfied or dissatisfied 6 7.3%
Dissatisfied 1 1.2%
Very Dissatisfied 0 0%
N/A 0 0%

Food served at the event [How would you rate:][edit]

# %
Excellent 28 33.7%
Good 34 41%
Fair 19 22.9%
Poor 1 1.2%
Very Bad 1 1.2%
N/A or didn't attend 0 0%

Wifi quality at Mission Bay [How would you rate:][edit]

# %
Excellent 16 19.3%
Good 37 44.6%
Fair 12 14.5%
Poor 15 18.1%
Very Bad 2 2.4%
N/A or didn't attend 1 1.2%

Wifi quality at WMF [How would you rate:][edit]

# %
Excellent 47 58%
Good 28 34.6%
Fair 4 4.9%
Poor 0 0%
Very Bad 0 0%
N/A or didn't attend 2 2.5%

Meeting Room equipment  [How would you rate:][edit]

# %
Excellent 27 32.9%
Good 42 51.2%
Fair 6 7.3%
Poor 0 0%
Very Bad 0 0%
N/A or didn't attend 7 8.5%

Anything else?[edit]

Attending this event was worth my time [Please tell us how much you agree or disagree with the following statements.][edit]

# %
Strongly Agree 46 54.8%
Agree 33 39.3%
Neither agree or disagree 1 1.2%
disagree 4 4.8%
strongly disagree 0 0%

The opportunity to meet fellow developers was valuable to me [Please tell us how much you agree or disagree with the following statements.][edit]

# %
Strongly Agree 58 69%
Agree 21 25%
Neither agree or disagree 5 6%
disagree 0 0%
strongly disagree 0 0%

I would like to attend this event again next year. [Please tell us how much you agree or disagree with the following statements.][edit]

# %
Strongly Agree 51 60.7%
Agree 24 28.6%
Neither agree or disagree 7 8.3%
disagree 2 2.4%
strongly disagree 0 0%