New Developers/Quarterly/2018-01

This is the second edition of the New Developers Quarterly, a report covering activities, metrics, surveys and lessons learned in the Onboarding New Developers program. This report covers October to December 2017.

Your questions and feedback are welcome in the Discussion page.

Subscribe to receive a notification when a new report is published.

Timeline
Newcomer-focused events, programs, and activities between October–December 2017:


 * October 5th How Technical Collaboration is bringing new developers into the Wikimedia movement, blogpost by Quim Gil
 * October 7th WikiFemHack, the first Wikimedia hackathon to encourage gender diversity in Greece, blogpost by Marios Magioladitis
 * October 13-15th Wikimedia mentors participated in the Google Summer of Code mentor summit at Google headquarters in Sunnyvale, California
 * October 19th Towards building an African Wikimedia Developer Community, blogpost by Felix Nartey and Derick Alangi
 * November 25th How two new Wikimedians spent their time at the Montreal Hackathon, blogpost by Melody Kramer
 * November 26th Wikimedia is taking part in Google Code-in 2017, mentoring students to January 17, 2018 on both coding and non-coding related tasks
 * December 1st Wikimedia Foundation funds six Outreachy interns for round 15, blogpost by Srishti Sethi
 * December 9-10th Wikimedia's Developer Relations organized a workshop around Google Code-in for students of the ChickTech organization in San Francisco (more details)

Key findings
1. Low response rates (~14%) for Wikimedia and Wikimania Hackathon follow-up emails

The follow-up emails included an invitation to participate in a post-event engagement research study and a question; if there is any way we could offer help. 14% newcomers responded to the email, and 6% participated in the research study. We would like to understand how useful this approach is, and more about newcomers who attend Wikimedia events. Thus, we would continue this strategy for a few iterations. In the next round, we plan to send emails immediately afterward to have higher response rates. The current study took us some time to prepare and was rolled out a few months after Wikimania.

2. New developers gain useful information through Wikimedia events

From a very few event participants that we chatted with through the post-event engagement research study, we learned that they got busy with their professional work after the Hackathon and were not contributing much to Wikimedia projects. But they had gathered enough information at the Hackathon that they could use if they want to start contributing again.

3. Newcomers continue to struggle in understanding the code contribution process

From the survey results of both the previous and the current report, it is apparent that new developers struggle initially in understanding the code contribution process, applying coding conventions, learning how to get around Gerrit, git-review, etc. We have been continuing to make progress on improving our related documentation for new developers. We will continue to watch for improvements in this area in the next editions.

4. Dedicated mentors time, a support channel, and self-contained tasks are some suggestions made by new developers

We have already piloted a test instance of Wikimedia Developer Support channel which is running on Discourse. We plan to monitor its use closely and also brainstorm on possible action items for other suggestions made by newcomers.

5. Completion rate for the new developers survey has improved by (~25%)

Shortening the survey, that we piloted among new developers for the previous edition of this report helped gather more responses.

6. Do new developers tend to be working professionals with over five years of experience developing software?

Similarly, in the previous report, the small sample of responses in the current one as well have shown a higher percentage of experienced professional developers than we expected. We will keep watching for these answers in future surveys.

7. Lack of an automated process to measure retained new developers who contribute to code repositories outside Gerrit

Our retention rates in general and more specifically of outreach programs and events are alarming. But, the data account for only those code contributions that have landed in Gerrit. We are currently not set up to measure code activity efficiently from outside Gerrit.

New developers metrics and trends
See also Technical Collaboration Metrics - Onboarding New Developers.

Volunteers contributing patches for review
140 volunteers contributed between October–December 2017. (Source)

QoQ : -6.0%. YoY : -14.6%.

New volunteers attracted
49 new volunteers contributed between October–December 2017. (Source)

QoQ : -15.5%. YoY : -31.0%.

New volunteers retained
Percentage of volunteers active one year (±3months) after their first contribution, out of all new volunteers attracted one year ago. (Source: Calculation on data)

QoQ : +6.2%. YoY : +44.6%.

Review of changesets by new volunteers
233 changesets were contributed by new volunteers. (Source: Calculation on data . Data as of January 19th, 2018.)

Projects with most new volunteers
49 new volunteers contributed to 23 repositories. (Source: "Repos by New Authors")

Outreach programs and events

 * Wikimedia Hackathon 2017 (May 2017):
 * Retained X new developers
 * Wikimania Hackathon 2017 (August 2017):
 * Retained XX new volunteers
 * Google Summer of Code 2017 (May–August 2017):
 * Out of 7 new developers who completed the internship, we retained 0 developers
 * Outreachy Round 15 (December 2017–March 2018):
 * Attracted 6 new developers

Survey analysis
Read through below for a quick summary:

We sent a survey to 36 new developers who submitted code for the first time between September–November 2017. We used the shorter version of the survey designed for the previous quarterly report to understand more clearly demographics and background, motivations, challenges and needs of our new developers.

Out of the 36 new developers to whom we reached out, 18 (50%) completed the survey.

Demographics and background information

 * Respondents of the survey were from United States of America (3), India (3), Germany (2), Cameroon, China, Finland, Italy, New Zealand, Pakistan and United Kingdom
 * A little over half the respondents said they belong to the working professionals category
 * 67% identified themselves as male and 27% as female

Motivations
New developers indicated that they first heard about Wikimedia as a place to contributing code through a MediaWiki workshop in Cameroon, their work colleagues, Outreachy, familiarity with Wikimedia tech (such as by using extensions, Toolforge, MediaWiki.org, or by hosting a wiki, etc.).

Challenges

 * Some challenges that new developers face around Wikimedia infrastructure are: understanding the codebase, Gerrit, coding conventions, non-availability of projects in Github, determining who is the project owner or code reviewer, etc. Other behavioral challenges are deciding what task to work on, learning a language, lack of knowledge of MediaWiki, hesitating to ask questions, etc.
 * Learning resources and channels that new developers refer to when they get stuck: IRC, Google, Stack Overflow, MediaWiki pages, Phabricator, PHP manuals, API docs, asking an experienced developer, etc.

Experience contributing to Wikimedia
More than 75% of the respondents said they are satisfied with their experience contributing to Wikimedia. More than half of the respondents to this survey said they are likely to recommend Wikimedia for code contribution to friends and colleagues.

Suggestions for improvement
Here are some suggestions made by new developers (responses below have been only slightly edited):
 * How-to documentation
 * An automated way to convert files to adhere to the coding style guide
 * GitHub pull requests
 * Dedicated mentors time to help newcomers initially when they begin contributing
 * An IRC channel for new contributors where everyone is there to help one another
 * Easy steps in the task description to reproduce a problem

Actions items from the previous report
We discussed the possible actions items from the previous report and made some progress on them:
 * T161901: Phab[mw:Gerrit/Tutorial is way too much information for new contributors ]] ✅
 * T173537: Can't view images on Phabricator but not on WP0 To be ✅ soon once deployed
 * T178633: Make Wikipedia Android app a featured project for new developers