New Developers/Quarterly/2017-10/Survey analysis

We sent a survey to new developers who submitted code for the first time during July - September 2017 timeframe. The survey was designed using the Qualtrics platform, with a goal of understanding more clearly demographics and background, motivations, challenges and needs of our new developers. We planned to pilot the survey in two rounds: Out of the 35 new developers to whom we reached out, 9 (25.7%) people completed the survey. Four people started filling out the survey but didn’t finish, indicating that the completion rate of our survey was 69%.
 * 35 new developers who submitted code in July or August (still ongoing)
 * XX new developers who submitted code in September (yet to do)

Note: Since we got quite a few responses, the analysis of this survey may not be entirely justified, and actions for next steps and recommendations indicated in this report (coming soon! ) may not have many conclusions from the survey.

Below are the results from the survey:

Demographics and background information
Geography

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 1}, {"x": 2,  "y": 1}, {"x": 3,  "y": 1}, {"x": 4,  "y": 1}, {"x": 5,  "y": 1}, {"x": 6,  "y": 1}, {"x": 7,  "y": 3} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Countries"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Countries
 * 1 – Canada (11.11%)
 * 2 – Cote d'Ivoire (11.11%)
 * 3 – Germany (11.11%)
 * 4 – India (11.11%)
 * 5 – Sweden (11.11%)
 * 6 – Turkey (11.11%)
 * 7 – United States (33.33%)

33.33% of respondents of this survey were from United States, and 11.11% from Canada, Cote d'Ivoire, Germany, India, Sweden, and Turkey

Age group

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 0}, {"x": 2,  "y": 1}, {"x": 3,  "y": 4}, {"x": 4,  "y": 1}, {"x": 5,  "y": 1}, {"x": 6,  "y": 1}, {"x": 7,  "y": 0},{"x":8, "y":1} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Age"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Age group
 * 1 – <18 (0.0%)
 * 2 – 18-24 (11.11%)
 * 3 – 25-34 (44.44%)
 * 4 – 35-44 (11.11%)
 * 5 – 45-54 (11.11%)
 * 6 – 45-54 (11.11%)
 * 7 – 65+ (0.0%)
 * 8 – I prefer not to answer (11.11%)

44.44% of respondents of this survey said they belong to the age group between 25-34

Gender

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 7}, {"x": 2,  "y": 1}, {"x": 3,  "y": 0}, {"x": 4,  "y": 1}, {"x": 5,  "y": 0}, {"x": 6,  "y": 0} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Gender"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Gender
 * 1 – Male (77.78%)
 * 2 – Female (11.11%)
 * 3 – Agender / None (0.0%)
 * 4 – Other / Non-Binary / Multiple Genders / Genderfluid (11.11%)
 * 5 – I prefer not to answer (0.00%)
 * 6 – Optionally tell us your gender (0.00%)

77.78% of respondents of this survey identify themselves as male

Current affiliation

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 1}, {"x": 2,  "y": 6}, {"x": 3,  "y": 1}, {"x": 4,  "y": 1}, {"x": 5,  "y": 0}, {"x": 6,  "y": 0},{"x": 7,  "y": 0},{"x": 8,  "y": 0} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Current affiliation"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }


 * Current affiliation
 * 1 – Student (11.11%)
 * 2 – Working Professional (66.67%)
 * 3 – Self-employed (11.11%)
 * 4 – Freelancer (11.11%)
 * 5 – Homemaker (0.00%)
 * 6 – Unemployed (0.00%)
 * 7 – I prefer not to answer (0.00%)
 * 8 – Other (0.00%)

66.67% of respondents of this survey are working professionals

Years of experience developing software

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 1}, {"x": 2,  "y": 0}, {"x": 3,  "y": 2}, {"x": 4,  "y": 0}, {"x": 5,  "y": 0}, {"x": 6,  "y": 6} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Years of experience"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Years of experience developing software
 * 1 – Less than a year (11.11%)
 * 2 – 1 to 2 years (0.00%)
 * 3 – 2 to 3 years (22.22%)
 * 4 – 3 to 4 years (0.00%)
 * 5 – 4 to 5 years (0.00%)
 * 6 – More than 5 years (66.67%)

66.67% of respondents of this survey said they have more than five years of experience developing software

Other professional development activities outside work

Contributing to open source, attending conferences, leading local Wikimedia community, organizing meetups, etc.

Contributions
Types of contributions

Participants could select more than one option

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 6}, {"x": 2,  "y": 5}, {"x": 3,  "y": 5}, {"x": 4,  "y": 4}, {"x": 5,  "y": 3}, {"x": 6,  "y": 2}, {"x": 7,  "y": 2}, {"x": 8,  "y": 1}, {"x": 9,  "y": 1}, {"x": 10, "y": 1}, {"x": 11, "y": 1} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Types of contributions"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Types of contributions
 * 1 – Fixing bugs (19.35%)
 * 2 – Reporting bugs (16.13%)
 * 3 – Editing articles (16.13%)
 * 4 – Uploading media to Wikimedia Commons (12.90%)
 * 5 – Task discussion (9.68%)
 * 6 – Feature development (6.45%)
 * 7 – Documentation (6.45%)
 * 8 – Organizing meetups (3.23%)
 * 9 – Planning technical implementation (3.23%)
 * 10 – Answering developer queries (3.23%)
 * 11 – Other (3.23%)

35.48% of respondents of this survey said they fix or report bugs

Frequency of making contributions

Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 4}, {"x": 2,  "y": 0}, {"x": 3,  "y": 3}, {"x": 4,  "y": 0}, {"x": 5,  "y": 1}, {"x": 6,  "y": 0}, {"x": 7,  "y": 1}, {"x": 8,  "y": 0} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Frequency of making contributions"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Frequency of making contributions
 * 1 – Once in a few months (44.44%)
 * 2 – Once a month (0%)
 * 3 – A few times a month (33.33%)
 * 4 – Does not apply to me because: (0%)
 * 5 – A couple times a week (11.11%)
 * 6 – Once a day (0%)
 * 7 – Multiple times a day (11.11%)
 * 8 – Other (0%)

44.44% of respondents of this survey said they contribute code to Wikimedia projects once in a few months and 33.33% a times a month

Motivations
New developers who participated in the survey indicated that they first heard about Wikimedia as a place to contributing code: First topics or projects new developers worked on: lexeme, wikiclass, bug fix, extension: ArticleToCategory2, Pywikibot and Wikimedia site request, newsletter, etc. New developers chose to work on a project as they:
 * Through Wikimedia members at a conference
 * From a friend
 * Through African Wikimedia developers project
 * Github
 * By installing MediaWiki themselves
 * Taking inspiration from bug fixes made by viewing them on Wikimedia's Gerrit
 * Found it interesting
 * Project was a good fit for their graduate studies
 * Could share their research by working on the project
 * Wanted to improve the project
 * Wanted to get started with fixing bugs, submitting patches, etc.
 * Were being asked to do so at work
 * Wanted to learn new things and to minimize burden on Wikimedia developers
 * Heard about it at a hackathon

Challenges
Understanding of Wikimedia's code contribution process

Participants (n) = 9 {	"version": 2, "width": 300, "height": 250, "data": [ {			"name": "table", "values": [ {					"x": 1, "y": 1 },				{					"x": 2, "y": 5 },				{					"x": 3, "y": 3 },				{					"x": 4, "y": 0 }			]		}	],	"scales": [ {			"name": "x", "type": "ordinal", "range": "width", "domain": { "data": "table", "field": "x" }		},		{			"name": "y", "range": "height", "domain": { "data": "table", "field": "y" }		}	],	"axes": [ {			"type": "x", "scale": "x", "title": "Understanding of Wikimedia's code contribution process" },		{			"type": "y", "scale": "y", "title": "No of developers" }	],	"marks": [ {			"type": "rect", "from": { "data": "table" },			"properties": { "enter": { "x": { "scale": "x", "field": "x" },					"width": { "scale": "x", "band": "true", "offset": -1 },					"y": { "scale": "y", "field": "y" },					"y2": { "scale": "y", "value": 0 },					"fill": { "value": "steelblue" }				},				"update": { "fill": { "value": "steelblue" }				}			}		}	] }
 * Understanding of Wikimedia's code contribution process
 * 1 – I have a clear understanding of Wikimedia's code contribution process (11.11%)
 * 2 – I am developing my understanding of Wikimedia's code contribution process (55.65%)
 * 3 – I struggle with Wikimedia's code contribution process (33.33%)
 * 4 – No opinion: (0%)

55.65% of respondents of this survey said they are developing their understanding of Wikimedia's code contribution process and 33.33% said they struggle with it

Some challenges that new developers face are: finding time, connectivity issues, documentation, git-review process and understanding conventions when working with Gerrit, ignorance to patch for a bug fix, not being able to access Phabricator due to IP ban, poor communication, lack of agile practices, poor requirements, project vision, etc.

Learning resources and channels that new developers refer to when they get stuck: Help pages on Wikitech, # wikimedia-devtools, IRC, Google, Github, Gerrit, other developers, teammates, unconferences, video tutorials, books, the contribution wiki, Stack overflow, documentation on MediaWiki, extension documentation page, etc.

Experience contributing to Wikimedia
Participants (n) = 9 { "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 5}, {"x": 2,  "y": 2}, {"x": 3,  "y": 1}, {"x": 4,  "y": 1} ,{"x": 5,  "y": 1} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Experience contributing to Wikimedia"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Experience contributing to Wikimedia
 * 1 – Extremely satisfied (50.00%)
 * 2 – Somewhat satisfied (20.00%)
 * 3 – Neither satisfied nor dissatisfied (10.00%)
 * 4 – Somewhat dissatisfied: (10.00%)
 * 5 – Extremely dissatisfied: (10.00%)

{ "version": 2, "width": 300, "height": 250, "data": [ { "name": "table", "values": [ {"x": 1, "y": 3}, {"x": 2,  "y": 2}, {"x": 3,  "y": 3}, {"x": 4,  "y": 0} ,{"x": 5,  "y": 1} ] } ], "scales": [ { "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "x"} }, { "name": "y", "range": "height", "domain": {"data": "table", "field": "y"} } ], "axes": [ {"type": "x", "scale": "x", "title":"Recommending Wikimedia for code contribution"}, {"type": "y", "scale": "y", "title": "No of developers"} ], "marks": [ { "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "y"}, "y2": {"scale": "y", "value": 0} }, "update": { "fill": {"value": "steelblue"} } } } ] }
 * Recommending Wikimedia for code contribution to friends and colleagues
 * 1 – Extremely likely (33.33%)
 * 2 – Somewhat likely (22.22%)
 * 3 – Neither likely nor unlikely (33.33%)
 * 4 – Somewhat unlikely: (0.00%)
 * 5 – Extremely unlikely: (11.11%)

''50.00% of respondents of this survey said they are extremely satisfied with their experience contributing to Wikimedia. 33.33% said that they are extremely likely to recommend Wikimedia for code contribution to friends and colleagues.''

Suggestions for improvement
Here are some suggestions made by new developers (responses below have been only slightly edited):
 * Standard toolbox over in-house or unusual systems (Gerrit, Phragile, ...)
 * More autonomy for development teams: build a trust-based DevOps culture allowing teams to adopt Continuous Integration/Continuous Delivery practices and deliver value in small iterations (current deployment guide mentions a lead time of over six weeks for new services!).
 * Breaking focused services out of the monolithic MediaWiki core.
 * Be bold with code changes. Approve more patches in a timely fashion. Enlarge the group of developers who are authorized to approve patches.
 * Surveying and documenting conventions on how to interact with other developers through Phabricator and Gerrit.
 * Git commit message guidelines suggests "To express that a commit resolves a bug, add Bug: Txxx in the footer at the end of the commit message." What do you do when the commit doesn't resolve the bug because the bug has multiple items? The "Bug: Txxx" footer should _always_ be there, because that automatically adds a change notice to Phabricator. That convention isn't made particularly clear from the guidelines.
 * Fix the IP ban that devoid access to Phabricator
 * Be better on follow up issues
 * Better guides. Make more documentation on Wikitech. So that we can contribute more.