📖 Developer Satisfaction 2023

The developer satisfaction survey is an annual survey of the Wikimedia developer community.

💯 Code review[edit]

We asked survey takers if, in the past year, they have had their code reviewed by someone else and/or reviewed someone else's code.

90%

🧑‍💻

90% of respondents either reviewed code, had their code reviewed, or both.

🔗 Hours spent on code review per week

We asked survey takers, "In the past year, on average, about how many hours per week do you spend reviewing others’ code as part of your role as a member of the Wikimedia Developer Community?"

1

😬

1 person responded that they spend 24+ hours per week on code review

45% of respondents said they spend 0-4 hours per week reviewing others' code
32% said they spend 4-8 hours per week
19% said they spend 8 or more hours per week
4% were unsure

One person noted in the open answers…

💬 This is probably a really misleading question, because many people do not do wikimedia developer activity full time.

🔗 Review tools used by respondents

🦊

💬 I'd love to use GitLab's CD/CI pipeline for more projects

🆚

🙅

💬 GitLab seems to be missing several features I depend on, including stacked patch sets […] and the ability to ask reviews from multiple people.

We asked survey takers, “Thinking about your work as a member of the Wikimedia Developer Community, have you used any of the following tools used to host code and conduct code review in the past year?"

Of the 116 survey takers who indicated they had participated in code review in the last year,

A large majority (91%) said they used Wikimedia's Gerrit
A smaller majority (66%) said they used GitHub.com
45% said they used Wikimedia's GitLab
10% said they used GitLab.com
7% said they used another review tool

🔗 Review tools used by respondents (by role)

Most roles used Wikimedia's Gerrit, GitHub, and Wikimedia's GitLab in descending order of use over the past year.

The exceptions are the following, who were more likely to use GitHub and Wikimedia's GitLab for their work.

Data engineers
Service/API developers
Test engineers

🔗 Review tools used by respondents (by tenure)

Respondents with 0-3 years tenure and respondents with 4+ years tenure showed similar distributions of review tools used.

🔗 Review tools used by respondents (by hours spent on code review)

Respondents who spent 0-8 hours per week reviewing code and respondents who spent 8+ hours per week reviewing code showed similar distributions of review tools used.

🔗 Code review tools to keep in 2023

We asked survey takers, “You indicated that you used the following tools used to host code and conduct code review in your role as a member of the Wikimedia Developer Community in the past year (2022). Given your experience, do you want to continue using that tool in the current year (2023)?”

If a respondent used a tool in 2022, we asked them if they wanted to keep using it in 2023:

Keep using in 2023
Tool used in 2022	Keep using %	Discontinuing/unsure %
Wikimedia's GitLab	66%	34%
Wikimedia's Gerrit	60%	40%
GitHub	55%	45%
GitLab	33%	66%

🔗 Code review tools to try in 2023

We asked survey takers, “You indicated that you did not use the following tools used to host code and conduct code review in your role as a member of the Wikimedia Developer Community in the past year (2022). Given your experience, do you want to start using that tool in the current year (2023)?”

If a respondent avoided a tool in 2022, we asked them if they wanted to use it in 2023:

Keep avoiding in 2023
Tool avoided in 2022	Keep avoiding/unsure	Ready to use
Wikimedia's Gerrit	100%	0%
GitLab.com	91%	9%
GitHub.com	84%	16%
Wikimedia's GitLab	66%	34%

🔗🕸️ Web of review tools (2022 vs 2023)

We examined the relationships between users' review tool use in 2022 vs. their desired review tool use in 2023.

In the diagram below,

Colors show the tool used in 2022
Numbers around the edge show number of respondents
Ribbons flow from tools respondents used in 2022 to tools they want to use in 2023
No ribbon is exclusive
Example: Green represents survey takers who used Wikimedia's GitLab in 2022. Green ribbons flow to the tools those respondents want to use in 2023. The green ribbon furthest to the left shows that approximately 30 users who used Wikimedia’s GitLab in 2022 would be happy to use Wikimedia’s Gerrit in 2023. The green ribbon furthest to the right shows that approximately 40 users who used Wikimedia’s GitLab in 2022 would be happy to continue using it in 2023.

*Excludes “Other”/write-in review tools used in 2022 (N=9)

🔗 Code review quality

13

⏳

There were 13 negative comments focused around either finding reviewers or getting timely review of their patches.

We asked survey takers how often the feedback on their code is kind, helpful, respectful, and/or provided in a reasonable amount of time (as a member of the Wikimedia Developer Community during the past year).

The majority of respondents indicated that feedback is respectful, helpful, and kind
- 92% said that feedback is always or often respectful
- 87% said that feedback is always or often helpful
- 84% said that feedback is always or often kind
Less than half (45%) said that feedback is always or often timely

🔗 Balancing code review

💬 Finding people to review my patches […] ends up taking more time than actually developing new features/fixing bugs for it.

We asked survey takers how easy or difficult it has been to balancing code review with other Wikimedia developer tasks in the past year as a member of the Wikimedia Developer Community.

34% of respondents said that balancing code review with their other Wikimedia developer tasks is easy or somewhat easy
32% said balancing was neither easy nor difficult
28% said balancing was difficult or somewhat difficult
6% were unsure

🔗 Code review and hosting features ranked by priority

We asked survey takers, “How would you rank the following code hosting and code review features by importance to your existing workflows over the past year? Rank the following features from 1 (most important) to 5 (least important)”

Respondents were asked to rank the following features:

Assigning multiple reviewers
Stacked patch sets
Cross-project dependent changes
Setting up or reconfiguring CI / CD for your repo
Onboarding new team members to your code-review workflow
Another priority not listed above

The feature with the highest average rank was "stacked patch sets", and "setting up or reconfiguring CI/CD for your repo" was the feature with the most #1 rankings.

Repeated responses in "Another priority not listed above" were:

Cross-project search
Syntax highlighting on diffs/patchsets/merge-requests
Attention sets

←⚙️ Phabricator

🔁 Continuous Integration→