Developer Satisfaction Survey/2026/Continuous integration
📖 Developer Satisfaction 2026 Report
🔁 Continuous Integration
[edit]tl;dr
- Queue time satisfaction – 46% respondents are satisfied with CI queue time in Gerrit: pretty much the same as last year.
- Gerrit CI satisfaction – 60%
New questions!
- GitLab CI satisfaction – 53%
- Blubber satisfaction – our buildkit-compatible container image builder – 40%
- Browser tests – Folks hate flaky browser tests!
- Visual regression tests – Only a few folks using them.
# CI use in Wikimedia Gerrit satisfaction
Of those who responded,
- 60% said they were satisfied
- 20% said they were neither satisfied nor dissatisfied
- 20% said they were dissatisfied
# CI queue time satisfaction
last year
Of the 87 survey takers who indicated that they did use Wikimedia's CI in the past year and responded,
- Less than half (46%) said they were satisfied with CI queue time
- 22% said they were neither satisfied nor dissatisfied
- 31% said they were dissatisfied
- 1% were unsure
# CI use in Wikimedia Gitlab satisfaction
“How satisfied are you with continuous integration in Wikimedia's Gitlab?”Of those who responded,
- 53% said they were satisfied
- 18% said they were neither satisfied nor dissatisfied
- 20% said they were dissatisfied
- 9% said they were unsure
# Blubber satisfaction
”How satisfied are you with your use of Blubber?”We asked respondents: "In the past year have you used Blubber to build container images?" For the 15 respondents who said Yes, we then asked how satisfied they were with Blubber.
- 40% said they were satisfied
- 20% said they were neither satisfied nor dissatisfied
- 20% said they were dissatisfied
- 20% said they were unsure
# MediaWiki browser tests use
”In the past year, have you used MediaWiki browser tests?”Respondents could select more than one answer.
Of the 54 who responded to this question,
- 72% said they have seen the output of MediaWiki browser tests in CI in the past year
- 41% said they run MediaWiki browser tests in the past year
- 35% said they have maintained MediaWiki browser tests in the past year
- 28% said they have written MediaWiki browser tests in the past year
# MediaWiki browser tests issues ranked
”Please rank the following browser test issues by how much each slows down your development speed (starting with the one that slows down your development speed the most).”The highest ranked issue (i.e., slowing down development the most) was browser test flakiness, followed by performance of the browser test running in CI.
# MediaWiki browser ease
”For browser tests, how would you rate the ease of… writing new browser tests / maintaining browser tests / running the existing browser tests locally ?”For all three actions, respondents said "difficult" more than any other response.
- 48% of respondents said maintaining is difficult
- 45% of respondents said that running is difficult
- 43% of respondents said the writing is difficult
# MediaWiki visual regression test use
”In the past year, have you used visual regression tests for MediaWiki?”Of the 99 respondents who answered, the majority (73%) said 'No', while 11% said 'Yes' and 15% weren't sure.
# MediaWiki visual regression test frequency
”Thinking about the past year, how often do you use visual regression tests for MediaWiki?”Of the 11 respondents, 45% said 'Very often' or 'Often', while 55% said 'Sometimes' or 'Rarely'.
# MediaWiki visual regression test agreement
”How much do you agree with the following statements about visual regression tests for MediaWiki? 'Visual regression tests are essential to my workflow', 'It is important to run visual regression testing on every proposed change to my code'”
63% agreed and 27% disagreed with the statement "It is important to run visual regression testing on every proposed change to my code."
63% agreed and 9% disagreed with the statement "Visual regression tests are essential to my workflow."
# MediaWiki visual regression test nonuse
”What is the main reason you don’t use visual regression tests for Mediawiki?”For respondents who said they hadn't used visual regression tests for MediaWiki in the past year, we asked them why they don't use it.
Of the 65 who answered,
- 42% said "I don’t work with visual interfaces, so visual regression tests are irrelevant to me"
- 29% said "I’m unfamiliar with the visual regression testing tooling for MediaWiki"
- 17% said "I’m unfamiliar with visual regression testing as a concept"
- 9% said "Other testing methods already meet my needs sufficiently"
- 3% said "They currently don’t meet my needs"
