Wikimedia Release Engineering Team/Goals/The next year

From MediaWiki.org
Jump to navigation Jump to search

This page tracks the next (at least) four quarters (moving over time) of the WMF Release Engineering goals/roadmap.

201718-Q1[edit]

#releng-201718-q1 (Phabricator project) -- All Technology team Q1 goals: Wikimedia_Technology/Goals/2017-18_Q1

Annual Program/Outcome Quarterly Objective Tech Goal Team Goal Dependencies Team members ETA Status
Program 1: Availability, performance, and maintenance

Outcome 1: All production sites and services maintain current levels of availability or better

Objective 1: Deploy, update, configure, and maintain production services

D: Tech Debt
  • Operations
  • Services
  • Discovery
  • Cloud Services
  • Tyler
  • Chad
  • Antoine
  • Mukunda
EOQ Yes Done
Program 1: Availability, performance, and maintenance

Outcome 5: effective and easy-to-use testing infrastructure and tooling

Milestone 1: Develop and migrate to a JavaScript-based browser testing stack

  • Migrate majority of developers to JavaScript based browser test framework (webdriver.io)
C. Improve our own feature set

D: Tech Debt

  • All developers
  • Notably: Wikidata, CirrusSearch
  • Zeljko
End of Q2 In progress In progress
Program 6: Streamlined service delivery

Outcome 2: unified pipeline towards production deployment.

Objective 2: Set up a continuous integration and deployment pipeline

  • Define functional tests for Mathoid running on the staging Kubernetes cluster for use in future gating decisions - task T170482
  • Define method for monitoring and reacting to the above functional tests - task T170483
C. Improve our own feature set

D: Tech Debt

  • Operations
  • Services
  • Lead: Tyler
  • Antoine
  • Dan
  • Jean-Rene
EOQ N Not done


201718-Q2[edit]

#releng-201718-q2 (Phabricator project) -- All Technology team Q2 goals: Wikimedia_Technology/Goals/2017-18_Q2

Annual Plan Program Annual Plan Outcome Milestone/Objectives Quarterly Work/Goal Tech Goal Dependency Team members ETA Status
Program 1: Availability, performance, and maintenance Outcome 1: All production sites and services maintain current levels of availability or better. Objective 1: Deploy, update, configure, and maintain production services (Traffic infrastructure, databases & storage, MediaWiki application servers, (micro)services, network, and miscellaneous sites & services) Work
  • workboard
  • Resolve some scap & git-fat technical debt.
  • Implement most-wanted features identified by users of scap.
D. Modernization, renewal and tech debt goals Technical Operations
  • Lead: Mukunda Modell
  • Chad Horohoe
Ongoing To do To do
Outcome 5: effective and easy-to-use testing infrastructure and tooling Objective 1: Maintain existing shared Continuous Integration infrastructure Work
  • Docker-CI
    • A generalized POC for a docker-based CI.
  • MW Nightlies
    • TBC
D. Modernization, renewal and tech debt goals Technical Operations, Cloud Services
  • Lead: Antoine Musso
Ongoing Yes Done
Milestone 1: Develop and migrate to a JavaScript-based browser testing stack Goal
  • Migrate majority of developers to JavaScript based browser test framework (webdriver.io)
C. Improve our own feature set

D: Tech Debt

All developers
  • Lead: Zeljko Filipin
EOQ Yes Done
Outcome 6: Engineering teams can effectively plan, track, and complete their work. Milestone 1: Maintain and improve existing shared code-review platform (Gerrit) Work
  • Ongoing maintenance and upgrades
D Modernization, renewal and tech debt goals Technical Operations
  • Chad Horohoe
Ongoing In progress In progress
Milestone 2: Maintain and improve existing shared project management platform (Phabricator) Work
  • on-going work to move to modern hardware and cross-datacenter resiliency
D Modernization, renewal and tech debt goals Technical Operations (Traffic)
  • Mukunda Modell
Ongoing Yes Done
Program 3: Addressing technical debt Outcome 1: The amount of orphaned code that is running Wikimedia “production” services is reduced. Objective 1: Define a set of code stewardship levels (from high to low expectations) - task T174090 Goal
  • Objective 1.
B: Features we build for others

D: Tech Debt

Code Health Group
  • Lead: Jean-Rene Branna
  • Greg Grossmeier
EOQ Yes Done
Objective 2: Identify and find stewards for high-priority/high use code segment orphans - task T174091 Goal
  • Objective 2.
B: Features we build for others

D: Tech Debt

Code Health Group, Current and future code/product stewards End of Q3 To do To do
Outcome 2: Organizational technical debt is reduced.

Objective 1: Define a “Technical Debt Project Manager” role that regularly communicates with all Foundation engineering teams regarding their technical debt

Goal
  • Objective 1.
B: Features we build for others

D: Tech Debt

WMF Product Mangers (or similar) EOQ Yes Done

Objective 2: Define and implement a process to regularly address technical debt across the Foundation

Goal
  • Objective 2.
B: Features we build for others

D: Tech Debt

End of Q3 To do To do
Program 6. Streamlined service delivery Outcome 2: Developers are able to develop and test their applications through a unified pipeline towards production deployment. Objective 2: Set up a continuous integration and deployment pipeline to publish new versions of an application to production via testing and staging environments that reliably reproduce production Goal
  • Complete build phase of release pipeline
    Build phase of the chart
    • Build test variant
    • Run test entrypoint w/developer feedback - services dependency
    • Build production variant w/developer feedback - services dependency
    • Tag production container
    • Push to production docker registry - ops dependency - staging namespace
C. Improve our own feature set

D: Tech Debt

Technical Operations, Services
  • Lead: Tyler Cipriani
  • Dan
EOQ Yes Done


201718-Q3[edit]

#releng-201718-q3 (Phabricator project) -- All Technology team Q3 goals: Wikimedia_Technology/Goals/2017-18_Q3

Annual Program/Outcome Objective Quarterly Work Tech Goal Dependency Team members ETA Status
Program 1: Availability, performance, and maintenance

Outcome 5: We have effective and easy-to-use testing infrastructure and tooling for developers.

Milestone 1: Develop and migrate to a JavaScript-based browser testing stack

Specific improvements to the now canonical framework, see: task T182421, notably:
  • Upgrade webdriverIO to version 4.9
  • Investigate replacing nodemw with mwbot
  • Video recording for Selenium tests in Node.js
B: For others
  • Lead: Zeljko
EOQ To do To do
Objective 1:  Maintain existing shared Continuous Integration infrastructure
  • Draft requirements for a CI k8s cluster in preparation for Ops to create it in the following quarter(ish)
  • Nodepool deprecation: npm and php jobs, but there will be a long tail of corner case jobs
B: For others
  • TechOps
  • Lead: Antoine
EOQ To do To do
Program 3: Tech Debt

Outcome 1: The amount of orphaned code that is running Wikimedia “production” services is reduced.

Objective 2: Identify and find stewards for high-priority/high use code segment orphans - task T174091 Obj. 2 C: Tech Debt All teams
  • Lead: Jean-Rene
  • Greg
EOQ In progress In progress
Program 3: Tech Debt

Outcome 2: Organizational technical debt is reduced.

Objective 2: Define and implement a process to regularly address technical debt across the Foundation  Obj. 2 C: Tech Debt
  • Lead: Jean-Rene
  • Greg
EOQ In progress In progress
Objective 3: Promote and surface important technical debt topics at large gatherings of Wikimedia developers (e.g., DevSummit and Hackathon(s)) Obj. 3 C: Tech Debt
  • Lead: Jean-Rene
  • Greg
EOQ4 To do To do
Program 6: Streamlined service delivery Objective 2: Set up a continuous integration and deployment pipeline Build a functional PoC to verify the basic functionality of the 'production' image in the context of an isolated k8s deployment initially targeting mathoid. C: Tech Debt

D. Improve our own feature set

  • Operations
  • Services
  • Lead: Tyler
  • Antoine
  • Dan
EOQ In progress In progress


201718-Q4[edit]

#releng-201718-q4 (Phabricator project) -- All Technology team Q4 goals: Wikimedia_Technology/Goals/2017-18_Q4

Annual Program/Outcome Objective Quarterly Work Tech Goal Dependency Team members ETA Status
Program 1: Availability, performance, and maintenance

Outcome 5: We have effective and easy-to-use testing infrastructure and tooling for developers.

Objective 1:  Maintain existing shared Continuous Integration infrastructure Migrate away from Nodepool - task T190097
  • Migrate MediaWiki PHPUnit tests to Shipyard (docker-based CI) (~40% of Nodepool usage) - task T183512
  • Add Composer support to Blubber - task T186547
  • Add Python/Tox support to Blubber - task T186545
  • Add Ruby/Gem/Bundler/Rake support to Blubber - task T188950
B: For others
  • SRE
  • WMCS
  • Lead: Antoine
  • Dan
  • Tyler
EOQ
Program 3: Tech Debt (aka Code Health)

Outcome 1: The amount of orphaned code that is running Wikimedia “production” services is reduced.

Objective 2: Identify and find stewards for high-priority/high use code segment orphans - task T174091 Broad role out of Code Stewardship model.
  • Update Maintainers/Developers page with currently known Code Stewards.
  • Use Code Stewardship review process to address gaps.
  • Deploy dashboard of Code Stewardship Coverage
C: Tech Debt
  • All engineering teams
  • CPO/CTO
Lead: JR EOQ
Program 3: Tech Debt

Outcome 2: Organizational technical debt is reduced.

Objective 2: Define and implement a process to regularly address technical debt across the Foundation Roll out of technical debt reduction approaches.
  • Identify early adopter engineering teams to test approaches.
C: Tech Debt
  • Select group of engineering teams
Lead: JR EOQ
Program 6: Streamlined service delivery

Outcome 2: Developers are able to develop and test their applications through a unified pipeline towards production deployment.

Objective 2: Prove viability of testing staged service containers alongside MediaWiki extension containers
  • Add Composer support to Blubber - task T186547
  • Small, standalone, MediaWiki containers built using Blubber
    • Limited scope with: Debian Stretch, php7, composer dependencies, Mariadb, Apache (or last two with standalone containers)
  • Build MW base containers upon branch cut at master branch point from core
  • Build ext specific containers using MW image as base
C: Tech Debt

D. Improve our own feature set

  • Lead: Tyler
  • Antoine
  • Dan
EOQ