Wikimedia Technology/Annual Plans/FY2019/TEC13: Code Health/Goals

= Program Goals and Status for FY18/19 =

TEC13: Code Health


 * Goal Owner: Greg Grossmeier
 * Program Goals for FY18/19: The program shares its goals/vision with the Code Health Group, which is: work on efforts that universally improve the lives of engineers and their ability to write products with shorter iteration time, decreased development effort, greater stability, and improved performance. Working closely with the Code Health Group, the Code Health Program will focus on the software engineering processes and practices that enable both experienced and junior technical contributors to deliver high value/quality features to the various Wikimedia projects in the most productive way possible
 * Annual Plan: TEC13: Code Health
 * Primary Goal is Knowledge as a Service: Evolve our systems and structures
 * Tech Goal: Sustaining



 = Q1 Goals =

Outcome 1 / Output 1.1
Increase software stewardship levels of our deployed code
 * Assess deployed code and prioritize stewardship gaps.

Dependencies on: Code Health Group

Goal(s)

 * Investigate and propose record of origin (ROO) for deployed code (currently Developers/Maintainers page)

Status
July 2018

August 10, 2018
 * should get started on this in the next week or so (trying to get to a single record of origin (automated if possible))

September 14, 2018
 * This is now and will need to do a few conversations with SRE folks for clarification and information for ROO.

October 2,, 2018
 * This is now and will wrap up in the next couple of days.

Outcome 1 / Output 1.3
Increase software stewardship levels of our deployed code
 * Using stewardship review process, create plan of action for top priority items each quarter.

Dependencies on: Development Teams

Goal(s)

 * Perform existing review process for Q1 cycle.

Status
July 2018

August 10, 2018
 * this work is scheduled to start in the next week.

September 14, 2018
 * There wasn't anything submitted so far this quarter, so it's still in a status until there is work to be done.

Outcome 2 / Output 2.1
We reduce the number of testable regressions from hitting our users
 * Integrate regression testability evaluation into our on-going post-mortem process.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

 * Add test evaluation to post mortem review process.

Status
July 2018

August 10, 2018

September 14, 2018
 * This is now ✅, we might need a slightly different way to collect this information, as right now the document is a bit difficult to use after an incident. We'll take a look at this in the future.

Outcome 2 / Output 2.2
We reduce the number of testable regressions from hitting our users
 * Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Development Teams

Goal(s)

 * Review existing e2e test coverage.
 * Define prioritization scheme.
 * Prioritize e2e testing gaps.

Status
July 2018

August 10, 2018
 * Need to find time to work on this, but it's a high priority task

September 14, 2018
 * This has been for the last couple of weeks — we're looking at the repos in production and noting the activity in the repos over time and will figure out which ones need the end to end coverage.

October 2, 2018
 * This has nearly finished, and is

Outcome 2 / Output 2.3
We reduce the number of testable regressions from hitting our users
 * Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Code Health Group

Goal(s)

 * Make current unit testing coverage more visible by reporting out to Engineering Management.
 * (maybe, TBC) set coverage goals

Status
July 2018

August 10, 2018
 * The unit testing coverage reporting is in progress but the actual report hasn't yet gone out quite yet.

September 14, 2018
 * This is still and will have the first report out by end of Sept.

October 2, 2018
 * This is now ✅

Outcome 3 / Output 3.2
Reduce Technical Debt
 * Tech Debt Management process rolled out.

Dependencies on: Development Teams

Goal(s)

 * Platform and Search Platform teams are using TDM PoC

Status
July 2018

August 10, 2018
 * Worked with both Platform teams to talk about their tech debt and how to manage them. Might want to revise / change up the existing tech debt tags in Phabricator.

September 14, 2018
 * This is currently ❌ due to the other teams being super swamped. Will roll over to Q2's work.

Outcome 3 / Output 3.4
Reduce Technical Debt
 * Reduce technical debt in the MediaWiki core, by refactoring and improving internal interfaces and policies.

Dependencies on: Platform Team

Goal(s)

 * Identify key Tech Debt areas
 * Put in place Tech Debt management process for PEP

Status
July 2018

August 10, 2018
 * Part of output 3.2's work, going slowly because PE Platform is still getting organized.

September 14, 2018
 * This is currently ❌ due to the other teams being super swamped. Will roll over to Q2's work.

Outcome 4 / Output 4.1
Increase visibility into Code Health
 * Define Code Health Metrics

Dependencies on: Code Health Group

Goal(s)

 * Define base Code Health metric set.

Status
July 2018

August 10, 2018
 * Gotten skeleton crew up and running, getting base set of metrics identified.

September 14, 2018
 * The working group has met for the first time, and have a follow-up meeting in another week or so. Everything is at this point and we might want to update the goal based on the working group's conversations to figure out 1 or 2 meaningful metrics.

October 2, 2018
 * This is now ✅

 = Q2 Goals =

Outcome 1 / Output 1.1
Increase software stewardship levels of our deployed code


 * Assess deployed code and prioritize stewardship gaps.

Goal(s)

 * Update/refresh review queue (review process for initial code deployment)

Status
October 2, 2018
 * This work has not yet started

November 8, 2018
 * Initial proposal was created and now and waiting to discuss with stakeholders.

December 6, 2018
 * This is still and we will meet up with SRE to go over initial proposal thoughts. This will continue in to Q3.

Outcome 2 / Output 2.1
We reduce the number of testable regressions from hitting our users


 * Integrate regression testability evaluation into our on-going post-mortem process.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

 * none

Status
October 2, 2018
 * This work has not yet started - will work on it when there is work to do.

November 8, 2018


 * No PMs thus far.

December 6, 2018
 * This goal was adding to post mortem reports, and can be considered ✅

Outcome 2 / Output 2.2
We reduce the number of testable regressions from hitting our users


 * Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

 * 5 of the 15 prioritized repositories have at least 1 end-to-end test.

Status
October 2, 2018
 * This work has not yet started

November 8, 2018


 * Prioritization data has been collected and starting to reach out to top 5 candidate teams.

December 6, 2018
 * We will contact stakeholders about creating tests for the prioritized repos, this goal is now.

Outcome 2 / Output 2.3
We reduce the number of testable regressions from hitting our users


 * Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Core Platform Team, Code Health Group

Goal(s)

 * Assess Platform unit test practices and define improvement plan

Status
October 2, 2018
 * This work has not yet started

November 8, 2018


 * This work has not yet started

December 6, 2018
 * We have now begun working with the Core Platform team on this

Outcome 3 / Output 3.2
Reduce Technical Debt


 * Tech Debt Management process rolled out.

Dependencies on: Development Teams, Core Platform Team, Search Team, Code Health Group

Goal(s)

 * Core Platform and Search Platform teams are using TDM PoC

Status
October 2, 2018
 * This work has not yet started

November 8, 2018
 * Had initial discussions with Editor group to understand their current approach to managing tech debt.

December 6, 2018
 * Met with the Core Platform team last week, much of the work currently with the Platform Evolution program, and is already oriented around this goal of tech debt.

Outcome 3 / Output 3.4
Reduce Technical Debt


 * Reduce technical debt in the MediaWiki core, by refactoring and improving internal interfaces and policies.

Dependencies on: Core Platform Team, Code Health Group

Goal(s)

 * Identify key Tech Debt areas
 * Put in place Tech Debt management process for PEP

Status
October 2, 2018
 * This work has not yet started

November 8, 2018
 * This work has not yet started

December 6, 2018
 * Met with the Core Platform team last week, much of the work currently with the Platform Evolution program, and is already oriented around this goal of tech debt.

Outcome 4 / Output 4.1
Increase visibility into Code Health


 * Define Code Health Metrics

Dependencies on: Code Health Group

Goal(s)

 * Metrics defined and deployed for all 4 Code Health areas.

Status
October 2, 2018
 * This work has not yet started

November 8, 2018


 * Code Health Metrics working group has been working steadily to review/assess code metrics tools. Current focus is on SonarQube.  PHPMetrics has also been deployed for review purposes.

December 6, 2018
 * We did an update in TEC3 for risk assessment; this goal has changed a bit this quarter, as we are now doing a proof of concept with SonarQube and is



= Q3 Goals =

Outcome 1 / Output 1.1
Increase software stewardship levels of our deployed code


 * Assess deployed code and prioritize stewardship gaps.

Goal(s)

 * Develop and communicate guidelines and best practices for successful Code Stewardship.
 * (Continued from Q2) Update/refresh review queue (review process for initial code deployment)

Status
January 10, 2019
 * Discussed that as we've just gotten back from our vacations, this work is ramping up and is

February 5, 2019
 * Have a basis for guidelines that should be published / reviewed in the coming weeks. Review queue work will start in another week or so.

March 12, 2019
 * We are and will wrap this up by end of this quarter (develop guidelines). Review queue work has gotten a bit bigger than expected and still, we do hope to be done by end of quarter.

April 12, 2019
 * The Code Stewardship Best Practices wiki page has been created with a skeleton of topics. This will be a living resource, but the work for Q3 is considered ✅

Outcome 2 / Output 2.2
We reduce the number of testable regressions from hitting our users


 * Jointly create smoke tests addressing high priority needs for 15 projects over the year.

Dependencies on: Release Engineering (re: TEC3, Outcome 2)

Goal(s)

 * 5 of the 15 prioritized repositories have at least 1 end-to-end simple sample test.

Status
January 10, 2019
 * Discussed that as we've just gotten back from our vacations, this work is ramping up and is

February 2019
 * We've been reaching out to the top 5 emails/users for this goal, we're waiting on feedback from folks now that All Hands is over.

March 12, 2019
 * This goal has now been indefinitely ❌ for a variety of reasons (possible death march)

Outcome 2 / Output 2.3
We reduce the number of testable regressions from hitting our users


 * Pro-actively add unit tests to MediaWiki core and deployed extensions.

Dependencies on: Development Teams, Core Platform Team, Code Health Group

Goal(s)

 * Evolve/develop tools and processes to support the PE refactoring effort to improve code health.
 * Develop common test strategy that enable teams to engage in more effective and efficient testing practices. (maybe should be output 2.4?)

Status
January 10, 2019
 * Discussed that as we've just gotten back from our vacations, this work is ramping up and is

February 5, 2019
 * Discussed this with the CPT team and will meet again this week for figuring out next action items. We've had additional talks with test strategy but will continue next week as we're getting over wikiflu.

March 12, 2019
 * Met last week with Core Platform team, to talk about addressing gaps and work is - we're making good progress.
 * The rough draft outline is ✅ for the test strategy but we need to finalize a few things, so still overall with expectations that we'll finish by end of quarter.

April 12, 2019
 * Groundwork is set for this, we worked with CPT to define gaps and priorities and a course of action as they progress through their refactoring efforts. This will be a continued effort throughout the refactoring activities. ✅
 * We'll need to do more discussion to develop a common test strategy, so this work will continue in Q4

Outcome 3 / Output 3.2
Reduce Technical Debt


 * Tech Debt Management process rolled out.

Goal(s)

 * Speak at All Hands on the status of Technical Debt
 * Engage and coach development teams on their approach to managing technical debt.

Status
January 10, 2019
 * Discussed that as we've just gotten back from our vacations, this work is ramping up and is

February 5, 2019
 * The talk did not happen at All Hands, will look for another forum for this for a technical audience to review with.

March 12, 2019
 * This goal has not been able to be completed and will probably be wrapped into the broader discussion of code health with various teams ❌. We hope to have a session at the Hackathon to talk about tech debt and code health.

Outcome 4 / Output 4.1
Increase visibility into Code Health


 * Define Code Health Metrics

Dependencies on: Code Health Group, Code Health Metric Working Group

Goal(s)

 * Code Health Dashboard with 50% of repositories covered.

Status
January 10, 2019
 * Discussed that as we've just gotten back from our vacations, this work is ramping up and is

February 5, 2019
 * Made great progress on this during All Hands and will continue on it, should be on track for finishing it soon.

March 12, 2019
 * We are on track to finish this in the quarter, with the sonarqube integration.

April 12, 2019
 * SonarQube is our tool of choice for our current set of code metric analysis and reporting. It's been integrated into the experimental pipeline and reporting for all extensions have been enabled. ✅



= Q4 Goals =

Outcome 1+3

 * Outcome 1: Increase software stewardship levels of our deployed code
 * Outcome 3: Reduce Technical Debt

Goal(s)

 * Presentation/session(s) at the Wikimedia Hackathon on the current state of Code Health projects (technical debt and code stewardship)

Status
April 8, 2019
 * This will happen in May 2019

May 7, 2019
 * This will happen next week.

June 4, 2019
 * Presentations were ✅ at the Hackathon and generated a lot of excitement

Outcome 1 / Output 1.1
Increase software stewardship levels of our deployed code
 * Assess deployed code and prioritize stewardship gaps.

Dependancies on: ___________

Goal(s)

 * Publish a re-imagination of the Review Queue process.
 * Develop and implement metrics around task and code-review responsiveness

Status
April 8, 2019
 * This is also

May 7, 2019
 * Publishing a re-imagination of the Review Queue is ❌ right now
 * Metrics is still

June 4, 2019
 * Review Queue is still ❌ right now, blocked on more urgent work right now.
 * Metrics are still - more work to come from the code review group and cross-team conversations. Work will continue in to next FY.

Outcome 4 / Output 4.2
Increase visibility into Code Health
 * Code Health Dashboard with 50% of repositories covered.

Dependancies on: _CI__________

Goal(s)

 * Expand SonarQube reporting into CI infrastructure
 * Perform SonarQube analysis on all extensions
 * Engage user communities in direct feedback solicitation

Status
April 8, 2019
 * Expanding and performing the SonarQube is and will engage the community later on in the quarter.

May 7, 2019
 * Most of the patches are ready, just need to be reviewed, still

June 4, 2019
 * First pass was to create a pipeline and we then added in a lot of extensions (more than 50%); created code health pipeline - currently at 10 extensions and probably will add about 80 more. We think that the rest of the work will be done by the end of the quarter.