Wikimedia Technology/Annual Plans/FY2019/CDP2: Platform Evolution/CDP Budget Segment 7/Goals

Program Goals and Status for FY18/19

Segment 7 - Core Platform

• Goal Owner: Corey Floyd
• Program Goals for FY18/19: Empower the Wikimedia Foundation to accomplish its goals of Knowledge Equity and Knowledge as a Service by evolving and investing in our technology stack to improve its flexibility, maintainability, and sustainability
• Annual Plan: Segment 7 - Core Platform
  • Primary Goal is Knowledge as a Service/Foundational Strength: Evolve our systems and structures

Note: Segment 8 has been folded into Segment 7 (this page) as of the beginning of Q3 FY18/19

Q1 Goals

Outcome 2 / Output 2.2

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Modularized RESTBase

Primary team: Services

Goal(s)

• Research, document and develop a specification for the storage API. N Postponed
• Propose an RFC. To do

Status

 Note: July 2018

• In progress

 Note: August 16, 2018

• In progress Spec for the storage API is still in progress.

 Note: September 19, 2018

• N Postponed This goal has been moved to Q2 work due to other issues having priority.

Q2 Goals

Outcome 2 / Output 2.3

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.

Session Management system

Dependencies on: Core Platform Team

Goal(s)

• Define and implement a session management service.

Status

 Note: November 20, 2018

• In progress

 Note: December 14, 2018

The RFC was approved last week and this portion is now  Done

Q3 Goals

Outcome 1 / Output 1.2

Engineers have a clear understanding of our technology stack and the plan to better scale, maintain and test it

Goal(s)

Develop 3-5 year road map.

This roadmap will include major platform goals such as initiatives around APIs, Storage, Parsing and other enabling technology improvements. It will also include short term refactoring work to enable some of those larger changes. The work outlined will be primarily based on the information gathered by the PE team before TechConf and the work done by attendees during TechConf.

Status

 Note: January 17, 2019

We're working on creating the documentation portal and getting things setup — defining outcomes and breaking them down into outputs.

 Note: February 25, 2019

we are breaking down projects and working on dependency tracking In progress

 Note: March 19, 2019

This is still In progress and in a draft state right now.

Outcome 2 / Output 2.3

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.

Session Management system

Dependencies on: Core Platform Team

Goal(s)

Production-ize and deploy session management service to unblock multi-DC

Status

 Note: January 17, 2019

RFC, language selection, and hardware was completed in Q2, this quarter, we are implementing servers and getting things set up.

 Note: February 25, 2019

CI is getting setup and ready to deploy soon, and also working on documentation - so fully In progress

 Note: March 19, 2019

We are still In progress with this and the work for deployment will extend into Q4

Outcome 2 / Output 2.6

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Goal(s)

Spec, plan, and implement MCR page layout enhancement.

Status

 Note: January 17, 2019

We are currently gathering requirements and writing the product plan.

 Note: February 5, 2019

We are a bit N Stalled with this work, but hope to be working on it again in March.

 Note: March 19, 2019

This goal has been N Postponed into next fiscal

Outcome 2 / Output 2.5

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Goal(s)

Design and begin integrating Parsoid into core

Dependencies on: Parsing Team

Status

 Note: January 17, 2019

We are just now starting development for this.

 Note: February 25, 2019

We are working on this, and is fully In progress with the migration and translating of file and getting things integrated back into core; we hope to get it done by end of this fiscal year.

 Note: March 19, 2019

We'd like to work further on calling Parsoid from MediaWiki, so this is still In progress with the goal of finishing up in Q4.

Q4 Goals

Outcome 1 / Output 1.2

Engineers have a clear understanding of our technology stack and the plan to better scale, maintain and test it

Goal(s)

Initial design of API documentation portal.

Status

 Note: May 29, 2019

• This is In progress and should have a prototype by the end of the quarter.

 Note: June 27, 2019

• This is In progress and we'll have a complete definition of done on July 1 (will wrap up by July 15)

 Note: July 9, 2019

• This is Partially done and will be completed by July 15, 2019.

Outcome 2 / Output 2.1

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.

MediaWiki REST API Infrastructure

Dependancies on: Parsing

Goal(s)

Implement Parsoid API

Status

 Note: May 29, 2019

• This is also In progress and coordination with the Parsoid team. Have an RFC for this work: https://phabricator.wikimedia.org/T221177

 Note: June 27, 2019

• This is In progress and we'll have a complete definition of done on July 1 (will wrap up by July 15)

 Note: July 9, 2019

• This is Partially done - the REST infra will be merged, we'll add the MW extension and more tests - all to be  Done by July 15, 2019.

Outcome 2 / Output 2.2

Modularized RESTBase

Dependancies on: Release Engineering, SRE

Goal(s)

Finish implementation of RESTBase split

Status

 Note: May 29, 2019

• This is also In progress and should still be done by end of the quarter with the ramping up of traffic to it next FY.

 Note: June 27, 2019

• This is now  Done

Outcome 2 / Output 2.3

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.

Dependancies on: SRE

Goal(s)

Deploy and serve sessions for group 0 wikis

Status

 Note: May 29, 2019

• This is currently N Stalled but still hoping to be done by end of quarter; we will do additional coordination with SRE team.

 Note: June 27, 2019

• This is In progress and will get deployed to testwiki in the next few days. We'll have a complete definition of done on July 1 (will wrap up by July 15)

 Note: July 9, 2019

• This is Partially done - this is deployed and can be considered  Done but it is not yet turned on on group 0 wikis. This can take up to 6 weeks to be fully rolled out.

Outcome 2 / Output 2.6

Refactoring output

Goal(s)

• Decouple Permission Manager
• Implement API integration testing infrastructure to run locally
• Finish MCR Storage changes to unblock DBAs Partially done
• Revision Table RFC
• Rank ordering of hooks based on usage
• Implement 2FA in OATHAuth
• Release automation: Generate 1.33 release tarball and announcement

Status

 Note: May 2019

• Decouple Permission Manager  Done
• Implement API integration testing infrastructure to run locally is currently N Stalled, conversation is ongoing (the MVP is  Done)
• Finish MCR Storage changes to unblock DBAs Partially done
• Revision Table RFC N Postponed to next FY
• Rank ordering of hooks based on usage N Postponed to next FY
• Implement 2FA in OATHAuth is still scheduled to be done this quarter, but slightly delayed due to an unforeseen incident.
• Release automation: Generate 1.33 release tarball and announcement is still In progress to be finished this quarter.

 Note: June 27, 2019

• The Revision table RFC is N Not done and will be investigated next year. 2FA for OATHAuth is currently N Blocked for now.
• The rest is In progress and we'll have a complete definition of done on July 1 (will wrap up by July 15)

 Note: July 9, 2019

• 2FA for OATHAuth is currently In progress and should be done by end July, pending security review.
• Release automation: Generate 1.33 release tarball is now  Done.

Outcome 2 / Output 2.5

Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Goal(s)

Integrating Parsoid into core

Dependencies on: Parsing Team

Status

 Note: May 2019

• About 75% of this is complete (initial port is  Done), and seems to be well on it's way to being completed by end of quarter.

 Note: June 27, 2019

• Parsoid is 100% ported now -  Done