Wikimedia Technology/Annual Plans/FY2019/CDP2: Platform Evolution/CDP Budget Segment 7/Goals

=Program Goals and Status for FY18/19=

Segment 7 - Core Platform
 * Goal Owner: Corey Floyd
 * Program Goals for FY18/19: Empower the Wikimedia Foundation to accomplish its goals of Knowledge Equity and Knowledge as a Service by evolving and investing in our technology stack to improve its flexibility, maintainability, and sustainability
 * Annual Plan: Segment 7 - Core Platform
 * Primary Goal is Knowledge as a Service/Foundational Strength: Evolve our systems and structures

''Note: Segment 8 has been folded into Segment 7 (this page) as of the beginning of Q3 FY18/19



 = Q1 Goals =

Outcome 2 / Output 2.2
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs
 * Modularized RESTBase

Primary team: Services

Goal(s)

 * Research, document and develop a specification for the storage API. ❌
 * Propose an RFC.

Status
July 2018

August 16, 2018
 * Spec for the storage API is still in progress.

September 19, 2018
 * ❌ This goal has been moved to Q2 work due to other issues having priority.



=Q2 Goals =

Outcome 2 / Output 2.3
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.
 * Session Management system

Dependencies on: Core Platform Team

Goal(s)

 * Define and implement a session management service.

Status
November 20, 2018 December 14, 2018
 * The RFC was approved last week and this portion is now ✅



=Q3 Goals =

Outcome 1 / Output 1.2
Engineers have a clear understanding of our technology stack and the plan to better scale, maintain and test it

Goal(s)
Develop 3-5 year road map.

This roadmap will include major platform goals such as initiatives around APIs, Storage, Parsing and other enabling technology improvements. It will also include short term refactoring work to enable some of those larger changes. The work outlined will be primarily based on the information gathered by the PE team before TechConf and the work done by attendees during TechConf.

Status
January 17, 2019
 * We're working on creating the documentation portal and getting things setup — defining outcomes and breaking them down into outputs.

February 25, 2019
 * we are breaking down projects and working on dependency tracking

March 19, 2019
 * This is still and in a draft state right now.

Outcome 2 / Output 2.3
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.


 * Session Management system

Dependencies on: Core Platform Team

Goal(s)
Production-ize and deploy session management service to unblock multi-DC

Status
January 17, 2019
 * RFC, language selection, and hardware was completed in Q2, this quarter, we are implementing servers and getting things set up.

February 25, 2019
 * CI is getting setup and ready to deploy soon, and also working on documentation - so fully

March 19, 2019
 * We are still with this and the work for deployment will extend into Q4

Outcome 2 / Output 2.6
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Goal(s)
Spec, plan, and implement MCR page layout enhancement.

Status
January 17, 2019
 * We are currently gathering requirements and writing the product plan.

February 5, 2019
 * We are a bit ❌ with this work, but hope to be working on it again in March.

March 19, 2019
 * This goal has been ❌ into next fiscal

Outcome 2 / Output 2.5
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Goal(s)
Design and begin integrating Parsoid into core

Dependencies on: Parsing Team

Status
January 17, 2019
 * We are just now starting development for this.

February 25, 2019
 * We are working on this, and is fully with the migration and translating of file and getting things integrated back into core; we hope to get it done by end of this fiscal year.

March 19, 2019
 * We'd like to work further on calling Parsoid from MediaWiki, so this is still with the goal of finishing up in Q4.



=Q4 Goals =

Outcome 1 / Output 1.2
Engineers have a clear understanding of our technology stack and the plan to better scale, maintain and test it

Goal(s)
Initial design of API documentation portal.

Status
May 29, 2019
 * This is and should have a prototype by the end of the quarter.

June 27, 2019
 * This is and we'll have a complete definition of done on July 1 (will wrap up by July 15)

July 9, 2019
 * This is and will be completed by July 15, 2019.

Outcome 2 / Output 2.1
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.

MediaWiki REST API Infrastructure

Dependancies on: Parsing

Goal(s)
Implement Parsoid API

Status
May 29, 2019
 * This is also and coordination with the Parsoid team. Have an RFC for this work: https://phabricator.wikimedia.org/T221177

June 27, 2019
 * This is and we'll have a complete definition of done on July 1 (will wrap up by July 15)

July 9, 2019
 * This is - the REST infra will be merged, we'll add the MW extension and more tests - all to be ✅ by July 15, 2019.

Outcome 2 / Output 2.2
Modularized RESTBase

Dependancies on: Release Engineering, SRE

Goal(s)
Finish implementation of RESTBase split

Status
May 29, 2019
 * This is also and should still be done by end of the quarter with the ramping up of traffic to it next FY.

June 27, 2019
 * This is now ✅

Outcome 2 / Output 2.3
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs.

Dependancies on: SRE

Goal(s)
Deploy and serve sessions for group 0 wikis

Status
May 29, 2019
 * This is currently ❌ but still hoping to be done by end of quarter; we will do additional coordination with SRE team.

June 27, 2019
 * This is and will get deployed to testwiki in the next few days. We'll have a complete definition of done on July 1 (will wrap up by July 15)

July 9, 2019
 * This is - this is deployed and can be considered ✅ but it is not yet turned on on group 0 wikis. This can take up to 6 weeks to be fully rolled out.

Outcome 2 / Output 2.6
Refactoring output

Goal(s)

 * Decouple Permission Manager
 * Implement API integration testing infrastructure to run locally
 * Finish MCR Storage changes to unblock DBAs
 * Revision Table RFC
 * Rank ordering of hooks based on usage
 * Implement 2FA in OATHAuth
 * Release automation: Generate 1.33 release tarball and announcement

Status
May 2019
 * Decouple Permission Manager ✅
 * Implement API integration testing infrastructure to run locally is currently ❌, conversation is ongoing (the MVP is ✅)
 * Finish MCR Storage changes to unblock DBAs
 * Revision Table RFC ❌ to next FY
 * Rank ordering of hooks based on usage ❌ to next FY
 * Implement 2FA in OATHAuth is still scheduled to be done this quarter, but slightly delayed due to an unforeseen incident.
 * Release automation: Generate 1.33 release tarball and announcement is still to be finished this quarter.

June 27, 2019
 * The Revision table RFC is ❌ and will be investigated next year. 2FA for OATHAuth is currently ❌ for now.
 * The rest is and we'll have a complete definition of done on July 1 (will wrap up by July 15)

July 9, 2019
 * 2FA for OATHAuth is currently and should be done by end July, pending security review.
 * Release automation: Generate 1.33 release tarball is now ✅.

Outcome 2 / Output 2.5
Engineers are able to access more functionality of the stack using well encapsulated components and well defined APIs

Goal(s)
Integrating Parsoid into core

Dependencies on: Parsing Team

Status
May 2019
 * About 75% of this is complete (initial port is ✅), and seems to be well on it's way to being completed by end of quarter.

June 27, 2019
 * Parsoid is 100% ported now - ✅