Core Platform Team/Initiative/Shard Revision Table/Initiative Description

Project Lead
Tim Starling

Current state
Not started

Expected start
April 2019

Summary
TBD

Significance and motivation
Splitting up storage over multiple servers will allow faster schema changes and possibly cheaper hardware. In some cases, this can increase our capacity. This will enable us to handle more applications like JADE.

Milestones and major tasks

 * Survey all the code that uses the revision table in order to get the full scope of the work
 * Investigate options for asynchronous/parallel MYSQL queries
 * Write RFC - Covering the whole project. This includes:
 * Possible Implementation options
 * abstract DB layer
 * have business logic be aware of multiple databases
 * Migration path
 * Depending on the implementation, potentially reach out to extension developers about changes
 * Implement
 * Migrate

Outcome
Increase the scalability of the platform for future applications and new types of content, as well as a growing user base and amount of content

Baseline

 * TBD

Target

 * TBD

Methodology and rationale

 * 1) Schema change speed. This process directly slows down any feature that requires database changes. The amount of improvement we get is dependent on the number of shards we introduce.
 * 2) Performance. We should have parity with current MYSQL access. We should have no performance degradation.

Time and resource estimate
TBD - we will know after analysis in the first step of the project. Should be comparable to the Actor/Comment table project

Dependencies
Actor and Comment Table Schema migrations

MCR (Slot and Content Table) Schema migrations

Collaborators
SRE

Stakeholders
Extension developers

Scoring

WMDE (Wikibase)

Open questions

 * Do we do parallel or serial MYSQL queries? PHP doesn’t support parallel MYSQL queries and so we will need to use a library.
 * How do you split up the table, over which indexes?
 * How much of as speed improvement do we want for schema changes? This will determine the number of shards.
 * How do we determine measure our metrics?

Phabricator
TBD

Plans and RFCs
TBD