Core Platform Team/Initiative/Unify Parsers-Phase 2/Initiative Description

Project Leads
Subbu Sastry

Current state
Blocked, waiting for phase 1 to be complete.

Some work is less defined until several tasks are complete which are expected to define the rest of the project. See milestones and major tasks below.

Expected start
FY1920 Q1

Summary
TBD

Significance and motivation
TBD

Milestones and major tasks

 * Fix known bugs in Parsoid
 * Finish implementing language variants support
 * Finish updating legacy PHP parser media output to match Parsoid
 * Connect with CL and engage with community (This can/will reveal new work)
 * Identify any other Parsoid feature gaps (This can/will reveal new work)
 * Establish regular visual diff QA runs to identify uncaught issues (this can/will reveal new work)
 * Decide on what compatibility is acceptable
 * Improve Parsoid performance (undefined until phase 1 is complete)
 * Establish regular visual diff QA runs to identify uncaught issues
 * Analyze results and file Parsoid bugs or identify any wikitext changes required on wikis
 * Finalize new parser hooks API + migrate over Wikimedia extensions
 * Enable Parsoid as default parser
 * Deprecate old parser

Outcome
Reduce complexity in core

Baseline

 * TBD

Target

 * TBD

Methodology and rationale
TBD

Time and resource estimate
12-18 months

3.75 FTE and .25 Engineering Manager for the duration

Possible augmenting of other engineers, but more clarity is needed

Dependencies
Build new HTTP API (Link TBD)

Reduce Extension Interface Surface Area

Collaborators

 * Parsing Team
 * Core Platform
 * Performance
 * SRE

Stakeholders

 * Client teams (Web, VE, iOS, Android)
 * Editing community
 * Core Platform

Open questions

 * To what extent do we want to refactor the Parsing Interface in Core? It is currently coupled to the templating engine.
 * What is acceptable feature parity between Parsoid and the PHP parser? How do we decide this? What qualitative analysis should be used.
 * What are our strategies for engaging with the community on this change?
 * What additional work is required on the linter?

Phabricator
https://phabricator.wikimedia.org/tag/parsoid-read-views/

Plans and RFCs
The Long And Winding Road To Making Parsoid The Default MediaWiki Parser