Content translation/Development Plan/Roadmap/CX01Release

Content Translation 0.01 release
Release Announcement

See below for the detailed development plan for each of these features

Increase language support (Labs->Beta->Prod)

 * 1) Languages with high-quality support through Apertium
 * 2) Catalan -> Spanish (reverse language pair)
 * 3) Spanish -> Portuguese
 * 4) Maltese -> Arabic (RTL)
 * 5) Languages well supported by Apertium, but will require better markup alignment support
 * 6) English - Spanish
 * 7) English - Catalan

Feature Set

 * 1) Entry Points
 * 2) Red Interlanguage Link (implemented; needs integration)
 * 3) New entry point showing in Wikidata.
 * 4) Entry points showing by: editing on other languages, Babel, browser language, Accept-Language
 * 5) Language tools
 * 6) Editing support (basic text styling)
 * 7) Existing paragraph translation - only design, not coding (improvement)
 * 8) Display disambiguation page links for link adaptation
 * 9) Increase number of tools Eg: MT backends, dictionary providers, Reference
 * 10) Translation dashboard
 * 11) continue ongoing translations, suggestions and impact (enhancement)
 * 12) Templates
 * 13) Pass through template support
 * 14) Inline templates adaptation - show placeholders instead of removing
 * 15) Category adaptation (not addition/editing)
 * 16) Infrastructure improvements (improvement)
 * 17) Improve Machine translation abuse algorithms
 * 18) Latest upstream Apertium requires for supporting more language pairs
 * 19) MT Infrastructure enhancement- Work on leftover technical items on MT backend
 * 20) Service provider registry
 * 21) Performance
 * 22) Test coverage
 * 23) Basic Analytics Dashboard: (enhancement)
 * 24) Collection
 * 25) Visualization

Development Plan
Mingle Story Board

CX Deployment Plan for 0.01 Release September 2014
Deployment date: TBD (September 20-25 2014)

Project: Content Translation Framework

Release: 0.01 (second release after the MVP release)

Long-term project roadmap: Content_translation/Roadmap

Language Pairs to be supported: Catalan-Spanish, Spanish-Portuguese, Maltese-Arabic, English-Spanish, English-Catalan

Release as: Beta Feature

System Architecture
See: https://www.mediawiki.org/wiki/Content_translation/Technical_Architecture

https://www.mediawiki.org/wiki/Content_translation#Workflow_and_Technical_Architecture

https://www.mediawiki.org/wiki/Content_translation

Caching Architecture
The following diagram includes the caching requirements for the CX framework.

https://www.mediawiki.org/wiki/Content_translation/Server_communications_workflow

https://commons.wikimedia.org/wiki/File:CX_ArchitectureV1.svg

Components to be provisioned for production
CX server installation and configuration: https://git.wikimedia.org/markdown/mediawiki%2Fservices%2Fcxserver.git/HEAD/README.md

See Setup: https://www.mediawiki.org/wiki/Content_translation/Setup for detailed information about component, installation and configuation and instructions.


 * Node.js


 * Dictd server (Also see: https://www.mediawiki.org/wiki/Content_translation/Dictionaries)


 * Apertium


 * Extension dependencies:
 * BetaFeatures
 * CLDR
 * EventLogging

Varnish:
 * Backend Services


 * External APIs called by CX
 * Wikidata
 * Parsoid API

Upstart and Systemd scripts are at: https://www.mediawiki.org/wiki/Content_translation/Setup
 * Configuration Scripts

Provisioning Plan
To be determined from discussion with ops To be determined from discussion with ops To be determined from discussion with ops
 * Storage Requirements
 * Hardware Requirements
 * Bandwidth Requirements
 * Performance expectations


 * MT TPS (Transactions per second)
 * User responsiveness
 * MT Round trip
 * General guidelines


 * https://www.mediawiki.org/wiki/Performance_guidelines
 * https://www.mediawiki.org/wiki/Performance_profiling_for_Wikimedia_code

Monitoring and metrics

 * EventLogging activity for CX
 * Number of users enabling the feature
 * Performance of S:CX, backend calls?
 * Check for node and varnish? Who to page?
 * Graph showing requests or timings for the WikiData API(s) we are calling
 * Graph showing requests or timings for the Parsoid API(s) we are calling

External Signoffs Required

 * Faidon - Ops
 * Gabriel - Infrastructure architecture
 * Ori - Performance
 * Chris Steipp - Security
 * Greg G - Release engineering
 * Mark - Ops
 * Tim - Platform

LE Team responsibilities

 * Kartik - Deployment, Engineer
 * Niklas - Engineer, Code Reviewer
 * Santhosh - Engineer, Code Reviewer
 * David - Engineer, Code Reviewer
 * Joel - Engineer, Code Reviewer
 * Runa - Team Scrum-Ninja / testing and communications
 * Pau - Feature UX reviewer, designer
 * Amir - Feature signoff
 * Alolita - Engineering coordination, Eng Manager