Content translation/Development Plan/Roadmap/CX01Release

From mediawiki.org

Content Translation 0.01 release[edit]

Release Announcement

See below for the detailed development plan for each of these features

Increase language support (Labs->Beta->Prod)[edit]

  1. Languages with high-quality support through Apertium
    1. Catalan -> Spanish (reverse language pair)
    2. Spanish -> Portuguese
    3. Maltese -> Arabic (RTL)
  2. Languages well supported by Apertium, but will require better markup alignment support
    1. English - Spanish
    2. English - Catalan

Feature Set[edit]

  1. Entry Points
    1. Red Interlanguage Link (implemented; needs integration)
    2. New entry point showing in Wikidata.
    3. Entry points showing by: editing on other languages, Babel, browser language, Accept-Language
  2. Language tools
    1. Editing support (basic text styling)
    2. Existing paragraph translation - only design, not coding (improvement)
    3. Display disambiguation page links for link adaptation
    4. Increase number of tools Eg: MT backends, dictionary providers, Reference
  3. Translation dashboard
    1. continue ongoing translations, suggestions and impact (enhancement)
  4. Templates
    1. Pass through template support
    2. Inline templates adaptation - show placeholders instead of removing
    3. Category adaptation (not addition/editing)
  5. Infrastructure improvements (improvement)
    1. Improve Machine translation abuse algorithms
    2. Latest upstream Apertium requires for supporting more language pairs
    3. MT Infrastructure enhancement- Work on leftover technical items on MT backend
    4. Service provider registry
    5. Performance
    6. Test coverage
  6. Basic Analytics Dashboard: (enhancement)
    1. Collection
    2. Visualization

Production Deployment - Resources & Provisioning[edit]


Milestones[edit]

Completion Date/Milestones Features Sprints
August 18 - August 29 2014
73
September 2014
74-75

Development Plan[edit]

Mingle Story Board

Feature Details
Entry Points
Editor
  • Tools column
  • Automatic Translation
Link Adaptation
Machine Translation Support (mt)
Dictionary Support
Templates Support
Architecture (technical feature)
Research and preliminary development

CX Deployment Plan for 0.01 Release September 2014[edit]

Deployment date: TBD (September 20-25 2014)

Project: Content Translation Framework

Release: 0.01 (second release after the MVP release)

Long-term project roadmap: Content_translation/Roadmap

Language Pairs to be supported: Catalan-Spanish, Spanish-Portuguese, Maltese-Arabic, English-Spanish, English-Catalan

Release as: Beta Feature

Overall Plan[edit]

System Architecture[edit]

See: https://www.mediawiki.org/wiki/Content_translation/Technical_Architecture

https://www.mediawiki.org/wiki/Content_translation#Workflow_and_Technical_Architecture

https://www.mediawiki.org/wiki/Content_translation

Caching Architecture[edit]

The following diagram includes the caching requirements for the CX framework.

https://www.mediawiki.org/wiki/Content_translation/Server_communications_workflow

https://commons.wikimedia.org/wiki/File:CX_ArchitectureV1.svg

Components to be provisioned for production[edit]

CX server installation and configuration: https://git.wikimedia.org/markdown/mediawiki%2Fservices%2Fcxserver.git/HEAD/README.md

See Setup: https://www.mediawiki.org/wiki/Content_translation/Setup for detailed information about component, installation and configuation and instructions.

  • Node.js
  • Apertium
  • Extension dependencies:
    • BetaFeatures
    • CLDR
    • EventLogging
  • Backend Services

Varnish:

  • External APIs called by CX
    • Wikidata
    • Parsoid API
  • Configuration Scripts

Upstart and Systemd scripts are at: https://www.mediawiki.org/wiki/Content_translation/Setup

Provisioning Plan[edit]

  • Storage Requirements

To be determined from discussion with ops

  • Hardware Requirements

To be determined from discussion with ops

  • Bandwidth Requirements

To be determined from discussion with ops

  • Performance expectations
    • MT TPS (Transactions per second)
    • User responsiveness
    • MT Round trip
    • General guidelines

Monitoring and metrics[edit]

  • EventLogging activity for CX
  • Number of users enabling the feature
  • Performance of S:CX, backend calls?
  • Check for node and varnish? Who to page?
  • Graph showing requests or timings for the WikiData API(s) we are calling
  • Graph showing requests or timings for the Parsoid API(s) we are calling

External Signoffs Required[edit]

  • Faidon - Ops
  • Gabriel - Infrastructure architecture
  • Ori - Performance
  • Chris Steipp - Security
  • Greg G - Release engineering
  • Mark - Ops
  • Tim - Platform

LE Team responsibilities[edit]

  • Kartik - Deployment, Engineer
  • Niklas - Engineer, Code Reviewer
  • Santhosh - Engineer, Code Reviewer
  • David - Engineer, Code Reviewer
  • Joel - Engineer, Code Reviewer
  • Runa - Team Scrum-Ninja / testing and communications
  • Pau - Feature UX reviewer, designer
  • Amir - Feature signoff
  • Alolita - Engineering coordination, Eng Manager