Content translation/Deployment Plan/Current Status

This document will list latest updates from Content Translation deployment process at the end of the day. To see live updates, use: http://etherpad.wikimedia.org/p/cx-deployment-notes

21/10/2014 (Tuesday)

 * Testing of Apertium Beta instance.
 * Use Port 80 instead of default port to simplify:
 * https://gerrit.wikimedia.org/r/167747
 * https://gerrit.wikimedia.org/r/167749
 * Added cleanup patch for Apertium Puppet:
 * https://gerrit.wikimedia.org/r/167748
 * Patch to update CXServer code is ready:
 * https://gerrit.wikimedia.org/r/167629

20/10/2014 (Monday)

 * All packages are merged.
 * Apertium puppet and CXServer puppet patches are updated.
 * Testing of Apertium puppet patch on deployment-apertium.
 * Created new instance: deployment-apertium02
 * Apertium Puppet patch merged by Alex.
 * Created proxy for Apertium instance: apertium-beta.wmflabs.org

1710/2014 (Friday)

 * Updated apertium-apy package (Use default port)
 * Updated Apertium puppet patch based on Alex's reviews.
 * All packages except apertium-apy are built and installed on apt.wikimedia.org
 * CXServer patches are merged (See updates of 14/10)

15/10/2014 (Wednesday)

 * More packages reviewed and merged by Alex.
 * Followup on other pending patches and updates.

14/10/2014 (Tuesday)

 * Fixed CXServer on Beta:
 * https://gerrit.wikimedia.org/r/166535 : Add missing link to init with upstart-job.
 * https://gerrit.wikimedia.org/r/166542 : Define cxserver port.
 * Reused CXServer init link fix in Apertium Puppet patch.
 * Fixed missing tag for lttoolbox and changelog. Package is merged (https://gerrit.wikimedia.org/r/163548)

13/10/2014 (Monday)

 * Asked to revert, https://gerrit.wikimedia.org/r/#/c/165528/ as it was accidentally pushed to master while pushing upstream branch.
 * Cleaning up for Puppet patch, https://gerrit.wikimedia.org/r/#/c/165485

10/10/2014 (Friday)

 * Cleaned up Apertium Puppet patch to depend on Apertium version we provide, not default one. https://gerrit.wikimedia.org/r/#/c/165485
 * Updated (and lots of cleanups!) Puppet patch based on Alex's feedback (https://gerrit.wikimedia.org/r/#/c/165485).
 * Cleaned up APY package.

09/10/2014 (Thursday)

 * Updated Packaging document: https://www.mediawiki.org/wiki/Content_translation/Apertium/Packaging
 * Updated Apertium-APY package and Puppet patch for Apertium.
 * Followup for CXServer recreation.
 * Discussion with Apertium upstream to fix minor issues with APY.

08/10/2014 (Wednesday)

 * Updated CXServer Beta and Labs diagram, https://www.draw.io/#G0BztYqLvFZZVncXR1eV84bmRQQXM to get more clearity on process workflow for CXServer.
 * Packaging:
 * apertium-pt-ca
 * apertium-es-pt
 * apertium-apy
 * Requested needed repository to Alex.
 * Co-ordination for Deployment plan.
 * All packaging done.
 * Added initial Puppet patch for Apertium: https://gerrit.wikimedia.org/r/#/c/165485

07/10/2014 (Tuesday)

 * Packaging:
 * apertium-en-es (with tests from David's ticket to upstream)

06/10/2014 (Monday)

 * Updated, https://www.draw.io/#G0BztYqLvFZZVncXR1eV84bmRQQXM
 * Update: apertium-es-ca (for unresponsive ca-es pair), cg3 in Debian.
 * Tested more language pairs.

02/10/2014 (Thursday)

 * Updated further on: https://www.mediawiki.org/wiki/Content_translation/Apertium/Packaging
 * Minor updates to all packages.
 * Started pushing new updates from upstream.
 * Work on 'rejected' cg3 in Debian.
 * Followup for CXServer Beta continue.

01/10/2014 (Wednesday)

 * Followup for CXServer Beta continue.
 * Co-ordination with Apertium upstream for apertium-lex-tools issue(s). Fixed.
 * Pushed apertium-lex-tools to Debian and Gerrit.
 * Updated packages for minor issues.
 * Followup on ca-es issues, packages will be updated after upstream declares fix.

30/09/2014 (Tuesday)

 * Followup with team for CXServer Beta updates issue (old code, can't update without mt.service variable)
 * Follow up with Alex for new project creation.
 * Upstart issues with apertium-apy (WIP)
 * Rebuilt package against trusty; up for reviews.

29/09/2014 (Monday)

 * Branch push issues in all repositories are fixed.
 * Code to apertium/apertium-es-ca/cg3/lttoolbox is pushed.
 * 8 new projects are being created by Alex.
 * Apertium-APY work continue in ground.

26/09/2014 (Friday)

 * Continuous work on Apertium-APY and other language pairs based upon upstream packaging efforts (by Tino).
 * TODO: Fix other branch push issue.

25/09/2014 (Thursday)

 * Packaging:
 * lttoolbox: https://gerrit.wikimedia.org/r/#/c/162620/
 * apertium-es-ca and work on other language pairs.
 * Apertium-APY
 * Follow up with Alex for Gerrit other branch push issues for packaging (master/upstream/pristine-tar)
 * Follow up with Tino for Packaging and issues related to pcre patch in Apertium/lttoolbox.
 * TODO: Updated wikitech document for above issue.

24/09/2014 (Wednesday)

 * Packaging Apertium:
 * Uploaded lttoolbox/apertium/cg3 to Debian experimental.
 * Started packaging Apertium-APY and language paris.
 * Fixing git review issues with operations/debs/ repo with Alex.
 * Santhosh pointed that we need more CPU/RAM than current Labs instance for Apertium service. Asked Alex to provide large instance at deployment-apertium01.eqiad.wmflabs
 * Followup with Alolita.

23/09/2014 (Tuesday)

 * Work on Apertium packaging continue (lttoolbox, apertium)
 * Alex created needed projects at, operations/debs/...
 * Alolita to follow up with Mark.
 * #4552: Apertium Puppet work scheduled.
 * Co-ordination with Tino continue.

22/09/2014 (Monday)

 * Apertium packaging:
 * Apertium/cg3/lttoolbox
 * Also read more about WMF's Python packaging notes.
 * Co-ordination with Tino for Debian packaging issues. All fixed in Debian.
 * Beta updates:
 * Alex: deployment-apertium01.eqiad.wmflabs is ready to serve.
 * Labs/Beta client configs are updated (Niklas, Kartik).
 * Followups with Alex.
 * WIP: Apertium Puppet module.

19/09/2014 (Friday)

 * Apertium packaging started.
 * Environment of Trusty is up and running. Difficult to use Labs due to use of Gerrit username/authentication.
 * Started with Apertium. Yet to figure out some issues with build.
 * TODO: Separate following notes in another page!
 * WMF Debian packaging (apt.wikimedia.org) documents and process:
 * https://wikitech.wikimedia.org/wiki/APT_repository
 * https://wikitech.wikimedia.org/wiki/Reprepro
 * https://wikitech.wikimedia.org/wiki/Git-buildpackage
 * Process:
 * Package goes via Gerrit patchset.
 * Repository: operations/debs/
 * Build package using 'git-buildpackage' in a clean chroot Precise/Trusty environment. Aim for Trusty.
 * Once reviewed, Package(s) will be 'build' by Ops member (Alex in our case).

18/09/2014 (Thursday)

 * Meeting with Alex K.
 * Discussed:
 * Current status in Beta cluster with Content Translation.
 * Why we need Apertium as service?
 * Questions from Alex about:
 * What will be situation with Google/Bing in future?
 * Google/Bing will have agreement on API uptime and reliability with us if we opt for such service in future. This will make sure that we're not affected.
 * How frequent Apertium service will be updated?
 * We will have mostly update on Apertium language pairs once in a month, which are data packages. Core Apertium packages will be only updated upon critical bugs found in upstream bug tracker. Any bug from Wikimedia dev/user will be track at Apertium upstream bug tracker.
 * Blockers from LE team:
 * Apertium need to run as service before we go ahead with deployment.
 * Blockers from Ops:
 * Ops running tight with server allocation.
 * Action items:
 * Kartik will start working with Alex for setting up Debian repositories in WMF.
 * Kartik will start work on packages and pushing them to Debian repositories.
 * Alex will provide machine in Beta cluster when Packaging is done (in a week).
 * Kartik/Niklas will start working on Apertium Puppetizing with Alex.
 * Alex will talk to Mark about production server estimate date.
 * Alolita to setup meeting with Mark/Ops.
 * Kartik will document everything that comes under the hood of Apertium/Puppet/Repository and CX deployment process!
 * ETAs:
 * Beta cluster machine for Apertium service: 1 week after Packaging.
 * Production server: 3 to 4 weeks.
 * Update Deployment plan document with daily updates.