Content translation/Deployments/How-to

This document describes deployment procedure for ContentTranslation, cxserver and Apertium.

ContentTranslation
Content Translation is updated via regular MediaWiki train. In case of manual update needed, follow the steps below.


 * 1) Update desired branches to desired commit/update using delete and create branches at Gerrit interface, https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/extensions/ContentTranslation,branches In case of emergency, note down old hash.
 * 2) Read and follow: https://wikitech.wikimedia.org/wiki/How_to_deploy_code#Updating_the_submodule Make sure you've clean copy of MediaWiki/core for this.
 * 3) You're ready for actual deployment.

Testing
Note image tag version from Gerrit patchset to be deploy. For eg: https://gerrit.wikimedia.org/r/#/c/mediawiki/services/cxserver/+/502964/ has  tag.

Run it:

Where, config.dev.yaml is local cxserver config file.

Endpoints can be tested at: http://localhost:4000/v2 etc

Deployment

 * 1) Clone deployment-charts repository (for first time).
 * 2) Modify helmfile.d folder of the service: deployment-charts/helmfile.d/services/staging/cxserver. We mostly need change in values.yaml file for cxserver update. When updating cxserver version, look at, version: and update it to patch that need to deploy.
 * 3) Make a CR (Example: https://gerrit.wikimedia.org/r/#/c/528618/) and after a successful review,  merge it.
 * 4) After merge, log in in a deployment server, there is a cron (1 minute) that will update the /srv/deployment-charts directory with the contents from git.
 * 5) Go to /srv/deployment-charts/helmfile.d/services/${CLUSTER}/cxserver where CLUSTER is one of (staging, eqiad, codfw).
 * 6) Execute:   This will show the changes that it will be applied on the cluster.
 * 7) Execute:   This will materialize the previous diff in the cluster and also will log into SAL the change.

Secrets
If secrets like API key or token need update, it need to be done via SRE at Private Puppet repository.

Status
This is done using helmfile:


 * 1) Change directory to /srv/deployment-charts/helmfile.d/services/${CLUSTER}/cxserver on a deployment server
 * 2) Unless you are mid un-applied changes the current values files should reflect the deployed values
 * 3) You can check for un-applied changes with:
 * 4) You can see the status with

Rolling back changes
If you need to roll back a change because something went wrong:


 * 1) Revert the git commit to the deployment-charts repo
 * 2) Merge the revert (with review if needed)
 * 3) Wait one minute for the cron job to pull the change to the deployment server
 * 4) Change directory to /srv/deployment-charts/helmfile.d/services/${CLUSTER}/cxserver where CLUSTER is one of (staging, eqiad, codfw).
 * 5) Execute   to see what you'll be changing.
 * 6) Execute

Apertium
Apertium deployment is done via updating apertium packages through Gerrit change and puppet.

Troubleshooting

 * 1) Check if package list are up-to-date in Puppet.
 * 2) Make sure to ask to restart 'apertium-apy' on scb1001 and scb1002 to any Ops member after adding new languages support for Apertium MT.
 * 3) Check logs at: