Deployment tooling/status

Last update on: 2014-05-monthly

2013-11-monthly

The DevOps sprint participants focused their efforts towards monitoring related work, specifically getting Logstash in production and puppetizing/migrating Graphite (both still in-progress). Cache related fixes were made to avoid users seeing outdated version of pages when using non canonical URL forms. A fix was made to the commons upload process to update all articles that use that page as users would except.

2013-11-30

The DevOps sprint refocused it's efforts on Monitoring from a wider range of activities. One particular item that time was put in on that will not be taken to full fruition is git-deploy.

2014-01-monthly

The Logstash service was deployed to production at https://logstash.wikimedia.org/. Work has started on an analysis of the current scap process which will be used to draft requirements for further deployment scripting work in the current quarter.

2014-02-monthly

The month of February saw a lot of work on WMF deployment tooling.

To see a real life example of what it looks like to deploy code on the WMF server cluster, watch this screencast created by Bryan Davis. That shows you what the person deploying the code sees when doing a localization (translations) update. A deployment that includes new changes to the code (e.g. MediaWiki and extensions) on the servers would be different.

The suite of tools that make up the current MediaWiki deployment tooling is continuing to be updated and rewritten in Python. You can see the work of this in [hthttps://phabricator.wikimedia.org/diffusion/MWTS/history/master/ the repository's history].

The updated Development and Deployment Process flowchart is now created using Blockdiag, a Python library for converting text into flow charts. You can see the current draft in the newly-minted Release Engineering repository.

There is now a matrix showing the requirements for deployment tooling for 3 projects (MediaWiki, Parsoid (and related), and ElasticSearch (and related)). This is not a fixed document and will grow/change as more is learned.

2014-04-monthly

We started investigating changes that may be needed to support the use of HHVM in production.

2014-05-monthly

Bryan Davis continued the Pythonification of our deployment tooling with the conversion of the remaining bits, namely all sync-* scripts (sync-common, sync-file, sync-dir, and sync-db). Work continues on modifying our deployment tooling to provide easier and more robust automatic access to our version information (available at Special:Version).