Heterogeneous deployment

WMF engineering activity

Heterogeneous deployment

A system allowing the deployment of different versions of MediaWiki to different production wikis

Group:	Platform
Start:	2011-05-02
End:	2011-09
Management:	Rob Lanphier
Team:	Tim Starling, Aaron Schulz

• [edit status] • [add new]
  2012-04-17: This feature is now deployed on Wikimedia's sites, and we use it to do staged rollouts of new versions of MediaWiki to our wikis.

Rationale

Part of the difficulty with deploying MediaWiki in a timely fashion is that many deployments require an all or nothing approach. We currently don’t have a systematic way of deploying different versions of MediaWiki to different production wikis (i.e. Hebrew Wikisource and English Wikipedia both must run the same version of MediaWiki).

The "Heterogenous deploy" project will give us the ability to stage new versions of our software to a production-like environment, and then deploy in waves to different parts of the cluster. This will allow us to deploy in such a way that we can receive and address reports from the first wikis we deploy to before moving on to other wikis that may have different problems to report.

Timeline

This project is currently in the planning phases. A provisional version of this feature was implemented by Roan Kattouw and Tim Starling during the 1.17 deployment. Tim Starling implemented a more robust starting point for implementation of this feature in April 2011 before moving on to other projects. Priyanka Dhanda is working on a more detailed plan which targets utilizing this feature for the 1.18 release. Tim and possibly others will join the project after some other parts of the General Engineering roadmap are completed.

Phase 1

Targeted for 1.18 release. Simple version that is good enough for 1.18, but not more ambitious than that.

This will be done in a series of partial deployments:

• Partial deployment 1 (completed 2011-07-21):
  • common/multiversion scripts
  • populateWikiversionCDB.php
  • mwversionsinuse
• Partial deployment 2A (completed 2011-07-21):
  • replaced live MWVersion.php, pointing most wikis to MWVersion-old.php
• Partial deployment 2B (completed 2011-07-28):
  • make all wikis use the new MWVersion.php
  • copy ExtensionMessages.php to ExtensionMessages-1.17.php
  • replace CommonSettings.php
  • modify MWScript.php
  • update addwiki.php
• Partial deployment 3 (in progress the week of 2011-08-01):
  • install & test mwscript
  • Update crontabs (and .sh files) for running maintanence scripts
  • Deploy l10nupdate and sync-l10nupdate changes
• Partial deployment 4 (scheduled for the week of 2011-08-01):
  • Deploy scap script changes
  • Deploy scap-* file changes in debs package

For detailed schedule, see wikitech:Software deployments

Phase 2

Version for HipHop.

Documents

• Software design document:wikitech:Heterogeneous deployment* Backlog/:etherpad:HetDeploy* Test plan:
• Release management plan:
• Communications plan:
• Status updates