Beta cluster

Wikimedia Foundation engineering activity

Beta cluster

Establish a functional, production-like environment in Wikimedia Labs suitable for final-stage testing of new features.

Group:
Start:	2012-03-01
End:
Team:	Antoine Musso, Sam Reed, Faidon Liambotis
Lead:	Chris McMahon
Status:	See updates

← Test2 wiki

The beta cluster aims to be a functional, production-like environment in Wikimedia Labs suitable for final-stage testing of new features. It runs off git master and has its MediaWiki core and extensions code updated about every three minutes. See the overview of its configuration.

Status

[edit status] • [add new]

2013-05-16:

Ariel Glenn has setup a redis instance on beta (bug 41530). Antoine fixed the jobrunner that was no more processing any jobs because it could not access to the Mediawiki files. Jobs are thus processing again, and from redis!

Rationale

The aim is to provide an exact copy of the WMF production cluster (including db hosts, apaches, squids, varnish, scalers, etc.) using Labs instances. For more rationale, see Test/Dev Labs.

Open tasks

Product tasks

TODO: Add tasks

Roadmap

Short term goal: stable beta labs commons in order to distinguish potential TMH errors from env/config errors

• Current activities
  • updating squid/puppet config => Antoine + Faidon
  • extracting beta labs config to public repos => Sam
  • information flow through image scaler => Faidon (first steps to getting correct thumbnails)
  • ongoing refinements for logging => Antoine
  • ongoing refinements for pre-deploy review => Faidon

• Near future activities
  • Swift integration => Faidon + Ben
  • true interwiki link emulation => ?
  • deploy known versions MediaWiki+Extensions from Jenkins => Antoine

• Beyond that
  • checking wikisource and enwiki for correctness

If we manage to get beta in a suitable state AND we get TimedMediaHandler seated there properly, then Antoine will be able to start working on migrating the continuous integration server to Labs. So, that won't happen till July 2012 at the earliest. Sumana Harihareswara, Wikimedia Foundation Engineering Community Manager (talk) 17:13, 11 June 2012 (UTC)

Roles

• Roles (who will be responsible for what)
  • Chris - lead customer / product manager
  • Antoine - lead dev
  • Sam - backup dev
  • Faidon - primary ops
  • Ryan - escalation on ops stuff
  • RobLa - peanut gallery

Documents

• User requirements:
• Specifications: Wikimedia Labs/Things to fix in beta
• Software design document: Configuration overview
• Test plan:
• Documentation plan:
• Schedule: see the old Labs plan (revisions 1, 2, and 3)
• Task management:
• Release management plan:
• Communications plan: