Wikimedia Engineering/Report/2012/May

 Engineering metrics in May:
 * 41 unique committers contributed code to MediaWiki.
 * The total number of unreviewed commits went from about 140 to 250.
 * About 34 shell requests were processed.
 * developers got developer access to Git and Wikimedia Labs, among which are volunteers.
 * Wikimedia Labs now hosts 97 projects, 177 instances and 431 users.

Major news in May include:
 * the publication of the [//blog.wikimedia.org/2012/05/11/book-architecture-mediawiki-open-source-applications/ Architecture of Open-Source Applications book], which contains a chapter on MediaWiki;
 * initial designs for a [//blog.wikimedia.org/2012/05/21/introducing-designs-for-the-universal-language-selector/ universal language selector];
 * a new and easier way to view a wiki's [//blog.wikimedia.org/2012/05/29/wikimedia-wikis-reveal-interwiki-map/ interwiki map];
 * [//blog.wikimedia.org/2012/05/29/1-million-media-files-uploaded-using-upload-wizard/ 1 million files] uploaded with our UploadWizard;
 * the [//blog.wikimedia.org/2012/05/31/wikidata-summit-kicks-off-in-berlin/ Wikidata/RENDER summit] in Berlin, followed by the [//blog.wikimedia.org/2012/06/02/diverse-wikimedia-tech-crowd-gathers-in-berlin/ hackathon].

Recent events
Berlin hackathon (1–3 June 2012, Berlin, Germany)

Upcoming events
Wikimania hackathon (10–11 July 2012, Washington, D.C., USA)

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Danielle Benoit joined the Platform Engineering team as a contractor working on development tutorials (announcement).
 * Ori Livneh joined the Editor Engagement Experiments (E3) engineering team as Software Developer (announcement).
 * Vibha Bamba joined the Product team as Interaction Designer (announcement).
 * James Forrester joined the Product team as Technical Product Analyst, focusing on the Visual Editor (announcement).
 * Subramanya Sastry joined the Features team as Senior Software Engineer (announcement).

Site infrastructure
Data Centers


 * Month of May has been a busy month for racking, stacking and provisioning of newly purchased servers, especially by Rob, Chris, Leslie and Mark. Recently, we purchased new  hardware for server refresh (many are out of warranty and over 4 years old), adding capacity and redundancy, and  for new projects, including servers for  Search, Analytics, Fundraising, OpenStreetMap (OSM), databases, varnish, memcached and backups.  Much effort were put into OS installation  and  servers networked, ready for the various systems and applications deployment.


 * IPv6 work went into full swing as well, in order to be ready for the June 6, IPv6 Launch Day. As of end of May, the database schemas were updated, and work started on refractoring LVS, PayBal, Varniish, Squid, DNS,  Nagios monitoring and puppetization.


 * Last month, we reported deploying the newly built Search cluster at our Ashburn datacenter and disabling the Tampa search cluster. This month, Peter went through the exercise of upgrading the Tampa search cluster and brought it back up. We do have a cross-datacenter hot standby for 'Search' service now.


 * With 11.4 (Ubuntu Precise) release available, we have packaged and started using it selectively in some of our systems, including Search@Tampa and half of the LVS servers. Next we will be setting up the Apache servers in Ashburn using Precise as well.


 * Recently, we were experiencing a few systems rebooting themselves and while we did not see them impacting our services, Faidon investigated and reported a bug with our kernel in 10.4 that caused our servers  to reboot after about 208 days of uptime. As a result, we applied the necessary kernel and security patches to those impacted servers.

Media Storage

Testing environment
Wikimedia Labs


 * The Labs infrastructure experienced several episodes of severe latency problems, and they all were traced the problem to the GlusterFS system. As a result, Ryan, Faidon and Andrew are working on a get well plan, which includes getting rid of GlusterFS and finding a suitable replacement. The short term plan is to eliminate GLusterFS but would expose us to a non-redundant infrastructure.

Backups and data archives
Data Dumps
 * We've been busy creating bundles of media in use per project and the first set of files is almost complete. For each wiki, there is now one or more files containing all media uploaded locally to the wiki, and one or more files containing all media used by the wiki but uploaded to Commons. We've also been preparing for the media back-end switch to Swift; since we won't be able to make copies of all media files in the usual way, some scripts were hacked together which will check the  and   tables and will retrieve and/or update media files via http as needed. Your.org and Masaryk University mirrors officially came online; we're still looking for other partners to host media backups and pageview statistics.

Internationalization and Editor Engagement Experiments
Editor Engagement Experiments (E3)

Wikidata

 * The Wikidata project is funded and executed by Wikimedia Deutschland.

Future
The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.