Wikimedia Engineering/Report/2012/December

Engineering metrics in December:
 * 113 unique committers contributed patchsets of code to MediaWiki.
 * The total number of unresolved commits went from about 535 to about 648.
 * About 39 shell requests were processed.
 * As of December 2012, users can self-register on Wikimedia Labs (and get access to git/Gerrit). It is no longer necessary to request an account for developer access.
 * Wikimedia Labs now hosts 148 projects, 847 users; to date 1378 instances have been created.

Major news in December include:
 * https://blog.wikimedia.org/2012/12/07/inventing-as-we-go-building-a-visual-editor-for-mediawiki/ https://blog.wikimedia.org/2012/12/12/try-out-the-alpha-version-of-the-visualeditor/
 * https://blog.wikimedia.org/2012/12/20/article-feedback-new-research-and-next-steps/
 * https://blog.wikimedia.org/2012/12/10/introducing-mediawiki-community-metrics/
 * https://blog.wikimedia.org/2012/12/11/welcome-to-floss-outreach-program-for-women-interns/
 * [https://blog.wikimedia.org/2012/12/12/translation-interface-makeover-in-progress/
 * [https://blog.wikimedia.org/2012/12/12/translation-interface-makeover-in-progress/

''Note: Like last month, we're proposing a shorter and simpler version of this report that does not assume specialized technical knowledge.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Matthew Flaschen joined the Wikimedia Features engineering team as Features Engineer (announcement).
 * Mike Wang joined the Operations team as part time Labs Ops Engineer (consultant) (announcement).

Technical Operations
Site Infrastructure
 * Since December is the Fundraising month, TechOps usually dials down its work on changing site infrastructure. Some of the rudimentary work performed include deploying the new parsoid cluster to support the Visual Editor project, rolling out doc.wikimedia.org, our auto-generated puppet documentation and using a new and unified SSL certificate for *wikipedia.org and *.m.wikipedia.org.
 * Asher migrated one of the main production English Wikipedia slaves, db59, to MariaDB 5.5.28. He has previously been testing 5.5.27 on the primary research slave, and on the current build on a slave in Ashburn datacenter. Taking the times of 100% of all queries over regular sample windows, the average query time across all enwiki slave queries is about 8% faster with MariaDB vs. our production build of 5.1-fb.  Some queries types are 10-15% faster, some are 3% slower, and nothing looks aberrant beyond those bounds.  Overall throughput as measured by qps has generally been improved by 2-10%. He wouldn't draw any conclusions from this data yet, more is needed to filter out noise, but it's positive.The main goal of migrating to MariaDB is not performance driven.  More so, I think it's in WMF's and the open source communities interest to coalesce around the MariaDB Foundation as the best route to ensuring a truly open and well supported future for mysql derived database technology.
 * Both Mark and Faidon have made tremendous progress in testing and deploying Ceph in our Ashbrun site. We are hopeful it would be robust and scalable.
 * TechOps focus for December was to prepare the production site 'failover' i.e., from Tampa to Ashburn datacenter. We deployed the new Ashburn monitoring server and service - icingna.wikimedia.org.

Currently we have most of the traffic (about 80%-90%) served out of Ashburn data center except for the application, memcache and database layers. We are targeting the week of Jan 22th 2013 to perform the switchover of the rest of the systems. For those interested in the planning that is going on, the Countdown meeting minutes are documented on http://etherpad.wmflabs.org/pad/p/EqiadMigration.

Fundraising
 * There were no major changes on the fundraising infrastructure because of the fundraiser itself. We ordered/received bastion hosts for which we're in the process of deploying. Monitoring got an overhaul and we're now sending send alerts to fundraising-tech and/or techops depending on metric.

Data Dumps

Wikimedia Labs

Kiwix
The Kiwix project is funded and executed by Wikimedia CH.

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.

New code and bugfixes have been deployed (detailed changes here and here and test2.wikipedia.org now gets language links from Wikidata. Changes on Wikidata that concern articles on test2 are shown in the recent changes of test2 as well. If there are no problems deployment on the Hungarian Wikipedia will be in January 14th. Other Wikipedias are going to follow later.

For the second phase of Wikidata representation of values is the central focus. We published a draft for this and discussions have started. We'd appreciate your feedback.

Additionally Denny and Lydia held office hours on IRC again. (logs in English and German)

More detailed summaries about what is happening around Wikidata are available here.

Future

 * The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.