Wikimedia Engineering/Report/2011/February

Major accomplishments this month include:
 * the build-out of of new datacenter in Virginia
 * the (painful) deployment of MediaWiki 1.17 to all Wikimedia wikis

Recent events

 * Data Summit (February 4, California) — working session about semantic data, analytics and research into data dumps. parsers, structured data and analytics.
 * FOSDEM 2011 (February 5-6, Brussels, Belgium) —
 * GNUnify 2011 (February 11-12, Pune, India) —

Upcoming events

 * Wikimedia Conference 2011 (May, Berlin) —
 * Wikimania (August 2-6, Haifa, Israel) — This year's Wikimania will be preceded by two days of hacking. Mark your calendar for August 2-3! You can also submit a talk or workshop for the Technology tracks of the actual conference (August 4-7).

Hiring
Are you looking to work for Wikimedia? We have a lot of hiring coming up this year, and we really love talking to active community members about these roles. The following positions are currently open:
 * Volunteer Development Coordinator
 * Performance Engineer
 * Software Developer (Features)
 * Software Developer (Mobile)
 * Data Analytics Engineer
 * Operations Engineer
 * Senior QA Engineer

In addition, we hope to post the following positions over the next few months:
 * Release Engineer
 * Technical Writer
 * Network Engineer (contractor)

Operations
Virginia Data Center — Installation of a world-class primary data center for Wikimedia Foundation websites.
 * Status: All the hardware has been delivered, and our team (Rob Halsell, Ryan Lane, Priyanka Dhanda and Ariel Glenn) has been building it out.
 * Program manager: Mark Bergsma

Media Storage — Improvement of our media storage architecture to accommodate expected increase in media uploads.
 * Status:


 * Program manager: Mark Bergsma

Monitoring — Operations and public monitoring system to improve overall uptime, prevent outages, increase transparency and support progress tracking.
 * Status:


 * Program manager: Mark Bergsma

Virtualization cluster — Environment to deploy temporary machines for testing and experimentation, for use by WMF staff and volunteers working on important projects (as capacity allows).
 * Status:


 * Program manager: Mark Bergsma

Backups — Improvement of backup coverage of Wikimedia-hosted data.
 * Status:


 * Program manager: Mark Bergsma

Data Dumps — Improvement of processes to create and provide public copies of public Wikimedia data.
 * Status: Data dumps have been put on hold until the new data center is up and running.
 * Program manager: Mark Bergsma

Content Quality Tools
Article Feedback — A feature to collaboratively assess article quality and incorporate reader ratings on Wikipedia.
 * Status: The deployment to our prototype has surfaced bugs that we've fixed. Now that MediaWiki 1.17 has been successfully deployed, we can release the latest version of the Article feedback tool on the English Wikipedia, as part of our pilot experiment
 * Program manager: Alolita Sharma

Pending Changes — A feature to allow changes made by logged-out and new users to be reviewed before they appear as the primary version of an article.
 * Status:


 * Program manager: Rob Lanphier

Threaded Discussions
Liquid Threads — A feature that brings threaded discussions capabilities to Wikimedia projects and MediaWiki.
 * Status: New design specifications have been published by Brandon Harris. A discussion was started on the "gender gap" mailing list about how this new discussion system could improve interactions between participants.
 * Program manager: Alolita Sharma

Multimedia Tools
Upload wizard — A feature that provides an easier way of uploading files to Wikimedia Commons, the media library associated with Wikipedia.
 * Status: Ryan Kaldari has joined Neil Kandalgaonkar to fix bugs and prioritize the work to be done for an UploadWizard 1.0 release
 * Program manager: Alolita Sharma

Media Projects — A set of features to improve media handling and key infrastructure support tools, many developed with Kaltura, such as Metavid, MwEmbed, and the Video Editor.
 * Status:


 * Program manager: Alolita Sharma

MediaWiki development
MediaWiki 1.17 deployment — Deployment of the latest MediaWiki version to Wikimedia sites
 * Status: In preparation for the planned deployment of MediaWiki 1.17 on February 8, all outstanding revisions were reviewed. The deployment was attempted twice that day, and eventually postponed because of major performance issues that caused an outage. The problems were investigated, and another plan was published, based on heterogeneous deployment (meaning not all wikis would run the same version of the software). On February 11, a first wave of small wikis were switched to MediaWiki 1.17. On February 16, other small and medium-sized wikis were switched. An attempt to deploy to our biggest wiki (en.wikipedia.org) resulted in a small outage. The English Wikipedia and all remaining wikis were successfully upgraded to MediaWiki 1.17 later that day. Many issues encountered this month were due to the large amount of code changes involved. In the future, software deployments should be smaller and happen more regularly, hence reducing the risk of repeated outages like this month.
 * Program manager: Rob Lanphier

MediaWiki 1.17 — The upcoming MediaWiki release.
 * Status: Now that MediaWiki 1.17 has been deployed to all Wikimedia wikis, remaining bugs are expected to surface and be fixed. MediaWiki 1.17 should officially be released in March for third-party users (see draft release notes). Its main feature will be the Resource loader, which improves the load times for JavaScript and CSS in MediaWiki, enabling faster loading of the Vector skin, media extensions, and anything else that makes extensive use of Javascript and CSS. It will also include category collation improvements. Developers are already discussing MediaWiki 1.18.
 * Program manager: Rob Lanphier

Test framework deployment — Creation of an automated test environment for MediaWiki using CruiseControl, Selenium, and PHPUnit.
 * Status:


 * Program manager: Rob Lanphier

Technical Documentation – Improvement of our technical documentation by making small, incremental improvements to the docs and docs process.
 * Status:


 * Program manager: Rob Lanphier

Wikimedia analytics
udp2log — A custom data analytics logging system.
 * Status:


 * Program manager: Rob Lanphier

OWA — Installation and customization of an Open Web Analytics (OWA) platform to process data to support decision making
 * Status:


 * Program managers: Rob Lanphier & Tomasz Finc

Fundraising
2010 Fundraiser — Engineering support for the yearly fundraiser (includes fraud prevention, CentralNotice, and the analytics upgrade).
 * Status:


 * Program manager: Tomasz Finc

Mobile
Mobile site rewrite — Port of our existing gateway to another framework for easier support & collaborative development.
 * Status:


 * Program manager: Tomasz Finc

Offline
Offline — Better support for offline reading of Wikimedia content.
 * Status:


 * Program manager: Tomasz Finc