Wikimedia Engineering/Report/2013/December

Engineering metrics in December: Major news in December include:
 * 152 unique committers contributed patchsets of code to MediaWiki.
 * The total number of unresolved commits went from around 1230 to about 1386.
 * About 25 shell requests were processed.
 * a retrospective on Language Engineering events, including the language summit in Pune, India;
 * the launch of a draft feature on the English Wikipedia, to provide a gentler start for Wikipedia articles.

''Note: We're also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

Upcoming events
There are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.

For a more complete and up-to-date list, check out the Project:Calendar.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Sherah Smith joined the Wikimedia Foundation as Fundraising Engineer in Features Engineering (announcement).
 * Kunal Mehta joined the Wikimedia Foundation as contractor in Features Engineering (announcement).
 * Andrew Green joined the Wikimedia Foundation as a contractor in Features Engineering, working on the Education Program (announcement).

Technical Operations
Datacenter RFP
 * As part of our ongoing work in selecting a new location for our next datacenter, members of the team traveled to several candidate locations throughout the US to tour facilities, meet facility staff, and otherwise continue the selection process. Following this process, we have been able to shortlist our bid proposals, and have begun the final selection process. We hope to complete bid selection and legal review in January.
 * Work continues on migrating our remaining services to our Ashburn datacenter. Consolidation and migration of databases, fundraising infrastructure, Labs, as well as progress on updating the configuration (puppetization) of several miscellaneous services was accomplished in December. Additionally, "triage" of the hardware within the facility was performed, with an eye towards what will be delivered to Ashburn, what will end up in our new facility, and what will be decommissioned.

Wikimedia Labs
 * Andrew Bogott purged of empty projects and stale instances, resulting in more accurate usage statistics for Labs:
 * Number of projects: 140
 * Number of instances: 403
 * Amount of RAM in use (in MBs): 1,592,832
 * Amount of allocated storage (in GBs): 21,525
 * Number of virtual CPUs in use: 797
 * Number of users: 2,425
 * Tool Labs saw a bump in usage as the winter holidays provided an opportunity for volunteers to migrate tools from the Toolserver and work on new projects; there are now 531 tools managed by 435 users, ranging from simple database queries to elaborate editing adjuncts using the new OAuth infrastructure.
 * Work for the impending migration of Labs to the Ashburn data center is well on its way: hardware is set up, the new storage servers are configured, and a lot of fresh OpenStack puppet manifests are in progress.

Language Engineering

 * Development of the TwnMainPage extension was completed; Translatewiki.net now has a new user registration process and a new dashboard for translators that provides insight in a user's activity compared to that of other users.
 * Plural rules for MediaWiki have been updated to comply with CLDR version 24. There were consequences for existing translations in Russian, languages that fall back to Russian, Serbian, Belarus and Ukrainian. These have been updated semi-automatically, and past contributors have been informed and asked to help in reviewing the updates.
 * MediaWiki Language Extension Bundle 2013.12 was released. It is compatible with MediaWiki 1.21 and MediaWiki 1.22. The MediaWiki language extension bundle provides easy way to bring ultimate language support to your MediaWiki. The bundle is a collection of selected few MediaWiki extensions needed by any wiki which desires to be multilingual.
 * A performance issue in the Translate extension that prevented use of the status field for translatable pages on Meta-Wiki was resolved.

Offline
Kiwix
 * The Kiwix project is funded and executed by Wikimedia CH.


 * We have released two new versions of Kiwix for Android this month (1.7 & 1.8), providing many new features; most of them were developed by young new developers as part of the Google Code-in program. Work continues around tools based on Parsoid output, especially as we need to rewrite the ZIM-related code for the MediaWiki offline toolchain, currently under heavy re-engineering. We have compiled download stats for 2013, and for the first time we have reached 700,000 downloads of the Kiwix app a year. Work to digitally sign the OSX and Windows binaries is ongoing and is the last step before releasing 0.9rc3. We have started experimenting with porting Kiwix-plug to RaspberryPi, and it looks good. Lots of new ZIM files were generated; we now generate a ZIM file of Wiktionary, as well as ZIM files without pictures.

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.


 * The Wikidata development team continued to work on quantity values, including localization, support for scientific notation and the user interface. They also worked on performance by improving caching and database handling. DataValues Serialization 0.1 was released, as well as Ask Serialization 1.0 and Wikibase DataModel 0.6. A new DataModel serialization component was started, which will allow authors and people analyzing dumps to have the deserialization task solved for them.

Future

 * The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.