Wikimedia Engineering/Report/2013/November

Engineering metrics in November: Major news in November include:
 * 146 unique committers contributed patchsets of code to MediaWiki.
 * The total number of unresolved commits went from around 1122 to about 1230.
 * About 29 shell requests were processed.
 * https://blog.wikimedia.org/2013/11/01/wikipedia-next-generation-discussion-system/
 * https://blog.wikimedia.org/2013/11/07/introducing-beta-features/
 * https://blog.wikimedia.org/2013/11/20/wikimedia-foundation-is-looking-for-a-vice-president-of-engineering/
 * https://blog.wikimedia.org/2013/11/22/developing-distributedly-part-2/
 * https://blog.wikimedia.org/2013/11/22/oauth-on-wikimedia-wikis/
 * https://blog.wikimedia.org/2013/11/25/wikidata-concept-cloud/
 * https://blog.wikimedia.org/2013/11/27/adding-musical-scores-to-wikimedia/

''Note: We're also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

Upcoming events
There are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.

For a more complete and up-to-date list, check out the Project:Calendar.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Jeff Hall joined the Platform engineering group as a member of the QA team (announcement).
 * Aaron Arcos joined the Platform engineering group as a volunteer developer working with the Multimedia team (announcement).
 * Dario Taraborelli was promoted to the position of Senior Research Scientist, Research and Data team lead. (announcement).
 * Aaron Halfaker was promoted to the position of Research Scientist (announcement).
 * Moiz Syed joined the User Experience team as User Experience Designer (announcement).

Technical Operations
Site infrastructure

Wikimedia Labs

A new dynamic proxy system has been deployed on Labs. This means that the admin of any project can arrange for public web access and a dedicated DNS hostname for a project instance without requesting an IP address. Labs staff and volunteers will now be reclaiming quite a few IPs as existing projects migrate to a dynamic proxy setup.

The WMF has hired a short-term contractor, Mike Hoover, to assist with the migration of Labs infrastructure from Tampa to our new datacenter in Ashburn. Mike has spent a lot of time exploring the existing infrastructure and running test setups; soon he will start to configure the new OpenStack nodes in production.

Andrew has been working on cleaning up stale and unused resources. He's working on some automatic documentation that will help users track the status of their projects and instances with an eye towards predicting the impact of the coming migration.

Labs suffered two brief outages: a brief, self-inflicted network failure, and a longer outage during which one of the virtualization hosts failed. Both outages were swiftly resolved, but there's a bit of lag as some tools and services failed to come back properly afterwards due to poor distribution of virtual servers (inter alia, both the grid master and the shadow (backup) master were on the same server).

Preparation for the move to the Ashburn data center is well in progress, with the new storage server being physically configured this week as well as the new hardware servers for user databases (including the new postgresql instance intended for OSM).

Engineering Community Team


In November, the Engineering community team held their second monthly showcase, as well as their quarterly review for the July–September period.

Kiwix
The Kiwix project is funded and executed by Wikimedia CH.


 * We have released this month two new versions (1.5 & 1.6) of Kiwix for Android providing many new features; Most of them developed by new young developers in the scope of the Google Code-in program. We have also released a new and unique tool to easily create ZIM file yourself from data on your hard disk; the tool is stable and can be used. Work continues with more resources than ever around tools based on Parsoid output, especially because we need to re-invent the ZIM related code for the Mediawiki offline toolchain which is currently in heavy re-engineering stage.

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.

The Wikidata developers held an office hour to give a status update and answer questions. The log is now available. In addition they worked on ranks, ordering of statements and the quantities datatype. The quantities datatype is needed to for example enter the number of inhabitants of a country in Wikidata. It is available for testing now on http://test.wikidata.org. Ranks will allow for certain statements to be marked as preferred or deprecated. This is for example useful to indicate a previous majors of a city or the number of inhabitants of a country in 1900.

Magnus Manske wrote a gadget that allows you to additionally show Wikidata search results when doing a search on Wikipedia. He also extended the Reasonator tool to now also work for cities. So far it only supported people and species.

Future

 * The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.