Wikimedia Engineering/Report/2012/July

 Engineering metrics in July:
 * The total number of unreviewed commits went from about 320 to about 360.
 * About 35 shell requests were processed.
 * About 80 developers got access to Git and Wikimedia Labs.
 * Wikimedia Labs now hosts projects,  instances and  users.

Major news in July include:

Recent events
Pre-Wikimania hackathon (10–11 July 2012, Washington, D.C., USA)

This year's pre-Wikimania Hackathon was special in that it had a full track for newcomers, going beyond tutorials. The Hackathon was a collaboration with OpenHatch, an open source teaching non-profit. The new efforts included identifying appropriate first-time tasks to orient newcomers into more advanced Wikipedia editing and tech contribution, creating a laptop setup guide that steps attendees through the process of configuring development environments, and providing constant in-person assistance to help people past problems they encountered. While at the event, we saw many people learning more about templates, editing Wikipedia, and using and modifying bots to improve the encyclopedia and media on it. At least 65 people signed in, with more surely more in attendance. A more full report on the Hackathon is forthcoming to the wikitech-l list.

Wikipedia Engineering Meetup
To showcase the interesting problems and products that Wikimedia engineering work on to the local developer community, the Tech group has created a Wikipedia Engineering Meetup. The Meetup plans meet every two months at WMF offices in San Francisco and tentatively consists of three short 15 minute engineering presentations followed by a question and answer period bracketed by mingling.

The inaugural meetup will be on August 15 and the talks scheduled will be:
 * Tomasz Finc and Jon Robson talking about Wikipedia Mobile
 * David Schonover talking about Analytics at the Foundation
 * Trevor Parscal and Roan Kattouw talking about the VisualEditor

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Peter Youngmeister, who was working as a contractor for the Operations team, was converted to full-time Technical Operations Engineer (announcement).
 * S Page joined the Editor engagement experiments team as Software Engineer (announcement).
 * S Page joined the Editor engagement experiments team as Software Engineer (announcement).

Site infrastructure


Data Centers

Object Store/Swift

Testing environment
Wikimedia Labs

Backups and data archives
Data Dumps
 * The YAS3 library for uploading to archive.org and to other s3-compatible sites, along with several command line clients, is now usable (though still under heavy development). This library handles 100 Continue correctly; this means that for large file uploads, the upload is only attempted once the client has been redirected to the right host, a great time saver. The library also supports uploads of large files in multiple chunks automatically, rather than requiring the user to split the file into separate pieces. That's a necessity for us since many of our dump files are quite large.

Offline
Kiwix


 * We finally released Kiwix 0.9 rc1 (see the CHANGELOG). All the binary files were compiled using our new continuous integration build platform. In collaboration with Wikimedia France (for the Afripedia project), we released a first version of kiwix-plug, a standalone WiFi hotspot using cheap plug computers. The Black&White project, contracted by Wikimedia CH, was completed; a recent achievement was the introduction of Kiwix in the official Debian package repository. Also in collaboration with Wikimedia CH, we started a new project called ZIM autobuild aiming to quickly and automatically generate ZIM files of our projects.

Wikidata

 * The Wikidata project is funded and executed by Wikimedia Deutschland.

The Wikidata team has made good progress towards their first roll-out. The initial deployment plans are being made and the Hungarian Wikipedia community stepped up to be the first to use the interwiki part of Wikidata in a few weeks. You can follow the deployment planning at Wikidata/Deployment. This also means the demo system needs to be tested more. If you have five spare minutes, have a look at the demo system and report any bugs you might find there so they can be fixed before the initial deployment.

The team also started to collect future use-cases of Wikidata that should be kept in mind during development. You can find the existing ones here and are invited to refine them or add your own. Additionally the team is looking for feedback on the third iteration of the storyboard for linking Wikipedia articles in the future.

Future
The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.