Wikimedia Engineering/Report/2011/August

Major news in August include:

Hover your mouse over the green question marks to see the description of a particular project.

Recent events

 * Wikimania (August 2-7, Haifa, Israel) —

Upcoming events

 * New Orleans hackathon (14-16 October, New Orleans) —
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/54940


 * Check out the Software deployments page on the wikitech wiki for up-to-date information on the upcoming deployments to Wikimedia sites.

Job openings
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. The following positions opened in August:

Open Requests for Proposals:
 * Article Feedback Feature

The following positions are still open: Product Manager (Analytics), QA Lead, Operations Engineer (Networking), Director of Features Engineering, Systems Engineer (Data Analytics), Networking Contractor (Amsterdam), Software Developer (Rich Text Editing, Features), Software Developer (Front-end) and Software Developer (Back-end), Product Manager (Mobile), Software Developer (Mobile), Product Manager (New Editor Engagement)

Short news

 * Jeremy Postlethwaite, software engineer, Fundraising (announcement)
 * The team for the Internationalization and localization tools project was put together in August. It consists of Siebrand Mazeland (Product Manager, Localization), Niklas Laxström (Internationalization Engineer) and Gerard Meijssen (Internationalization / Localization Outreach Consultant).

Training and Process Improvement

 * Operations staff meeting — The Operations team got together the week of the 22nd. The goals of the meeting were to: improve and share site recovery knowledge (documentation and training); share knowledge of new project designs; review & prioritize operations projects; document and communicate our EQIAD data center buildup milestones; and to develop the RT management process.

Site infrastructure

 * Tampa Data Center — Mark Bergsma put into production the second Tampa router, which means we have now router redundancy in our Tampa network infrastructure. Follow-up work is underway to fully implement auto-router (hot) failover, which should be completed by mid September. Mark also standardized our LVS implementation and puppetized the configuration. Since our data center contractor in Tampa left us, the installation of new application servers was delayed. Other highlights include software upgrade to our Squid servers, and upload performance issues (now solved).


 * Virginia Data Center — Asher Feldman deployed the new Mobile Varnish servers in our Eqiad data center. All six LVS servers are ready and two of them are in production now, load-balancing the mobile Varnish servers. Ben Hartshorne and Asher also created two new database servers for the Summer of Research interns, in addition to the original one created last month. Last, the team updated the backup procedures documentation to reflect Eqiad being our (current) key backup and recovery store.


 * Media Storage — Russ Nelson refactored code in the SwiftMedia extension following Tim Starling's code review feedback; he also asked users for peculiar use cases involving file manipulation. Ben Hartshorne started to plan for the deployment and switchover on Wikimedia sites.


 * HTTPS & IPv6 — Roan Kattouw and Ryan Lane have been fixing issues that surfaced during the internal testing period. New servers were ordered for the AMS and TPA to handle SSL termination processing. In the meantime, Ryan has been setting up SSL servers in eqiad, enhancing Varnish to deal with x-forwarded-for and x-forwarded-proto HTTP headers, and making necessary changes to Squid.

Testing environment

 * Virtualization test cluster —

Backups and data archives

 * Data Dumps — The July runs were completed, as well as 7z files for May and the August run for the English Wikipedia. NFS mount was changed to "hard mount" in an attempt to fix file truncation issues. Preliminary code for the production of checkpoint file is ready for testing.

Editing tools

 * Visual editor —
 * Internationalization and localization tools —

Content Quality and Editorial Tools

 * Article feedback —

Participation and editor retention

 * WikiLove —
 * MoodBar Dashboard —
 * GlobalProfile —
 * QuickComments —

Multimedia Tools

 * UploadWizard —

MediaWiki infrastructure

 * ResourceLoader —

Wikimedia Labs

 * TimedMediaHandler —
 * MediaWiki.next —

Mobile

 * Mobile Research — Mani Pande and Parul Vora continued to synthesize the findings from field research in India and Brazil; a research page was also created on meta. They launched user experience research in US with AnswerLab, and have started recruiting readers and editors for ethnographic research to be conducted in San Francisco, Dallas and Chicago. The mobile survey was prepared in LimeSurvey, and translations are ongoing.


 * MobileFrontend —
 * http://blog.wikimedia.org/2011/08/17/calling-mobile-testers-for-round-two/
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/54875
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55009
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55056
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55098
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55194

Fundraising support

 * 2011 Fundraiser —
 * http://blog.wikimedia.org/2011/08/18/fundraiser-engineering-update-sprint4/
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55064

Offline

 * Wikipedia version tools —
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55065
 * Kiwix UX initiative —
 * http://blog.wikimedia.org/2011/08/16/come-beta-test-offline-wikipedia/
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55217


 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/55141

Platform Engineering

 * http://blog.wikimedia.org/2011/08/17/what-is-platform-engineering/

MediaWiki Core

 * MediaWiki 1.18 —
 * Code review management —
 * Heterogeneous deployment —
 * API maintenance —
 * Shell requests —
 * Continuous integration —
 * Wikitext scripting —
 * Projects on hold — The HipHop deployment, AcademicAccess and Disk-backed object cache projects were on hold in August.

Wikimedia analytics

 * Wikimedia Report Card 2.0 —

Technical Liaison; Developer Relations

 * Bug management —
 * Summer of Code 2011 —
 * Engineering project documentation —
 * Volunteer coordination and outreach —
 * http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/54843
 * MediaWiki architecture document —