Wikimedia Engineering/Report/2011/September

Major news in September include:
 * The deployment of protocol-relative URLs, paving the way for native HTTPS;
 * Major progress by our new Internationalization & localization team;
 * The deployment of our new mobile front-end, now the default mobile experience;
 * The start of the deployment of MediaWiki 1.18 to Wikimedia sites;
 * The completion of the Google Summer of Code project;
 * Ongoing preparation for the upcoming coding events in the USA, the UK, India and online;
 * A revamped, and now maintained, roadmap for Wikimedia engineering efforts.

Hover your mouse over the green question marks to see the description of a particular project.

Upcoming events

 * New Orleans hackathon (14–16 October, New Orleans, USA) — Ryan Lane and Sumana Harihareswara continued to plan this coding event, which will focus on tools and infrastructure, and Wikimedia Labs. Discussions about the upcoming migration to git are also planned.
 * Weekend of Code (18 October - 8 November, online) — Weekend of code is a planned hacking contest targeting developers worldwide. Awards will be presented for the best gadget, extension, and mobile app. The contest will be advertised via different channels.
 * India hackathon (18–20 November, Mumbai, India) — A hackathon will be held concurrently with the WikiConference India 2011, and near its venue. The main focus will be on language, mobile and offline support for MediaWiki content. About 100–150 participants are expected (read more).
 * Brighton hackathon (19–20 November, Brighton, England) — Lewis Cawte is preparing a hackathon in Europe dedicated to general MediaWiki hacking.
 * Check out the Software deployments page on the wikitech wiki for up-to-date information on the upcoming deployments to Wikimedia sites.

Job openings
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.

The following positions are open: Product Manager (Analytics), QA Lead, Director of Features Engineering, Systems Engineer (Data Analytics), Software Developer (Rich Text Editing, Features), Software Developer (Front-end) and Software Developer (Back-end), Product Manager (Mobile), Software Developer (Mobile), Product Manager.

The following requests for proposals are open: Networking Contractor Amsterdam, Internationalization and Localization Feature Development, Logging Analysis, Article Feedback Feature, Development and Operations Engineer, Systems & Operations Engineer, XML Dumps Help.

Site infrastructure

 * Tampa Data Center — Our new data center contractor in Tampa, Chris Johnson, has finalized the installation of three new racks, which will be used for networking, application serving, caching and data storage. Two racks (72 servers) worth of application servers have also been installed. With each server having 12 CPU cores, this is almost doubling our existing application server capacity. To-date, we have deployed and added 58 (of the 72) new application servers to production capacity.


 * Virginia Data Center — Ben Hartshorne has completed replicating External storage (article texts) data to our new data center in Ashburn, for disaster recovery and usage by eqiad application servers in the future. Preparations are also being made to deploy bits.wikimedia.org in eqiad using Varnish 3. Jeff Green has been building up the new EQIAD fundraising infrastructure and securing those boundary servers.


 * Media Storage — Russ Nelson has continued work on the SwiftMedia extension for support of Swift as a MediaWiki foreign media file repository, and image thumbnail scaling. The integration of Swift with MediaWiki will also be one of the topics of the upcoming Hackathon in New Orleans.


 * HTTPS — Protocol-relative URLs have been enabled on all sites by Roan Kattouw. SSL termination servers have arrived and are in the process of being installed. HTTPS should be fully enabled in early October.


 * Lowlights — There were three short outages on September 26th, 2011.

Testing environment

 * Virtualization test cluster — All services except for DNS are up. The puppet repository has been released in a public repository. We've switched to using git/gerrit for our production puppet process. New instances have been tested building from scratch using puppet. We've started giving accounts out to some early testers of Labs via labsconsole.

Backups and data archives

 * Data Dumps — The September English Wikipedia run completed in under 8 days. The dump of all text revisions is now available in a number of smaller files rather than several giant ones. The dump system has been converted to work with heterogeneous deployment and has undergone some testing with the upcoming MediaWiki 1.18 codebase.

Editing tools

 * Visual editor —
 * Internationalization and localization tools —

Participation and editor retention

 * Article feedback —
 * LiquidThreads 3.0 —
 * MoodBar —
 * Feedback Dashboard —
 * Article creation and patrol — In response to a request by a majority of English Wikipedia community members to restrict new page creation to autoconfirmed users, discussions are underway to improve the article creation workflow and the user interface for new page patrolling as alternative strategies to cope with the new page creation backlog and reduce high-friction interactions with new users.

Multimedia Tools

 * UploadWizard —

MediaWiki infrastructure

 * ResourceLoader —

Wikimedia Labs

 * Multimedia —

Mobile

 * Mobile Research — Mani Pande and Parul Vora continued to work on the report on their field research in India and Brazil. They completed phone interviews in San Francisco, Chicago and Dallas (with partner firm Answerlab) and started to schedule in-person interviews.


 * MobileFrontend —

Fundraising support

 * 2011 Fundraiser —

Offline

 * Kiwix UX initiative — The Kiwix team built out a beta portal for feedback this month, where users can now easily submit their thoughts on using Kiwix. Next, we'll be adding some in-app feedback to offer multiple ways of sending ideas and feedback.

MediaWiki Core

 * MediaWiki 1.18 —
 * Code review management —
 * API maintenance —
 * Shell requests —
 * Continuous integration —
 * Wikitext scripting —
 * Git conversion —

Wikimedia analytics

 * Wikimedia Report Card 2.0 —

Technical Liaison; Developer Relations

 * Bug management —
 * Summer of Code 2011 —
 * Engineering project documentation —
 * Volunteer coordination and outreach —
 * MediaWiki architecture document —