New Orleans hackathon (14–16 October, New Orleans, USA) — About 30 MediaWiki developers and friends attended this coding event (organized by Ryan Lane, Sumana Harihareswara and Dana Isokawa) to work on Wikimedia's technical infrastructure. Work focused on the virtualization cluster / Wikimedia Labs, media storage (using Swift), Puppet, and continuous integration. The event was also used to discuss and prepare for the upcoming migration to git as our primary version control system.
San Francisco event (21-22 January 2012) — Erik Moeller and Sumana Harihareswara worked on planning for an outreach-focused developers' weekend. Staff and volunteer developers will help interested technologists learn to make awesome things with Wikimedia, by improving the Wikimedia platform, creating new functionality for Wikimedia users, or integrating Wikimedia content into their own products and services.
Data Centers [?] — Chris Johnson, Rob Halsell and Peter Youngmeister have racked and brought online new servers in Tampa to enhance the capacity of the Bits, MediaWiki and API application servers. Leslie Carr worked on partitioning with PartMan and documented the process. An external firm performed a security audit on our infrastructure on our behalf. The team also addressed several high priority break fixes, like the external data storage failures, and the analytics packet loss due to heavy load with udp2log processing. Peter Youngmeister upgraded all Apache servers to Ubuntu 10.04; Squid servers have also been upgraded, and the Linux kernels have been updated with the latest security patches. Ben Hartshorne continued to set up servers for external storage. Mark Bergsma and Asher Feldman deployed two new servers running a new and improved version of the Bits caching server in Amsterdam, to prepare for the deployment in Ashburn.
Media Storage [?] — During the New Orleans hackathon, developers agreed to create an abstraction layer (FileBackend) which will allow MediaWiki users to use SwiftMedia to store media files, or another filesystem of their choice. They also agreed to re-architect the FileRepo module in Mediawiki.
HTTPS — HTTPS support has been enabled on all Wikimedia sites; mobile support is still to come. At some point in the future, we'll be switching the log-in link to default to logging in via https.
Virtualization test cluster [?] — Wikimedia Labs has been launched. We are starting to install instances to duplicate our production environment. We are also bringing up some instances for non-MediaWiki related work. At the NOLA hackathon, we worked with a team from Canonical to give the juju project access to our environment. A new project from Wikimedia Deutschland, Wikidata, announced they'd be working in the Labs environment as well.
Data Dumps [?] — This month's dump for the English Wikipedia started later in the month (on October 11), since we were waiting for the production roll-out of MediaWiki 1.18. The run completed without issues, and all other servers are also using 1.18 without problems. We set up our first mirror site, hosted by the C3SL at Universidade Federal do Paraná in Brazil, whom we thank for their support. The mirror provides a copy of the last 5 successful dumps of each project and is updated as new dumps complete.
Article feedback — Work has started on Version 5 of this feature, which will focus on reader engagement, for example by offering new feedback forms and free-text comment fields; a comment moderation system is also planned. This project is led by contractor Fabrice Florin, and will be implemented by an external contracting company, OmniTI.
UploadWizard — Ian Baker implemented multi-file selection. Neil Kandalgaonkar did research on how licenses are used on Commons, added custom wikitext licenses with various safeguards to ensure this wouldn't result in more poorly licensed files, and implemented wording changes on the advice of Wikimedia general counsel. Ben Hartshorne implemented a simple way to add geocoding templates. Jeroen De Dauw also worked on this feature.
Mobile Research [?] — Mani Pande and Parul Vora conducted user experience research in the US in three cities (San Francisco, Chicago and Dallas) assisted by UX firm AnswerLab. They met with current and potential mobile readers, as well as editors. They are also finalizing the report on their field research in India and Brazil.
Android Wikipedia App [?] — We partnered with nitobi to create the first official Wikipedia app for Android devices. The app is based on the PhoneGap framework and should be added to the Android market soon. The team is still looking for developers.
2011 Fundraiser — October was a very busy month for fundraising engineering, as the annual fundraiser is about to start. The biggest achievement was the rewrite and deployment of the credit card processing software, the DonationInterface extension. It now supports GlobalCollect, a new payment processor, in addition to the existing ones. Work is underway to add support for non-credit card payments. Another major feature of the extension is the ability to easily create new forms simply by inserting template tags into HTML markup; it also leverages MediaWiki's message translation system using translatewiki.net. Changes were made to the LandingCheck extension to determine whether to send users to donate.wikimedia.org (used by the Wikimedia Foundation) or wikimediafoundation.org (used by some Wikimedia chapters). Donators can now opt out from e-mails directly from the thank-you message they receive after their donation. Last, from an operations perspective, we increased the monitoring capacity of the servers in the payment cluster, as well as the redundancy of systems; we're currently working to implement automatic failover between our data centers.
Kiwix UX initiative [?] — The last critical bugs in version 0.9 of Kiwix (especially on Windows) were fixed, and the release candidate cycle will soon begin. A Kiwix activity was released in alpha for Sugar, a learning platform for children. The first two prototypes of the Kiwix-plug have been built; the Kiwix-plug is a standalone wifi hotspot providing offline Wikipedia content, based on a plug computer.
Code review management — The focus on 1.18, as well as events and conferences, have allowed the backlog of unreviewed commits to increase in October (see chart). There are currently about 350 unreviewed revisions in trunk/phase3/, which means the deployment of MediaWiki 1.19 might not happen until January 2012.
Git conversion — Chad Horohoe started to prepare for the migration to git; he wrote a script to compare LDAP accounts to the information stored in Code review, and prepared for conversion tests from the current Subversion repository. An important step in the conversion is having names and email addresses for all committers, since neither Git nor Gerrit function well without them. As of this writing, 26 committers do not have any contact information, and 25 have a name but no e-mail address. Please help by filling in the names of committers you know.
Wikimedia Report Card 2.0 — Erik Zachte discovered inconsistencies in the report card numbers, which were investigated and attributed to packet loss of up to 25%. This has now been fixed; further steps to prevent similar issues include adding monitoring to the process, and versioning the configuration files.
Summer of Code 2011 — Sumana Harihareswara, Neil Kandalgaonkar and Max Semenik attended the GSoC mentor summit in Mountain View, California. Students Salvatore Ingala and Kevin Brown made progress on integrating their summers' work into MediaWiki trunk, during the New Orleans hackathon; much of the GSoC students' work remains to be integrated into MediaWiki core or extensions.
MediaWiki architecture document — Guillaume Paumier finished the write-up of the document, based on the input provided by developers, the existing documentation on mediawiki.org, the auto-generated documentation from doxygen, and deep dives into the actual code. The MediaWiki community reviewed the document, which was submitted to the book's editors. This project is considered to be mostly completed. Further work will include minor polishing and follow-up on the document and its publication, and integration of the content into the relevant pages on mediawiki.org.
The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.