Wikimedia Engineering/Report/2012/June

 Engineering metrics in June:
 * unique committers contributed code to MediaWiki.
 * The total number of unreviewed commits went from about 250 to about 320.
 * About 53 shell requests were processed.
 * 45 developers got developer access to Git and Wikimedia Labs.
 * Wikimedia Labs now hosts projects,  instances and  users.

Major news in June include:
 * the [//blog.wikimedia.org/2012/06/02/diverse-wikimedia-tech-crowd-gathers-in-berlin/ Berlin hackathon], the largest gathering of Wikimedia technologists to-date, co-organized with Wikimedia Deutschland;
 * the [//blog.wikimedia.org/2012/06/21/help-us-shape-wikimedias-prototype-visual-editor/ June milestone release of the Visual Editor and Parsoid] to mediawiki.org;
 * the launch of IPv6 support for all Wikimedia projects.

Recent events
Berlin hackathon (1–3 June 2012, Berlin, Germany)
 * Approximately 104 participants from 30 countries came to Berlin, including MediaWiki developers, Toolserver users, systems administrators, bot writers and maintainers, Gadget creators, and other Wikimedia technologists. The community also learned more about the Wikidata and RENDER projects. More updates, links to videos, and followups are on the talk page.

Upcoming events
Pre-Wikimania hackathon (10–11 July 2012, Washington, D.C., USA)
 * Open source teaching nonprofit OpenHatch will be aiding in organizing and running this two-day event, with Katie Filbert, Gregory Varnum, and Sumana Harihareswara. Experienced Wikimedia technologists will collaborate on their own projects, while interested new developers will be able to learn introductory MediaWiki development. Accessibility will be one of the event themes.  The event is free to attend even for those not attending Wikimania itself.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Munaf Assaf joined the Product team as UX Designer, mainly working on the Editor engagement experiments (announcement)
 * Adam Wight joined the Features team as Fundraising Engineer (announcement)



Site infrastructure

 * June was another busy month for racking, stacking and provisioning of newly purchased equipment. In the works are additional servers to clusters such as External Storage, Memcached, Parser Cache, Object Store and Labs. Meantime, new servers were rolled out in EQIAD for analytics, DNS resolver, UDP2Log and for Storage3 replacement.


 * IPV6 Launch day (6/6/12) came and went without much fanfare. Much work was put into the infrastructure and system-stack  by Mark, Faidon, Ryan and Asher, especially into  LVS, PayBal, Varnish, Squid, DNS, database, Nagios monitoring and puppetization. We also took this opportunity to update those technology as well as run them on Precise (12.04) where possible. We are have been keeping IPV6 traffic on since.


 * During the Berlin Hackathon, the TechOps team got together for about 2 hours to review the year's progress. A blog on this will follow soon. In summary, the team completed xxx number of priority 1 projects that were identified at the beginning of the year. A blog with more details on this will follow soon. In addition to working on IPv6-related work, the team did a major cleanup of jobs creating cronspam.


 * We have identified a new colocation to be the new West Coast caching center, and it is located at 200 Paul Street, San Francisco. Work on setting up the infrastructure is planned to begin this coming August/September. With this caching center, we will be able to improve users' site experience for US west coast and Asia Pacific.

Data Centers

Object Store / Swift

 * A sever bottleneck has been identified in doing container listings in Swift and Ben Hartshorne is adding SSD drives to the swift back end storage nodes to provide faster container listings. Testing has been completed to verify that this change will solve the problem and it is being deployed to production this month.  Additionally, integration of the SwiftStack monitoring improvements was accepted to the mainline Swift codebase last month and will be deployed to our environment in July.

Testing environment
Wikimedia Labs

Backups and data archives
Data Dumps

Other news

 * We had our fair share of several short site incidents in the month of June. We experienced  ...

Wikidata

 * The Wikidata project is funded and executed by Wikimedia Deutschland.

Future
The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.