Wikimedia Engineering/Report/2012/October

Engineering metrics in October:
 * 110 unique committers contributed patchsets of code to MediaWiki.
 * The total number of unresolved commits remained stable around 440.
 * About 42 shell requests were processed.
 * About 57 developers got access to Git and Wikimedia Labs.
 * Wikimedia Labs now hosts 137 projects, 694 users; to date 1268 instances have been created.

Major news in October include:
 * a redesign of the mobile site emphasizing readability and navigation;
 * the launch of a Wikipedia app for Windows RT and Windows 8 tablets;
 * a test of a redesigned account creation page;

''Note: As of last month, we're proposing a shorter and simpler version of this report for less technically savvy readers.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Željko Filipin joined the Platform engineering team as QA Engineer (announcement).
 * Andre Klapper joined the Platform engineering team as Bug Wrangler (announcement).
 * Michelle Grover joined the Mobile engineering team as a QA contractor (announcement).
 * Luke Welling joined the Features engineering team as Senior Features Engineer (announcement).
 * Brad Jorsch joined the Platform engineering team as Software Engineer, working in the MediaWiki Core group (announcement).
 * Steven Bernardin joined the Operations team as Data Center Technician, working in our Tampa data center.

Technical Operations
Site Infrastructure
 * Mark Bergsma has successfully implemented range seeking feature in Varnish, fixed several video streaming bugs, and finally redeployed Varnish at Eqiad, replacing the upload Squids in Tampa. Mark is now working on replacing upload Squid at Esams. He has provisioned 8 servers, and based on early testing, less may actually be needed. We are currently using 23 older servers for upload Squid at Esams. In addition, Mark deployed 4 new Varnish servers to serve bits from Esams; The existing 2 are being redeployed for other uses. This will provide higher throughput and added redundancy for the coming Fundraising season.


 * Due to Swift cross-datacenter replication issues, we have moved originals to our nas1 server in Tampa as a stopgap measure, and replicated the contents to nas1001 at Eqiad. Currently, we have only 'originals' in nas1/nas1001. Mark, Aaron Schulz and Faidon Liambotis will work on copying over thumbnails next. Faidon upgraded Swift to 1.74 to address the Swift proxy memory leak issue.


 * Tim Starling has deployed limited php-redis on the Apache servers, and Redis is now capturing session data (mc1). The new Memcached servers at Tampa are ready, and Asher Feldman has started testing, and working on putting them into production in the coming weeks. He was originally hoping to use Redis to replace memcached/parser cache, but had to stop due to performance/latency issues with their replication method. Meantime, Asher has put into use the captured session data and identified some areas for performance improvement. The developers have been notified and are working on resolving the issues now.

West Coast caching center
 * We've started building out a new caching center in the San Francisco area, called ULSFO. In October, Leslie Carr and Daniel Zahn racked and stacked networking equipment. Next, we are going to purchase the caching servers once Mark Bergsma confirms the configuration.

Data Dumps
 * Compressed multistream format files of current articles are now being produced for all dumps; researchers working with content from the larger wikis may find these helpful. We're working with Amazon on hosting the most recent dumps for EC2 users, thanks to Diederik van Liere. We've encountered some performance issues with media bundle generation off-site and are investigating; we're also working on moving that from experimental to production status.

Wikimedia Labs
 * Home directories are being migrated to glusterFS: pam_mkhomedir has been enabled, the home directory creation script has been disabled, and /home is now a direct autofs mount, rather than having indirect mounts for each user under /home. Nova, Glance, and Keystone have been upgraded to latest Essex stable release. On all instances, Salt has been installed and puppet has been run. Numerous bugs have been fixed in OpenStackManager (project creation bug, removing tiny flavor from interface) and some features have been added as well (add user to bastion when shell group added). A patch was sent into Nova to enforce unique instance names. The Labs team attended the OpenStack summit in San Diego. One takeaway is that we are leading a team to push a DNS service into OpenStack incubation; Andrew Bogott's Nova DNS code will be merged into this project.

Fundraising
 * Existing and new payments clusters were fully integrated, with common puppet, code propagation, logging. Eqiad payments cluster was successfully tested with several hours of production traffic. SPF was rolled out for the wikimedia.org domain, which improved deliverability of fundraising email. A failing database machine was replaced, and one additional machine deployed for the duration of the fundraiser. A lot of work with Fundraising Tech on measurement and improvement of banner and landing page performance.

Readers
Windows 8 app
 * Brion Vibber released a native Wikipedia application for Windows 8. This app was not part of the regular Wikimedia product roadmap; instead, it was the result of a 1-day iteration that each team member gets for research time.

Offline
Kiwix
 * Release 0.9rc2 is almost finished. The highlight of this release is kiwix-serve for MS/Windows, directly available from the Kiwix UI. A first version of kiwix-plug was installed on 15 devices with the Afripedia project. With Wikimedia France, USB sticks containing the French-language Wikipedia were made available for purchase for the first time in France; they were sold out after only a week.

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.

The Wikidata team has worked on initial parts of Phase 2 of Wikidata (Infoboxes) and worked together with the WMF to get Wikidata deployed on http://www.wikidata.org. A big step towards this deployment was the merge of the content handler branch into MediaWiki core. This allows MediaWiki to handle other content types besides just wikitext. In addition, the team is looking for help with the initial design of the Main Page of wikidata.org. A draft was also published by the team discussing how the propagation of changes from a repository to the clients should work. Feedback and questions are welcome during the IRC office hours, on the mailing list and on meta.

Future
The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.