Wikimedia Engineering/Report/2012/August

 Engineering metrics in July:
 * The total number of unreviewed commits went from ~360 to ~.
 * About shell requests were processed.
 * About developers got access to Git and Wikimedia Labs.
 * Wikimedia Labs now hosts projects,  instances and  users.

Major news in July include:

Recent events
Wikipedia Engineering Meetup (15 August 2012, San Francisco, USA)

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Srikanth Lakshmanan joined the Internationalization and localization team as outreach coordinator / QA engineer, contractor (announcement).
 * Daniel Zahn moved over from Germany to San Francisco office and joined us as a full time Technical Operations Engineer
 * Andrew Bogott was converted from a contract to a full time Dev/Ops Engineer, working on Labs development.
 * Andrew Bogott was converted from a contract to a full time Dev/Ops Engineer, working on Labs development.

Operations
Site infrastructure
 * Continuing from his earlier MySql work, Asher built additional MySql servers for each of the clusters in Ashburn, all in preparation for the primary data center migration in the coming quarter. In addition, he  deployed an additional server to the En cluster and replaced the EN master database server too. The lastest information on our  database clusters can be found here.
 * Fundraising infrastructure buildup@ EQIAD [Jeff G]
 * Fundraising infrastructure buildup@ EQIAD [Jeff G]


 * provision and deploy Boron (build host)
 * provision and deploy Indium (logging host)
 * build pxeboot, preseed, and puppet config
 * provision pay-lvs hosts
 * deploy nagios-nsca monitoring for new hosts
 * Limited Apache-on-Precise deployment @ Tampa [Asher/Peter/Mark]
 * NetApp /home deployment [Mark]

Install new passive optics system between the 2 datacenters in Tampa CWDM Optics to increase bandwith capacity Varnish deploy Varnish with persistent store [Mark] [testing on mobile]
 * Moved Apache error logs to nfs to a dedicated server in the new data center, and patched the wmerrors extension (Tim Starling)

Object Store/Swift
 * In addition to thumbnails, Swift is now the primary object store for Images and multimedia contents (aka Originals). Those contents were successfully copied over to Swift from the NFS filer and Mediawiki is now updating both Swift and the NFS server (ms7). In the coming months, we will be disabling the content updates to ms7 and run solely on Swift. Users reads for those contents are now being served from Swift instead from ms7.

Wikimedia Labs
 * Labs OpenStack release upgrade
 * Labs OpenStack release upgrade

Data Dumps

Other news
 * Site issue Aug 6 2012
 * Asher Feldman has been promoted to Site Architect.

Offline
Kiwix

We mostly have worked on the 0.9 RC2 (see CHANGELOG) which should be released soon after the portage of kiwix-serve to MS/Windows. Kiwix UI localisation was improved, thanks to the Translatewiki Rally, four new languages are supported. For the ZIM autobuild project, we have migrated the server to a datacenter in Zurich, Switzerland and coding work is ongoing. Otherwise, pretty much energy is involved in 2013 projects planning, we need volunteers and ideas... Join us!.

Wikidata

 * The Wikidata project is funded and executed by Wikimedia Deutschland.

Future
The engineering management team continues to update the Software deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the engineering roadmap, listing ongoing and future Wikimedia engineering efforts.