Wikimedia Engineering/Report/2014/September

Major news in September include:
 * a call for candidates for the Free and Open Source Software Outreach Program for Women;
 * a roundtable discussion between the Language engineering team and editors from the Catalan language Wikipedia, focusing on the Content Translation tool.

Engineering metrics in September:
 * 151 unique committers contributed patchsets of code to MediaWiki.
 * About 27 shell requests were processed.

Upcoming events
There are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.

For a more complete and up-to-date list, check out the Project:Calendar.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.

* Senior Software Engineer - Services
 * Software Engineer - Maps & Geo - Mobile
 * Software Engineer - Mobile - iOS
 * Release Engineer
 * Technical Writer
 * Full Stack Developer - Analytics
 * Research Analyst
 * Agile Coach/ScrumMaster - Team Practices Group
 * Senior Technical Product Manager
 * Community Liaison
 * Community Liaison (PT Contract)
 * Operations Security Engineer
 * UX Senior Designer
 * UX Senior Design Researcher
 * UX Visual Design Fellowship
 * Mobile Partnerships Regional Manager

Announcements

 * Damon Sicore joined the Wikimedia Foundation as Vice President of Engineering (announcement).
 * Rachel Farrand joined the Engineering Community Team as Events Coordinator (announcement).
 * Jeff Hobson joined the Wikipedia Zero engineering team (announcement).
 * Daisy Chen joined the UX Research team as Associate Design Researcher (announcement).

Technical Operations
Dallas data center
 * In September we have setup (backup) replication of most project data, including core databases and external storage. Work on Swift images and system backups was still ongoing into October. Essential system infrastructure such as an installation server, DNS, LVS, NTP etc. has been deployed as well.

Tampa data center
 * We started the last push to get the remaining services & systems out of our Tampa data center, with a deadline for shutdown of all systems on October 1st. The remaining services included PDF generation, mail servers, noc.wikimedia.org and LDAP.

Labs metrics in September:
 * Number of projects: 146
 * Number of instances: 415
 * Amount of RAM in use (in MBs): 1,996,288
 * Amount of allocated storage (in GBs): 20,435
 * Number of virtual CPUs in use: 977
 * Number of users: 4,083

 Wikimedia Labs
 * Wikitech (the Labs web interface) is now managed via the standard WMF deployment system. This should allow for more frequent MediaWiki updates and overall greater stability.
 * The last historic remaining dependencies on our old Tampa datacenter (e.g. LDAP and Labs DNS backup servers) were finally stamped out and replaced with dependencies on Dallas hardware.
 * One of the labs virtualization hosts (virt1006) was suffering intermittent problems, so all affected instances were migrated to other hosts in order to stave off possible future disaster. Consequently, Labs is a bit short on virtualization space, but new hardware procurement is under way.
 * Several long-unused instances and projects were cleaned up in order to free up more space.
 * The last of the ToolLabs replica DB servers was upgraded to MariaDB 10.

Kiwix
The Kiwix project is funded and executed by Wikimedia CH.




 * We made progress in our work with our partner Bookeen to get an e-ink reader able to read Wikipedia offline. We managed to get a first version of the device firmware working, and it will be tested in the field as part of the Malebooks pilot deployment.


 * As a consequence of a bug fixing sprint with Parsoid and Wikisource developers at Wikimania, we were finally able to generate usable ZIM files from Wikisource (example with fr.wikisource.org).


 * Work on the offline project Gutenberg continued and we are now almost ready to release. A few ZIM files are in testing, for example in German and in Spanish.


 * Kiwix was represented at the Selenium conference where we held a 2-day bug hunting session: 120 bugs were reported, of which 50% were fixed.


 * mwoffliner was improved to make it easier to use for everyone, in particular to make ZIM files for only a selection of articles. As a demonstration, we prepared a ZIM files containing all Wikipedia articles about medicine.


 * After many years, a new version of a tool to generate a live CD including Kiwix and Wikipedia content was released.

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.


 * In September, the Wikidata team focused on improving performance, doing groundwork for the new user interface design, and making it possible to track where data from Wikidata is used. Next to that, they worked on tests and prepared for a week-long meeting with the WMF multimedia team and volunteers to discuss and plan structured data support for Wikimedia Commons.

Future

 * The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.