Wikimedia Engineering/Report/2014/January

Engineering metrics in January: Major news in January include:
 * 159 unique committers contributed patchsets of code to MediaWiki.
 * The total number of unresolved commits went from around 1386 to about 1320.
 * About 22 shell requests were processed.
 * the transition of our search engines on Wikimedia sites to ElasticSearch;
 * a presentation of how the Tech newsletter works, including a historical perspective;
 * an invitation to comment on the 2-year vision of the multimedia engineering team;
 * a request for comments on whether the MP4 video format should be supported on Wikimedia sites.

''Note: We're also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

Upcoming events
There are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.

For a more complete and up-to-date list, check out the Project:Calendar.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.



Announcements

 * Shahyar Ghobadpour joined the Wikimedia Core features team as Software Engineer (announcement).
 * David Chan joined the Language Engineering team as Software Engineer (announcement).
 * Gilles Dubuc joined the Multimedia team as Senior Software Engineer and Tech Lead (announcement).
 * Charles Salvia joined the Analytics team (announcement).
 * Sam Smith joined the Features Engineering team as Software Engineer, working with the Growth team (announcement).
 * Preteek Saxena joined the UX Design team as UX Prototyping contractor (announcement).

Technical Operations
Datacenter RFP
 * The Wikimedia Operations team is in the final stages of the selection process. A short list of 4 bids has been created and final negotiations are underway. The winner of the bid will be selected in February based on the technical criteria listed in the RFP, and pricing.

Labs metrics in January: Wikimedia Labs
 * Number of projects: 131
 * Number of instances: 441
 * Amount of RAM in use (in MBs): 1,734,144
 * Amount of allocated storage (in GBs): 23,505
 * Number of virtual CPUs in use: 867
 * Number of users: 2,595
 * The Labs Migration team, consisting of Andrew Bogott and Marc-Andre Pelletier, have made good progress with testing the newest version of Openstack (called Havana) and with Neutron, an OpenStack project to provide "networking as a service". The plan is to upgrade the Openstack software when we migrate the Labs infrastructure out of the Tampa data center.

Language Engineering
Content Translation

The language engineering team kicked off development of a prototype version of context translation workflow. This functionality aims to create a workspace for helping editors bootstrap new articles in non-Latin language Wikipedias. In the prototype, Russian and Welsh are being used for initial concept verification. Technical architecture details for this project can be found at http://www.mediawiki.org/wiki/Content_translation

Universal Language Selector

This feature was disabled on January 21 2014 in production for all language Wikimedia sites (other than wikidata.org) for font delivery performance issues. Users can still enable ULS for their language needs by going to their user profile preferences and enabling ULS from the internationalization settings. Development is in progress for a solution to enable ULS when a user logs in and selects their language preferences explicitly to enable webfonts.

Visual Editor Language Support

David Chan continued his work on language support integration for the visual editor for phase 5 languages (see http://www.mediawiki.org/wiki/VisualEditor#Timeline)

Mediawiki Architecture Summit

Niklas Laxstrom and Santhosh Thottingal participated in the architecture summit in San Francisco in January in RFC discussions and jsonification of i18n support for the Visual Editor. See https://wikimedia.mingle.thoughtworks.com/projects/scrum_of_scrums/cards/58 for more details.

New members on team

David Chan joined the Language Engineering team in January. See announcement on wikitech-l on January 6 2014

Collaboration and outreach

Team continued their collaborative projects with Google, Twitter, Microsoft internationalization and MT teams on webfonts, input tools and machine translation.

Blogs posts on language engineering related topics include: OpenDyslexic font now available on Polish Wikipedia http://blog.wikimedia.org/2013/12/17/opendyslexic-now-available-in-polish/

Offline
Kiwix
 * The Kiwix project is funded and executed by Wikimedia CH.


 * Much time this month was spent planning for 2014. We mainly worked on mwoffliner and almost managed to create a full English Wikipedia ZIM file with thumbnails. The upgrade of our main storage platform allowed us to start our automatic ZIM file generation system.

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.


 * In January, the team worked mainly on performance improvements around Wikidata. The Quantities datatype was deployed so it is now possible to enter data like the number of inhabitants of a country. Wikisource can now manage its language links via Wikidata as well just like Wikipedia, Wikivoyage and Commons could already. Two new front-end developers, Adrian and Thiemo joined the team to help improve Wikidata's user interface. Last but not least, the team released their plan for the development of Wikidata in 2014 and beyond.

Future

 * The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.