Wikimedia Engineering/Report/2011/May

Major news this month include:

Recent events

 * Berlin Hackathon 2011 (May 13-15, Berlin) —
 * Wikimedia tech crowd and MediaWiki developers gather in Berlin
 * Wikimedia developers start second day of Berlin hackathon
 * Berlin hackathon continues with group coding, discussions and bug squashing
 * Developers go home after productive Berlin hackathon


 * CiviCRM code sprint (May 24-25, Berlin) — The Wikimedia Foundation office in San Francisco hosted a coding sprint for about 8 CiviCRM developers in May. Participants squashed many bugs, and also improved contact/contribution search performance by 15-25x. This is particularly useful for major users of CiviCRM with large databases, like the Wikimedia Foundation and its donors database. The Wikimedia Foundation, who endeavors to contribute to the free software ecosystem, had already hosted meet-ups for the CiviCRM community in the past.

Upcoming events

 * Wikimania (August 2-7, Haifa, Israel) —


 * Check out the Software deployments page on the wikitech wiki for up-to-date information on the upcoming deployments to Wikimedia sites.

Job openings
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.

The following positions have opened this month:
 * Operations Engineer — Special projects
 * Software Developer Front-end — General
 * Software Developer Back-end — General
 * Software Developer Back-end — General

The following positions are still open:
 * Engineering Program Manager — Data Analytics
 * Software Developer — Features
 * Systems Engineer — Data Analytics (previously Data Analytics Engineer)
 * Operations Engineer
 * Senior QA Engineer
 * Networking Contractor — Amsterdam
 * Software Engineer — Community R&D
 * Software Developer, Rich Text Editing — Features
 * Product Manager — Features

In addition, we hope to post the following positions over the next few months:
 * Release Engineer
 * Technical Writer

Short news

 * Visitors —
 * Hires —
 * Asher Feldman, Performance Engineer

Operations

 * Program manager: Mark Bergsma

Site operations
Virginia Data Center — Installation of a world-class primary data center for Wikimedia Foundation websites.
 * Status:

Media Storage — Improvement of our media storage architecture to accommodate expected increase in media uploads.
 * Status:

Testing environment
Virtualization test cluster — Environment to deploy temporary machines for testing and experimentation, for use by WMF staff and volunteers working on important projects (as capacity allows).
 * Status:

Backups and data archives
Backups — Improvement of backup coverage of Wikimedia-hosted data.
 * Status:

Data Dumps — Improvement of processes to create and provide public copies of public Wikimedia data.
 * Status:

Features Engineering

 * Program manager: Alolita Sharma

Editing tools
Visual editor 0.1 — Exploratory work to identify & prototype initial ideas for a visual editor for MediaWiki.
 * Status: Trevor Parscal and Neil Kandalgaonkar have done exploratory work on the visual editor project. Neil worked with developers of HackPad (a custom version of real-time collaborative editing software Etherpad) on a proof of concept of integration between Etherpad and MediaWiki (read more). They're now working on turning it into a MediaWiki extension. Work on the visual editor is also intersecting with the groundwork done on the new parser.

Content Quality and Editorial Tools
Article Feedback — A feature to collaboratively assess article quality and incorporate reader ratings on Wikipedia.
 * Status: Version 3 was deployed to the English Wikipedia on May 9 with new features, like the article feedback dashboard, a summary page showing general rating trends. The experiment was expanded to 100,000 articles and may be expanded further after analysis of the results. The article feedback feature is now in maintenance and bugfixing mode.

Discussions and Interactions
Liquid Threads — A feature that brings threaded discussions capabilities to Wikimedia projects and MediaWiki.
 * Status: Lead developer Andrew Garrett continued to work on his new object model for the back-end. Timo Tijhof or Jan Paul Posma  be working with Andrew on the new front-end.

WikiLove 1.0 — An extension to encourage praise and virtual gifts between users.
 * Status: Ryan Kaldari and Jan Paul Posma completed the first version of the WikiLove extension, including documentation for the API. They also made changes to the code in order to add parameters to the configuration to add new gifts. The code is now pending review.

MoodBar 0.1 — A feature to encourage new users to provide feedback.
 * Status: Brandon Harris published initial designs for this feature allowing new users to quickly provide feedback to the community. Erik Möller reviewed the designs with the team, and prototyping began.

Multimedia Tools
Upload wizard — A feature that provides an easier way of uploading files to Wikimedia Commons, the media library associated with Wikipedia.
 * Status: The Upload wizard was enabled as the default upload system on Wikimedia Commons on May 9. It was disabled shortly after that because of issues believed to come from a bug in ResourceLoader. It was re-enabled on May 17 after further investigation. This project is now in maintenance and bugfixing mode.

Other projects

 * ResourceLoader 2.0 — ResourceLoader 1.0 is now in maintenance mode. There are currently no engineering resources available to work on ResourceLoader 2.0.
 * Non-Roman character set localization — Alolita Sharma and Erik Möller are currently gathering requirements on this project with the help of possible customers, including the language committee.
 * Community feature prototyping —
 * German Wikipedia editor survey support — Wikimedia Deutschland will be running its own editor survey in mid June, to assess community health. The survey will run on the English and German Wikipedia. Wikimedia Deutschland will handle the development work needed to integrate CentralNotice with the user profile information. The Features team of the Wikimedia tech department is helping with code review and deployment.
 * Mobile survey support — The Global Development department of the Wikimedia Foundation is planning to run a survey about mobile usage on the English Wikipedia in early June. Ryan Kaldari and Arthur Richards provided engineering support for the survey; Nimish Gautam took over the project as he transitioned to the Global Development department.

Wikimedia Labs
Media projects — A set of features to improve media handling and key infrastructure support tools, many developed with Kaltura, such as Metavid, MwEmbed, and the Video Editor.
 * Status:


 * Program manager: Alolita Sharma

Special projects

 * Program manager: Tomasz Finc

Mobile
Mobile projects — All things Mobile and Wikimedia.
 * Status:

Mobile Research — A research project to help determine our Mobile strategy.
 * Status: India fieldwork in Bangalore and Delhi, consisting of ~30 2 hour interviews, was conducted from April 25th to May 6th. A follow up workshop with interview participants as well as community members took place in Bangalore on May 15th.  Recruiting and prepping for the parallel study in Brazil is underway and the fieldwork, consisting of ~20 2 hour interviews, will be conducted from June 6th - June 23rd.  The RFP for the third mobile research study in the United States was sent out to ~15 vendors and proposals will be accepted until mid-June.  The mobile survey launch has been delayed due to reallocation of resources, but is planned to go live in mid July.

Mobile site rewrite — Port of our Ruby-based mobile gateway to PHP.
 * Status: We demoed the mobile extension at the Berlin hackathon and led a lively debate about implementing it as a skin vs extension. For those following that discussion a good wrapup of it can be found here. Our detailed project status can be found here. As we finish up all the loose ends were preparing to start an intensive qa cycle that we'll need volunteers for. Stay tuned for a more detailed blog post about how to get involved. As always, for those wanting a bleeding edge look at development .. check out the code and play with it.

Fundraising support
2011 Fundraiser — Support and development for the annual fundraiser of the Foundation.
 * Status:

Offline
Wikipedia version tools — Support and development of a series of tools to select Wikipedia content for offline use.
 * Status: Yuvi Panda has begun work on porting the WP 1.0 Bot to a MediaWiki extension. He has drafted a project plan and has begun development on a way to parse and track assessment data found in articles.

OpenZim for Collections — Integration of openZim into the Collections extension.
 * Status: Having finalized all the existing critical bugs were talking with PediaPress and numerous others about where to take the projects next.

Kiwix — Improvement of the user experience of the Kiwix app to access offline Wikimedia content.
 * Status:

General Engineering

 * Program manager: Rob Lanphier

MediaWiki development and tools
MediaWiki 1.17 release — The upcoming MediaWiki release.
 * Status:

Code review management — Review of changes made to the MediaWiki code.
 * Status:

Bugmeistering — Management of our bug tracker.
 * Status:

Summer of Code 2011 — A sponsored community program allowing students to join the community as developers.
 * Status: In May, our eight Summer of Code students learned how MediaWiki works, as software and as a community. They checked out IRC and the mailing lists, talked with their mentors, asked for commit access, and started investigating the components that would include their project areas. On May 23rd, they started working on their projects full-time. WMF staffers and community members are mentoring the students, assisted by Sumana Harihareswara.

Parser & gadgets — Groundwork for the next generation visual editor of MediaWiki.
 * Status:

Performance optimization
PoolCounter — A MediaWiki extension to avoid parser deadlocks on high-traffic pages.
 * Status:

Disk-backed object cache — Deployment of a disk-backed object cache to increase the parser cache hit ratio.
 * Status:

Wikimedia analytics
udp2log — A custom data analytics logging system.
 * Status:

A/B testing — A set of tools to perform A/B testing on Wikimedia sites.
 * Status:

Wikimedia Report Card 2.0 — Usability improvements and streamlining of the creation of this monthly report of key metrics measuring community health
 * Status:

Technical communications
Engineering project documentation — An activity to ensure that project documentation of Wikimedia engineering projects is complete and up-to-date.
 * Status: The Berlin Hackathon and Wikimedia tech days were an opportunity to start catching up on missing project pages. Stubs were created using the new, lighter format, and some existing pages were transitioned to the new format.

Other projects

 * API maintenance —


 * Shell bugs —


 * Access to Subversion —


 * Migration to Git —


 * Heterogeneous deployment —


 * HipHop support —