Wikimedia Engineering/Report/2010/October

(to be posted on techblog.wikimedia.org by RobLa-WMF)

(intro)

(insert the article break here)

Operations
 Virginia Data Center - Setting up a world-class primary data center for Wikimedia Foundation properties.
 * Status:
 * Program manager: Mark Bergsma

 Media Storage - Re-vamping our media storage architecture to accommodate expected increase in media uploads.
 * Status:
 * Program manager: Mark Bergsma

 Monitoring - Enhancing both Operations and public monitoring to a) notice potential outages sooner, b) increase transparency to the community, c) support progress tracking required in the 5-year plan.
 * Status:
 * Program manager: Mark Bergsma

 Virtualization cluster - more easily deploy temporary machines for testing and experimentation. This cluster is intended for use not just by WMF staff, but will be available to volunteers working on important projects as capacity allows.
 * Status:
 * Program manager: Mark Bergsma

Content Quality Tools
 Article Feedback - Working on feature to collaboratively assess article quality and incorporate reader ratings on Wikipedia
 * Status: The first version of this feature was rolled out at the end of September, and we spent the month of October collecting and analyzing the feedback from the tool. We are currently designing the second version of this feature, and starting development later this month.  We plan to roll out the next phase in sync with the Public Policy Initiative Phase 2 in January.
 * Program manager: Alolita Sharma

 Pending Changes - Pending Changes is a new review feature recently deployed to en.wikipedia.org, which allows changes made by anonymous and new users be reviewed before they appear as the primary version of an article.
 * Status: Recently we completed two large features related to Pending Changes - speeding up the display of diff pages and adding a "Reject" button. We are currently on track to deploy a new version of Pending Changes on November 16 with further incremental improvements to be rolled-out as we complete development.


 * Program manager: Rob Lanphier

Threaded discussions
 Liquid Threads - LiquidThreads is an extension that brings threaded discussions capabilities to Wikimedia projects and MediaWiki.
 * Status: Development on this feature slowed in October due to lack of staff availability. However, we anticipate continued maintenance work occuring through the end of the year, with development picking back up in January.


 * Program manager: Alolita Sharma

Multimedia tools
 Upload wizard - The upload wizard is an extension for MediaWiki providing an easier way of uploading files to Wikimedia Commons, the media library associated with Wikipedia.
 * Status: We've extended this project one more month (until the end of November) to have a more robust implementation and time for testing before launching this on Wikimedia Commons. As part of this project, we've been working on an online licensing tutorial, which will be integrated into the website.  The feature is in testing on http://commons.prototype.wikimedia.org/uwb/
 * Program manager: Alolita Sharma

 Add media wizard - The Add-media wizard is a gadget to facilitate the insertion of media files into wiki pages. Development effort for this labs feature is supported by Kaltura (consider re-working this section to a "MediaLabs" section that can contain notes on Michael Dale's gadgets).
 * Status: The current phase of development is focused on integrating the existing components with ResourceLoader to decrease load time for the gadgets. Other components in development include a media sequencer, embedPlayer ( timedMediaHandler, WebM integration ), and SwarmPlayer. xxxxAdd URLs for each of these componentsxxxx. For more details, also see this techblog post. (https://techblog.wikimedia.org/2010/10/video-labs-universal-subtitles-on-commons/)
 * Program manager: Alolita Sharma

MediaWiki Infrastructure
Resource loader - The resource loader aims to improve the load times for JavaScript and CSS components on any wiki page. The intention of this work is to support faster loading of the Vector skin, media extensions, and anything else that makes use of Javascript.
 * Status: The feature is largely complete and checked into trunk.  The team has been working on unit testing the framework and performance testing (and tuning).  We are also ensuring compatibility with existing extensions and modifying extensions to use the new messaging system.  We hope to have this work wrapped up some time in December, with a deployment sometime after that.
 * Program manager: Alolita Sharma

General Engineering
 Analytics Revamp - Incorporate an analytics solution that can grow and answer the questions that the Wikimedia movement has. Includes udp2log revamp.
 * Status: our primary focus has been on analytics tools for the Fundraiser, allowing that team to make data-driven decisions to refine our donation workflow. We've also kept an eye on long-term development, so that we can use these tools for studying and improving other parts of our user interface. Much improvement has been made on OWA/MediaWiki integration, as well as scaling OWA to work in our environment.
 * Program managers:  Rob Lanphier &  Tomasz Finc

Test framework deployment -
 * Program manager:  Rob Lanphier

 Process improvement - Increase transparency and generally organize Wikimedia Foundation's engineering efforts more efficiently
 * Status:
 * Program manager: Rob Lanphier

Code review - improving the way we provide code reviews for MediaWiki
 * Status:
 * Program manager: Rob Lanphier

Technical Documentation – Improve our technical documentation by making small, incremental improvements to the docs and docs process.
 * Status:
 * Program Managers: Rob Lanphier /  Zak Greant

Fundraising
2010 Fundraiser - The engineering tasks necessary to run a successful fundraiser, with sub-projects involving fraud prevention, CentralNotice, and the analytics upgrade.
 * Status: This month has been full of content creation to allow us to optimize our donation workflow. We've seen gains as high as five percent in between tests. Our current plan is to launch November 8, though this date is subject to change as we make our final plans.


 * Program manager: Tomasz Finc

 Credit card server upgrade - Upgrading our current payments infrastructure to support 1-click donations.
 * Status: We've been successful in adding caching and optimizing mediawiki. In our second stage we added a squid caching layer which has given us an 8x throughput increase.
 * Program manager: Tomasz Finc

Mobile
WMF Projects/Porting Mobile Gateway - Porting our existing gateway for easier support, development, and participations
 * Stats: Our current mobile gateway is very popular and a very valuable resource for many (including much of the staff). However, it has proven difficult to support operationally.  We're currently contemplating a redesign of this feature (see the Mobile site rewrite page).  We're planning to hire an engineer to start work on the rewrite, and hope to start in earnest in the beginning of 2011.
 * Program manager: Tomasz Finc


 * On going maintenance

Offline
Offline
 * Status: We are starting to ramp back up on this activity as the development team starts to become available again (having wrapped up most of the devleopment work on the Fundraiser). Ariel Glenn is working on parallelizing the generation of offline XML dumps.  Additionally, we're exploring adding additional export formats to the collection extension. We're also exploring new clients for offline reading of Wikipedia.
 * Program manager: Tomasz Finc

Hiring
We have a lot of hiring coming up before the end of the year. Job descriptions are already posted for the following:
 * Director of Technical Operations
 * Bugmeister
 * User Interface Designer

In addition, we hope to post the following positions soon: (Note: all of these positions may change as Foundation requirements evolve)
 * Software Developer (Mobile)
 * Software Developers (Features)
 * Data Analyst
 * Senior QA Engineer
 * Release Engineer
 * Storage Engineer (contractor)

Misc
 Hack-A-Ton - held October 22–24, 2010 just outside Washington, D.C. in the United States. This was our first developer event on the east coast of the United States. We had 27 attendees total (16 Foundation staff and 11 non-staff). The engineers in attendance were able to fix 44 bits of code marked "fixme" (roughly one-third of the total) We had lots of great conversations about release engineering and code review, and plenty of discussion and coordination of future feature work. There were many short presentations and discussions about WikiBhasha, the Selenium framework, the PHPUnit framework, and sentence level editing among others. See the summary blog posts (first day and second day) for more information.