Wikimedia Labs

Wikimedia Labs is a two-part project aimed at helping volunteers get involved in Wikimedia operations and software development. The first part of this project is Test/Dev Labs, and the second part is Tool Labs. (For an explanation of various different usages of the term "labs", see Labs labs labs.)

Background
Wikipedia's infrastructure and software was built and originally completely administered by community volunteers. As the site has grown over time the operations infrastructure and software development has been dominated by Wikimedia Foundation staff. While the Foundation's increased involvement has had a positive effect, we would like to reverse the trend of less volunteerism and more staff. Wikimedia Labs was started in 2011/12 to help our community to contribute code, by creating a virtualized environment that anyone can use to make changes to infrastructure or software, demo the changes, have the code reviewed, and ultimately pushed to production.

It can also be used for small projects that are Wikimedia-related but distinct from the software used in production. For example, a project might present information from the MediaWiki API of a WMF project in a new way.

Access
You can create a Labs account by creating an account on wikitech.

Having a Labs account gives you access to Labs, Gerrit (our code review system) and a few other developer-related tools.

Implementation
The architecture is described on Wikitech. The software for controlling this environment is implemented as a MediaWiki extension, and is described on mediawiki.org.

Roadmap
For the roadmap, please see Wikimedia Labs's goals for 2014-15 (past roadmaps: 2012-13, 2013-14; see also /Tool Labs/Roadmap en which describes the migration from Toolserver to Tool Labs in 2013/14).

Open tasks
We'd love help with all of the below!


 * Development and sysadmin tasks
 * Enable IPv6
 * Integrate RT with Labs LDAP (is this still relevant?)
 * Create a second zone in eqiad (the zone exists but not yet in operation)
 * Push OpenStackManager changes to show SSH fingerprints for instances
 * Add DNS support to Nova
 * We added support for this in essex, but there's now an OpenStack project called Moniker we'll use instead. Waiting for it to enter incubation.

...
 * Product tasks

Proposals

 * Development Process
 * Puppet learning mode
 * Toolserver features needed / wanted in Tool Labs (contains even some features which are not on toolserver)
 * Investigate Pentaho
 * Find/test OTRS replacement, or upgrade/puppetize OTRS (with security patches)
 * Puppetize PDF server
 * Package mwlib dependencies
 * Create an init script (preferably upstart) for mwlib daemon
 * Mentions of init script here (wikitech) but last I heard not in use. Peachey88 05:38, 21 December 2011 (UTC)
 * Add puppet syntax highlighting to vim
 * Create shared sql service for all projects
 * Package and puppetize lilurl for use as a url shortening service
 * Write Documentation for console
 * Fix puppet repo so that it runs a complete first run of the puppet catalog on instance creation for all services
 * Java App stack
 * Package JDK 1.6 (confused about this, can we not use openjdk or sun's packaged jdk?)
 * Apache Ant
 * Maven
 * Jetty App Servers
 * Apache Solr
 * Hadoop
 * Sbt
 * Per-project saltstack remote execution
 * Integrate a global queuing system (to replace tools' use of SGE when they migrate from the toolserver)
 * Authentication improvement project
 * Instance creation improvement project
 * Interface usability improvement project
 * Stability improvement project
 * Communication improvement project

Completed

 * Shared home directories per project
 * Package adminbot, with an init script
 * Puppet repository branch per project or instance
 * Nagios management without exported puppet resources
 * Package gerrit
 * Create a log bot for the #wikimedia-labs channel
 * Reverse proxy for web services

Documents

 * Status updates
 * /Terms of use/
 * /Agreement to disclosure of personally identifiable information/
 * /Account creation text/
 * /Things to fix in beta/

Communications

 * IRC channel on Freenode
 * labs-l mailing list
 * inventory of Labs's total capacity as of late September 2012 (compute, database and storage nodes)
 * Quarterly review, April 2015