Wikimedia Labs provides cloud computing infrastructure. The virtual machines are powered by the OpenStack software, administered through special pages on the Wikitech wiki. The environment includes a live replica of databases of all public Wikimedia Foundation wikis.
The project is aimed at helping volunteers get involved in Wikimedia operations and software development.
Initial focus of this project was on two parts: Tool Labs, and Beta Cluster. For more about these, see Wikitech.
Background[edit | edit source]
The Wikipedia website infrastructure and software was built and originally completely operated by community volunteers. As the site has grown, over time the infrastructure and software development became more dominated by Wikimedia Foundation staff. While the Foundation's increased involvement has had a positive effect, we would like to reverse this trend and encourage more volunteerism.
When Wikimedia Labs started in 2011, initial focus was on two key projects:
- Beta Cluster, a small wiki farm running the latest alpha version of the MediaWiki software.
- Tool Labs, a platform for volunteers to easily run web services, bots and other applications.
The Beta Cluster runs in a virtualized environment that closely resembles Wikimedia's production cluster. It can be used by anyone to contribute changes to backend infrastructure or software, demo the changes, have the code reviewed, and ultimately pushed to production.
In Tool Labs, users have the comfort of not needing to become system administrators of any (virtual) servers or hardware, instead the platform is entirely provided for. It can be used for small projects that are Wikimedia-related but distinct from the software used in production. For example, a project might present information from the MediaWiki API of a Wikimedia wiki in a new way.
Open access[edit | edit source]
Anyone can create a Labs account by creating an account on wikitech.
Having a Labs account grants you access to Wikitech, Wikimedia Labs cloud, Gerrit (our code review system), and a few other developer-related tools.
Implementation[edit | edit source]
Roadmap[edit | edit source]
Open tasks[edit | edit source]
We'd love help with all of the below!
Development and sysadmin tasks[edit | edit source]
- Enable IPv6
Proposals[edit | edit source]
- Development Process
- Puppet learning mode
- Toolserver features needed / wanted in Tool Labs (contains even some features which are not on toolserver)
- Investigate wikitech:Pentaho
- Find/test OTRS replacement, or upgrade/puppetize OTRS (with security patches)
- Puppetize PDF server
- Add puppet syntax highlighting to vim
- Create shared sql service for all projects
- Package and puppetize lilurl for use as a url shortening service
- Write Documentation for console
- Fix puppet repo so that it runs a complete first run of the puppet catalog on instance creation for all services
- Java App stack
- Package JDK 1.6 (confused about this, can we not use openjdk or sun's packaged jdk?)
- Apache Ant
- Jetty App Servers
- Apache Solr
- Per-project saltstack remote execution
- Integrate a global queuing system (to replace tools' use of SGE when they migrate from the toolserver)
- Authentication improvement project
- Instance creation improvement project
- Interface usability improvement project
- Stability improvement project
- Communication improvement project
Completed[edit | edit source]
- Shared home directories per project
- Package adminbot, with an init script
- Puppet repository branch per project or instance
- Nagios management without exported puppet resources
- Package gerrit
- Create a log bot for the #wikimedia-labs channel
- Reverse proxy for web services
Documents[edit | edit source]
- Status updates
- Agreement to disclosure of personally identifiable information
- Account creation text
- Things to fix in beta
Communications[edit | edit source]
- IRC channel on Freenode
- labs-l mailing list
- inventory of Labs's total capacity as of late September 2012 (compute, database and storage nodes)
- Quarterly review, April 2015