Wikimedia Labs/Tool Labs/Roadmap en

= Roadmap for the setup of Tool Labs and the migration from Toolserver (Draft) =

The German version is available here: Tool_Labs/Roadmap_de

Expectations of an infrastructure for volunteer software development
The Toolserver is an infrastructure that hosts software supporting Wikipedia and its sister projects. Our expectations of such an infrastructure are the following:
 * Any volunteer can request an account to develop tools to support, improve or analyze Wikimedia projects.
 * It makes sharing of code and knowledge as easy as possible.
 * The infrastructure is well maintained and available. Technical failures should be an exception.
 * The tools developed in such an infrastructure are running in an environment that is similar to the productive Wikimedia projects. The data of all Wikimedia projects and OpenStreetMap are easily queriable. By data, we mean the subset that is visible to normal, unprivileged registered users.
 * It offers user databases and shared storage for projects.
 * Tool maintainers can take part in the definition of features and software needed in such an infrastructures.
 * Ideally, it offers a staging process for software that can be used in production.
 * There are admins available, ideally in different timezones, that respond to the tool maintainers’ requests in a reasonable time. They should deal with maintainers with different amounts of technical knowledge and be open and welcoming to newcomers.

Crucial differences between the Toolserver and Labs/Tool Labs

 * Technical concept: The Toolserver is mainly used by individual developers. Though there is a concept of multi-maintainer projects, this is not considered to be the default. The ideas of sharing software and of offering a staging process are at the core of Labs’ concept though.
 * Responsibility: The Toolserver has been maintained by very few people for years. They were only partly paid and essential parts of the work have been done by a few volunteers. This has not only led to an immense work load for them but it has entailed a lack of documentation, too. This makes it more difficult to bring new sysadmins in to better share responsibilities. Labs/Tool Labs have been designed and built by paid WMF staff from the beginning. Volunteer admins are welcome but the responsibility for the project as a whole is assumed by an institution that has got more hands/heads to share the knowledge and more time for documentation. A single person leaving would thus not endanger the whole project.
 * Labs' dedication to OSI-approved licenses/free and open source software fits well into Wikimedia’s and Wikipedia’s concept of freeing knowledge. The Toolserver also hosts closed source software on both the level of infrastructure and tools.

General announcements of Wikimedia Germany
WMDE clarifies the following things:
 * The Toolserver is a community project. WMDE is in a coordinating role to support it, but the migration itself has to be done by the active community.
 * No tool maintainer willing to migrate his or her software shall be left alone. WMDE will try to assist them as much as possible. All tool maintainers who want support to migrate their tools are asked to tell us what exactly they need until the end of September 2013. (“Us” being either the Toolserver contact persons at WMDE or the WMF ops working on Tool Labs.) After this deadline, WMDE will not be able to take calls for support anymore.
 * The above mentioned commitment to free/open source software means though that no tool maintainer can claim the 1:1 transfer of his or her tools to Labs. It is possible that some tools will have to be ported to open source alternatives by the tool maintainers. WMDE does not accept responsibility for the preservation of all tools and WMDE will not be able to take care of proprietary software running on the Toolserver. According to WMF Legal, it is possible to negotiate exceptions: "If there are particularly important tools impacted by this rule, then on a case-by-case basis we may be willing to take a license to specific software. For example, if a university owns specific software, and are willing to license it to the Wikimedia Foundation and our volunteers (not just one specific volunteer), then we could discuss that. But the licenses would be very broad, and we would not be willing to negotiate them at great length or detail."
 * WMDE has run the Toolserver, WMDE is coordinating the transition process. We are not responsible for Tool Labs though - that is a WMF project.

Migration from toolserver - what WMDE offers to tool maintainers

 * Technical support for migration: Our software developer Johannes Kroll can be asked for help if tools need adaptations to run in Tool Labs. He will accept support requests until September 30th, 2013, he will help to estimate the amount of programming work needed. If desired, he can help to modify tools. A part of his working time is dedicated to this between October and December 2013. This is why he will prioritize, depending on the number of requests.
 * documentation of the migration process from Toolserver to Tool Labs

Status of Tool Labs (as of May 10th, 2013)
Tool Labs consists of two projects inside Wikimedia Labs: "Tools" and "Bots".

"Tools" is a stable production environment for tools. This project is built and maintained by three administrators, two of them being WMF staff and one a volunteer. More volunteers can be added, if they are ready to sign an NDA with WMF (like on the Toolserver). Root access is quite restricted here to keep the environment itself stable.

"Bots" is a more experimental environment for tools. This project is community-maintained and more flexible in its architecture so that tool maintainers can experiment with changes in the environment.

Schedule
This road map clarifies
 * when Tool Labs is technically ready to replace the Toolserver,
 * which steps have to be undertaken to migrate.

Links

 * Overview of features needed in Labs/Tool Labs (community): Wikimedia_Labs/Toolserver_features_needed_in_Tool_Labs
 * List of Tools (Silke and community, incomplete): Toolserver/List_of_Tools
 * Migration Docs/FAQ (Silke, to be continued): Wikimedia_Labs/Migration_Of_Toolserver_Tools
 * WMF Engineering goals: Wikimedia_Engineering/2012-13_Goals
 * Tool Labs Database Plan: ToolLabsDatabasePlan
 * List of tools hosted in Tool Labs: http://tools.wmflabs.org/