Wikimedia Services

One of outcomes of the architecture summit in January 2014 was a consensus for moving towards a service-oriented architecture. Major reasons for this are:


 * enabling new engineers to be more productive by letting them concentrate on a part of the stack
 * improved reliability and security through fault and security context isolation
 * ease of testing, debugging, monitoring and scaling
 * supporting new uses of our data by rich clients and mobile through a well-structured and efficient public content API

A storage service and a REST content API are particularly good candidates for first steps in this direction. There has been some preliminary work on the Rashomon revision storage service by the Parsoid team and a generic PHP service interface with support for parallel operations by Aaron.

This has happened as side projects, with other team responsibilities taking priority. We need to do more if we really want to reap the benefits of a service-oriented architecture.

We are taking on this challenge with the new services team, which aims to


 * guide existing work to think in terms of independent services and versioned APIs,
 * help to develop internal and external REST APIs (ex: Flow),
 * help to set up automated tests against those APIs,
 * identify and implement general backend services in support of other teams (ex: storage service), and
 * support packaging for internal and third party distribution.

In the short term, this can help other teams by removing some of their workload, avoiding duplicate work and helping with API implementations.

In the longer term, it will
 * build up a consistent REST content API to enable mobile and innovative feature projects,
 * provide back-end services like the Rashomon storage service, and
 * improve testing and thus the ability to deploy more often without breaking things.

Ultimately, it should help us make real progress towards a service-oriented architecture with strong APIs. It will hopefully also result in improved developer productivity, testability, security and agility as discussed in the SOA RFC.

Division of labor between PHP and REST API
The PHP and REST APIs have a different focus and will be largely complementary. The PHP API offers powerful features including generators, but is not designed to support serving content at high request rates. Per-request overheads are high, and caching the output is not generally possible. Mostly due to its performance characteristics it is not commonly consumed by MediaWiki itself.

The REST API on the other hand focuses on simple but high-volume interfaces with a small per-request overhead. It further supports caching, which makes it suitable for directly serving content at a larger scale.

Developers will need clear guidance on which interface they should use for specific tasks. The REST interface will start out fairly small and won't overlap with the PHP API. Over time we can consider gradually migrating simple and high-volume end points (like opensearch) to the REST interface. This will happen in close cooperation with the PHP interface team.

Also see these meeting notes from the API roadmap meeting last September.

Contact the team
Contact the team:
 * services@lists.wikimedia.org mailing list
 * #wikimedia-services IRC channel