RESTBase



RESTBase is a caching / storing API proxy. Its configuration is based on Swagger specs, and its primary storage backend is using Cassandra. It powers rest.wikimedia.org, the Wikimedia REST content API just entering beta production.

As a proxy, RESTBase does not perform any significant content processing itself. Instead, it requests content transformations from backend services when needed, and typically (depending on configuration) stores it back for later retrieval. For high-volume static end points most requests will be satisfied directly from storage.

Its storage backends expose a RESTful table storage API similar to Amazon DynamoDB and Google DataStore. The primary implementation uses Apache Cassandra. Notable features include automatically maintained secondary indexes and some lightweight transaction support. A SQLite backend is under development.

RESTBase automatically emits statsd metrics about all storage and backend requests. This provides a good baseline level of performance and error instrumentation in a micro-service architecture.

Use cases
Our first use case is speeding up VisualEditor by reducing HTML size, and eliminating Varnish cache misses. In the current iteration, we store parsoid metadata separately, reducing HTML size by about 40%. This reduces network transfer and processing latency significantly. In the longer term, we are aiming to bring down the size of the HTML to that of current PHP parser output to make it suitable for regular page views. This has the potential to make switching to visual editing instantaneous and free of any scrolling.

Another use case we are strongly interested in is providing a section-level edit API for micro-contributions and extremely fast VisualEditor saves, even faster than wikitext.

Documentation

 * Overview
 * Architecture
 * Browse the documentation
 * Browse the API endpoints