RESTBase



RESTBase is a caching / storing API proxy. Its configuration is based on Swagger specs, and its primary storage backend is using Cassandra. It powers "rest_v1", the Wikimedia REST content API currently in production and used by VisualEditor to retrieve page HTML for editing. The server is rest.wikimedia.org. For performance (T95229) some service endpoints are also available on each wiki, e.g. [//api/rest_v1/page/html/Main_Page /api/rest_v1/page/html/Main_Page] on this wiki.

As a proxy, RESTBase does not perform any significant content processing itself. Instead, it requests content transformations from backend services when needed, and typically (depending on configuration) stores it back for later retrieval. For high-volume static end points most requests will be satisfied directly from storage.

Its storage backends expose a RESTful table storage API similar to Amazon DynamoDB and Google DataStore. The primary implementation uses Apache Cassandra. Notable features include automatically maintained secondary indexes and some lightweight transaction support. A SQLite backend is under development.

RESTBase automatically emits statsd metrics about all storage and backend requests. This provides a good baseline level of performance and error instrumentation in a micro-service architecture.

Use cases
Our first use case is speeding up VisualEditor by reducing HTML size, and eliminating Varnish cache misses. RESTBase stores Parsoid metadata separately from the HTML of the page, which reduces the size of the latter by about 40%. RESTBase provides only this HTML to VE, which reduces network transfer and processing latency significantly. In the longer term, we are aiming to bring down the size of the HTML to that of current PHP parser output to make it suitable for regular page views. This has the potential to make switching to visual editing instantaneous and free of any scrolling.

If parse time is not a pressing concern for your wiki (for example it does not have complex templates or large transclusion counts), then accessing Parsoid directly may make more sense than introducing a dependency on RESTBase.

Another use case we are strongly interested in is providing a section-level editing API for micro-contributions and extremely fast VisualEditor saves, even faster than wikitext.

Documentation

 * Browse the API endpoints
 * Overview
 * Architecture
 * Browse the documentation
 * Deployment process
 * List of current users