RESTBase/deprecation

From mediawiki.org


This document summarises the rationale and plan for deprecating RESTbase.

Decision[edit]

The RESTbase service should be deprecated and removed from WMF sites. Services that use RESTbase should be migrated to a different platform.

Rationale[edit]

This was already decided in 2019, see Core Platform Team/Initiatives/Sunsetting RESTBase. So quote: "Remove the need to support a complex piece of service architecture replacing it with a well supported third party application reducing cost of ownership".

See also the notes from the Technical Conference 2019.

In summary:

  • The rationales for the design of RESTbase are obsolete. Needs fulfilled by RESTbase are better met by other means. Examples:
    • Load balancing and life cycle management should by done with Kubernetes.
    • API routing and instrumentation should be done with Envoy. See also the design document for rate limiting in the API gateway.
    • Response caching should be done with Varnish.
    • Parsing wikitext should be done by Parsoid in MediaWiki core.
    • Persistent storage should be done by each service individually, supported by a generalized data storage platform.
    • Event handling should be done by individual services or using a stream processing framework.
  • RESTbase has major architectural problems. Most importantly, it serves as a central hub that may be hit multiple times recursively while handling a single request. This creates a cyclic dynamic dependency which incurs latency, makes errors hard to analyze, and causes issues with one service to impact other services.
  • Several of the ideas that drove and informed the development of RESTbase, never materialized. Examples:
    • include storing HTML for all versions of a page
    • ...
  • The RESTbase code base is unmaintained. All engineers with a deep knowledge of the code base have left WMF. This makes it a critical risk factor to operational reliability.
  • RESTbase is in need of an update. It depends on old versions of npm packages. Updating it would be a significant effort, especially since all engineers with a deep knowledge of the code base have left WMF.
  • The RESTbase code is complex. The effort level that would be required for a team of engineers to develop a level of understanding sufficient to update and maintain RESTbase is at least comparable to the effort level required to retire it in favor of alternatives that are technically preferable anyway.
    • The storage abstraction layer seems to add more complexity rather than removing it.

Migration Plan[edit]

A checklist for individual services is available under RESTBase/service migration.

A migration plan for each service has been layed out in the Services Architecture Recommendations (2019).

An overview can be found in the Services Re-architecture document.