Core Platform Team/Initiatives/Remove storage from RESTBase


Initiative Description

< Initiatives


RESTBase acts as the main entry point for our REST API hierarchy. In addition to acting as an API proxy, it also incorporates a storage engine (backed by Cassandra) that either caches or permanently stores responses generated by back-end services. Such a set-up makes it hard to reason about and creates a direct dependency on RESTBase for the back-end services providing the content.

Significance and Motivation

The goal of the project is to reduce the complexity of the REST API sub-system and give control of operating the storage to individual services.


Reduce the complexity of the platform

The primary measure of the success of this project is that API proxying and storage engine service are deployed separately in production.

Baseline Metrics
  • RESTBase code simplified and API proxying and storage engine separated
Target Metrics
  • Both services deployed to production (one on Cassandra nodes, the other in Kubernetes)
  • Individual services use the storage engine service directly
  • Core Platform
  • SRE
Known Dependencies/Blockers
  • SRE (for k8s deployment)

Epics, User Stories, and Requirements

< Initiatives

  • Simplify Parsoid Storage
  • Simplify MCS Storage
  • Rework Storage Modules to separate them from routing
  • Separate services (1 new service, 1 old)
  • Deploy new service
  • Connect services
  • Create client libraries for accessing the storage
  • Instrument mathoid, parsoid and MCS to access storage directly
  • Document the changes made, especially for configs

Time and Resource Estimates

< Initiatives

Estimated Start Date

April 2019

Actual Start Date

None given

Estimated Completion Date

None given

Actual Completion Date

None given

Resource Estimates
  • 2.5 FTE for 3 months
  • 1.5 FTE for the next 3-6 months
  • Core Platform
  • Individual service owners: Parsing team, Reading Infrastructure