Core Platform Team/Initiative/Remove storage from RESTBase/Initiative Description

Project Lead
Makro Obrovac

Current state
In planning

Expected start
April 2019

Summary
RESTBase acts as the main entry point for our REST API hierarchy. In addition to acting as an API proxy, it also incorporates a storage engine (backed by Cassandra) that either caches or permanently stores responses generated by back-end services. Such a set-up makes it hard to reason about and creates a direct dependency on RESTBase for the back-end services providing the content.

Significance and motivation
The goal of the project is to reduce the complexity of the REST API sub-system and give control of operating the storage to individual services.

Milestones and major tasks

 * Simplify Parsoid Storage
 * Simplify MCS Storage
 * Rework Storage Modules to separate them from routing
 * Separate services (1 new service, 1 old)
 * Deploy new service
 * Connect services
 * Create client libraries for accessing the storage
 * Instrument mathoid, parsoid and MCS to access storage directly
 * Document the changes made, especially for configs

Outcome
Reduce the complexity of the platform

Baseline

 * RESTBase code simplified and API proxying and storage engine separated

Target

 * Both services deployed to production (one on Cassandra nodes, the other in Kubernetes)
 * Individual services use the storage engine service directly

Methodology and rationale
The primary measure of the success of this project is that API proxying and storage engine service are deployed separately in production.

Time and resource estimate
2.5 FTE for 3 months

1.5 FTE for the next 3-6 months

Dependencies
SRE (for k8s deployment)

Collaborators
Core Platform

Individual service owners: Parsing team, Reading Infrastructure

Stakeholders
Core Platform

SRE

Open questions
TBD

Phabricator
TBD

Relevant materials, plans and RFCs
TBD