Flow/Architecture/Memcache

Flow is written to take full advantage of a memcache caching layer. To this end we have implemented something very roughly equivalent to queryable indexes in memcache. We have considered moving this to redis, but for now its using the memcache infrastructure.

What is being cached?
All of the flow specific data models. The reverse-proxy cluster serving up most of the wiki content to visitors does not apply to editors. To provide the editors the responsiveness they deserve we are aggressively caching within the application. We have decided to cache at the data model level rather than options like caching view fragments to simplify cache invalidation and variances between what different editors see based on their roles.

What is actually written in the cache?
The Flow "Indexes" are always nested arrays of arrays. e.g. array( array( /* first data model */ ), array( /* second data model */ ), ... ); Each index is defined with a memcache key prefix (e.g. 'flow_definition:pk' ). This is combined with an equality condition like array( 'definition_id' => 12345 ) to generate a cache key such as 'flow_definition:pk:12345'. When the index is defined it is told what keys within the data model are used for indexing such that it can always build cache keys with properties in the same order.

Currently we have two main index implementations, both of which build off a common parent(FeatureIndex). We have the UniqueFeatureIndex which is mostly used for the primary index of each data model and holds exactly 1 data model matching a unique set of features. We additionally have a TopKIndex which stores the top K items matching a defined group of features(properties) sorted by a pre-defined feature of the data model. In addition to these two there are a few indexes that extend TopKIndex and add additional related data to the model to allow indexing models based on data not explicitly a part of the data model.

For every data model there is exactly one primary index. This primary index generally uses the same keys as the primary key in MySQL. The primary index should be (but is not specifically restricted) the only index that contains the full data models. The primary index still stores its index as a nested array of arrays for consistency, but there is always exactly one data model stored in this index.

Most data models then use supplementary secondary(i.e. not primary) indexes. All secondary indexes limit their data model storage to the piece of the data model necessary for looking up the full data model in the primary index, along with whatever values are necessary for sorting the index. These secondary indexes are always pre-sorted and limited in length. From a developers perspective querying a data model's primary or secondary index is the same, both always return the full data model.

Will this affect other users of memcache, like the parser cache?
Yes. Depending on how long(or if) timeouts are set on the keys, and how much data is cached this could affect the parser cache and cause more requests to fall back to querying the DB layer parser cache. We are not sure how to measure this, or determine what level of impact it will have.

How long is it cached?
We are undecided on cache time. Currently we are not setting any kind of timeout allowing Memcache to perform LRU eviction. We are not tied to this setting, valid arguments are being accepted for what this should be. Initial review suggests a few possibilities:
 * Use a different memcache server(possibly on same machines) for flow, such that it uses LRU eviction but has a pre-defined and expected limit to its memory usage, preventing unexpected effects on the parser cache.
 * Set a timeout on the keys such that they get evicted over time.
 * Continue using LRU and pray™
 * Add other options - this means you

Memcache usage estimates?
We have not yet estimated how much memory flow will use within memcache. For reference the current production memcache cluster is 16 machines with 96G memory each, for a total of 1.5T. This (i think) is primarily, in respect to % of memory, used for the parser cache.

Reads
All reads in flow hit the memcache infrastructure first. Queries to storage are almost entirely done in terms of equality conditions(along with options that specify details such as the prefered sorting order) so we can use those equality conditions as a cache key. Reads within flow always prefer to multi-get content from memcache to reduce the number of round trips required. This should work well combined with twemproxy which receives the full multi-get and parallelizes those requests to the appropriate memcache servers.

We prefer there to be a single source of truth within the memcache cluster. This is accomplished by using a single primary index with the full data model along with multiple secondary indexes which point to the primary index. This is a convention and not enforced by the implementation. We can see the possibility of future use cases having different requirements.

Writes
For the most part wiki's dont delete anything. Even things that are deleted are only really hidden from view except in specific very limited cases. This helps to limit(but not remove) our exposure to race conditions.

Updating data in memcache is done after the transaction to MySQL has been committed. To update an existing index we utilize a CAS operation to:
 * 1) Pull the current index value from memcache
 * 2) Add the new value to the index
 * 3) Sort the index
 * 4) Limit the index to the predefined size
 * 5) Write it back to memcache