Requests for comment/CentralNotice Caching Overhaul - Frontend Proxy

Overview
The current CentralNotice architecture is unsatisfactory for several reasons:
 * 1) It uses massively more cache objects than is desired (a worst case of about 200GB of on disk usage)
 * 2) We do not load the banner until the ResourceLoader controller has loaded; which results in a page bump

The proposed solution to these issues is to have a static javascript blob in every content namespace page that will call up to a special CentralNotice proxy. This proxy will map the request (which may have a space size greater than 400k) into approximately 200 different mincut allocations in a lookup table.

The lookup table will have specific details about banners in that allocation which will allow, with the usage of a random number generator, a direct request by banner name to the backend varnish cache. This allows the backend varnish to much more elegantly handle objects as it only needs to cache a banner by three varying parameters .



The JSONP Call in HEAD
A small snippet of Javascript in the head of a page will call a new subdomain entitled banners.wikimedia.org (or similar). The request parameters will be a combination of static variables known at page generation time on a wiki and dynamic variables that will come from a cookie or local storage. Dynamic variables are required where the user state will have changed from the defaults on the wiki; aka they logged in, or they changed their UI language. It is also required so that we may bucket users for A/B testing. These variables are:
 * Static project name (e.g. wikipedia)
 * Static project content language
 * Static user login status (may be overriden by cookie)


 * Dynamic UI language
 * Dynamic bucket (1 thru 4 currently)
 * Dynamic user login status

Additional Variables added by Proxy
CentralNotice needs additional variables in order to know what banner is suitable to serve to a user. These are unique per client and should initially be calculated on the proxy and then passed down for usage in a dynamic cookie. These are:
 * The user's device (e.g. desktop, iphone, android, ...)
 * The user's location as determined by IP address

Proxy Mincut Mapping
CentralNotice will routinely produce a mincut map of banner allocations and distribute them to the proxy servers. In the current plan this will be a series of three sets of tables, several offset lookup tables, a map table, and several map line->banner entry tables. Presuming that the proxy server is a node.js server string lookups will probably be the most efficient map format and the lookup would look something like:


 * 1) For each variable, determine the map string offset
 * 2) In each map line, check if offset is set, if so keep map line in future lookups
 * 3) Repeat until you have gone through all variables or map lines; there should be either 1 or no viable map lines

Simplified example:

Random Banner Choice
Using a random number generator in the proxy we select a random banner and then request that from the backend proxy. We optionally composite the request (e.g. feeding back the detected country/device) with the returned banner which is set in a JavaScript variable ready for consumption by CentralNotice.