Flow/Architecture/HTML caching

Flow uses memcache to cache individual data models and query rollups from the database.

The team is considering caching HTML for entire topics, and possibly entire Flow boards, for anonymous and logged-in users. Users with special rights (sysops and oversighters) would always get a special view of the board (relatively few hits for these users). The performance benefit is fairly straight forward, instead of querying the database, serializing the data models into its api representation, and formatting that api representation into html we just get the html from memcache and hand it out.

There is also the possibility of allowing Edge Side Includes from varnish to incorporate the header and ten topics. We would have to do a good bit of testing and performance checks to determine if its reasonable. The idea here is that a board would not actually render the topics, but would instead output ESI tags for each of the topics. Varnish will then

A worry is that perhaps a varnish server goes from 15k requests/s down to 5k or 2k req/s because it does more processing handling the ESI tags...but on the other hand the php servers are probably beefier and do ~50 req/s.

Concerns: Is this scalable? What about other features we have in the roadmap? Would all of those be cacheable in the same way?

Things that currently vary on each request
Caching HTML doesn't work if the desired HTML varies with user or with time. Dynamic features would have to be dropped, some re-added in JavaScript.


 * The user's username/IP appears above textareas for any reply the user can potentially make.
 * approach: don't show these in non-JS version


 * "A minute ago" dates
 * PHP outputs human readable times with timestamps attach to a data attribute. Javascript sees these timestamps and updates them to match the delta from the browsers current time.


 * Can only Edit your own posts (depends on Flow permissions)
 * Could have an Edit button next to Reply everywhere, in no-JS it could either fail on submit or show wikitext of item. JavaScript would hide this on the user's own posts.


 * no Thank if anonymous
 * approach: have a class="no-anonymous" on Thank that is hidden if the user is known.


 * no Thanking yourself
 * approach: have the Thank link everywhere, non-JS would submit to Thanks form and fail if thanking yourself. JavaScript could hide Thank on thanks to yourself.


 * no blue background on the user's own name
 * JS enhancement