Thread:Talk:Requests for comment/Caching Overhaul - Frontend Proxy/IRC meeting 2013-10-02

&lt;gwicke>	mwalker: re load, did you do further benchmarking on real hardware? &lt;mwalker>	gwicke: no; not yet -- more on that later &lt;gwicke>	k &lt;TimStarling>	I am reading the CentralNotice one, I haven't seen it before &lt;TimStarling>	btw RFCs should generally be subpages of https://www.mediawiki.org/wiki/Requests_for_comment/ &lt;TimStarling>	otherwise my scripts will get confused &lt;mwalker>	yep yep - I started it as a brainstorming page on the extension &lt;mwalker>	I can add a redirect to it -- would that make things work better? &lt;Elsie>	Instead of moving it? &lt;TimStarling>	and also having RFCs there makes sure that the page is clear about its purpose -- i.e. definitely an RFC rather than some other kind of design page &lt;mwalker>	Elsie: I'm not opposed to moving it -- but I dont have those rights &lt;legoktm>	mwalker: anyone can move a page &lt;Elsie>	It's a wiki, man. &lt;legoktm>	(if not, someone will just give you sysop rights) &lt;mwalker>	oh hey -- moving is allows -- don't know why I thought it wasnt &lt;TimStarling>	so there will be a script tag in the header that provides a JSON blob &lt;TimStarling>	how does the data from the JSON blob get into the actual page content? &lt;mwalker>	the rest of the CentralNotice JS will be delivered via resourceloader &lt;TimStarling>	but that part will be semi-static JS? &lt;mwalker>	but having the banner content already available reduces the amount of time it takes to display -- and will also get rid of a round trip (to get the geoiplookup) &lt;Elsie>	Where does the 200GB figure come from? &lt;mwalker>	"but that part will be semi-static JS?" -- yes; the bit in the head will be as small, simple, and static as I can make it &lt;mwalker>	"Where does the 200GB figure come from?" CN has a potential space of all projects, languages, countries, user states, buckets, and slots -- which comes to a large number which is then multiplied by the average size of a fundraising banner and varnish overhead &lt;mwalker>	*trying to find my worksheet on that now &lt;TimStarling>	presumably some of those dimensions would have to be fairly large &lt;TimStarling>	is that what you mean by "worst case", that they are all as large as possible? &lt;mwalker>	yes &lt;TimStarling>	e.g. buckets and slots, there are not always a lot of those, right? &lt;mark>	i wonder how many we've actually got cached right now &lt;mwalker>	so right now the space is 14 projects * ~300 languages * ~200 countries * 3 device types * 30 slots * 4 buckets * 2 uesr states &lt;mark>	not entirely trivial to figure out though &lt;mwalker>	mark: most of them :( &lt;mwalker>	the timeout is 15minutes &lt;mwalker>	and for everything but wikipedia they're empty &lt;TimStarling>	ok &lt;gwicke>	mwalker: I believe that your performance estimation for node might be about right, but it would definitely be good to establish a baseline on a real machine &lt;gwicke>	I'm getting about 7k req/s on my laptop with a trivial http server &lt;TimStarling>	so what is the reason for using a separate domain name? &lt;TimStarling>	connection setup is expensive &lt;mark>	so, we don't need to do that &lt;mark>	in the discussion page I argue it shouldn't be a separate (node.js) server, but should probably just use varnish and be a backend to or a plugin of that &lt;mark>	and then we also have the option to do this on one of the existing host names/clusters &lt;TimStarling>	it seems pretty similar to the mobile varnish stuff &lt;TimStarling>	the problem is that it is dynamic in various annoying ways, right? &lt;TimStarling>	and we want to resolve that to some smaller number of cacheable objects &lt;mwalker>	TimStarling: it was a naive suggestion thinking that I could separate the infrastructure entirely from the rest of the site so that if it goes down nothing else suffers &lt;mwalker>	but I agree with mark that bits varnish could 'pass' a request to bits.wm.o/banners or something to a backend &lt;MaxSem>	separate LB and varnish boxes? &lt;mwalker>	TimStarling: and yes; your summary of the problem is correct &lt;TimStarling>	have you considered redirecting? &lt;mwalker>	it costs a round trip; and additionally still has to be cached &lt;mwalker>	the round trip is important because something a lot of users complain about is the 'page bump' that happens when a banner loads &lt;TimStarling>	I mean, have the dynamic part redirect to the static part &lt;mark>	ESI is one of the options on the table for that &lt;TimStarling>	presumably the way to avoid a page bump is to load early &lt;TimStarling>	right? &lt;mwalker>	ya; that's the thought process &lt;TimStarling>	then you could use document.write &lt;TimStarling>	like advertising, advertising usually uses document.write, doesn't it? &lt;mwalker>	potentially yes -- banners are fairly dynamic though so it's not the best solution &lt;TimStarling>	what is the difference between this and what you have proposed? &lt;mwalker>	not much; in my original proposal I had the proxy being the machine LVS directed to; in marks suggestion LVS directs to bits.wm.o which will vcl_pass to a backend &lt;mark>	bits or anything else &lt;mark>	can be a separate cluster, but doesn't have to be &lt;TimStarling>	you are saying that JS in the head will fetch some dynamic data &lt;mwalker>	yes &lt;TimStarling>	then a script on the client side will use this dynamic data to form a URL to request some static data &lt;TimStarling>	then that static data will be used by another script to actually display the HTML &lt;TimStarling>	is that fair? &lt;mwalker>	no; everything dynamic is fetched in the first call; which is then used by a resourceloader script to actually display &lt;TimStarling>	so the banner HTML is delivered in the first call? &lt;mwalker>	yes -- that's the plan &lt;TimStarling>	ok, I agree with mark that doing this in varnish would be better than doing it in node.js &lt;TimStarling>	I think a node.js frontend server would add quite a lot of complexity &lt;mwalker>	do you have thoughts on the backend technology? &lt;gwicke>	I see not reason why varnish can't do the front-end, with all requests being forwarded to a node backend &lt;mwalker>	if the backend should be written as a VMOD to eventually move into the frontend -- or if it should be a node server? &lt;gwicke>	should keep the backend simple &lt;mark>	I think I prefer to have it as a backend &lt;TimStarling>	I think we should continue this on the talk page &lt;mark>	there's not much reason to integrate it into varnish itself &lt;mark>	other than that it needs to see every request &lt;mwalker>	ok -- I will update the RfC and we can continue with other topics &lt;mark>	varnish can't cache anything there