Reading/Web/Performance/Removal of srcsets

From mediawiki.org

As of 17th February 2016 (00:54) the mobile web site no longer serves srcset attributes.

Impact on single page[edit]

Impact on bytes[edit]

For the Barack Obama article this halved the amount of bytes of images we shipped to users on initial load from 1.12mb to 450kb.

Bytes shipped to users for the Barack Obama article halved.

Impact on render and fully loaded time[edit]

Fully loaded time was somewhat impacted by the change on a 2G connection

Using webpagetest we can see the impact on production for the loading of Barack Obama simulated under 2G connections.

There seems to be a minor improvement, but despite the big difference in bytes there are still the same amount of HTTP requests competing against one another. It is expected that lazy loading images will have a much greater impact on these graphs.

Generally this looks positive, but not massively impactful. Measuring the average of all median fully loaded times before and after the change, using a script [note 1], it seems to have improved fully loaded time by 5%.

Property Before (avg) after (avg) Delta (Avg) % decrease (Avg) Before (median) After (median) Delta (median) % decrease (median)
html.bytes 189120.9 189555.9 -435.0 -0.23% 189211.0 190000.0 -789.0 -0.42%
TTFB.median 4881.6 4905.8 -24.2 -0.50% 5252.0 5254.0 -2.0 -0.04%
render.median 9768.6 11283.0 -1514.3 -15.50% 11705.0 11789.0 -84.0 -0.72%
fullyLoaded.median 24989.8 23797.0 1192.8 4.77% 24599.0 23478.0 1121.0 4.56%

Global impact[edit]

This change to be accurately measured, due to internal caching would take 30 days to completely roll out across the entire site. It should be noted that other changes post this change could have improved or reduced performance. Since a change was made on the 16th March we consider data collected up to this date and post the roll out of the change. Given the leap day in this period this is a 27 day period.

Impact on bytes[edit]

Given the text served to users barely changed in this period, I think we can attribute the 0.5GB/s drop in images served to users to the removal of the srcset attribute.

Before the change we were serving roughly 1.6GB of images to users per second, 30 days afterwards we were serving 0.9GB of images per second. Given the text served didn't see a drop I believe we can probably attribute this to the srcset removal.

Impact on fully loaded time[edit]

That said looking at `frontend.navtiming.totalPageLoadTime.mobile.anonymous` for global traffic the fully loaded time median time also appears to have improved by around 5% over a 27 day period [note 2]

Property Before (avg) after (avg) Delta (Avg) % decrease (Avg) Before (median) After (median) Delta (median) % decrease (median)
median 2131.7 2005.2 126.6 5.94% 2138.0 2019.1 118.9 5.56%
p95 14474.7 13639.8 834.9 5.77% 14472.3 13649.5 822.8 5.69%
upper 57167.6 56756.7 410.8 0.72% 57966.0 57695.5 270.5 0.47%

That said, the fully loaded times we are currently collecting are sub 5s so do not reflect 2G connections. It is suspected that on 2G, page loads take so long that these users are currently not represented in the data we collect.

Impact on page views[edit]

According to the page view API, there was no significant and obvious spikes in page view traffic relating to this change a week after it had occurred.

Notes[edit]

  1. using command node wptreporter.js "webpagetest.enwiki-mobile-2gslow.anonymous.Barack_Obama.us-east-1.Google_Chrome-emulateMobile.firstView" 17 2 2016 1 0 "" 27 wikitext
  2. Using command node wptreporter.js "frontend.navtiming.totalPageLoadTime.mobile.anonymous" 17 2 2016 21 0 "median" 27 yes