Reading/Web/Projects/Performance/Lazy loading images

Lazy loading images was enabled on the production beta channel on the 24th March. As an example, this led to the overall bytes served to users for images on the Barack Obama article dropping from 657kb to 169kb.

There was a small positive impact on the 95th percentile of beta anonymous and authenticated users in our global metrics.

Impact was less clear in our controlled tests on Barack Obama.

There are many problems with using beta to predict performance impact in production. It should not be used for this purpose:
 * Beta has a split cache compared to production so is generally not a reliable environment to predict performance changes - pages often load from an unpopulated cache.
 * In beta there is a banner experiment running which greatly increases bytes loaded by introducing an additional image at the top of the page. This does not run in production
 * The 95th percentile of beta users have a fully loaded time of 4s, in production it is closer to 11s. This suggests beta users are more likely to use a much faster connection than the types of connection we are targeting in production.

Predicting impact on stable
Static files emulating the lazy loaded images experience vs the current experience were uploaded to the cluster.

First view load was 38.541s for the current experience of the Barack Obama page on a 2G browser. With images lazily loaded under the same conditions first view load was 25.210s - a saving of around 13s and a 35% speed improvement.

Interestingly, our own webpagetest jobs simulating a 2G connection record fully loaded time as already being around the 25s mark. When run here the first load time was 22.787s for the current experience and 18.942s with lazy loaded images - a saving of around 4s - a 17% speed improvement.

The speeds simulated are supposed to be the same, so it is currently not clear which is a more accurate indication of how this change will benefit 2G users and this is being investigated. When you look at the 95th percentile for fully loaded time per day across the site in the last month it ranges from 11.5s to 15s. This is somewhere between a 3G Fast and 3G Slow connection. When pushing to stable if our visitor speed profile does not change we would hope to see it range between 7s and 11s. However, it is possible given the large difference in 2G before and after we might see it double if it leads to us obtaining more samples from users on slower profiles. When you look at the 95th percentile for first paint per day across the site in the last month it ranges from 4.4 and 5.8s. Again this seems to match a 3G profile. When pushing to production we'd hope this would shift to either 3s or leap to 13s if new samples are collected.