Reading/Web/Lazy loading of images on Bengali Wikipedia

On May 9th 2016 (4pm PST) we rolled out lazy loading of images to all users of Bengali Wikipedia. The purpose of this action was to collect data around page views, global NavigationTiming results from a project to guide a future roll out.

Analysis
Rollout of lazy loaded images on bn.m.wikipedia.org suggests a nontrivial speed improvement in page fully loaded time (initial lag excluded) on HTTP1, likely no speed improvement on HTTP2, and again a significant reduction in image bytes shipped per pageview, leading to lighter weight pages.

Note well that the x-axis in the speed graphs is presented in log2n increments.

To reduce noise, navigation timing events are filtered to ensure key fields are present, with an emphasis on anonymous, non-redirected, plain article pageviews.

To simplify analysis for bytes shipped calculations, eligible source HTML pages are constrained as to a relative path  (on language variant wikis this path would be different, but that's out of scope here) for a given wiki without a colon ":" character in the remainder of the path, with a restriction that responses must be HTTP 200s, in order to avoid overcounting of 30x, 40x, or other such spurious responses. This aids in narrowing down the analysis to requests likely to be plain article pageviews. Image bytes are constrained to those served from upload.wikimedia.org with an eligible Referer for the same restriction as page paths.

Changes introduced to support lazy loaded images required modified (increased) JavaScript/CSS/HTML.

bnwiki

Navigation timing data for bnwiki were sparse, making analysis difficult.

Based on data from 5-11 May 2016, slightly before the initial lazy loaded images went into force, and 23-29 June 2016, the week corresponding to the latest Thursday to Wednesday week with the most up-to-date loading technique 9 (the same week used for the latter week for ukwiki and fkwiki in above analysis), pages with lazy loaded images loaded faster at the 10th, 50th (median), and 90th percentiles on HTTP1. But pages with non-lazy loaded images loaded faster at the 10th, 50th (median), and 90th percentiles on HTTP2.

A certain level of chaos in the events is evident in the line chart. Comparison of 14-May-2016 - 11-May-2016 (prior to lazy loaded images) versus 2-29 June 2016 (well after lazy loaded images implemented) paints a slightly more complete picture, at the expense of more general time based trends potentially complicating the data. Taking the data at face value, it again appears that HTTP1 lazy loaded pages loaded faster at the 10th, 50th (median), and 90th percentiles. On HTTP2 lazy loaded pages loaded faster at the 90th percentile, but slower at the 10th and 50th (median) percentiles. Something interesting occurred with bnwiki. The relative amount of HTTP1 traffic was considerably greater (72.94% of share) on lazy loaded images than without them (61.13%) for the 4 week window comparison. This same trend was observed with comparison windows closer to the switchover of varying lengths. This suggests that perhaps lazy loaded images had a larger impact on the relatively slower connections for bnwiki (twice as slow at the median prior to the change), many originating from Bangladesh.

Data transfer comparisons were more straightforward. Based on examination of two 3-day periods (3-5 May 2016 vs 28-30 May 2016), image bytes per pageview were reduced by about 40.24%. This contributed to a decrease of about 22.73% in bytes shipped for the modified JavaScript/CSS/HTML plus images as compared to the baseline JavaScript/CSS/HTML and images.

Data Transfer Queries

The following query was used to derive the lag-excluded load time, roughLoadTimeInitialLagExcluded. The following query was used to derive image bytes transferred using the constraints described above. The following query was used to derive page and JavaScript/CSS bytes (pre- and post-modification for lazy loading) transferred using the constraints described above.

The following query was used to derive pageviews for the using the constraints above. Practically all matching records were qualified as pageviews, largely ruling out the possibility of image byte transfer counts with proper Referer values being derived from anything other than qualified pageviews. Caveats

As with any data spanning time series and the myriad complexities involved with different devices and environments, data are subject to fluctuation. However, the data transfer savings are unambiguous, and the larger event sampling pool with ukwiki and fawiki lend a degree of confidence that pages are actually loading faster.