Reading/Web/Lazy loading of images on Japanese Wikipedia

From MediaWiki.org
< Reading‎ | Web
Jump to: navigation, search

What happened[edit]

On Tuesday 5th July lazy loaded images was enabled on Japanese Wikipedia for all users. The immediate impact we noticed is that images served to users on w:ja:日本 (Japan) dropped from 1.443 MB to 142 kB.

Impact of lazy loaded images on fully loaded time on the Japan article on Japanese Wikipedia. Graph from Grafana.

Bytes saved[edit]

The bytes saved varies per article. Articles with more images obviously will be reduced more. We can analyze bytes saved by looking at raw access logs and looking at images with an internal referrer. We can also look at specific pages to gauge impact.

Bytes went down.

Analysis of weeks worth of data[edit]

A database dump of the EventLogging data for Japanese Wikipedia was taken from the period 1st June to 12th July. Data 10 days before and after the change was compared and sample size was not normalized. Given the change was deployed on 5th July at 4:47pm PST the following query was used.

select * from NavigationTiming_15485142 where wiki = 'jawiki' and event_mobileMode = 'stable' and event_action ='view' and timestamp > 20160625004700 and timestamp < 20160715004700 

This SQL query gave a sample of x rows of data. With the result of the following query, using a Python script the following command was run

python navtimingcsv.py jawiki.tsv 20160705004700 10

Fully loaded time[edit]

Label Sample Size 95th percentile median
Before change 120906 8142.75 2183.0
After change 119855 7104.0 2146.0
Before change (anons) 120781 8146.0 2183.0
After change (anons) 119722 7107.0 2147.0
Before change (http2) 85043 5690.0 1916.0
After change (http2) 84614 5026.0 1887.5
Before change (http1) 35863 11638.8 3381.0
After change (http1) 35241 10036.0 3268.0

First paint[edit]

Label Sample Size 95th percentile median
Before change 36279 3428.0 1042.0
After change 35978 3219.15 1042.0

DomInteractive[edit]

Label Sample Size 95th percentile median
Before change 120906 3585.75 1000.0
After change 119855 3232.0 998.0

Analysis of 30 days of data[edit]

A database dump of the EventLogging data for Japanese Wikipedia was taken from the period 5th June to 4th August. Data 30 days before and after the change was compared. Given the change was deployed on 5th July at 4:47pm PST the following query was used.

select * from NavigationTiming_15485142 where wiki = 'jawiki' and event_mobileMode = 'stable' and event_action ='view' and timestamp > 20160601004700 and timestamp < 20160804004700

This SQL query gave a sample of 1249001 rows of data. With the result of the following query, using a Python script the following command was run

 python navtimingcsv.py japanwiki.tsv 20160705004700 30

Fully loaded time[edit]

Label Sample Size 95th percentile median
Before change 360470 7924.0 2178.0
After change 357042 7475.0 2127.0
Before change (anons) 360068 7926.0 2178.0
After change (anons) 356604 7476.0 2128.0
Before change (http2) 251404 5384.85 1902.0
After change (http2) 254774 5339.35 1869.0
Before change (http1) 109066 11345.0 3356.0
After change (http1) 102268 10692.65 3281.0

First paint[edit]

Label Sample Size 95th percentile median
Before change 107238 3367.0 1072.0
After change 107643 3377.0 1041.0

DomInteractive[edit]

Label Sample Size 95th percentile median
Before change 360470 3435.0 1003.0
After change 357042 3467.0 999.0


Week against week comparisons and longterm data transfer graph[edit]

Rollout of lazy loaded images on ja.m.wikipedia.org suggests a nontrivial speed improvement in page fully loaded time (initial lag excluded) and a significant reduction in image bytes shipped per pageview, leading to lighter weight pages.

Note well that the x-axis in the speed graphs is presented in log2n increments.

To reduce noise, navigation timing events are filtered to ensure key fields are present, with an emphasis on anonymous, non-redirected, plain article pageviews. Logged in pageviews for the NavigationTiming queried window comprised only 0.117% of the pageviews in the set and are not used in the NavigationTiming derived calculations.

To simplify analysis for bytes shipped calculations, eligible source HTML pages are constrained as to a relative path /wiki/ (on language variant wikis this path would be different, but that's out of scope here) for a given wiki without a colon ":" character in the remainder of the path, with a restriction that responses must be HTTP 200s, in order to avoid overcounting of 30x, 40x, or other such spurious responses. This aids in narrowing down the analysis to requests likely to be plain article pageviews. Image bytes are constrained to those served from upload.wikimedia.org with an eligible Referer for the same restriction as page paths.

Changes introduced to support lazy loaded images required modified (slightly increased) JavaScript/CSS/HTML.

jawiki page load speed[edit]

Based on weeks just before and just after the transition to lazy loaded images, pages with lazy loaded images loaded faster at the 10th, 50th (median), and 90th percentiles on both HTTP1 and HTTP2. A basic Kolomogorov-Smirnov statistical significance test suggested that the pre- and post-change cohort speed differences were statistically significant.

The following table is based on anonymous pageviews using the query below.

Percentile HTTP1 w/o lazy HTTP1 LAZY HTTP2 w/o lazy HTTP2 LAZY H1 LAZY Improvement H2 LAZY Improvement
10% 818 783.7 398 384 4.19% 3.52%
50% 2021 1936 1346.5 1328 4.21% 1.37%
90% 5492.8 4561 3206.1 2968 16.96% 7.43%
Jawiki-lazy load-histogram.jpg

jawiki data transfer[edit]

Based on examination of two 7-day periods (24-30 June 2016 vs 8-14 July 2016), image bytes per pageview were reduced by about 51.37%. This contributed to a decrease of about 9.18% bytes shipped for the modified JavaScript/CSS/HTML plus images as compared to the baseline JavaScript/CSS/HTML and images.

The following graph shows the general trends of images bytes shipped per in-scope pageview, as well in-scope pageviews, core HTML/CSS/JavaScript bytes, and the sum of core HTML/CSS/JavaScript bytes plus image bytes per pageview.

Jawiki lazy loaded images.jpg

Queries[edit]

The following query was used to derive the lag-excluded load time, roughLoadTimeInitialLagExcluded. Note this query also captured data on tlwiki, although the scope of the analysis on this page is jawiki mobile

select
left(timestamp,8) as ts,
event_isAnon,
event_lazyLoadImages,
event_isHttp2,
event_loadEventEnd-event_responseStart as roughLoadTimeInitialLagExcluded,
event_responseEnd-event_responseStart as roughNetworkTimeInitialLagExcluded,
event_requestStart,
event_responseStart,
event_responseEnd,
event_firstPaint,
event_domInteractive,
event_domComplete,
event_loadEventStart,
event_loadEventEnd,
webHost,
event_originCountry,
event_mediaWikiVersion
from NavigationTiming_15485142
where
timestamp > '20160623'
and timestamp < '20160721'
and event_action = 'view'
and event_mobileMode = 'stable'
and event_namespaceId = 0
and event_redirectCount is null
and event_loadEventEnd is not null
and event_domComplete is not null
and event_domInteractive is not null
and event_responseStart is not null
and wiki in ('jawiki', 'tlwiki')
order by
wiki,
event_isAnon,
event_lazyLoadImages,
event_isHttp2,
ts
;

The following queries were used to derive image bytes transferred using the constraints described above, as well as to support the longer term graph with trend lines.

select
month,
day,
substr(referer,1,26),
sum(response_size)
from
webrequest
where
year = 2016
and month = 6 and day > 4
and uri_host = 'upload.wikimedia.org'
and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'
and content_type rlike '^image'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
substr(referer,1,26)
;
select
month,
day,
substr(referer,1,26),
sum(response_size)
from
webrequest
where
year = 2016
and month = 7
and uri_host = 'upload.wikimedia.org'
and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'
and content_type rlike '^image'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
substr(referer,1,26)
;


select
month,
day,
substr(referer,1,26),
sum(response_size)
from
webrequest
where
year = 2016
and month = 8 and day < 5
and uri_host = 'upload.wikimedia.org'
and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'
and content_type rlike '^image'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
substr(referer,1,26)
;


The raw data received were as follows:

month   day     _c2     _c3
6       30      https://ja.m.wikipedia.org      124180180469
6       5       https://ja.m.wikipedia.org      135720072139
6       6       https://ja.m.wikipedia.org      108178906611
6       7       https://ja.m.wikipedia.org      111401108523
6       8       https://ja.m.wikipedia.org      106164037819
6       9       https://ja.m.wikipedia.org      105406723448
6       10      https://ja.m.wikipedia.org      104839062684
6       11      https://ja.m.wikipedia.org      141886296285
6       12      https://ja.m.wikipedia.org      170638150504
6       13      https://ja.m.wikipedia.org      146692782536
6       14      https://ja.m.wikipedia.org      134084883702
6       15      https://ja.m.wikipedia.org      133562013207
6       16      https://ja.m.wikipedia.org      142537562128
6       17      https://ja.m.wikipedia.org      133251860232
6       18      https://ja.m.wikipedia.org      151938833556
6       19      https://ja.m.wikipedia.org      167860625982
6       20      https://ja.m.wikipedia.org      136443890103
6       21      https://ja.m.wikipedia.org      136692664905
6       22      https://ja.m.wikipedia.org      147929702365
6       23      https://ja.m.wikipedia.org      139453947620
6       24      https://ja.m.wikipedia.org      156084306064
6       25      https://ja.m.wikipedia.org      168950872999
6       26      https://ja.m.wikipedia.org      171282566058
6       27      https://ja.m.wikipedia.org      135543126283
6       28      https://ja.m.wikipedia.org      136833152633
6       29      https://ja.m.wikipedia.org      135462758117
7       1       https://ja.m.wikipedia.org      124684212453
7       2       https://ja.m.wikipedia.org      164418677635
7       3       https://ja.m.wikipedia.org      161752597537
7       4       https://ja.m.wikipedia.org      149993526937
7       5       https://ja.m.wikipedia.org      134035063705
7       6       https://ja.m.wikipedia.org      103235144867
7       7       https://ja.m.wikipedia.org      67828813390
7       8       https://ja.m.wikipedia.org      66337742258
7       9       https://ja.m.wikipedia.org      80564160978
7       10      https://ja.m.wikipedia.org      85920579820
7       11      https://ja.m.wikipedia.org      68779837487
7       12      https://ja.m.wikipedia.org      67698073684
7       13      https://ja.m.wikipedia.org      69773350604
7       14      https://ja.m.wikipedia.org      67633588058
7       15      https://ja.m.wikipedia.org      67486541269
7       16      https://ja.m.wikipedia.org      78015793312
7       17      https://ja.m.wikipedia.org      86470922402
7       18      https://ja.m.wikipedia.org      86544561954
7       19      https://ja.m.wikipedia.org      66590125244
7       20      https://ja.m.wikipedia.org      64112347900
7       21      https://ja.m.wikipedia.org      66050339575
7       22      https://ja.m.wikipedia.org      63700245787
7       23      https://ja.m.wikipedia.org      72510674705
7       24      https://ja.m.wikipedia.org      82103567189
7       25      https://ja.m.wikipedia.org      66774017408
7       26      https://ja.m.wikipedia.org      67152344637
7       27      https://ja.m.wikipedia.org      65026808541
7       28      https://ja.m.wikipedia.org      62521007017
7       29      https://ja.m.wikipedia.org      62639495316
7       30      https://ja.m.wikipedia.org      70456033383
7       31      https://ja.m.wikipedia.org      79507268276
8       1       https://ja.m.wikipedia.org      67572125043
8       2       https://ja.m.wikipedia.org      66961618961
8       3       https://ja.m.wikipedia.org      67092175391
8       4       https://ja.m.wikipedia.org      65062330557


The following queries were used to derive page and JavaScript/CSS bytes (pre- and post-modification for lazy loading) transferred using the constraints described above, as well as to support the longer term graph with trend lines.

select
month,
day,
uri_host,
sum(response_size)
from
webrequest
where
year = 2016
and month = 6 and day > 4
and uri_host = 'ja.m.wikipedia.org'
and ((uri_path rlike '^/wiki/([^:])+$') or (uri_path = '/w/load.php' and uri_query rlike 'skins\.minerva\.icons\.images\.scripts&skin=minerva' and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'))
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host
;
select
month,
day,
uri_host,
sum(response_size)
from
webrequest
where
year = 2016
and month = 7
and uri_host = 'ja.m.wikipedia.org'
and ((uri_path rlike '^/wiki/([^:])+$') or (uri_path = '/w/load.php' and uri_query rlike 'skins\.minerva\.icons\.images\.scripts&skin=minerva' and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'))
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host
;


select
month,
day,
uri_host,
sum(response_size)
from
webrequest
where
year = 2016
and month = 8 and day < 5
and uri_host = 'ja.m.wikipedia.org'
and ((uri_path rlike '^/wiki/([^:])+$') or (uri_path = '/w/load.php' and uri_query rlike 'skins\.minerva\.icons\.images\.scripts&skin=minerva' and referer rlike '^https://ja\.m\.wikipedia\.org/wiki/([^:])+$'))
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host
;


The raw data received were as follows:

month   day     uri_host        _c3
6       5       ja.m.wikipedia.org      584043509098
6       6       ja.m.wikipedia.org      453573407856
6       7       ja.m.wikipedia.org      443560946176
6       8       ja.m.wikipedia.org      445979910759
6       9       ja.m.wikipedia.org      500639200613
6       10      ja.m.wikipedia.org      488962065693
6       11      ja.m.wikipedia.org      547200189342
6       12      ja.m.wikipedia.org      593204311330
6       13      ja.m.wikipedia.org      492918438444
6       14      ja.m.wikipedia.org      447036568383
6       15      ja.m.wikipedia.org      453638629857
6       16      ja.m.wikipedia.org      479841968101
6       17      ja.m.wikipedia.org      517480911709
6       18      ja.m.wikipedia.org      551939360543
6       19      ja.m.wikipedia.org      591493939800
6       20      ja.m.wikipedia.org      458977191045
6       21      ja.m.wikipedia.org      454923880318
6       22      ja.m.wikipedia.org      492636495002
6       23      ja.m.wikipedia.org      456603025665
6       24      ja.m.wikipedia.org      490341384015
6       25      ja.m.wikipedia.org      537929362713
6       26      ja.m.wikipedia.org      545503508434
6       27      ja.m.wikipedia.org      431615776632
6       28      ja.m.wikipedia.org      448780648978
6       29      ja.m.wikipedia.org      451773538098
6       30      ja.m.wikipedia.org      423185112608
7       3       ja.m.wikipedia.org      537992180969
7       4       ja.m.wikipedia.org      448322594045
7       5       ja.m.wikipedia.org      445007695821
7       6       ja.m.wikipedia.org      435472597765
7       7       ja.m.wikipedia.org      430814756332
7       8       ja.m.wikipedia.org      461334758600
7       9       ja.m.wikipedia.org      547280892717
7       10      ja.m.wikipedia.org      600062076779
7       11      ja.m.wikipedia.org      473057269970
7       12      ja.m.wikipedia.org      477262829644
7       13      ja.m.wikipedia.org      471436878948
7       14      ja.m.wikipedia.org      472398332176
7       15      ja.m.wikipedia.org      548724119441
7       16      ja.m.wikipedia.org      616527657391
7       17      ja.m.wikipedia.org      639723642046
7       18      ja.m.wikipedia.org      638694292375
7       19      ja.m.wikipedia.org      481106407202
7       20      ja.m.wikipedia.org      473148500657
7       21      ja.m.wikipedia.org      466261658125
7       22      ja.m.wikipedia.org      465249325390
7       23      ja.m.wikipedia.org      492041513150
7       24      ja.m.wikipedia.org      563837017528
7       25      ja.m.wikipedia.org      445503057133
7       26      ja.m.wikipedia.org      447799358192
7       27      ja.m.wikipedia.org      447035047673
7       28      ja.m.wikipedia.org      432588647022
7       29      ja.m.wikipedia.org      447436227804
7       30      ja.m.wikipedia.org      503914286671
7       31      ja.m.wikipedia.org      553840973015
7       1       ja.m.wikipedia.org      424109984113
7       2       ja.m.wikipedia.org      535223235640
8       4       ja.m.wikipedia.org      463721744365
8       1       ja.m.wikipedia.org      465305749921
8       2       ja.m.wikipedia.org      459082686410
8       3       ja.m.wikipedia.org      468718936637


The following queries were used to derive pageviews using the constraints above, as well as to support the longer term graph with trend lines. Practically all matching records were qualified as pageviews, largely ruling out the possibility of image byte transfer counts with proper Referer values being derived from anything other than qualified pageviews.

select
month,
day,
uri_host,
content_type,
is_pageview,
count(1)
from
webrequest
where
year = 2016
and month = 6 and day > 4
and uri_host = 'ja.m.wikipedia.org'
and uri_path rlike '^/wiki/([^:])+$'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host,
content_type,
is_pageview
;
select
month,
day,
uri_host,
content_type,
is_pageview,
count(1)
from
webrequest
where
year = 2016
and month = 7
and uri_host = 'ja.m.wikipedia.org'
and uri_path rlike '^/wiki/([^:])+$'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host,
content_type,
is_pageview
;
select
month,
day,
uri_host,
content_type,
is_pageview,
count(1)
from
webrequest
where
year = 2016
and month = 8 and day < 5
and uri_host = 'ja.m.wikipedia.org'
and uri_path rlike '^/wiki/([^:])+$'
and agent_type = 'user'
and http_status = '200'
group by
month,
day,
uri_host,
content_type,
is_pageview
;


The raw data received were as follows. Note that the low count "text/html" and "-" content types, as well as the non-pageview records are not used in the calculations on this page. These were included in this query to gauge the magnitude of their potential effect. It would appear their effect was minimal.

6       19      ja.m.wikipedia.org      text/html; charset=UTF-8        true    21683504
6       28      ja.m.wikipedia.org      application/pdf false   1
6       17      ja.m.wikipedia.org      -       false   4
6       17      ja.m.wikipedia.org      image/png       false   9
6       21      ja.m.wikipedia.org      text/html; charset=UTF-8        false   1
6       21      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17370675
6       17      ja.m.wikipedia.org      text/css; charset=utf-8 false   6
6       20      ja.m.wikipedia.org      text/html       true    2
6       17      ja.m.wikipedia.org      text/javascript false   2
6       23      ja.m.wikipedia.org      text/html; charset=UTF-8        false   9
6       23      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17673155
6       25      ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
6       25      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20560909
6       6       ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
6       6       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17427607
6       17      ja.m.wikipedia.org      image/vnd.microsoft.icon        false   7
6       27      ja.m.wikipedia.org      text/html; charset=UTF-8        false   3
6       27      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16981545
6       8       ja.m.wikipedia.org      text/html; charset=UTF-8        false   3
6       8       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17168195
6       26      ja.m.wikipedia.org      text/html       true    2
6       29      ja.m.wikipedia.org      text/html; charset=UTF-8        false   1
6       29      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17880569
6       10      ja.m.wikipedia.org      text/html; charset=UTF-8        false   12
6       10      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17312428
6       12      ja.m.wikipedia.org      text/html; charset=UTF-8        false   2
6       12      ja.m.wikipedia.org      text/html; charset=UTF-8        true    21701144
6       14      ja.m.wikipedia.org      text/html; charset=UTF-8        false   15
6       14      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16960827
6       16      ja.m.wikipedia.org      text/html; charset=UTF-8        false   24
6       16      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17600084
6       17      ja.m.wikipedia.org      application/json; charset=utf-8 false   6
6       18      ja.m.wikipedia.org      text/html; charset=UTF-8        false   11
6       18      ja.m.wikipedia.org      text/html; charset=UTF-8        true    19381891
6       20      ja.m.wikipedia.org      text/html; charset=UTF-8        false   11
6       20      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17291241
6       22      ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
6       22      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18665374
6       30      ja.m.wikipedia.org      text/javascript; charset=utf-8  false   1
6       24      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
6       24      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18789507
6       5       ja.m.wikipedia.org      text/html; charset=UTF-8        false   8
6       5       ja.m.wikipedia.org      text/html; charset=UTF-8        true    21686129
6       26      ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
6       26      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20798241
6       7       ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
6       7       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17492334
6       17      ja.m.wikipedia.org      text/javascript; charset=UTF-8  false   5
6       22      ja.m.wikipedia.org      text/javascript false   1
6       28      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
6       28      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17811344
6       9       ja.m.wikipedia.org      text/html; charset=UTF-8        false   3
6       9       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17059342
6       17      ja.m.wikipedia.org      text/javascript; charset=utf-8  false   27
6       30      ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
6       30      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16896524
6       11      ja.m.wikipedia.org      text/html; charset=UTF-8        false   7
6       11      ja.m.wikipedia.org      text/html; charset=UTF-8        true    19594972
6       13      ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
6       13      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18410111
6       21      ja.m.wikipedia.org      text/javascript; charset=utf-8  false   1
6       30      ja.m.wikipedia.org      text/css; charset=utf-8 false   1
6       15      ja.m.wikipedia.org      text/html; charset=UTF-8        false   1
6       15      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17176324
6       17      ja.m.wikipedia.org      image/svg+xml   false   7
6       17      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
6       17      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17645960
6       17      ja.m.wikipedia.org      text/css; charset=UTF-8 false   1
6       19      ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
7       6       ja.m.wikipedia.org      text/html       true    3
7       28      ja.m.wikipedia.org      text/html; charset=UTF-8        false   14
7       28      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16285305
7       9       ja.m.wikipedia.org      text/html; charset=UTF-8        false   10
7       9       ja.m.wikipedia.org      text/html; charset=UTF-8        true    20806034
7       30      ja.m.wikipedia.org      text/html; charset=UTF-8        false   2
7       30      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18485500
7       11      ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
7       11      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18072054
7       13      ja.m.wikipedia.org      text/html; charset=UTF-8        false   8
7       13      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17788981
7       3       ja.m.wikipedia.org      application/pdf false   12
7       12      ja.m.wikipedia.org      text/html       true    1
7       15      ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
7       15      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17055753
7       14      ja.m.wikipedia.org      text/html       true    1
7       17      ja.m.wikipedia.org      text/html; charset=UTF-8        false   11
7       17      ja.m.wikipedia.org      text/html; charset=UTF-8        true    21657938
7       28      ja.m.wikipedia.org      image/vnd.microsoft.icon        false   1
7       7       ja.m.wikipedia.org      application/pdf false   1
7       25      ja.m.wikipedia.org      text/javascript; charset=utf-8  false   1
7       8       ja.m.wikipedia.org      text/javascript; charset=UTF-8  false   7
7       19      ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
7       19      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17341474
7       28      ja.m.wikipedia.org      application/pdf false   2
7       8       ja.m.wikipedia.org      text/javascript; charset=utf-8  false   1
7       21      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
7       21      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17240735
7       2       ja.m.wikipedia.org      text/html; charset=UTF-8        false   2
7       2       ja.m.wikipedia.org      text/html; charset=UTF-8        true    20680721
7       23      ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
7       23      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17927678
7       4       ja.m.wikipedia.org      text/html; charset=UTF-8        false   70
7       4       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17534848
7       25      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
7       25      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16528308
7       6       ja.m.wikipedia.org      text/html; charset=UTF-8        false   8
7       6       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17320422
7       27      ja.m.wikipedia.org      text/html; charset=UTF-8        false   6
7       27      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16801517
7       8       ja.m.wikipedia.org      text/html; charset=UTF-8        false   11
7       8       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17586299
7       25      ja.m.wikipedia.org      image/png       false   1
7       29      ja.m.wikipedia.org      text/html; charset=UTF-8        false   30
7       29      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16521987
7       4       ja.m.wikipedia.org      text/javascript false   1
7       25      ja.m.wikipedia.org      text/css; charset=utf-8 false   1
7       10      ja.m.wikipedia.org      text/html; charset=UTF-8        false   14
7       10      ja.m.wikipedia.org      text/html; charset=UTF-8        true    22014268
7       25      ja.m.wikipedia.org      text/javascript false   1
7       31      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
7       31      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20046905
7       2       ja.m.wikipedia.org      image/vnd.microsoft.icon        false   1
7       12      ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
7       12      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17971521
7       21      ja.m.wikipedia.org      application/pdf false   1
7       10      ja.m.wikipedia.org      -       false   1
7       14      ja.m.wikipedia.org      text/html; charset=UTF-8        false   8
7       14      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17191513
7       16      ja.m.wikipedia.org      text/html; charset=UTF-8        false   27
7       16      ja.m.wikipedia.org      text/html; charset=UTF-8        true    19963165
7       18      ja.m.wikipedia.org      text/html; charset=UTF-8        false   8
7       18      ja.m.wikipedia.org      text/html; charset=UTF-8        true    22374981
7       20      ja.m.wikipedia.org      text/html; charset=UTF-8        false   10
7       20      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17164645
7       1       ja.m.wikipedia.org      text/html; charset=UTF-8        false   23
7       1       ja.m.wikipedia.org      text/html; charset=UTF-8        true    16583764
7       10      ja.m.wikipedia.org      application/pdf false   1
7       22      ja.m.wikipedia.org      text/html; charset=UTF-8        false   9
7       22      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16624345
7       3       ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
7       3       ja.m.wikipedia.org      text/html; charset=UTF-8        true    20609077
7       24      ja.m.wikipedia.org      text/html; charset=UTF-8        false   7
7       24      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20685545
7       5       ja.m.wikipedia.org      text/html; charset=UTF-8        false   109
7       5       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17714516
7       25      ja.m.wikipedia.org      application/json; charset=utf-8 false   1
7       26      ja.m.wikipedia.org      text/html; charset=UTF-8        false   2
7       26      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16840162
7       7       ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
7       7       ja.m.wikipedia.org      text/html; charset=UTF-8        true    16916233
8       1       ja.m.wikipedia.org      text/html; charset=UTF-8        false   4
8       1       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17212992
8       3       ja.m.wikipedia.org      text/html; charset=UTF-8        false   11
8       3       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17545536
8       1       ja.m.wikipedia.org      application/pdf false   1
8       2       ja.m.wikipedia.org      text/html; charset=UTF-8        false   9
8       2       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17167953
8       4       ja.m.wikipedia.org      text/html; charset=UTF-8        false   5
8       4       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17291588




As with any data spanning time series and the myriad complexities involved with different devices and environments, data are subject to fluctuation. However, the data transfer savings are unambiguous, and the large event sampling pool lends a degree of confidence that pages are actually loading faster.

Statistical significance


The Kolomogorov-Smirnov test suggests the differences in the distributions for lazy loaded compared to non-lazy loaded image based page load times for each of HTTP/1 and HTTP/2 are material and we can reject the null hypothesis.

from scipy.stats import ks_2samp

# h1_conventional.txt contains 24-30 June 2016 HTTP/1 convetional loading rough load times
with open('h1_conventional.txt') as f:
        a = [int(i) for i in f.readlines()]

# h1_lazy.txt contains 8-14 July 2016 HTTP/2 lazy loading rough load times
with open('h1_lazy.txt') as f:
        b = [int(i) for i in f.readlines()]

len(a)
15517
len(b)
15898

ks_2samp(a,b)
(0.037166639427270387, 7.1355334748281838e-10)

# result is ks statistic, p-value. Null hypothesis rejected.



with open('h2_conventional.txt') as f:
        c = [int(i) for i in f.readlines()]

with open('h2_lazy.txt') as f:
        d = [int(i) for i in f.readlines()]

len(c)
53735
len(d)
55437

ks_2samp(c,d)
(0.020979426651792699, 7.149451368361895e-11)

# result is ks statistic, p-value. Null hypothesis rejected.

The data used for the two month line graph for bytes shipped, pageviews, etc. is provided here:

In-scope pageviews                                      
month   day     uri_host        content_type    is_pageview     pvs
6       5       ja.m.wikipedia.org      text/html; charset=UTF-8        true    21686129
6       6       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17427607
6       7       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17492334
6       8       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17168195
6       9       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17059342
6       10      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17312428
6       11      ja.m.wikipedia.org      text/html; charset=UTF-8        true    19594972
6       12      ja.m.wikipedia.org      text/html; charset=UTF-8        true    21701144
6       13      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18410111
6       14      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16960827
6       15      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17176324
6       16      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17600084
6       17      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17645960
6       18      ja.m.wikipedia.org      text/html; charset=UTF-8        true    19381891
6       19      ja.m.wikipedia.org      text/html; charset=UTF-8        true    21683504
6       20      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17291241
6       21      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17370675
6       22      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18665374
6       23      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17673155
6       24      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18789507
6       25      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20560909
6       26      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20798241
6       27      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16981545
6       28      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17811344
6       29      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17880569
6       30      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16896524
7       1       ja.m.wikipedia.org      text/html; charset=UTF-8        true    16583764
7       2       ja.m.wikipedia.org      text/html; charset=UTF-8        true    20680721
7       3       ja.m.wikipedia.org      text/html; charset=UTF-8        true    20609077
7       4       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17534848
7       5       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17714516
7       6       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17320422
7       7       ja.m.wikipedia.org      text/html; charset=UTF-8        true    16916233
7       8       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17586299
7       9       ja.m.wikipedia.org      text/html; charset=UTF-8        true    20806034
7       10      ja.m.wikipedia.org      text/html; charset=UTF-8        true    22014268
7       11      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18072054
7       12      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17971521
7       13      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17788981
7       14      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17191513
7       15      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17055753
7       16      ja.m.wikipedia.org      text/html; charset=UTF-8        true    19963165
7       17      ja.m.wikipedia.org      text/html; charset=UTF-8        true    21657938
7       18      ja.m.wikipedia.org      text/html; charset=UTF-8        true    22374981
7       19      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17341474
7       20      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17164645
7       21      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17240735
7       22      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16624345
7       23      ja.m.wikipedia.org      text/html; charset=UTF-8        true    17927678
7       24      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20685545
7       25      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16528308
7       26      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16840162
7       27      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16801517
7       28      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16285305
7       29      ja.m.wikipedia.org      text/html; charset=UTF-8        true    16521987
7       30      ja.m.wikipedia.org      text/html; charset=UTF-8        true    18485500
7       31      ja.m.wikipedia.org      text/html; charset=UTF-8        true    20046905
8       1       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17212992
8       2       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17167953
8       3       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17545536
8       4       ja.m.wikipedia.org      text/html; charset=UTF-8        true    17291588







image bytes                     
month   day     _c2     _c3
6       5       https://ja.m.wikipedia.org      135720072139
6       6       https://ja.m.wikipedia.org      108178906611
6       7       https://ja.m.wikipedia.org      111401108523
6       8       https://ja.m.wikipedia.org      106164037819
6       9       https://ja.m.wikipedia.org      105406723448
6       10      https://ja.m.wikipedia.org      104839062684
6       11      https://ja.m.wikipedia.org      141886296285
6       12      https://ja.m.wikipedia.org      170638150504
6       13      https://ja.m.wikipedia.org      146692782536
6       14      https://ja.m.wikipedia.org      134084883702
6       15      https://ja.m.wikipedia.org      133562013207
6       16      https://ja.m.wikipedia.org      142537562128
6       17      https://ja.m.wikipedia.org      133251860232
6       18      https://ja.m.wikipedia.org      151938833556
6       19      https://ja.m.wikipedia.org      167860625982
6       20      https://ja.m.wikipedia.org      136443890103
6       21      https://ja.m.wikipedia.org      136692664905
6       22      https://ja.m.wikipedia.org      147929702365
6       23      https://ja.m.wikipedia.org      139453947620
6       24      https://ja.m.wikipedia.org      156084306064
6       25      https://ja.m.wikipedia.org      168950872999
6       26      https://ja.m.wikipedia.org      171282566058
6       27      https://ja.m.wikipedia.org      135543126283
6       28      https://ja.m.wikipedia.org      136833152633
6       29      https://ja.m.wikipedia.org      135462758117
6       30      https://ja.m.wikipedia.org      124180180469
7       1       https://ja.m.wikipedia.org      124684212453
7       2       https://ja.m.wikipedia.org      164418677635
7       3       https://ja.m.wikipedia.org      161752597537
7       4       https://ja.m.wikipedia.org      149993526937
7       5       https://ja.m.wikipedia.org      134035063705
7       6       https://ja.m.wikipedia.org      103235144867
7       7       https://ja.m.wikipedia.org      67828813390
7       8       https://ja.m.wikipedia.org      66337742258
7       9       https://ja.m.wikipedia.org      80564160978
7       10      https://ja.m.wikipedia.org      85920579820
7       11      https://ja.m.wikipedia.org      68779837487
7       12      https://ja.m.wikipedia.org      67698073684
7       13      https://ja.m.wikipedia.org      69773350604
7       14      https://ja.m.wikipedia.org      67633588058
7       15      https://ja.m.wikipedia.org      67486541269
7       16      https://ja.m.wikipedia.org      78015793312
7       17      https://ja.m.wikipedia.org      86470922402
7       18      https://ja.m.wikipedia.org      86544561954
7       19      https://ja.m.wikipedia.org      66590125244
7       20      https://ja.m.wikipedia.org      64112347900
7       21      https://ja.m.wikipedia.org      66050339575
7       22      https://ja.m.wikipedia.org      63700245787
7       23      https://ja.m.wikipedia.org      72510674705
7       24      https://ja.m.wikipedia.org      82103567189
7       25      https://ja.m.wikipedia.org      66774017408
7       26      https://ja.m.wikipedia.org      67152344637
7       27      https://ja.m.wikipedia.org      65026808541
7       28      https://ja.m.wikipedia.org      62521007017
7       29      https://ja.m.wikipedia.org      62639495316
7       30      https://ja.m.wikipedia.org      70456033383
7       31      https://ja.m.wikipedia.org      79507268276
8       1       https://ja.m.wikipedia.org      67572125043
8       2       https://ja.m.wikipedia.org      66961618961
8       3       https://ja.m.wikipedia.org      67092175391
8       4       https://ja.m.wikipedia.org      65062330557
                        
                        
page bytes                      
month   day     uri_host        _c3
6       5       ja.m.wikipedia.org      584043509098
6       6       ja.m.wikipedia.org      453573407856
6       7       ja.m.wikipedia.org      443560946176
6       8       ja.m.wikipedia.org      445979910759
6       9       ja.m.wikipedia.org      500639200613
6       10      ja.m.wikipedia.org      488962065693
6       11      ja.m.wikipedia.org      547200189342
6       12      ja.m.wikipedia.org      593204311330
6       13      ja.m.wikipedia.org      492918438444
6       14      ja.m.wikipedia.org      447036568383
6       15      ja.m.wikipedia.org      453638629857
6       16      ja.m.wikipedia.org      479841968101
6       17      ja.m.wikipedia.org      517480911709
6       18      ja.m.wikipedia.org      551939360543
6       19      ja.m.wikipedia.org      591493939800
6       20      ja.m.wikipedia.org      458977191045
6       21      ja.m.wikipedia.org      454923880318
6       22      ja.m.wikipedia.org      492636495002
6       23      ja.m.wikipedia.org      456603025665
6       24      ja.m.wikipedia.org      490341384015
6       25      ja.m.wikipedia.org      537929362713
6       26      ja.m.wikipedia.org      545503508434
6       27      ja.m.wikipedia.org      431615776632
6       28      ja.m.wikipedia.org      448780648978
6       29      ja.m.wikipedia.org      451773538098
6       30      ja.m.wikipedia.org      423185112608
7       1       ja.m.wikipedia.org      424109984113
7       2       ja.m.wikipedia.org      535223235640
7       3       ja.m.wikipedia.org      537992180969
7       4       ja.m.wikipedia.org      448322594045
7       5       ja.m.wikipedia.org      445007695821
7       6       ja.m.wikipedia.org      435472597765
7       7       ja.m.wikipedia.org      430814756332
7       8       ja.m.wikipedia.org      461334758600
7       9       ja.m.wikipedia.org      547280892717
7       10      ja.m.wikipedia.org      600062076779
7       11      ja.m.wikipedia.org      473057269970
7       12      ja.m.wikipedia.org      477262829644
7       13      ja.m.wikipedia.org      471436878948
7       14      ja.m.wikipedia.org      472398332176
7       15      ja.m.wikipedia.org      548724119441
7       16      ja.m.wikipedia.org      616527657391
7       17      ja.m.wikipedia.org      639723642046
7       18      ja.m.wikipedia.org      638694292375
7       19      ja.m.wikipedia.org      481106407202
7       20      ja.m.wikipedia.org      473148500657
7       21      ja.m.wikipedia.org      466261658125
7       22      ja.m.wikipedia.org      465249325390
7       23      ja.m.wikipedia.org      492041513150
7       24      ja.m.wikipedia.org      563837017528
7       25      ja.m.wikipedia.org      445503057133
7       26      ja.m.wikipedia.org      447799358192
7       27      ja.m.wikipedia.org      447035047673
7       28      ja.m.wikipedia.org      432588647022
7       29      ja.m.wikipedia.org      447436227804
7       30      ja.m.wikipedia.org      503914286671
7       31      ja.m.wikipedia.org      553840973015
8       1       ja.m.wikipedia.org      465305749921
8       2       ja.m.wikipedia.org      459082686410
8       3       ja.m.wikipedia.org      468718936637
8       4       ja.m.wikipedia.org      463721744365



Page views[edit]

To rule out bytes saved being due to a drop in page views, we checked the traffic to mobile site, in ja.wikipedia (i.e., ja.m.wikipedia.org) in the months of June and July. In June the user traffic was 585 millions and in July 587 millions. (normalized for a 30-day month). The two months are pretty much the same in terms of traffic for practical purposes so we concluded that the savings in bytes was not likely due to a macro change in traffic.

See also[edit]

  • A decrease in the number of requests would make mediacounts more truthful.