Extension:EventLogging/SendBeacon

Intro
This document describes some of our recent experiments of reagrding EventLogging and the new sendBeacon method  present in more modern browsers.

The mean advantage of using sendBeacon is is true async nature, it allows us to log events without disturbing the user workflow. For details of functionality requested that might benefit from sendBeacon you can go to this bug: T44815 https://phabricator.wikimedia.org/T44815

Goal
Asses how sendBeacon is working in browsers that claim to support it.

Methodology
In order to know if sendBeacon works as advertised we setup a minor experiment using EventLogging itself. We designed an schema that allows us to "measure" whether a browser that says to support SendBeacon really does:

If sendBeacon is available, event data is logged using both the sendBeacon wrapper (logPersistentEvent) and the img approach (logEvent). Otherwise, it only uses logEvent. Thus, if sendBeacon is working properly for the browser in question we should see two events one tagged as "logEvent" and other as "logPersistentEvent" with the same id. If "logEvent" is present but "logPersistenEvent" is not when the browser says to support sendBeacon, the browser support is faulty .

Code for our experiment events can be found here:.

Disclaimer
These conclusions are obtained with preliminary data, we expect to have data for our users at large from January 10 onwards when cache expires make our code available to all users. Data thus far (about 400K records) comes from editors (which see fresh files, not cached ones) and from user visits to pages that have been evicted from the cache.

Results
The bulk majority of browsers that say that support sendBeacon really do and we see our events coming as we would expect. The number of records for browsers that seem to have faulty support is quite small. We see less than 100 faulty events (out of about 200k).

Browsers that support sendBeacon as expected
Note that we found a bug in the random function that is used to produce random event identifiers and that we have removed these records from analysis. The number of duplicates is small but significant, there are 2000 events with duplicate ids out of 200K. (TODO: add link to bug once phabricator comes back online)

We first select all user agents that have two events: 'logEvent' and 'logPersistenEvent' per eventId:

-- get all browsers that support sendBeacon, remove duplicated ids for logEvent use log; select count(userAgent), userAgent from SendBeaconReliability_10735916 where event_method='logPersistentEvent' and event_logId in (select logId from (select count(event_logId) as c, event_logId as logId from SendBeaconReliability_10735916 where event_method='logEvent' group by event_logId) as A where c=1) group by userAgent;

We process these to see which are the most used browsers of the ones that support send beacon: https://gist.github.com/nuria/9af8c6edbf44becbc6ac

Of the browser set that supports send beacon well these are the browsers that have more than 0.5% share:

0.52% Chrome Mobile 39 Android 5.0 0.52% Firefox 34 Mac OS X 10.10 0.54% Firefox 34 Mac OS X 10.9 0.58% Firefox 31 Windows 7 0.65% Chrome 39 Android 4.4 0.67% Chrome Mobile 39 Android 4.3 0.67% Firefox 34 Ubuntu 0.70% Firefox 34 Windows 8 0.86% Chrome Mobile 39 Android 4.1 1.02% Chrome Mobile 39 Android 4.2 1.03% Firefox 34 Windows Vista 1.32% Firefox 33 Windows 7 1.45% Opera 26 Windows 7 1.65% Chrome 39 Mac OS X 10.9 1.70% Chrome 39 Windows Vista 1.88% Chrome 39 Mac OS X 10.10 2.42% Chrome 39 Windows 8 2.86% Firefox 34 Windows XP 3.75% Firefox 34 Windows 8.1 4.23% Chrome 39 Windows XP 6.40% Chrome Mobile 39 Android 4.4 10.19% Chrome 39 Windows 8.1 13.57% Firefox 34 Windows 7 31.57% Chrome 39 Windows 7

Browsers with faulty support of SendBeacon
Faulty support is present in less than 100 events of our 200K event dataset, thus we will refrain of drawing conclusions until we have a bit more data.