Extension:Popups/Fundraising test 2

Following on from the first test in Hungary, in October/November 2016 the Fundraising and Reading teams ran a second experiment to see if enabling Hovercards had an effect on banner fundraising.

Methodology
We selected the Italian Wikipedia for our next experiment, since it was a larger Wikipedia where the community was receptive to a test of Hovercards, and we had already planned fundraising in Italy for October/November.

Hovercards were enabled on the Italian Wikipedia for 20% of anonymous users on September 23. On October 11 the fundraising team began running their planned banners on the Italian Wikipedia desktop site in Italy, with the usual campaign setup: We used our standard fundraising banners for this test, translated into Italian and with some minor technical modifications to log whether Hovercards were enabled or disabled for the user.
 * anonymous users only
 * users see one large banner (example) on their first visit to the site, and then smaller banners (example) on subsequent visits
 * two weeks after launch (October 25) the number of banners a user will see is limited to a maximum of 3
 * four weeks after launch (November 8) the campaign ends and banners are disabled

Results
Full data can be seen here.

Large banners
Performance comparison - Hovercards gives between 1.1% decrease and 2.8% increase in number of donations / impressions (with 95% confidence) i.e. likely no change.

Small banners
Performance comparison - Hovercards gives between 0.18% and 5.3% increase in number of donations / impressions (with 95% confidence)

Total
Performance comparison - Hovercards gives between 1.1% and 4.2% increase in number of donations / impressions (with 95% confidence)