Reading/Multimedia/December 2015 cross-wiki upload A/B test

In December 2015, Multimedia team ran an A/B test of four different interfaces for cross-wiki upload tool.

Background
Cross-wiki upload functionality was first announced on the multimedia@ list (as "Upload tool strategy change") and then developed at T91717, with the first deployment of uploads in VisualEditor happening on 2015-10-21. (Uploads in wikitext editor followed, per T115748, on 2015-10-28.) It was announced afterwards in Tech News 2015/42 and on Commons village pump.

The dialog allows any registered user to upload a self-made file under the CC BY-SA license from any Wikimedia wiki to Commons, guiding them through the process. Its interface is inspired by UploadWizard's, but much more limited (in particular, it only allows uploading one file at a time and only allows providing a simple description in one language).

After a while, it was noticed on 2015-12-07 that while the tool is successful at producing uploads (~1000 a day), many of them are copyright violations uploaded by users who didn't notice, ignored or misunderstood the warnings. T120867 was filed and we set out to design an interface providing more information to new users and more resistant to being blindly clicked through, while not too discouraging to good contributors.

For Commons editors
All new cross-wiki uploads are tagged with  and the list can be seen at Special:Log/upload?tagfilter=cross-wiki-upload on Commons. These tags are also visible in file edit history, upload log and recent changes.

Cross-wiki uploads uploaded during the A/B test are tagged with  through. These tags are not ordinarily visible, but you can still use them for filtering. The number of files uploaded with each tag (including deleted files) can be seen at Special:Tags on Commons.

The test is not enabled on all wikis (in particular, it's not enabled on ones in languages that were not translated in time), so you may still see uploads without the numbered tags while the test is ongoing.

Implementation
The test was implemented using change I557056b8 (which implemented the new interfaces and a method to select which one the user will see) and change I90cb12c5 (which recorded the interface used for given upload server-side using change tags).

Users are bucketed into four groups by taking their user ID number modulo 4 (the test only runs for registered users, as only registered users can upload files). For testing and debugging, one can add '?uploadbucket=N' to the URL to force given interface option to appear (any upload completed this way won't count towards the bucket). The test can be enabled by setting  to , and disabled without code changes by setting it back to. To change the default after the test is over,  can be set to the number of the winning option.

The test was enabled with change If10f3e54 (for English-language wikis, responsible of ~43% of cross-wiki uploads so far) and change I12a7ab2a (nine other languages, responsible for ~17% of uploads), started on 2015-12-16 and is expected to run for at least a week (possibly longer if results are inconclusive).

Timeline

 * 2015-12-16 – test begins
 * 2015-12-23 (planned) – test ends

Options
After brainstorming at T121021, we settled on four options:

Each of them will be seen by ~25% of users. Any given user will only see one of the options throughout the duration of the test.

Results
Some results, with graphs, have been posted to T120867. The third option discouraged both good and bad uploads. None seems clearly superior. I commented, explaining what I see as the problem they all have in common, as well as data problems I see.--Elvey (talk) 23:52, 4 January 2016 (UTC)