Account creation user experience

This is a test of whether improving the registration process increases reader to editor conversion. Additional iterations will examine whether an improved conversion process increases edit rate for new editors.

Background
There is quite a bit of old testing and infrastructure, some of it still live on English Wikipedia.

Account Creation Improvement Project / CustomUserSignup extension
The Account Creation Improvement Project in 2011 had similar goals. Extension:CustomUserSignup was developed in 2011 to A/B test changes to account creation. As of July 2012, this extension is still active on en wiki, but only (?) for the user flow _Log in_ > Don't have an account? _Create one_.

To see the different experiences, go to Log in and reload until you see campaign=ACP1 /2/3 in the URL query string, then click _Create one_. Or try this

The separate "Create account" user flow in place on en wiki (what drives that?) does not "[icon] Log in"

TODO: Turn off this extension TODO: Harvest anything useful from its A/B testing.

SignupAPI
This was a Google Summer of Code project to improve the signup experience and develop an API, per the description at Requests for comment/Account creation. It's in Extension:SignupAPI; the code was recently blanked out in git for code review, there are code review comments in https://gerrit.wikimedia.org/r/#/c/8002/

TODO: Evaluate current status of Signup API, develop spec for it. In process in subpage /SignupAPI.

A/B testing issues
The initial experiment should mirror what we do now and make targeted improvements.

"Conversion"
The key metric is conversion: users who navigate to the account creation page but fail to complete the action vs. users who successfully create an account on English Wikipedia.

"Defining Blocks"
After successful account creation, Patrollers and bots find usernames against policy and block them.

from research topic $16, We want to relate how many blocks during the campaign and whether the user went through A/B/control (or none),

To do this we would have to log username rather than the hash. But user is making herself well-known by registering.

This information won't be directly provided by extension code,

nice to haves...

 * The experiment will not track failures within the account creation.

Following links out of AC?
Track people following links like _Help_ during account creation. >> do they return to page or abandon?

abandoned account creation
We think there's a lot of abandon created accounts.

With the experiment, will we be able to measure... ?

How to track where they go after?
Maybe we leave something in ArticleSaveComplete for *all* experiments one that works for tracking all experiments.

Ryan thinks there could be a table of hash, event name, and whether in earlier experiment.

Not at first.

Technical notes

 * Q Works without JavaScript?
 * A: The experiment won't no, user with JavaScript disabled won't be bucketed so will not be directed into the test. But account creation will continue to work without JavaScript, though without some of whatever improvements arise from the experiment.


 * Q Works without cookies?
 * A: ?? TBD, testing won't follow through without cookies enabled. Unclear if the code checks for cookies disabled or just fails to set the userbucket.


 * Q Which browsers?
 * Firefox 12 Chrome 19 IE 8 Opera 11.64 Safari 534.47
 * Browsers that fail to meet the minimum requirements will probably be ineligible for the experiment (rather than letting them in possibly with degraded experience)

Where does Account creation traffic come from?
TODO: Need to verify experiment makes sense for all incoming traffic flows:

Mainly from "Create account" on every single page

There used to be only a "Login" on every page, and from there uses can Create an account. On this flow, CustomUserAccount (Lennart's project) may appear

Templates for shared IPs say "Create an account" and login.

Any anonymous edit has a login / create account banner.

Data collection and analysis
See Meta

Approximately 200 new accounts are created in English Wikipedia every hour. This volume of new accounts created gives us ample opportunity to test many separate, small changes in hourly or daily tests.

It should be trivial to add new tracking codes to the custom user config file. However, MediaWiki doesn't parse  elements; therefore, the only way to track link-specific events is to wrap them in a span or div and assign an ID to the wrapper. All changes will be prototyped and subject to code review.

We will additionally have any squid filters reviewed before deployment. Emery does not currently have disk space issues and the experiment should not generate more than 1K events per hour, which it can easily handle. As long as other admins don't mess around with the markup of the AC templates, it's safe to enable clicktracking and collect the data we need.