Growth/Analytics updates/Welcome survey editor activation rate

When we deployed the Welcome survey to Czech and Korean Wikipedia, our main concern was that it would lead to fewer newly registered users become editors within the first 24 hours after registering (what we call "editor activation"). This concern is described in our experiment plan, and is why we ran the survey as a randomized A/B test over the course of a month. Half of new registrations got the survey, half did not. In this update, we describe the results of that A/B test, answering the question: does the welcome survey affect editor activation rate?

The survey was deployed to Czech and Korean Wikipedias on November 19, 2018, shortly after 19:00 UTC. In this analysis, we use data from deployment until December 25, so as to use whole weeks. While we had one more week of data available at the time of analysis, we discarded it due to a spambot attack affecting registrations on Korean Wikipedia.

In addition to limiting accounts by time of registration, we also apply several other filters. First, the survey is only shown to users who register on the given wiki, so we filter out users who already had accounts on another wiki (also known as "autocreated accounts"). Secondly, we filter out accounts created through Wikipedia's API as those are mainly app accounts, and the survey is not running on either of the Wikipedia apps. Lastly, we remove known test accounts created by Growth Team members.