Growth/Personalized first day/Newcomer tasks/Experiment analysis, November 2020

In November 2019, the Growth team added the "newcomer tasks" feature to the newcomer homepage for the wikis we work with. Newcomer tasks provide a feed of suggested articles to edit, tailored to the area of interests of the newcomer. The objective was to give newcomers easy edits that they are interested in when they first arrive on the wiki. We believed that this would make it more likely for newcomers to start editing, to learn editing skills, see their impact, and then continue editing.

Six months after the feature was added to the newcomer homepage, we then collected data from Arabic, Vietnamese, Czech, and Korean Wikipedias to analyze the overall impact of the Growth features, including newcomer tasks. This was possible because the Growth features are deployed in a controlled experiment, in which 80% of newcomers have the features and 20% do not.

Summary of findings
In general, the analysis showed that the Growth features improve outcomes for newcomers. Below are the most important points.

We believe that these results validate that the Growth features, especially newcomer tasks, cause newcomers to edit more and stay on the wiki for longer. Therefore, we confidently encourage all Wikipedias to consider implementing these features. We also believe that these results validate that the Growth team should continue our work on structured tasks, meant to create new kinds of easy editing workflows for newcomers.
 * Newcomers who get the Growth features are more likely to be activated (i.e. make a first article edit).
 * We believe they are also more likely to be retained (i.e. come back and make another article edit on a different day).
 * The features also increase edit volume (i.e. number of edits) and constructiveness (i.e. whether edits are reverted).

Activation
Activation is defined as a newcomer making their first edit. For this experiment, we focused on edits to the Article and Article Talk namespaces.


 * Activation: newcomers with Growth features are 11.6% more likely to make a first article edit.  The baseline activation rate across the four wikis in the experiment is 21.6% making a first article edit. The Growth features brings this to 24.1%.


 * Constructive activation: the effect is larger when looking only at constructive activation. Constructive activation is a newcomer making their first unreverted edit.  Newcomers with Growth features are 26.7% more likely to make a first constructive article edit.  The baseline constructive activation rate across the four wikis in the experiment is 16.1% of newcomers making a first constructive article edit.  The Growth features brings this to 20.4%.  This means that the features nudge newcomers toward edits that are less likely to be reverted.

Retention
Retention is defined as a newcomer who made their first edit then coming back on a different day in the following two weeks and making another edit. Because retention is much more rare than activation, it is harder to detect changes. Though we do not detect it directly, we estimate that retention is increased to a similar degree that activation is increased, i.e. by about 11.6%. The idea is that the Growth features primarily impact activation, in that they get more people to make their first edit. But then those additional people who made their first edit are retained at similar rates to others, leading to an increase in retention caused by the increase in activation.

Other metrics
Edit volume: beyond just the first edit, the Growth features also lead to higher numbers of article edits by newcomers in their first two weeks. On average, a newcomer without the Growth features makes 0.34 article edits in their first two weeks, while a newcomer with the Growth features makes 0.63 article edits. In other words, 1,000 newcomers without the Growth features would make 340 article edits, while 1,000 newcomers with the Growth features would make 630.