Translations:Growth/Personalized first day/Newcomer tasks/Experiment analysis, November 2020/32/en

Because retention is much more rare than activation, it is harder to detect changes. In this experiment, we did not detect any changes directly. Instead, we estimate that retention is increased to a similar degree that activation is increased, i.e. by about 11.6%. This comes from the idea that activity during the first day affects activity on the following days, something we account for in our statistical models. Since the Growth features are found to increase the number of users who are active on their first day, and we find no change in the probability that activated users are retained, it follows that we can expect the increase in activation to translate into a similar increase in retention. In other words, the Growth features appear to lead to an increase in retention caused by the increase in activation: some of the users that the Growth features activated would naturally go on to being retained.