Growth/Analytics updates/Key metrics and power analysis

The Growth Team is planning on running interventions aiming to increase new editor retention, as stated in our goals and metrics. However, the standard way of measuring retention (Surviving new editor) uses a two month window. We would prefer to be able to understand the impact of our interventions sooner than two months. This led us to dig more closely into our data on new contributors to our target Wikipedias (Czech and Korean) to understand what the typical contributor behavior is. Next, we used those insights to define three key metrics, followed by a power analysis to estimate the duration of our interventions depending on their magnitude of change.

Our key metrics relate strongly to the definition of "surviving new editor" linked above, but define the path towards retention more clearly and with specific time constraints. Where "surviving new editor" typically has an loosely specified period of time for exploration, our metrics define a clear three-step path: activation, short-term retention, and long-term retention. These three steps form a funnel, each being a requirement for the next. This means that a long-term retained editor also is a short-term retained editor, and similarly that any retained editor was once activated.

Next, we'll walk through our three key metrics and describe more about how we came to define them.

Activation Rate
In order to be a retained editor, a user first needs to become an editor. We call this event "activation". One way to increase the number of retained editors might be to increase the number of users who become editors, thus one of our key metrics is the proportion of registered accounts who make an edit.

The question is, how much time should we give them to make that edit?