Contributors/Metrics

Like any team, the Editing department will do better if we have a set of clear, well-thought-out performance indicators that we can use to evaluate our work. Comments and edits welcome.

Current strategy
Many of the metrics that the Editing department might use as strategic indicators, like global active editors and global mobile edits, are mostly out of our control. If we see, say, the global active editors number trend upwards over 6 months around the time we roll out a headline new feature, we have very little way to know if the new feature was responsible and many reasons to suspect that it wasn't (because so many other thing influence the metric). Whether they go up or down or stay flat, such metrics are interesting but not actionable. As Aaron Halfaker puts it, we might build a great windmill and write it off as a failure because the wind isn't blowing (or the reverse).

In addition, Editing contains a fairly heterogenous group of teams. It not clear that one metric, or four, can be an actionable guide for all of them at once.

Instead, for the moment, we're mostly tabling work on strategic, depatment-level indicators and focusing on team-specific indicators instead. These are both conceptually easier to design and more likely to be immediately actionable. For example, focusing on rates (like the rate at which editors who've registered become contributors) instead of absolute numbers could take the high level "winds" (e.g. people's interest in Wikimedia projects) as given and focuses instead on how efficiently we convert them into our desired outcomes.

Barometer projects
Sometimes it's necessary or desirable to track individual projects rather than global numbers. However, it's always impossible to track hundreds of projects individually, so it's helpful to have a list of specific projects which we focus on. There are a number of considerations involved: One very simple solution the six Wikipedias in the official languages of the United Nation.
 * Large size, which makes the data less noisy.
 * Mix of mature and developing projects.
 * Global diversity
 * Mix of Latin and non-Latin scripts

List, plus May 2015 active contributors and rank thereby:
 * English Wikipedia (31 601, 1st)
 * French Wikipedia (4 602, 3rd)
 * Spanish Wikipedia (4 318, 4th)
 * Russian Wikipedia (3 315, 6th)
 * Chinese Wikipedia (2 378, 8th)
 * Arabic Wikipedia (944, 12th)
 * Should we also include Commons? It's not clear that most metrics will apply well to it.

Requirements
These are open to debate and prioritization.
 * Editor numbers are globally deduped.
 * Statistics are backfilled through, say, 2012.

Open questions

 * "Top" projects which we focus on tracking? (May 2015 active contributors; rank)
 * English Wikipedia (31 601)
 * German Wikipedia (5 807)
 * French Wikipedia (4 602)
 * Spanish Wikipedia (4 318)
 * Japanese Wikipedia (4 295)
 * Russian Wikipedia (3 315)
 * Chinese Wikipedia (2 378)
 * Portuguese Wikipedia (1 534)
 * Arabic Wikipedia (944)
 * English Wikitionary (352)
 * French Wikisource (113)
 * Wikidata or Commons? But they're idiosyncratic.
 * Do we need to break down every global metric for all 800-odd wikis? It's not clear how much of a extra burden that imposes when we're already measuring global metrics.
 * How should we measure VE adoption?
 * (How) should we save intermediate indicators: e.g., for editing success rate, the number of init events, the number of those which had saveSuccess events, and the number excluding the specific saveFailures?