Product Analytics/Movement metrics

The Product Analytics team is responsible for defining and calculating a set of metrics for the overall health of the movement.

For current definitions, see Wikimedia Audiences/Data dictionary. For our most recent movement metrics report, see Wikimedia Audiences.

We report these metrics monthly, so that they can be used in the following way:


 * Reviewed in the Audience Leads meeting on the 4th Tuesday of the month.
 * Posted on Mediawiki for wider availability
 * Compiled on a quarterly basis for presentation in the Board Meeting
 * Compiled and evaluated on a quarterly basis, along with explorations of trends and drivers, for the Quarterly Insights Meeting

Reporting process
We compile each monthly report by the 15th of the following month (or the next business day if the 15th falls on a weekend or holiday).


 * 1) Update the numbers in the movement metrics tables preparation spreadsheet, which has exactly the format used in the final report. When you paste the new numbers in, make sure to "paste values only" so the formatting is preserved.
 * 2) Copy the tables from the spreadsheet into the appropriate places in the key product metrics: board slide deck.If asked, paste them unlinked so we can keep changing the spreadsheet without unintentionally changing the slides for past months.
 * 3) Briefly identify any key trends (the "what") next to the tables; detailed analysis (the "why") can wait until the quarterly insights meeting.
 * 4) Once the slides are completed,
 * 5) write an email to the reports key consumers letting them know it's finished. As of June 2019, these are Kate Zimmerman, Jon Katz, Danny Horn, and Toby Negrin.
 * 6) upload the month's slides to Commons as a PDF (using the March 2019 report as a guide).
 * 7) update Wikimedia Audiences to include the PDF you just uploaded.

Contributors metric calculation process

 * 1) Clone the wikimedia-research/Editing-movement-metrics repo from GitHub onto one of the SWAP hosts.
 * 2) In any order, run the three notebooks numbered
 * 3) 01a-editor-month-table.ipynb: creates or updates an intermediate editor-month table in the  Hive database.
 * 4) 01b-new-editor-table.ipynb: creates or updates an intermediate table of new editors in the   Hive database.
 * 5) 01c-diversity-metrics.ipynb: just calculates Global South new editor retention. This needs to integrated into 02-calculation and past numbers backfilled into metrics/metrics.tsv (T225625).
 * 6) Run the notebook 02-calculation.ipynb, which actually calculates the metrics (some of them using the editor month and new editor tables calculated in the previous step) and inserts them into metrics.tsv.
 * 7) Run the notebook 03-report.ipynb, which does a few simple transformations on the metrics and produces the table of values needed for the final report, as well as a graph of each metric.
 * 8) Do any analysis you need to understand major trends (drawing on the analysis notes in past months' slides if needed). The analysis folder in the metrics repo has a variety of notebook you could reuse; if you do new analysis, considering keeping it in an existing or new notebook in this folder, so it can be reused in the future.