Analytics/Reportcard/2.0/Requirements

From MediaWiki.org
Jump to: navigation, search
"Report card 2.0" is running at http://reportcard.wmflabs.org/

Rationale[edit | edit source]

There is a need for both the foundation and community to see 'at-a-glance' what the current standing is on various projects and a need for a common reporting infrastructure within the foundation for measures we find important. The current report card, as it exists is a monolithic structure and process which ends with very specific data points being summarized for the board and ED. There are parts of data in this report that can be reused and/or combined with other reports to get very specific, relevant reports to other interested parties.

Goals[edit | edit source]

This project will seek to improve the general metrics reporting process in the following ways:

  • Data: extraction, transformation and loading is fully automated, there will be zero manual steps.
  • Granularity: we start with daily aggregated data.
  • Targets: a measurement is benchmarked against a target, this applies to certain reader, mobile and diversity measures.
  • API: there will be a simple API so people can fetch the data and analyze and visualize it themselves.
  • Embedded: there will be a Mediawiki extension that allows you to embed a particular chart in a Wiki page.
  • Modular: the frontend and backend will be very loosely coupled so that it will be possible to just replace the backend in one shot.
  • Interactivity: the charts will offer basic interactivity: zoom-in / zoom-out, indexed vs raw count, add other projects to compare, etc.

User Groups[edit | edit source]

1) The monthly metric meeting where Erik will use it to give a high level overview of the state of the community. User Group 1: C-level.

2) The different departments (Community, Global Dev) and different teams (Mobile) want to have more fine-grained control over their charts and want to be able to 'write' their own queries. User Group 2: Different WMF teams

3) The community at large (community members, admin, researchers, whoever), they will probably want to download the raw JSON data and do extra things. User Group 3: Community (broadly defined)

Devices[edit | edit source]

We will limit the reportcard to modern web-browsers on a desktop / laptop.


Metrics[edit | edit source]

In total, there are six themes that will be visualized on the Dashboard: Reader metrics, Editor metrics, Device metrics, Diversity metrics, Media metrics and API metrics.

Reader-centered measurements[edit | edit source]

  • Pageviews
    • Breakdown by project_language
    • Breakdown by project
  • Unique visitors
    • Breakdown by project_language
    • Breakdown by project
  • Unique visitors competing web properties (Google, Facebook)

Editor-centered measurements[edit | edit source]

  • Count of edits
    • Breakdown by project_language
    • Breakdown by project
  • Count of new editors
    • Breakdown by project_language
    • Breakdown by project
  • Count of active editors (5+ edits per month)
    • Breakdown by project_language
    • Breakdown by project
  • Count of very active editors (100+ edits per month)
    • Breakdown by project_language
    • Breakdown by project

Already supported by WikiPride[edit | edit source]

Currently only supports Wikipedia by language by month. Could be easy to change.

  • Count of edits
  • Count of active editors
  • Count of very active editors

Low priority[edit | edit source]

  • Count of reverted edits
  • Count of bot edits
  • Logins and/or edit sessions per day/week/month

Article-centric measures[edit | edit source]

  • Count of
    • total articles
    • deleted articles
    • new articles per day
  • Breakdown by project_language
  • Breakdown by project

Device-centered measurements[edit | edit source]

  • Count of mobile devices
    • Breakdown by manufacturer
    • Breakdown by project_language
    • Breakdown by project
    • Breakdown by geography
    • Breakdown by partner (Wikimedia Zero)
    • Breakdown by official apps (iOS, Android, Symbian, etc)

Diversity-centered measurement[edit | edit source]

  • Percentage of editors from the Global South
  • Editors / readers from India (data at state level and for the following languages: English, Hindi, Kannada, Malayalam, Bengali, Marathi, Gujarati, Tamil and Telugu)
  • Editors / readers from Brasil
  • Percentage of female editors

Media-centered measurements[edit | edit source]

(This applies mainly to Commons)

  • Count of binary files (jpg, png, svg, ogg, gif, tiff, pdf, djvu, ogv, mid)

API-centered measurements[edit | edit source]

  • Count of different API actions
    • Breakdown by language
    • Breakdown by project

Proposed Database Design[edit | edit source]

You can find the proposed database design at Analytics/Reportcard/Database_design.

Shelved Features[edit | edit source]

This is a list of features that we want but are outside of the current scope

  • Raw data: Instead of storing aggregates, we want to be able to store the raw data.
  • Granularity: Hourly data is something to consider.


External Links[edit | edit source]