Community metrics

How is the MediaWiki / Wikimedia tech community doing? Let's analyze the data available in order to highlight the contributors and areas setting an example, and also the bottlenecks or inactive corners requiring our attention.

Your feedback and requests are welcome in Phabricator (project Analytics-Tech-community-metrics). You can also comment at the discussion page and at the Analytics mailing list.

Median age of open changesets
"Time from last patchset" in days at "Age of open changesets (monthly snapshots)".

{ "width": 200, "height": 350, "padding": {"top": 10, "left": 30, "bottom": 30, "right": 10}, "data": [ {     "name": "table", "values": [ {"x": 1502, "y": 52.1}, {"x": 1503, "y": 61.0}, {"x": 1504, "y": 63.7}, {"x": 1505, "y": 44.2}, {"x": 1506, "y": 57.3}, {"x": 1507, "y": 64.7}, ]   }  ],  "scales": [ {     "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "data.x"} },   {      "name": "y", "range": "height", "nice": true, "domain": {"data": "table", "field": "data.y"} } ],  "axes": [ {"type": "x", "scale": "x"}, {"type": "y", "scale": "y"} ], "marks": [ {     "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "data.x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "data.y"}, "y2": {"scale": "y", "value": 0} },       "update": { "fill": {"value": "steelblue"} },       "hover": { "fill": {"value": "red"} }     }    }  ] }

Open changesets waiting for review
"Waiting for review" at "Backlog of open changesets (monthly snapshots)"

{ "width": 200, "height": 350, "padding": {"top": 10, "left": 40, "bottom": 30, "right": 10}, "data": [ {     "name": "table", "values": [ {"x": 1502, "y": 1162}, {"x": 1503, "y": 1213}, {"x": 1504, "y": 1230}, {"x": 1505, "y": 1364}, {"x": 1506, "y": 1226}, {"x": 1507, "y": 1250}, ]   }  ],  "scales": [ {     "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "data.x"} },   {      "name": "y", "range": "height", "nice": true, "domain": {"data": "table", "field": "data.y"} } ],  "axes": [ {"type": "x", "scale": "x"}, {"type": "y", "scale": "y"} ], "marks": [ {     "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "data.x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "data.y"}, "y2": {"scale": "y", "value": 0} },       "update": { "fill": {"value": "steelblue"} },       "hover": { "fill": {"value": "red"} }     }    }  ] }

New changesets submitted per month
The "submitted" line in "submitted vs. Merged changes vs. Abandoned".

{ "width": 200, "height": 350, "padding": {"top": 10, "left": 40, "bottom": 30, "right": 10}, "data": [ {     "name": "table", "values": [ {"x": 1502, "y": 2758}, {"x": 1503, "y": 3484}, {"x": 1504, "y": 3257}, {"x": 1505, "y": 3044}, {"x": 1506, "y": 3316}, {"x": 1507, "y": 2833}, ]   }  ],  "scales": [ {     "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "data.x"} },   {      "name": "y", "range": "height", "nice": true, "domain": {"data": "table", "field": "data.y"} } ],  "axes": [ {"type": "x", "scale": "x"}, {"type": "y", "scale": "y"} ], "marks": [ {     "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "data.x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "data.y"}, "y2": {"scale": "y", "value": 0} },       "update": { "fill": {"value": "steelblue"} },       "hover": { "fill": {"value": "red"} }     }    }  ] }

Active Gerrit code review users per month
Uploaders, Reviewers, Committers

{ "width": 400, "height": 350, "padding": {"top": 10, "left": 30, "bottom": 30, "right": 10}, "data": [ {     "name": "table", "values": [ {"id": "uploaders", "date": "Feb 2015", "amount": 211}, {"id": "uploaders", "date": "Mar 2015", "amount": 221}, {"id": "uploaders", "date": "Apr 2015", "amount": 207}, {"id": "uploaders", "date": "May 2015", "amount": 214}, {"id": "uploaders", "date": "Jun 2015", "amount": 212}, {"id": "uploaders", "date": "Jul 2015", "amount": 197}, {"id": "reviewers", "date": "Feb 2015", "amount": 190}, {"id": "reviewers", "date": "Mar 2015", "amount": 185}, {"id": "reviewers", "date": "Apr 2015", "amount": 186}, {"id": "reviewers", "date": "May 2015", "amount": 183}, {"id": "reviewers", "date": "Jun 2015", "amount": 190}, {"id": "reviewers", "date": "Jul 2015", "amount": 194}, {"id": "committers", "date": "Feb 2015", "amount": 127}, {"id": "committers", "date": "Mar 2015", "amount": 122}, {"id": "committers", "date": "Apr 2015", "amount": 122}, {"id": "committers", "date": "May 2015", "amount": 125}, {"id": "committers", "date": "Jun 2015", "amount": 132}, {"id": "committers", "date": "Jul 2015", "amount": 132}, ],   }  ],  "scales": [ {     "name": "x", "type": "ordinal", "range": "width", "domain": { "data": "table", "field": "data.date" }   },    {      "name": "y", "type": "linear", "range": "height", "nice": true, "domain": { "data": "table", "field": "data.amount" }   },    {      "name": "color", "type": "ordinal", "range": "category10" } ],  "axes": [ {     "type": "x", "scale": "x", "tickSizeEnd": 0 },   {      "type": "y", "scale": "y" } ],  "marks": [ {     "type": "group", "from": { "data": "table", "transform": [ {           "type": "facet", "keys": [ "data.id" ]         }        ]      },      "marks": [ {         "type": "line", "properties": { "enter": { "x": { "scale": "x", "field": "data.date" },             "y": { "scale": "y", "field": "data.amount" },             "stroke": { "scale": "color", "field": "data.id" },             "strokeWidth": { "value": 2 }           }          }        },        {          "type": "text", "from": { "transform": [ {               "type": "filter", "test": "index==data.length-1" }           ]          },          "properties": { "enter": { "x": { "scale": "x", "field": "data.date", "offset": 2 },             "y": { "scale": "y", "field": "data.amount" },             "fill": { "scale": "color", "field": "data.id" },             "text": { "field": "data.id" },             "baseline": { "value": "middle" }           }          }        }      ]    }  ] }

Active users in Phabricator
Monthly active users in Bugzilla (from 2013-02 to 2014-10) and in Phabricator (from 2014-12 to last month).

{ "width": 1000, "height": 350, "padding": {"top": 10, "left": 30, "bottom": 30, "right": 10}, "data": [ {     "name": "table", "values": [ {"x": 1302, "y": 450}, {"x": 1303, "y": 495}, {"x": 1304, "y": 511}, {"x": 1305, "y": 532}, {"x": 1306, "y": 505}, {"x": 1307, "y": 542}, {"x": 1308, "y": 530}, {"x": 1309, "y": 492}, {"x": 1310, "y": 539}, {"x": 1311, "y": 508}, {"x": 1312, "y": 540}, {"x": 1401, "y": 537}, {"x": 1402, "y": 527}, {"x": 1403, "y": 533}, {"x": 1404, "y": 478}, {"x": 1405, "y": 470}, {"x": 1406, "y": 509}, {"x": 1407, "y": 501}, {"x": 1408, "y": 515}, {"x": 1409, "y": 475}, {"x": 1410, "y": 491}, {"x": 1411, "y": 0}, {"x": 1412, "y": 619}, {"x": 1501, "y": 669}, {"x": 1502, "y": 733}, {"x": 1503, "y": 838}, {"x": 1504, "y": 766}, {"x": 1505, "y": 775}, {"x": 1506, "y": 794}, {"x": 1507, "y": 797}, {"x": 1508, "y": 831}, ]   }  ],  "scales": [ {     "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "data.x"} },   {      "name": "y", "range": "height", "nice": true, "domain": {"data": "table", "field": "data.y"} } ],  "axes": [ {"type": "x", "scale": "x"}, {"type": "y", "scale": "y"} ], "marks": [ {     "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "data.x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "data.y"}, "y2": {"scale": "y", "value": 0} },       "update": { "fill": {"value": "steelblue"} },       "hover": { "fill": {"value": "red"} }     }    }  ] }

New accounts in Phabricator
New Phabricator accounts created every month. FIXME: find data from September and October 2014.

{ "width": 1000, "height": 350, "padding": {"top": 10, "left": 30, "bottom": 30, "right": 10}, "data": [ {     "name": "table", "values": [ {"x": 1409, "y": 0}, {"x": 1410, "y": 0}, {"x": 1411, "y": 299}, {"x": 1412, "y": 395}, {"x": 1501, "y": 305}, {"x": 1502, "y": 317}, {"x": 1503, "y": 388}, {"x": 1504, "y": 260}, {"x": 1505, "y": 248}, {"x": 1506, "y": 265}, {"x": 1507, "y": 264}, {"x": 1508, "y": 288}, ]   }  ],  "scales": [ {     "name": "x", "type": "ordinal", "range": "width", "domain": {"data": "table", "field": "data.x"} },   {      "name": "y", "range": "height", "nice": true, "domain": {"data": "table", "field": "data.y"} } ],  "axes": [ {"type": "x", "scale": "x"}, {"type": "y", "scale": "y"} ], "marks": [ {     "type": "rect", "from": {"data": "table"}, "properties": { "enter": { "x": {"scale": "x", "field": "data.x"}, "width": {"scale": "x", "band": true, "offset": -1}, "y": {"scale": "y", "field": "data.y"}, "y2": {"scale": "y", "value": 0} },       "update": { "fill": {"value": "steelblue"} },       "hover": { "fill": {"value": "red"} }     }    }  ] }

korma.wmflabs.org
korma.wmflabs.org is the Wikimedia Tech community metrics dashboard. It has been under development since June 2013.

The data is refreshed on a daily basis. The data sources include Git and Gerrit repositories, Phabricator's Maniphest (and Bugzilla before), mediawiki.org, mailing lists, and IRC.

Korma is powered by open source projects Metrics Grimoire and Viz Grimoire. You can find the development specific to the Wikimedia tech dashboard in GitHub.

Bugs and feature requests can be reported under the Analytics-Tech-community-metrics project.

Git
ssh -p 29418 gerrit.wikimedia.org gerrit ls-projects | grep "mediawiki/extensions
 * The source code repos analyzed are mediawiki/core and all the mediawiki extensions:
 * FIXME: This is only a portion (a big one, yes) of all the repositories we need to scan. The default is everything at gerrit.wikimedia.org but let's look at every repo before adding it just in case.

Code review
Korma offers code review data based on a selection of repositories scanned on a daily basis at gerrit.wikimedia.org. Specifically:
 * gerrit_trackers.conf contains a list of repositories retrieved using a command similar to: . We are planning to maintain this list automatically (T104845).
 * gerrit_trackers_blacklist.conf contains a list of blacklisted repositories that is manintained manually. These are repositories that we don't want to compute against our metrics, and they will be ignored as soon as they are added to the blacklist: upstream projects, empty or deprecated repositories, and other exceptional cases.

In addition to this, if a project was removed from the gerrit list, the script will detect the change and the project will be removed from Korma's database automatically.

Issue tracking
Since the move from Bugzilla to Phabricator in 2014, only basic data is available in the Metrics dashboard. For potential future data, see T28.

mediawiki.org

 * The wiki activity is analyzed using an analyzer tool including editions, editors and pages. Results and discussion.

Mailing lists

 * FIXME - mailing lists missing?
 * FIXME - Is it possible to specify the number of subscribers?

IRC

 * Pending to better define the channels to be added to the dashboard in T56230.

How to update user data
Our goal is to provide a tool allowing users to edit their own data directly (T60585). Meanwhile, users can request updates to their personal data creating a Phabricator task including:
 * real name
 * username(s) and email address(es) used for your contributions
 * current and previous affiliations, with the dates of change of affiliation
 * Current location (country)

At the moment we can only process single affiliations (T95238). If you are contributing from different affiliations (i.e. Wikimedia Foundation as part of your work, Independent in your free time), then we recommend you to use different usernames and email addresses.

Managing identities
SortingHat is the tool to manage identities. This helps in the following way:
 * To centralize all information in a database.
 * To deal with several identities: a developer may have several identities depending on the data source she is working on. This tool helps to identify for each identity of a developer where that information came from.
 * To avoid the use of direct database: a command line interface deals with ITS.
 * To manage extra developer attributes: it has support for managing affiliations and other developer attributes such as nationalities or bot activity.
 * To manage black lists: this is typically used in cases where bots are committing changes, or too generic names or emails addresses such as "root".

The process to merge all of the identities into one database could be done in two ways: a more detailed one, or an incremental one. The first process is done through the use of extra scripts to parse such information. However this is a heavy-time process and this is typically used in the first identities database creation. Later updates of the database typically follows the second step.

SortingHat also provides a way to export all of this data. This helps to look for other developer identities and merge them through the command line.

These exported JSON files follows the same structure:

As an example, if an identity is required to be merged with another identity, the command "sortinghat merge" is used and the original ".identities.id" is merged into the specified "".

The most useful SortingHat commands to deal with identities are the following ones:
 * sortinghat merge: to merge unique identities
 * sortinghat affiliate: to affiliate an identity to some organization
 * sortinghat show: to show information about an identity
 * sortinghat profile: to show profile information of that unique identity

The user pages of contributors are linked from the contributor names in the top tables for each data source section.

Once unique people exist, other categories are created using them. For example, affiliation classification is done initially with a script that uses email domains if available. The classification supports periods of time to cover that a unique people has worked for several companies. There is some experimental support also for countries.
 * FIXME - Contributors must be linked to WMF and other orgs.
 * FIXME - Is By country relevant? Do we want to gather that data?
 * FIXME - Plan for linking this data to user profiles? Where?

Other data sources and tools
Git
 * Wikimedia stats in OpenHub/Ohloh including many projects.
 * "How many unique contributors submitted unique pull requests to a https://github.com/wikimedia/ repo" - Python script by marktraceur.

Gerrit
 * Gerrit/Navigation
 * MediaWiki Gerrit stats  (Is it working? 2013-06-28)  and how to query Gerrit data.
 * Number of gerrit committers (marktraceur's bash script)
 * cmd-query for Gerrit.

Phabricator
 * "Phabricator monthly statistics" emails on the wikitech-l mailing list - see its archives.

mediawiki.org
 * monthly Statistics of page views and how the data is gathered.

Mailman
 * Wikimedia Mail Stats: PowerPosters.

Team
Quim Gil and Andre Klapper from the Wikimedia Engineering Community team are coordinating the Metrics Dashboard project, which is being implemented by Bitergia as contractors.

The Bitergia team working in the MediaWiki dashboard is formed by Daniel Izquierdo, Luis Cañas and Jesus Gonzalez Barahona and Alvaro del Castillo as project manager.

The ownership of this project might get transfered to the Wikimedia Analytics team at some point.