Extension:WikiLambda/Metrics

This page describes the usagecs that are currently collected from the WikiLambda extension. These metrics are collected to provide information about the level of usage of Wikifunctions and its various features, and the degree of success or difficulty that users may have in working with the different features, which in turn can provide valuable guidance for the further evolution of Wikifunctions.

The datasets from which these metrics are computed are not currently made public. Because these datasets contain a few types of information that are personally identifiable, such as WMF usernames and device identifiers, they are carefully protected and accessible only to individuals with appropriate permissions. If and when datasets are shared, they will be anonymized; that is, personally identifiable information will be omitted.

Links to relevant Phabricator tickets are given below (e.g., T340786) for most of the metrics categories.

= Interaction Metrics = These metrics count up specific types of interactions in the Wikifunctions user interface. They are organized by Wikifunctions’ Critical User Journeys (such as Evaluate a function and Create a function definition).

1. Evaluate a function (T340786)

 * Number of clicks on Run Function button
 * Breakdown by page:
 * Function page
 * Implementation page
 * Special evaluation function page
 * Testing page
 * Function call completion rate (the first divided by the second)
 * Number of successful calls to functions
 * Number of all calls to functions
 * “Function popularity”: How many times each called function gets called

2. Create a function definition (T340160)

 * Number of function-creation attempts started
 * Number of function-creation attempts ending with Publish
 * Number of function-creation attempts ending with Cancel

3. Edit an existing function definition (T340779)

 * Number of function-edit attempts started
 * Number of function-edit attempts ending with Publish
 * Number of function-edit attempts ending with Cancel

4. Create tests (T340783)

 * Number of test-creation attempts started
 * Number of test-creation attempts ending with Publish
 * Number of test-creation attempts ending with Cancel

5. Create implementations (via code or composition) (T340784)

 * Number of implementation-creation attempts started
 * Number of code implementation attempts started
 * Number of code implementation attempts ending with Publish
 * Number of code implementation attempts ending with Cancel
 * Number of composition implementation attempts started
 * Number of composition implementation attempts ending with Publish
 * Number of composition implementation attempts ending with Cancel

= Product Metrics =

Content (T341748, T341733, T341734)

 * Number of functions created
 * Number of implementations created
 * Number of connected implementations
 * Number of tests created
 * Number of connected tests

Performance

 * Pageload time (patch 924177)
 * Calculations:
 * Average for all pages
 * Average for each type of view (currently 4 distinct types)
 * Averages for viewing existing objects and for editing new objects

= Global Metrics =

Adoption

 * Number of new users
 * Number of active users
 * Number of users whose accounts become inactive
 * Number of unique devices accessing Wikifunctions

Community diversity

 * Number of different languages used in Wikifunctions
 * [Languages with at least one label on at least one Function]
 * Number of different countries represented by users
 * Number of (new, active) users per country