Extension:WikiLambda/Metrics

This page describes the metrics that are currently collected from the WikiLambda extension, and the dashboards on which most of these metrics may be viewed. These metrics are collected to provide information about the level of usage of Wikifunctions and its various features, and the degree of success or difficulty that users may have in working with the different features, which in turn can provide valuable guidance for the further evolution of Wikifunctions.

The datasets from which these metrics are computed are not made public. Because these datasets contain a few types of information that are personally identifiable, such as SUL usernames and device identifiers, they are carefully protected and accessible only to individuals with appropriate permissions. If and when datasets are shared, they will be anonymized; that is, personally identifiable information will be omitted.

Links to relevant Phabricator tickets are given below (e.g. T340786) for most of the metrics categories.

Interaction Metrics
These metrics count up specific types of interactions in the Wikifunctions user interface. They are organized here by Wikifunctions’ Critical User Journeys (such as Evaluate a function and Create a function definition).

1. Evaluate a function (T340786)

 * Number of clicks on Run Function button
 * Breakdown by page:
 * Function page
 * Implementation page
 * Special evaluation function page
 * Test page
 * Function call completion rate (the first divided by the second)
 * Number of successful calls to functions
 * Number of all calls to functions
 * “Function popularity”: How many times each called function gets called

2. Create a function definition (T340160)

 * Number of function-creation attempts started
 * Number of function-creation attempts ending with Publish
 * Number of function-creation attempts ending with Cancel

3. Edit an existing function definition (T340779)

 * Number of function-edit attempts started
 * Number of function-edit attempts ending with Publish
 * Number of function-edit attempts ending with Cancel

4. Create a test (T340783)

 * Number of test-creation attempts started
 * Number of test-creation attempts ending with Publish
 * Number of test-creation attempts ending with Cancel

5. Create an implementation (via code or composition) (T340784)

 * Number of implementation-creation attempts started
 * Number of implementation-creation attempts ending with Publish
 * Number of published implementation-creations using code
 * Number of published implementation-creations using composition
 * Number of implementation-creation attempts ending with Cancel
 * Number of canceled implementation-creation attempts using code
 * Number of canceled implementation-creation attempts using composition

Content (T341748, T341733, T341734)

 * Number of functions created
 * Number of implementations created
 * Number of connected implementations
 * Number of tests created
 * Number of connected tests

Performance

 * Pageload time (patch 924177)
 * Calculations:
 * Average for all pages
 * Average for each type of view (currently 4 distinct types)
 * Averages for viewing existing objects and for editing new objects

Adoption

 * Number of new users
 * Number of active users
 * Number of users whose accounts become inactive
 * Number of unique devices accessing Wikifunctions

Community diversity

 * Number of different languages used in Wikifunctions
 * [Languages with at least one label on at least one Function]
 * Number of different countries represented by users
 * Number of (new, active) users per country