Analytics/Epics/Accurate Pageviews for Wikipedia Zero

From mediawiki.org

Goals[edit]

Wikipedia Zero is a program where we work with Mobile Carriers to provide access to Wikipedia resources without data charges.

We are contractually obligated to provide data on the performance on Wikipedia Zero (W0) to our partners. The W0 Team would also like information about the program's usage. We are currently providing dashboards for integrated Carriers here.

It's not totally clear to me what we are actually obliged to provide and we should work with that team to come up with the list. For the time being, we'll use the functionality that we currently expose as a proxy for requirements.

Detailed Tracking Links[edit]

Development (Mingle)

Users[edit]

User Description
W0 Management The W0 Management team who makes commitments to the Carriers and uses the data to measure effectiveness of the programs
Carriers The Mobile Telecom companies who agree to provide Wikipedia free of data charges
Analytics Developers The people who write the software that counts and displays the W0 Requests
Analytics Operators The people who ensure the counting software is running and the data is updated
W0 Developers The people who integrate the Carriers with the W0 serving infrastructure
Operators The people who work with W0 developers integrating the Carriers with the W0 serving infrastructure
Communications WMF personnel who use the data in monthly reports
Community The Wikipedians who look at the data to assess their success and the health of the community

Prioritized Use Cases[edit]

High Probability[edit]

  1. As W0 Management, I need to provide my Carrier partners with usage data about their W0 Integrations.
    1. The makeup of this usage data is detailed in the metrics section below.
  2. As a Carrier, I want to know how successful my W0 programs are.
  3. As a Analytics Developer, I need definitions of W0 metrics that are relatively static.
  4. As a Analytics Developer, I need a heads-up when these definitions changes so I can deal with any impact to the metrics.
  5. As a Analytics Developer and W0 Developer, I need a well-documented process to bring new carriers on line.
  6. As an Analytics Operator, I need to documentation of the data flows so I can accurately monitor the service.
  7. As an W0 and Analytics Developer and a Carrier, I need to know that the usage data is accurate.

Need Confirmation[edit]

  1. As a Carrier, I want my usage information to be private.
  2. As W0 Mgmt and a Carrier, I want my data to be visualized.
  3. As W0 Mgmt, I want historical data to be maintained in a format that can be queried easily.

Not Prioritized[edit]

  1. As an W0 Developer, I want near real-time data to help debug configuration operations.

Non functional requirements[edit]

  1. All dashboards should be updated daily.
  2. Once we have signoff on the basic issues described below, all issues with dashboards should be addressed within 2 "business" days of the problem reports.
  3. Data be retained indefinitely

Metrics[edit]

The required metrics for the project are the following:

  • Total Mobile Pageviews
  • Total Mobile Pageviews not delivered by W0
  • Total Mobile Pageviews delivered by W0

I am just looking at the dashboards and trying to figure out what the metrics mean. We need to add some more documentation to the dashboards

The breakdowns required are:

  • Total
  • By Carrier

There is a list of the current carriers here -- it was last updated in July so I'm not sure what the freshness is:

https://office.wikimedia.org/wiki/Wikipedia_Zero_Partner_Dashboard_Status

Outstanding Business Questions for W0 Team[edit]

  1. What are we actually required to deliver to Carriers?
  2. Where is the current list of Carriers? (both integrated and upcoming)

Outstanding Technical Issues for W0 Team[edit]

  1. What is the definition of a page view?
  2. How can we streamline/document the proxy handling mechanism?
  3. How do we test the new tagging mechanism before deployment?

See Also[edit]

Mingle Requests