Analytics/Development Process

From mediawiki.org

Getting in Touch[edit]

analytics@lists.wikimedia.org is the best way to initiate communications with the team.

This is a public list and is archived at http://lists.wikimedia.org/pipermail/analytics/ .

Another way to make requests is to create a task in the "Analytics" project in Phabricator.

Classification of Work in Backlog[edit]

Description Level of detail
Projects
  • They should have a name that reflects business value and is comprehensible to (non-technical) managers at the WMF
  • They have a single owner (but may have multiple stakeholders)
  • They are the core unit of prioritization (as opposed to execution)
  • Should generally be achievable in a quarter
  • Prioritized once a quarter
low
Epics
  • Describe features of functionality at a high level
  • They are too large to complete in a single sprint
  • Epics are decomposed into User Stories
low
User Stories
  • User Stories are written from the end user's perspective
  • They are discrete units of value driven delivery
  • They follow the syntax "as role, I want to be able to do action, so that benefit"
  • The product owner prioritizes and grooms the backlog of User Stories
  • Their level of effort is small enough that they can be completed in a single sprint
  • Story size & complexity is estimated in points and uses relative values from the Fibonacci sequence {1, 2, 3, 5, 8, 13, 21...}.
high
Tasks the core work units belonging to a User Story
  • Assembled by development team during the story tasking meeting.
  • Measured in hours to implement.
high
Defects bugs high
Spikes units of development reserved for research to support point estimation of a feature or tasking. low

For Example:

Project: Editor Engagement Vital Signs Minimum Viable Product

Epics:

  • CronUser runs report on entire project
  • User runs report for Newly Registered User
  • User runs report for New Editor
  • Analyst selects metrics to graph
  • Analyst selects projects to graph
  • etc.

User Story

  • As an Analyst, I need visualization of session length KPI over period of 1 day, so that I can assess effectiveness of UI

Task

  • implement session start count
  • implement session end threshold
  • calculate session length

Production Issues[edit]

  • Production issues are always top priority
  • If you’re not sure if an issue is a production issue ask peers/managers/directors/VPs
  • We track these against a special Epic called Production Issues - Q(1-4)
  • We track them in a Epic so we can identify patterns, development metrics, etc.
  • Fix the problem first, create the card later

Bug Triage[edit]

  • Defects are logged in Phabricator under the "Analytics" project.
  • Non urgent bugs will be dealt with as part of the regular backlog and backlog grooming process
  • Urgent bugs will be tagged as production issues and dealt with immediately
Priority Reference
Priority Meaning
Needs Triage New backlog item that Product Owner needs to set priority
Unbreak Now! Need to start next sprint
High Need to start in the next sprint
Normal Likely to be started within a month
Low To do, but later
Needs Volunteer new task not considered yet

Backlog States[edit]

The backlog workboard in Phabricator indicates states of items by what swim lane or column the items is in.

New items are collected in the "Incoming" swim lane with a priority of "Needs Triage"

The next column to the right is "Epics." This column is a visual representation of the various Epics, and its placement is unrelated to any backlog state.

The next column to the right is "In Tech Review." Items here have been triaged by the Product Owner and are in review with the Dev team.

The final column on the right is "Ready for Dev." These items have been reviewed by team and are ready for discussion in tasking meeting with the goal of identifying all the underlying tasks and estimating a point value for the story.

From this column, stories will be pulled into sprints.

Once committed to for a given sprint, stories are pulled from this board an into the sprint board where each sprint is an individual project..

Agile Scrum[edit]

Quirks[edit]

  • We have many products and many stakeholders - most Agile perspectives assume a single product
  • Team is completely distributed
  • Team does not work on same project at the same time
  • Infrastructure tasks are not well suited to Agile

Ceremonies[edit]

Ceremonies are meetings that adhere to Agile/Scrum methodology and help coordinate the team, deepen understanding of the requirements and prioritize features. They also set the team's cadence to two week iterations or sprints. Note that a sprint ends on a Tuesday and a new sprint starts on a Thursday. The time in-between is for developers 10% time.

Ceremony Description Frequency
Daily Standup
  • timeboxed 15 minutes
  • every developer answers:
    • what have you done since the last standup?
    • what are you working on until the next standup
    • do you have any blockers?
  • This is a meeting for the team to reinforce commitment to sprint's goals and for Product Owner to ensure that the team is working on the right stuff
  • Outsiders may observe, but they are not allowed to speak
  • This is not an opportunity for management to bring up new requests
Daily - 2pm UTC
Story Tasking
  • User Stories that have gone through Tech Review by the team are prioritized by the Product Owner
  • the team thinks through the tasks to implement the User Story
  • the team votes by a delphi technique to establish a relative estimate of effort to implement the User Story
Weekly - Thursday 2:30pm UTC
Sprint Planning
  • The Product Owner makes request for prioritized User Stories to be pulled into the coming sprint
  • The team decides which stories they can commit to delivering in the sprint
  • The team locks the commitment
Fortnightly - Thursday at 5:00pm UTC
Showcase
  • The Product Owner selects the functionality to demo
  • Stakeholders can see progress and value that sprint produced
Fortnightly - Tuesday at 5:00pm UTC
Sprint Retrospective
  • this is a meeting for just the team (no stakeholders)
  • team reflects on how process worked in the sprint
    • celebrate what went well
    • suggest improvements for what did not go well
Fortnightly - Tuesday at 5:30pm UTC

Planning Poker[edit]

The product owner and scrum master are responsible for having stories prioritized to be estimated by the team at the weekly tasking meeting.

We estimate stories in relative points or effort and tasks in hours. We use a pseudo-Fibonacci sequence to estimate (0,1,1,2,3,5,8,13...). The idea is that the Fibonacci number represents the size of the story. Complexity is a factor when estimating the story but so is logistical and busywork as those add to size.

Examples: A simple story made up of a simple coding task is consider to be a 3. We always include writing tests and testing in staging in our estimation of the coding task. The simplest task you could think of would be a '1' but if it is worthy of a story it should be estimated.

The points help us calculate our velocity or the amount of work we can complete per sprint and shows how are we doing as time and projects move along.

Every story is made of tasks and we estimate tasks in hours, we use hours cause we want to decouple the size of a story (measured in points) versus the duration (measured in hours). Obviously size and duration are related but their ratio is not always identical per story.


Points Scale
34 one person sprint if that person is not interrupted at ALL
21 one person sprint if that person is interrupted a moderate amount
13
8
5
3 simplest coding task
2 task involving some documentation
1 simplest task

Kanban[edit]

phabricator:tag/analytics-kanban/

Tooling[edit]

Tool Purpose
Wiki
  • Documentation
Phabricator
  • Stories/Features, their description, scope and point estimate.
  • Task
  • Defect Tracking
Etherpad
  • Tasking out stories and estimating number of hours per task.
Google Docs Spreadsheet
  • Rough count of number of context switches from committed sprint work to take care of unplanned work. This document is updated during the standup by the scrum master.