Team Practices Group/Improving burndown charts

From mediawiki.org
The Team Practices Group (TPG) was dissolved in 2017.

Introduction[edit]

This page is a work in progress, reflecting current thinking. It is not authoritative in any way.

Raw kickoff meeting notes: https://etherpad.wikimedia.org/p/PhabGraphKickoff

Terminology[edit]

  • Burndown Chart
  • Burnup Chart
  • Phabricator
    • The issue-tracking system used by the WMF
  • Phabricator Sprint Extension
    • Adds "Sprint" type projects, which add a "Story Points" field to any task in that project
    • Written by the WMF (?)
  • Phragile
    • A tool that generates graphs of data pulled from phabricator's API
  • Release
    • A "marketing" feature-driven release, OR a timeboxed release
    • For feature-driven, the question we want to answer is "When?"
    • For timeboxed, the question is "How much can we get done?"
  • Release Burnup Chart (or Product Burnup Chart)
    • A burnup chart whose scope spans multiple sprints
  • Release Cumulative Flow Chart
  • Sprint Burnup Chart
    • A burnup chart whose scope is limited to a single sprint

Assumptions[edit]

  • Burnup charts are fine (in lieu of Burndown charts)
  • Product/Release charts are the focus of this initiative (as opposed to Sprint charts)
  • Forward-looking prediction is the focus of this initiative (as opposed to retrospective)
  • An external chart generation tool is acceptable (it doesn't have to be built into phab)
  • At least for now, it would be acceptable for phab to export raw data, to allow fancy charts to be generated by a spreadsheet
  • We have some (unknown $) budget to spend on upstream Phabricator coding, as needed
  • We have the ability to configure our phab instance however we wish
  • We have limited WMF human hours available for phab coding
  • Any phab coding we do should be in the form of patches pushed upstream
  • Is backward-looking important, given that projects wouldn't have used the necessary conventions???
  • Long-term reliance on the Sprint Extension is problematic
    • We need Task.Estimate (aka "Story Points"), and Project.StartDate/EndDate
    • (Not sure what else Sprint Extension provides that we need

Current State[edit]

  • Sprint extension
    • Developed by Christopher at WMDE
    • Requires ongoing maintenance to remain compatible with upstream changes
    • Therefore requires testing with each new upstream release
    • Christopher doesn't have time to maintain it
    • Mukunda can maintain as needed, but not ideal
    • We really need to get sprint-y features in upstream (Story points, start/end dates)
  • Phabricator's built-in burnup charts (part of Sprint Extension):
    • Reports/Burnup, filter by a single project
    • Does not track when tasks were added/removed to/from the project
    • Odd pink color choice
    • In FF on Linux, scrolling down leaves the graph grid behind
  • Phabricator's built-in burndown charts (phact?):
    • Work OK within a single Sprint, but require a ton of work to span multiple sprints
    • Trigger on a task being in a "Done" column, which not all teams use
      • But note that "resolved" is also problematic
    • Generates Burndown charts, not burnup charts
    • Does not show the scope line moving up or down as scope changes--it always just shows as a flat line, at the *current* scope level
    • Handles weekends in ways that some people dislike, but that's really a Sprint chart issue more than a Release chart issue
  • Phact:
    • Upstream reporting tool, with a "burnup chart" feature
    • Disabled in our instance.
    • Chase: Phact is in "prototype" mode, effectively disowned by upstream
    • Generally phab doesn't have good reporting, so phact was a response to that
    • But they didn't have clear requirements, so it has gone stale
    • We could push for phact to take on a reporting role, but it doesn't seem to have much of use to build on
  • Phragile:
    • Is being developed by wmde
    • Provides ??? charts (I haven't seen examples yet)
    • Is an external tool, which would be acceptable
    • Would need to interface with phab authentication/authorization system?
      • To be able to handle security issues, yes, but for normal projects, is this true?
    • Currently provides BOTH graph generation AND sprint creation
      • Should these be separated?
  • Robla's "wbstatus" script:

Project Roles[edit]

  • Kevin: Product Owner.
  • Mukunda:
    • could code in phab
    • maintain phab in general (features, integration, deployment)
    • could maintain/fix the sprint extension as needed
      • Complexity of integrating Sprint extension with upstream changes (we have no Continuous Integration of Phab+Sprint extension)
  • Greg:
    • OK'ing Mukunda's time
    • Delegating interface w/Evan
    • Neither an implementor nor a (knowledgeable) consumer
  • Arthur:
    • Shepherding, supporting
    • Conduit/buffer as necessary for exec requirements
  • Chase:
    • Ops eng
    • mostly 'historically significant'
    • dealt with deployments previously (handing off to Mukunda)
    • much experience dealing with upstream
    • is pregnant ;)

Issues to Remember[edit]

  • Backlog stories may not have estimates
    • Assign arbitrary value (e.g. average story size)?
  • "Release" might mean fixed-date/train, or feature-driven
  • Not all teams will have "sprints"--some are Kanban
  • Need to avoid double-counting when task and subtasks were both estimated
    • Could be an issue both for "work completed" and for "target scope" calculations

Data Model[edit]

  • It is "relatively easy" to expose other db elements via new custom Conduit API calls
  • Helpful phab queries, if they are/were possible:
    • List all tasks that were in Column C of Project P on Date D (NO?)
    • List all projects matching wildcard name search (YES?) [Maybe not necessary]
    • List all tasks which were EVER in Project P (NO?)
    • List all transactions for Task T, with timestamps (YES except not sure about timestamps)
    • List all transactions for Project P, with timestamps (NO?)
  • If we did a nightly snapshot of all tasks in Project P,Q,R..., then where would we store that data
  • Can a Release project also be a Sprint project? (YES)
    • (would be nice to have estimated future epics only be in the Release project, and not to force them to also exist in a Sprint project that isn't a real Sprint)

Desired Datasets/Charts[edit]

  • Points "done" per sprint over time (velocity)
    • Note: Should scale by sprint length, to accommodate Kanban teams
    • Note: Is this based on a "done" column, or marked "resolved"?
  • Current points of all tasks in all columns of all sprints PLUS non-sprint backlog
  • Historical points of all tasks in all columns of all sprints PLUS non-sprint backlog
  • Average task estimate across multiple sprints (or maybe just within each sprint?)
    • To provide a placeholder size for unestimated tasks
  • Average Lead Time per story (from entering a sprint backlog until being "done")
    • Especially helpful for Kanban projects

Possible Solution(s)[edit]

  • Completed points are calculated from a set of wildcard-filtered Sprint projects
    • CONVENTION: Tasks are considered completed if they are in a workboard column named "Done"
      • Graph-requesting user could specify the column name when they give the wildcard spec
    • Brute-force algorithm:
      • For each matching Sprint project,
      • Sum story points of all tasks in the "Done" column
      • Note the timestamp at which that Sprint ended (assuming it is in the past)
  • Scope of release is calculated from a Release project
    • FEATURE REQUEST: New Phab API call: List all transactions that refer to a specific project
    • CONVENTION: The Release project must contain *every* task that is part of that release
    • Any task with subtasks must exclude any subtask estimates from its own current estimate
    • Brute-force algorithm:
      • Given a Release project, find all transactions affecting it
      • Create event history of tasks being added/removed on that project
      • Replay the history to know which tasks were in that project at any timestamp
      • Record the story point sum at desired timestamps
  • Phragile, with the following changes:
    • Ability to view graphs for sprints not created by phragile (e.g. existing sprints)
    • Make sure it will work with a months-long sprint
    • Burnup charts instead of burndown
      • NOTE: Should use historical estimates, as well as historical statuses
      • Must include tasks which used to be in the sprint, but no longer are
    • Calculate scope line based on points in the sprint at that time
      • NOTE: Must use the historical estimates, as well as historical statuses
      • Must include tasks which used to be in the sprint, but no longer are
      • Need a rule to handle unestimated tasks (TBD)
    • Ability to export underlying data that built the chart
      • Allows power users to generate alternative graphs
    • Make the "ideal points" line optional
    • Would prefer subset of the app which only does anonymous queries
      • For increased security
    • Would prefer subset of the app that doesn't require its own database
      • For simpler deployment and less ongoing maintenance
      • Only feature to drop would be snapshots?
Retrieved from "https://www.mediawiki.org/w/index.php?title=Team_Practices_Group/Improving_burndown_charts&oldid=5212418"