VisualEditor/Backlog design

We track our team backlog using the VisualEditor project in Phabricator. This page explains how exactly we structure it.

Tranches
We use fixed-scope tranches to group our upcoming work. The lower the tranche number, the higher its priority. Tranche 0 will never be closed (because there will always be bugs and breaking changes to our dependencies), but all the others eventually will when the work is finished. Once that happens, we won't reuse tranche numbers, but instead create a tranche 5, tranche 6, and so on.

Tranche 0: Interrupt
This covers the constant flow of work needed to keep the production service running within our quality criteria. These are:
 * No loss/corruption of existing article data
 * No loss/corruption of the user's contribution
 * No security issues
 * No regressions in performance
 * No regressions in usability

Tranche 1: Release support
Support community-requested deployments, and urgent key usability improvements as they arise

Tranche 2: Mobile MVP
Improve VisualEditor for mobile tablets and phones

Tranche 3: Language support
Expand VisualEditor's content language support to cover: … and then others from "phase 6" languages.
 * 1) Japanese
 * 2) Korean
 * 3) Arabic
 * 4) Persian
 * 5) Hindi
 * 6) Cantonese

Tranche 4: Link editor tweaks
Improve the link editing experience

Assigning priority
Key issues: Rating: Features:
 * Functionality regression
 * Performance regression
 * Data corruption
 * Security issue
 * Crash of entire editor
 * Crash of bits of editor (e.g. copy & paste stops working)
 * Very common -> Unbreak now! if a key issue, Normal if not
 * Common -> High if a key issue, Normal if not
 * Sometimes -> Normal if a key issue, Low if not
 * Rare / very rare -> Low if a key issue, Lowest if not
 * On list for this quarter -> High
 * On list for this year -> Normal
 * On list for next year -> Low
 * Not on list -> Lowest

Story points
We give each task story points as a very rough way of quantifying how much person-time we expect the task to take (including the time it takes to triage, investigate, and respond to the bug).
 * 1 point: 1–3 person-hours
 * 8 points: 1–3 person-days
 * 40 points: 1–3 person-weeks
 * 160 points: 1–3 person-months

Process

 * 1) Make the minimum number of changes to the VE Phabricator data necessary to produce:
 * 2) A stack-ranked backlog suitable for discussion and planning by most stakeholders
 * 3) A Product Burnup chart
 * 4) Don't (yet) try to do anything with orthogonal categorization of VE tasks (for example, don't try to tag all mobile tasks vs all desktop tasks, and/or all security vs all performance).  May come back to this later but not needed yet.
 * 5) Don't try to create a new project to contain a full stack-ranked list of all VE bugs (well, stack-ranked from 1-100).
 * 6) Reason 1: huge amount of Phab spam if we bulk-move 1300+ active and 3000+ closed tasks.
 * 7) Reason 2: Don't explicitly need it yet
 * 8) Do create tranches in order to organize the backlog.
 * 9) A tranche is an ordered set of all Stories and Epics (Tasks, in Phab) which the VE team intends to complete prior to all Stories and Epics in all lower-ranked tranches.
 * 10) For now, each tranche is intended to address a group of related issues.  (I.e., each tranche is effectively the top-priority tasks from a set of all tasks within a category, but the full list of all tasks in each category is not yet explicitly documented within Phab).
 * 11) Tranches are implemented as columns within the VisualEditor project in Phabricator.
 * 12) VisualEditor shall always have an open Tranche 0, defined as "Interrupt", tasks which are urgent enough to jump ahead of all other planned work.
 * 13) Tranche 0 will also be stack-ranked.
 * 14) Tranche 0 will include (among other things) critical bugs for Wikitext, VisualEditor, and any other projects the VE team is responsible for.
 * 15) Logically, therefore, work on Tranches 1+ will only occur when Tranche 0 is empty or when the VE team is sufficiently staffed to have capacity available after engaging with all Tranche 0 tasks.
 * 16) All numbered tranches should be maintained fully stack-ranked
 * 17) TR0, and the next two active Tranches should be carefully ranked; other tranches may be loosely ranked.
 * 18) As a tranche is completed, it is hidden but not deleted/emptied. So Tranche 1 will disappear soon, and eventually "Tranche 20" will be the new "Tranche 1".  To be clearer, we will use "Interrupt Tranche" for Tranche 0, "Primary Tranche" for the next open Tranche, "Secondary Tranche" the one after that, etc.
 * 19) There is no intentional ranking for any tasks that are not in Tranches
 * 20) The existing VisualEditor columns (of the form "Next Up:"  in Phabricator) may serve as a card mapping area for items before they are assigned up to a tranche or down to the unordered lower backlog.
 * 21) This area may come and go.  Currently, we don't see a need for it in this SOP.
 * 22) The complete, stack-ranked list of VisualEditor tasks may therefore be derived from this algorithm:
 * 23) All open tasks in Tranche (0), in "natural order" as stored on the Phabricator board
 * 24) All open tasks in Tranche 1, in natural order
 * 25) All open tasks in Tranche 2, in natural order.
 * 26) Etc through all numbered tranches.
 * 27) Note that technically, Tranche 1 is still more important than Tranche 20 even when tranches 1-19 are all Done and closed and hidden.  This seems purely theoretical but may matter for reporting.
 * 28) All open tasks in any "Next Up" columns, in no order (no order with columns and no order between columns)
 * 29) All open tasks in To Triage and in Backlog, in no order
 * 30) We will try to construct a Product burnup from a Phab data dump of this arrangement and refine the plan if that is not possible

Use Cases

 * 1) Developer: Use backlog to determine next work to do
 * 2) Product Owner: Prioritize work to meet goals
 * 3) Any Stakeholders: view burnup to assess capability, work load, and forecast times for team work.
 * 4) Active Stakeholders: view backlog identify work coming up and plan around it. (via roadmap?)
 * 5) Goals-based burnup, to compare with tranche-based burnup, to see if they align
 * 6) For gap analysis of goals vs tasks
 * 7) Goals with no task:
 * 8) Should we be doing more to accomplish this goal?
 * 9) Should we cut this goal?
 * 10) Tasks with no goal:
 * 11) Should we be doing this task?  At this priority?
 * 12) Should we have a new goal (make explicit a goal that is currently implicit)?
 * 13) If so, what should its priority be, and how does that affect resourcing?
 * 14) Any stakeholder: use a URL to have permanent reference to work requested or planned by the Foundation or movement.

Open questions

 * 1) How far can an Epic go before it must be decomposed?
 * 2) * Option: No epics in TR0, Primary Tranche, Secondary Tranche. <-- Decided
 * 3) * After an Epic is broken out, the Epic tag should be removed and it should become either closed (since it's wholly juiced out) or a Tracker (if it retains some integration work, which must still be <= 1 task worth of work)
 * 4) What's the definition of an Epic for VE?
 * 5) * Will take one member of VE staff more than a few days of solid work. In practice, over 8 points.  Decided.
 * 6) How is pre-coding work (requirements, design) accounted for?
 * 7) * Option A: Part of continuous "Backlog Grooming" that is assumed to take a fixed percentage of Product Owner's time as overhead. Not counted in points.
 * 8) * Option B: separate "Grooming" story for preparing one or more other stories to be shovel-ready. Pointed and forecast.
 * 9) * Option C: considered part of the lifecycle of a story, so that a story should be created as early as possible after basic scope definition, and the story assigned to appropriate people in sequence (such as designer, User Research, etc) and pointed and forecast appropriately. <-- Decided
 * 10) ** Some of option A is inevitable; e.g., bulk work to prioritize or define 50 tasks in one 30-minute sitting is not pointed or tracked.
 * 11) ** If anyone non-Product Person (James or Neil) is spending time grooming or working on a Task (good rubric: is it assigned to them?), it should be a Task or part of another Task, be pointed, and be estimated.
 * 12) * Option D: Spikes, which are unpointed stories that are identified and prioritized, but not estimated or forecasted.
 * 13) What are we doing with the Doing column?
 * 14) * keep it? Then how does it relate to tranches?
 * 15) * move those tasks back into Tranches?
 * 16) * Treat that as tranche -1, and then stop using it after current tasks are done <-- Decided
 * 17) What are we doing with the Done column?
 * 18) stop using it after current tasks are done  <-- Decided
 * 19) What is an activity period?  A monthish? (Decide Later)
 * 20) Don't need to nail this down - will be informed by data.
 * 21) How much of the backlog has to be well-ordered to have a realistic plan 4-6 months out? (Decide Later)
 * 22) Guessing 100 items.
 * 23) As of 18 June 2015, there are about 50 fully stack-ranked items and 1300+ tied for 51st.
 * 24) Will come back to this in a few weeks or months.
 * 25) Can/should/will we do any ordering outside of the top hundred (or fifty)?
 * 26) Option A: use vague tranches
 * 27) Option B: use "Next Up" columns to hold things above the 1300+ but below the numbered tranches
 * 28) Option C: Use priority field
 * 29) Option D: actually stack-rank (where - in what Projectcolumn? could do this in Backlog column in VisualEditor)
 * 30) Option E: Like Option B but with Epics and Super-Epics
 * 31) What does the Priority field mean for VE tasks?
 * 32) Option A: use the priority field for priority (and if so, how will it interact with the stack-ranking)
 * 33) Option B: severity as perceived by reporter <-- Decided
 * 34) Option C: something else
 * 35) Option D: ignore it

Goals

 * 1) What exactly is “goals”?
 * 2) Objective + Key Result(s)
 * 3) What is the relationship between milestones and KRs?
 * 4) Option A: Milestone is another kind of Key Result
 * 5) Option B: Milestone is a combination of Key Results
 * 6) Option C: Milestone is a combination of
 * 7) What are the useful queries for different stakeholder groups, and does this design support them?
 * 8) See Use Cases above.
 * 9) What are the edge cases?  E.g., Tasks which are technically small but have complex implications or dependencies - treat as Task and pass-through Epic, or Task is Epic, or Task rolls up into catch-all Epic?
 * 10) How exactly do we get the roadmap back from this?
 * 11) Hierarchical Backlog: List of Epics
 * 12) Heterogeneous backlog: ?
 * 13) How do we relate OKRs to tasks?
 * 14) Leave OKR in Wiki pages, don't link tasks to OKRs
 * 15) Leave objective in wiki page, create a tranche projectcolumn for each key result
 * 16) Put tasks into the related tranche
 * 17) Leave objective in wiki page, create a task for each KR
 * 18) Make related tasks blockers of of the KR tracking task
 * 19) Create task for each objective and create subtasks for each tracking task
 * 20) Make related tasks blockers of of the KR tracking task
 * 21) Have to retain a usable presentation on the wiki

Estimation

 * 1) How are Epics estimated?
 * 2) *Option A: Use "Epic Points", either 1/2/4/8/16 or Fibonacci, which are separate from Story Points but calibrated to them
 * 3) *Option B: Use very big numbers of Story Points <-- Decided
 * 4) *Option C: Use T-shirt sizing (S/M/L/XL), correlate to Story Points via calibration sample
 * 5) How far through the backlog should we estimate?
 * 6) Option A: All stack-ranked <-- Decided (need to try for 1-4 weeks and see if this satisfies Option E)
 * 7) Option B: All stack-ranked plus all Next Ups
 * 8) Option C: All stack-ranked, and others as touched/needed
 * 9) Option D: all 1300 :)
 * 10) Option E: Enough to get 3-5 months' visibility
 * 11) who does it for initial catchup? James, Done.
 * 12) Who does it on an ongoing basis?
 * 13) James
 * Ed
 * 1) Neil
 * 2) Rotating
 * 3) Some two-level pass of initial + validation
 * 4) Are there homogeneous groups of tasks that can be assigned the average estimate from a subset?
 * 5) The full 1300, based on average of all estimated tasks (including history, not including history?
 * 6) No <-- Decided
 * 7) some other set
 * 8) What range of values should we use for velocity? (Decide Later)
 * 9) come back to this