VisualEditor/Backlog design

From mediawiki.org

We track our team backlog using the VisualEditor project in Phabricator. This page explains how exactly we structure it.

 Tranches[edit]

We use open-scope tranches to group our upcoming work. The lower the tranche number, the higher its priority. Tranche 0 will never be closed (because there will always be bugs and breaking changes to our dependencies), but most of the others eventually will eventually close when the work is finished, or be de-prioritized once urgent milestones are met. Once that happens, we won't reuse tranche numbers, but instead create a tranche 5, tranche 6, and so on.

Last updated: 2017-01-03.

 Current tranches[edit]

Tranche 0: Interrupt[edit]

Maintaining the production service at a suitable quality level:

  • No loss/corruption of existing article data
  • No loss/corruption of the user's contribution
  • No security issues
  • No regression in performance
  • No regression in usability

Tranche 1: Release support[edit]

Support community-requested releases/deployments, and urgent key usability improvements as they arise

  • Goal: Provide improvements to the Beta Feature of the wikitext edit mode — T142523
  • Goal: Provide a single edit tab interface, and switch all WMF wikis over to using it (without any other change in configuration) — T102398
  • Goal: Run an A/B test of providing the visual editor by default rather than the wikitext editor on clicking 'edit' for a small proportion of anonymous users on the English Wikipedia — T119269
  • Goal: Enable by default for all users on all Wikisources – T138966 for bugs, then T138391 for deployment

Tranche 6: Visual diffs[edit]

Provide an alternative to wikitext two-column diffs for users, in case they are not fluent in wikitext, and where it would be more appropriate. This covers:

  • Goal: Letting users choose between visual and wikitext views in the "Review your changes" tab inside the visual and new wikitext editors, on desktop and mobile — T143350
  • Goal: Letting users choose between visual and wikitext views in MediaWiki's diff page, on desktop and mobile — T105173
  • Exploring letting users "play" over the history of a page, seeing a stack of visual changes animate

Lower-priority tranches[edit]

Tranche 2: Mobile MVP[edit]

Improve VisualEditor for mobile tablets and phones to the point where it is credible to provide it by default

  • Goal: Support sub-documents inside the editor stack, and use this to support loading parts of the page asynchronously and only when needed — T76544
  • Goal: Load parts of the editor asynchronously and only when needed — T54365
  • Goal: Provide a way to edit parts of a page (paragraphs, tables, images) individually — T50429

Tranche 3: Language support[edit]

Expand VisualEditor's content language support

  • Goal: Enable by default for all users on the remaining, language-variant Wikipedias – T93388
    • Support for "language conversion" blocks in Parsoid – T43716
    • Design and provide an interface to editing "language conversion" blocks – T49411

Tranche 5: Rich media tools[edit]

Improve the editing of various 'rich' editing tools so that particularly-difficult tasks are simplified – specifically:

  • Images, including uploads (part of MediaWiki core),
  • Formulæ ("Math" extension),
  • Charts ("Graph" extension),
  • Galleries (part of MediaWiki core), and
  • Sheet music ("Score" extension).

Decision tree[edit]

Priority issues:

  • Functionality regression
  • Performance regression
  • Data corruption
  • Security issue
  • Crash of entire editor
  • Crash of bits of editor (e.g. copy & paste stops working)

Rating:

  • Very common -> Unbreak now! if a priority issue, Normal if not
  • Common -> High if a priority issue, Normal if not
  • Sometimes -> Normal if a priority issue, Low if not
  • Rare / very rare -> Low if a priority issue, Lowest if not

Features:

  • On list for this quarter -> High
  • On list for this year -> Normal
  • On list for next year -> Low
  • Not on list -> Lowest

Former tranches, now complete[edit]

Note: Former goals in active and former tranches are not listed for brevity.

Tranche 4: Link editor tweaks[edit]

Improve the link editing experience

Active for "FY2016Q2" (completed in October–December 2015).

Story points[edit]

We give each task story points as a very rough way of quantifying how much person-time we expect the task to take (including the time it takes to triage, investigate, and respond to the bug).

  • 1 point: 1–3 person-hours
  • 8 points: 1–3 person-days
  • 40 points: 1–3 person-weeks
  • 160 points: 1–3 person-months

Process[edit]

  1. Make the minimum number of changes to the VE Phabricator data necessary to produce:
    1. A stack-ranked backlog suitable for discussion and planning by most stakeholders
    2. A Product Burnup chart
  2. Don't (yet) try to do anything with orthogonal categorization of VE tasks (for example, don't try to tag all mobile tasks vs all desktop tasks, and/or all security vs all performance). May come back to this later but not needed yet.
  3. Don't try to create a new project to contain a full stack-ranked list of all VE bugs (well, stack-ranked from 1-100).
    1. Reason 1: huge amount of Phab spam if we bulk-move 1300+ active and 3000+ closed tasks.
    2. Reason 2: Don't explicitly need it yet
  4. Do create tranches in order to organize the backlog.
    1. A tranche is an ordered set of all Stories and Epics (Tasks, in Phab) in a specific area of functionality that are largely independent of work in other tranches.
    2. For now, each tranche is intended to address a group of related issues. (I.e., each tranche is effectively the top-priority tasks from a set of all tasks within a category, but the full list of all tasks in each category is not yet explicitly documented within Phab).
    3. Tranches are implemented as columns within the VisualEditor project in Phabricator.
    4. VisualEditor shall always have an open Tranche 0, defined as "Interrupt", tasks which are urgent enough to jump ahead of all other planned work.
      1. Tranche 0 will also be stack-ranked.
      2. Tranche 0 will include (among other things) critical bugs for Wikitext, VisualEditor, and any other projects the VE team is responsible for.
      3. Logically, therefore, work on Tranches 1+ will only occur when Tranche 0 is empty or when the VE team is sufficiently staffed to have capacity available after engaging with all Tranche 0 tasks.
    5. All numbered tranches should be maintained fully stack-ranked - OBSOLETE, NEED TO UPDATE
    6. TR0, and the next two active Tranches should be carefully ranked; other tranches may be loosely ranked. - OBSOLETE, NEED TO UPDATE
    7. As a tranche is completed, it is hidden but not deleted/emptied. So Tranche 1 will disappear soon, and eventually "Tranche 20" will be the new "Tranche 1". To be clearer, we will use "Interrupt Tranche" for Tranche 0, "Primary Tranche" for the next open Tranche, "Secondary Tranche" the one after that, etc.
    8. There is no intentional ranking for any tasks that are not in Tranches
  5. Within tranches, work is grouped by Milestones - INCOMPLETE, NEED TO UPDATE
  6. The existing VisualEditor columns (of the form "Next Up:" in Phabricator) may serve as a card mapping area for items before they are assigned up to a tranche or down to the unordered lower backlog.
    1. This area may come and go. Currently, we don't see a need for it in this SOP.
  7. The complete, stack-ranked list of VisualEditor tasks may therefore be derived from this algorithm: - OBSOLETE, NEED TO UPDATE
    1. All open tasks in Tranche (0), in "natural order" as stored on the Phabricator board
    2. All open tasks in Tranche 1, in natural order
    3. All open tasks in Tranche 2, in natural order.
    4. Etc through all numbered tranches.
      1. Note that technically, Tranche 1 is still more important than Tranche 20 even when tranches 1-19 are all Done and closed and hidden. This seems purely theoretical but may matter for reporting.
    5. All open tasks in any "Next Up" columns, in no order (no order with columns and no order between columns)
    6. All open tasks in To Triage and in Backlog, in no order
  8. We will try to construct a Product burnup from a Phab data dump of this arrangement and refine the plan if that is not possible

Use Cases[edit]

  1. Developer: Use backlog to determine next work to do
  2. Product Owner: Prioritize work to meet goals
  3. Any Stakeholders: view burnup to assess capability, work load, and forecast times for team work.
  4. Active Stakeholders: view backlog identify work coming up and plan around it. (via roadmap?)
  5. Goals-based burnup, to compare with tranche-based burnup, to see if they align
  6. For gap analysis of goals vs tasks
    1. Goals with no task:
      1. Should we be doing more to accomplish this goal?
      2. Should we cut this goal?
    2. Tasks with no goal:
      1. Should we be doing this task? At this priority?
      2. Should we have a new goal (make explicit a goal that is currently implicit)?
      3. If so, what should its priority be, and how does that affect resourcing?
  7. Any stakeholder: use a URL to have permanent reference to work requested or planned by the Foundation or movement.

Open questions[edit]

  1. How far can an Epic go before it must be decomposed?
    • Option: No epics in TR0, Primary Tranche, Secondary Tranche. <-- Decided
    • After an Epic is broken out, the Epic tag should be removed and it should become either closed (since it's wholly juiced out) or a Tracker (if it retains some integration work, which must still be <= 1 task worth of work)
  2. What's the definition of an Epic for VE?
    • Will take one member of VE staff more than a few days of solid work. In practice, over 8 points. Decided.
  3. How is pre-coding work (requirements, design) accounted for?
    • Option A: Part of continuous "Backlog Grooming" that is assumed to take a fixed percentage of Product Owner's time as overhead. Not counted in points.
    • Option B: separate "Grooming" story for preparing one or more other stories to be shovel-ready. Pointed and forecast.
    • Option C: considered part of the lifecycle of a story, so that a story should be created as early as possible after basic scope definition, and the story assigned to appropriate people in sequence (such as designer, User Research, etc) and pointed and forecast appropriately. <-- Decided
      • Some of option A is inevitable; e.g., bulk work to prioritize or define 50 tasks in one 30-minute sitting is not pointed or tracked.
      • If anyone non-Product Person (James or Neil) is spending time grooming or working on a Task (good rubric: is it assigned to them?), it should be a Task or part of another Task, be pointed, and be estimated.
    • Option D: Spikes, which are unpointed stories that are identified and prioritized, but not estimated or forecasted.
  4. What are we doing with the Doing column?
    • keep it? Then how does it relate to tranches?
    • move those tasks back into Tranches?
    • Treat that as tranche -1, and then stop using it after current tasks are done <-- Decided
  5. What are we doing with the Done column?
    1. stop using it after current tasks are done <-- Decided
  6. What is an activity period? A monthish? (Decide Later)
    1. Don't need to nail this down - will be informed by data.
  7. How much of the backlog has to be well-ordered to have a realistic plan 4-6 months out? (Decide Later)
    1. Guessing 100 items.
    2. As of 18 June 2015, there are about 50 fully stack-ranked items and 1300+ tied for 51st.
    3. Will come back to this in a few weeks or months.
  8. Can/should/will we do any ordering outside of the top hundred (or fifty)?
    1. Option A: use vague tranches
    2. Option B: use "Next Up" columns to hold things above the 1300+ but below the numbered tranches
    3. Option C: Use priority field
    4. Option D: actually stack-rank (where - in what Projectcolumn? could do this in Backlog column in VisualEditor)
    5. Option E: Like Option B but with Epics and Super-Epics
  9. What does the Priority field mean for VE tasks?
    1. Option A: use the priority field for priority (and if so, how will it interact with the stack-ranking)
    2. Option B: severity as perceived by reporter <-- Decided
    3. Option C: something else
    4. Option D: ignore it

Goals[edit]

  1. What exactly is “goals”?
    1. Objective + Key Result(s) 
  2. What is the relationship between milestones and KRs?
    1. Option A: Milestone is another kind of Key Result
    2. Option B: Milestone is a combination of Key Results
    3. Option C: Milestone is a combination of
  3. What are the useful queries for different stakeholder groups, and does this design support them?
    1. See Use Cases above.
  4. What are the edge cases?  E.g., Tasks which are technically small but have complex implications or dependencies - treat as Task and pass-through Epic, or Task is Epic, or Task rolls up into catch-all Epic?
  5. How exactly do we get the roadmap back from this?
    1. Hierarchical Backlog: List of Epics
    2. Heterogeneous backlog: ?
  6. How do we relate OKRs to tasks?
    1. Leave OKR in Wiki pages, don't link tasks to OKRs
    2. Leave objective in wiki page, create a tranche projectcolumn for each key result
      1. Put tasks into the related tranche
    3. Leave objective in wiki page, create a task for each KR
      1. Make related tasks blockers of the KR tracking task
    4. Create task for each objective and create subtasks for each tracking task
      1. Make related tasks blockers of the KR tracking task
      2. Have to retain a usable presentation on the wiki

Estimation[edit]

  1. How are Epics estimated?
    • Option A: Use "Epic Points", either 1/2/4/8/16 or Fibonacci, which are separate from Story Points but calibrated to them
    • Option B: Use very big numbers of Story Points <-- Decided
    • Option C: Use T-shirt sizing (S/M/L/XL), correlate to Story Points via calibration sample
  2. How far through the backlog should we estimate?
    1. Option A: All stack-ranked <-- Decided (need to try for 1-4 weeks and see if this satisfies Option E)
    2. Option B: All stack-ranked plus all Next Ups
    3. Option C: All stack-ranked, and others as touched/needed
    4. Option D: all 1300 :)
    5. Option E: Enough to get 3-5 months' visibility
  3. who does it for initial catchup? James, Done.
  4. Who does it on an ongoing basis?
    1. James
    2. Ed
    3. Neil
    4. Rotating ([1])
    5. Some two-level pass of initial + validation
  5. Are there homogeneous groups of tasks that can be assigned the average estimate from a subset?
    1. The full 1300, based on average of all estimated tasks (including history, not including history?
    2. No <-- Decided
    3. some other set
  6. What range of values should we use for velocity? (Decide Later)
    1. come back to this