Phlogiston/Configuring Phabricator to be reportable

Making Phabricator tasks reportable in Phlogiston means two things: there must be a way within Phabricator to identify all of the tasks within scope for a report (e.g., all of the tasks belonging to a team), and there must be a way within Phabricator to designate which tasks belong to which categories. Phlogiston can usually create a report from a team's existing Phabricator setup without any changes, but because most teams group work only by status, not by category, this initial report will be less valuable compared to reports that can be grouped by category.

What is the scope of the team's work?
How does your team identify its work? Phlogiston will need a list of Phabricator projects that include all of the tasks that your team works on, and all of the tasks that you consider to be part of your backlog, and no other tasks. Especially, no other tasks that other teams work on.

How do you identify categories of work?
Most teams group their work in Phabricator in terms of sequence of work, such as tasks to be designed, tasks to be tested, tasks ready to be released. For purposes of reporting and for answering the kinds of questions that Phlogiston reports answer, however, it is generally much more useful to group work in terms of purpose: goals, milestones, fixed-scope projects, ongoing projects, etc. Phlogiston supports several different approaches in Phabricator to grouping work.

Group by Project
This is the simplest: all tasks with a particular project tag can be grouped into one category in Phlogiston. No further configuration is required in Phabricator.

One example configuration is: In this configuration, the only detail relevant to Phlogiston reporting is the last bullet;  is a Phabricator project, and all tasks that belong to that project could be grouped into, for example, a   category in Phlogiston reports. (Category names in Phlogiston reports can be customized and do not have to exactly match the Phabricator names.)
 * All work that the team does is tagged with a project tag, such as .  This could also be a fixed-scope tag, such as.
 * The columns of the workboard for the  project mark the state of the work, such as ,  ,.
 * Different categories of work, such as  and , each have their own Phabricator project tag.  These tags are visible within the task cards on the   workboard.

Group by Project Column
All tasks in a particular column in a particular project can be grouped.

Group by Parent Task
All tasks that are "children" of a parent task, meaning they block the parent task, can be grouped. This works for all levels of a blocking relationship: tasks that block tasks that block a particular task can be grouped under that task.

Some teams already configure their tasks this way, making this a natural fit for Phlogiston reporting. However, because there is no easy way in Phabricator to see if a task has  parent task, it is very difficult to maintain this data perfectly in Phabricator. Additional overhead, such as regularly running a Phlogiston report to see if there are any uncategorized tasks, is required.

For this to work in Phlogiston:
 * The parent task must be tagged with the  project tag.  This tag is hard-coded into Phlogiston to denote tasks whose descendents constitute a category.
 * All of the tasks in the family tree must be included in projects included in the Phlogiston scope.
 * For example, if the parent task is in the Phabricator project  and has a sub-task in the project , and the sub-task has a sub-sub-task in the project  , and the project   is not included in the list of projects to be reported on but   is, then the sub-sub-task would be included in the report but would not be recognized as a sub-sub-task of the parent task, because the intermediate task is not present in the report data.

Group by Intersection of two Projects
EXPERIMENTAL. Phlogiston can group all tasks that belong to two specific projects into a single category.

Example configuration
 * All work that the team does is tagged with.
 * The columns of the workboard for the  project mark the state of the work, such as ,  ,.
 * Different categories of work, such as  and , each have their own Phabricator project tag.  These tags are visible within the task cards on the   workboard.
 * also includes many other tasks that other teams are working on, and which should not be included in Phlogiston reports. These tasks are not on the   board.
 * Phlogiston counts all tasks that have both the  and the   project tags as the reporting category.

Using heterogeneous categorization rules
A Phabricator configuration can use more than one of these categorization rules simultaneously. Phlogiston applies the categorization rules in a fixed sequence, so that all tasks that are not categorized by the first rule are available to the second and following rules.

Example:
 * All work that the team does is tagged with
 * The columns of the Branding Team board correspond to different long-term initiatives, such as  and   and
 * Within each column, specific milestones are denoted with tracking tasks such as  and   and  .  These tasks are tagged as   and all of the tasks related to them are marked as blocking them.
 * With this Phabricator configuration, Phlogiston could categorize tasks from most to least specific, such as,  ,  ,  ,  ,  ,.

How to debug categorization rules

 * 1) Set up Phabricator according to the above instructions.
 * 2) Wait for the nightly data dump, and then run Phlogiston.
 * 3) Look at the debugging reports linked from the top right part of the Phlogiston report page.
 * 4) Full List of Possible Categories. Example.  This report shows all of the information about tasks in Phabricator that Phlogiston can use to categorize tasks.  If Phabricator is configured with one of the ways described above to categorize a task, and there are tasks in that category, then the Phabricator category titles (e.g., the Project title, Project Column title, etc) should be present in this report.  If it is not, then that category will be empty in Phlogiston.
 * 5) Rules for recategorizing tasks.  Example.  This is a link to the github file that defines the categorization rules for this report.
 * 6) Recently Closed Tasks.  Example.  This is list of all tasks resolved in the last 2 weeks. It should include all of the tasks that Phlogiston was told to track that were changed from open to resolved.  It therefore can be used to confirm that Phlogiston is seeing what the Product Owner thinks it should.  It can also be used to identify tasks that did not get categorized correctly, to ensure that there are no uncategorized resolved tasks in the data.
 * 7) All open tasks, sorted by category. Example.  This is a list of all open tasks in this scope, grouped by category.  It can also be used to figure out if Phlogiston is categorizing tasks as expected.
 * 8) Histogram of tasks by points. Example. This shows patterns in how a team has been assigning point estimates to their tasks.  It is a histogram of the point value of all resolved tasks, grouped by Phabricator Priority field.  Each column is a different point value.  So the highest bar in each row is the most common point value for tasks in that row.  Each row is a different Phabricator priority, From Lowest priority at the top to Unbreak Now! at the bottom.  (10 = lowest, 25= low, 50 = Normal, 80 = High, 90 = Untriaged, 100 = Unbreak Now!).  The last row is a summary of all tasks by point, so the highest bar in the bottom row is the most common (modal) point value for all tasks in this report.
 * 9) If a report includes a mix of pointed and unpointed stories, it may be helpful to set a default point value for unpointed tasks.  Otherwise, the points-based reports may be misleading because they will not show the unpointed stories.  Phlogiston will assign this point value to all stories with no points; it will not change stories with zero points.  If the histogram shows that most tasks get the same point value, that would be a good default value.
 * 10) The same line in Phlogiston that links to this report also shows the default value that is currently set in the configuration file for this report.