Bug management/How to triage

From MediaWiki.org
Jump to: navigation, search
This triage guide is work in progress and in an early stage. There are still several sections with a preceding "TODO" which have not been written yet.

This page is a guide how to triage bug reports in Wikimedia Bugzilla. It is part of information on Triaging.

When we triage bugs, we review each bug's initial report, comments, reproducibility, and any attached patches. We discuss its priority and status and decide whether to change them, and sometimes recruit developers to fix the bug or implement the enhancement.

Contents

Before you start [edit]

Make sure that you have an account in Wikimedia Bugzilla - this account is separate from your wiki accounts. If you don't have one yet, you will need to register an account in Wikimedia Bugzilla. For more information see the wikipage about Bugzilla.

If you are not familiar with Bugzilla yet, you may want to take a look at the Bugzilla page on this wiki first, and at Bugzilla/Fields to understand what all those options on a bug report page are used for.

For a basic understanding of the triaging workflow and the steps to perform, we recommend to take a look at the Triaging flow graphic of the KDE project which is very similar.

Getting Started: Find reports to work on [edit]

There are many different techniques and approaches to find reports to triage. You can find some ideas at Bug management/Triage tasks.

If at any point you feel like you do not know what to do with a certain report, please first ask developers or contributors before changing something.

Is it a duplicate? [edit]

Some reports in Bugzilla have already been reported before so you can search for an already existing report. We do not recommend to spend too much time on it; if a bug is filed twice, someone else will mark it as a duplicate later. If the bug is a duplicate, mark it as a duplicate in the resolution box below the comment field by setting the "RESOLVED DUPLICATE" status, and shortly explain your action in a comment for the reporter (or see "Help tools" at the bottom of this page).

If you think that you have found a duplicate but you are not totally sure, just add a comment like "This bug looks related to bug XXXXX" (and replace XXXXX by the bug number) so somebody else can take a look and help judging.

You can also take a look at Bugzilla's list of popular duplicates.

Tips for searching:

  • As it is often hard for reporters to find the right place (product and component) where to file a report, also search for duplicates outside same product and component of the bug report you are triaging.
  • Use common words and try several times with different combinations, as there could be several ways to describe the same problem. If you choose the proper and common words, and you try several times with different combinations of those, you ensure to have matching results.
  • Drop the ending of a verb (e.g. search for "delet" so you get reports for both "delete" and "deleting"), and also try similar words (e.g. search both for "delet" and "remov").
  • Search using the date range delimiter: Most of the bug reports are recent, so you can try to increase the search speed using date delimiters by going to "Search by Change History" on the search page. Example: search from "2011-01-01" or "-730d" (to cover the last two years) to "Now".

Is it an outside problem? [edit]

If the problem happens because of a piece of software that is not developed by MediaWiki/Wikimedia developers the problem should be forwarded to the so-called "upstream" developers (MediaWiki/Wikimedia is called "downstream" in this context, as we are just users of that piece of software). You can find a list of software which is tracked elsewhere and used by Wikimedia, plus addresses of the corresponding bugtrackers here: Bug management/Upstream bugtrackers.

After forwarding a bug report to an upstream bugtracker, please add the address of the upstream report to the "See Also" field in the bug report, add the "upstream" keyword to the report, and explain in a comment that the problem has been reported to its original developers.

Is there enough information? [edit]

To make a report useful, the same rules apply as for How to report a bug.

It's hard to generalize what makes a good report. For "average" reporters is definitely often helpful to have good steps to reproduce, a web address (URL) where to see the problem, or a MediaWiki software version in case it's their own wiki installation, and information about the browser and version used. If the reporter is a developer, steps to reproduce can sometimes be omitted as context is obvious (however this can create a problem for contributors that need to find their way).

Other tips:

  • If the report is not in English, an online translation service can sometimes help.
  • There should be only one issue per report.
  • It should be possible to call the described problem fixed at some point. "Improve the documentation" or "It runs slow" could never be called fixed, while "Documentation should cover the topic Embedding" or "The page at http://en.wikipedia.org/wiki/Example should load in less than five seconds" would have a criterion.
  • If the bug is a graphical problem, you may want to ask for a screenshot to attach to the bug report. Make sure to ask that the screenshot should not contain any confidential information.
  • You can get the technical ID of a string in the browser by attaching ?uselang=qqx to the web address. You can also append ?uselang=en to get a page in English.
  • If a page does not load you can get an initial trace by attaching ?printable=yes to the web address

Are the fields correct? [edit]

Summary [edit]

Sometimes the summary does not summarize the bug itself well. You may want to update the bug summary to make the report distinguishable. A good title may contain:

  • A brief explanation of the root cause (if it was found)
  • Some of the symptoms people are experiencing

Adding people to the "CC" or changing the "Assigned to" field [edit]

Sometimes, reports describe general issues or are filed against common bugzilla products. Only if you know developers who work in the area covered by the bug report, and if you know that these developers accept getting CCed or assigned to certain reports, you can add that person to the CC field or even assign the bug report to her/him.

To get an idea who works in which area, querying changes in Gerrit (see Gerrit/Navigation) or in case of MediaWiki Extension bugs taking a look at the corresponding Special:AllPages/Extension: Extension homepage can be helpful.

Severity [edit]

Please see mw:Bugzilla/Fields#Severity for information on the available values and their meanings.

Keywords and tracking bugs [edit]

In order to "tag" certain types or categories of bugs all across the different products and components, keywords and tracker bugs are used. You can find a list of all keywords here. You can add keywords in the "Keywords" field.

Sometimes so-called "blocker bugs" are used instead of keywords. There are many of them in use and you can find a list of blocker bugs here. If a report fits to what a blocker bug is for, add the number of the blocker bug in the "Blocks" field of the bug report.

Try to reproduce [edit]

This is an optional step. Only try if you have time.

When you try to reproduce a bug, write down the exact steps to reproduce it, plus mention the operating system, browser and browser version that you used for it. This is very similar to the guidelines How to report a bug.

A template could be:

I can reproduce the problem with the URL https://en.wikipedia.org/wiki/ExamplePage with InternetBrowser version 9.2 on Microsoft Windows XP3.
1. Go to the address above
2. Log in into the wiki
3. In the upper right corner, click the Link "Settings"
4. In the following page, click the checkbox in front of "Use Highlighting"
5. The checkbox should be enabled, but it stays empty. The checkbox is not greyed out either.

In general you don't change the version field, unless it is to remove "unspecified" and replace it with the version the bug was reported against or the version that you could reproduce it with. You can find out about the version of a wiki by going to the page Special:Version on that wiki.

If you cannot reproduce the problem and the report is a bit older (more than 18 months; also see the MediaWiki [Version lifecycle]), you could add a similar comment like the one above (describing your setup and the steps that you tried) and kindly ask the reporters if this problem still happens, in case the reporter has time to check again.

If you get feedback from the reporter that the problem does not exist anymore you can close the report as RESOLVED WORKSFORME (or RESOLVED FIXED if you can identify a specific code change that fixed the problem).

It is recommended to test on test2.wikipedia.org. You can find out which exact MediaWiki or extension software version a wiki runs by going to the page "Special:Version" (for example: http://test2.wikipedia.org/wiki/Special:Version.

Setting priority [edit]

TODO: This section needs an update and be synced with Bugzilla/Fields#Priority.

You should only do this if you are an experienced triager. Skip this section if you are not yet.

During triage, a Priority is assigned to a bug. Project managers can use this field, in cooperation with the Bugwrangler, in order to drive development of fixes or features to address the bug.

In order for these fields to retain meaning and effectiveness, communication is essential. For example, the priority Highest has no meaning if no one has agreed to get it resolved quickly.

With that in mind, here are some guidelines for basic communication that should take place:

Highest [edit]

This priority should be reserved for bugs that affect Wikimedia's servers in production. In other words, the effect of this bug can be seen by an end user of Wikipedia or its sister sites. Usually, this is a problem that affects a number of users and impedes their use of the site. For anything that isn't marked with a severity of enhancement, these are regressions in site behavior — in other words, some functionality that worked before is not currently working.

Bugs with this priority should have someone assigned to them so that they can be addressed as soon as possible. If you think a bug deserves this priority, but do not know the developer who should work on it, then assign it to the manager for the affected product (i.e. Visual editor, Mobile, Page Triage, etc.). If you don't have any idea who to assign the bug to and you are still sure that the bug deserves this priority, then ask in IRC (#wikimedia-devconnect) for some help before giving it that priority.

Immediately after marking a bug up to Highest priority, notify the following people in IRC (#wikimedia-devconnect) or via email: the director of platform engineering, the management of 20% time, the management of QA. This way problems can be addressed quickly.

High [edit]

A list of bugs marked High should be collated on a regular basis and distributed to wikitech-l. This will encourage discussion of the problems and remind people of their existance. The discussion may also help you understand why a bug does not deserve this priority and improve your triageing abilities.

Non-enhancement bugs marked with a High priority should be addressed after Highest priority bugs. These should be good targets for WMF developers 20% time, so it is good to make sure the manager of 20% time is aware of the bugs.

Normal [edit]

Any bugs that have a severity of major, critical, or blocker that is not marked with Highest or High should probably be marked with Normal priority.

Low [edit]

All other bugs that merit any attention should be marked Low priority.

Lowest [edit]

Bugs that are genuine issues with the software (i.e. the software is doing something it shouldn't) but that do not affect many people and will not be fixed by developers but only if a patch is contributed by a volunteer should be marked Lowest priority.

Add yourself to the CC list [edit]

By adding yourself to the CC list of bug reports that you change, you will receive followup emails with all comments and changes by anybody on that individual report. This helps learning what further investigations others make. (You can change the settings which actions you will receive bugmail about here)

Helper tools [edit]

Andre Klapper has shared some Greasemonkey scripts that he uses (Greasemonkey is a clientside Javascript extension that runs in Firefox and some other browsers). These scripts are available under "wikimedia/bugzilla/triagescripts" in Wikimedia's code repository (see Download from Git if you have not used Gerrit/Git before) and everybody can install them to save some time!

These scripts include for example stuff like

  • provide one-click stock comments,
  • embed several links (to Extension product homepage; query tickets previously filed by the reporter)
  • display email addresses of people and not just the name
  • color people based on a manual list (e.g. green = "I know this person and don't need to look at this report")
  • color MWExtensions based on a manual list (e.g. "green = deployed")

Please contact Andre Klapper if you need help installing or understanding the code. Contributions are welcome!

Other triaging documentation sources [edit]

Much has been written about triaging by other teams and Bugzilla users.

Retrieved from "http://www.mediawiki.org/w/index.php?title=Bug_management/How_to_triage&oldid=672024"