Phabricator/versus Bugzilla

BUGZILLA LAND IS UNSTABLE RIGHT NOW We are moving Bugzilla to https://old-bugzilla.wikimedia.org We are setting automatic redirects from bugzilla.wikimedia.org reports to Phabricator tasks Don't be surprised if your browser warns you about old-bugzilla being insecure while we get the CERT to work. Don't be surprised if redirects bring you to unexpected places during the weekend. Thank you for your patience. We are keeping our weekend migration schedule.--Qgil-WMF (talk) 19:24, 22 November 2014 (UTC)

Wikimedia is deprecating Bugzilla for bug reporting and replacing it by Phabricator. All the Bugzilla reports will be migrated, and the process for reporting and managing bugs will be basically the same. However, some things will change.

Instead of trying to use Phabricator exactly how you use Bugzilla, we recommend you to get used to how Phabricator works. Get your account at https://phabricator.wikimedia.org

Some of the features described below are planned but not implemented today. In these cases you can follow the link to the task to learn about their current status.

Timeline
This sequence of steps refers specifically to the Bugzilla migration. For a wider scope, see Phabricator.

Important: during the Bugzilla migration, emergencies can be reported at Project:Support desk and. We will announce this properly closer to the migration.


 * 1) ✅ phabricator.wikimedia.org open to all Wikimedia users
 * 2) * Bugzilla users can register adding their Bugzilla email address(es) to claim their activity automatically.
 * 3) ✅ Test instance containing Bugzilla reports automatically migrated
 * ✅, 2014-11-12: Go-NoGo meeting. (minutes coming soon)
 * ✅, 2014-11-13: Andre to send second and last announcement email to latest active Bugzilla users, see T618
 * ✅, 2014-11-18: two IRC office hours hosted
 * ✅, 2014-11-19: Reminder sent to wikitech-l and forwarded to mediawiki-l, wikitech-ambassadors, and wmfall.
 * 1) ✅ Andre to set up Security project permissions in Phabricator.
 * ✅, started 2014-11-21 00:30 UTC: Bugzilla migration starts
 * 1) * ✅ Andre to restrict Bugzilla access to read-only.
 * 2) * ✅ Chase to pull phabricator.wikimedia.org and disable incoming email
 * 3) * ✅ Chase to disable email Phabricator interaction
 * 4) * ✅ Chris to Switch off Gerrit Notification bot
 * 5) * ✅ Arthur to Switch off Bingle/Bugello
 * 6) ✅ Data migration
 * 7) * ✅ Chase to apply the Bugzilla patch ref XML-RPC API issue
 * 8) * ✅ Chase to set ext_ref field as editable
 * 9) * ✅ Chase to ensure mysql file size is set at an appropriate level for migration
 * 10) * ✅ Chase to run the migration fetch script (tasks and comments)
 * 11) * ✅ Chase to revert the Bugzilla patch ref XML-RPC API issue
 * 12) * ✅ Chase to run the migration create script (attachments)
 * 13) * ✅ Daniel and Mukunda to move Bugzilla to old-bugzilla.wikimedia.org & Andre to update sslbase and urlbase values in Bugzilla
 * 14) * ✅ Chase to configure cron jobs to start assigning Bugzilla activity to Phabricator users.
 * 15) * Service sanity testing (security issues are hidden, app is responsive, issue lookup works, etc)
 * 16) * ✅ Mukunda to set up redirects from bugzilla.wikimedia.org URLs to phab.wm.o
 * 17) * ✅ Daniel to Switch off / update Watchmouse
 * 18) * Testing (T1244)
 * 19) * Put phabricator.wikimedia.org back in service
 * 20) ✅ Updating documentation
 * 21) * ✅ Andre to update Bug Management related documentation on wiki pages
 * 22) * ✅ Quim to update mw:Template:Extension and test it
 * 23) * ✅ Quim to update links to bugtracker in sidebar on mediawiki.org
 * , planned for 2014-11-24: phabricator.wikimedia.org reopens with the Bugzilla content migrated.

Improvements
Phabricator comes with many improvements over Bugzilla.

Interface

 * The desktop UI looks contemporary.
 * Unlike Bugzilla, the Phabricator UI is primarily designed for software development and not for average non-technical users (according to upstream). On this basis, Phabricator refused to implement some bugzilla features intended to make life easier for inexperienced users. The Phabricator UI hides more complexity than Bugzilla (e.g. by having separate modes for displaying and editing certain report information; while bugzilla only collapses advanced fields in bug filing and bug searching windows).
 * Most features are mobile friendly as well.
 * Interacting via email is possible. In some special cases (Operations procurement) non-registered users can also interact via email (T52).

User account
Phabricator/Help
 * Log in with your Wikimedia (SUL) or wikitech.wikimedia.org (LDAP) credentials (T346).
 * Your email address is finally private.

Task-centric environment

 * Tasks can be bugs, and they can be tasks. No more "this is not a bug, doesn't belong here" or "this report is being tracked in Mingle/Trello"
 * Reporters, developers, designers, and product managers use the same tool to discuss bugs/features and report progress.
 * We get project boards (example) and mockups with notes (example).
 * Explicit titles of tasks, code reviews, and mockups related to the task, without needing to click or memorize numbers.
 * You can also see in which column/status of the project board the task is currently.

Flexible taxonomy

 * Projects are like tags or keywords; there is no rigid tree structure.
 * Tasks can be assigned to multiple projects or to none. No more fights to decide where should a bug live when there are several candidates.

Creating tasks
Phabricator/Help
 * Simple form for newcomers with title, project, CCed users (subscribers), description, assignee and priority.
 * Type-ahead to select projects, instead of having to browse products/components to find the destination.
 * The description ("comment 0") of a task can be edited by multiple users (with history/diffs available) to evolve the task and summarize long discussions.
 * A toolbar and a specific markup allows formatting of descriptions.

Commenting
Phabricator/Help
 * Enjoy auto-saved comments while you type.
 * A toolbar and a specific markup allows formatting of comments.
 * You can add files and inline images to comments with a simple drag & drop.
 * You can edit your own comments (with history).
 * You can hide your own comments (admins can make them visible or delete them for good).
 * No mid-air collision when someone adds a comment while you are writing yours.

Notifications

 * Web notifications show activity relevant to the tasks you are subscribed to (once enabled, see T1047).
 * You can watch and subscribe to a project in Phabricator. Hence you do not need to ask Bugzilla admins anymore if they can add you to the default CC list of a Bugzilla component to receive mail notifications for all changes.
 * With Herald you can create custom notifications based on queries, e.g. "email me when someone mentions (your regexp here)". Note that Herald is currently disabled (T493).
 * Also see Phabricator/Help for more information.
 * Note that the wikibugs-l mailing list does not yet receive update notification emails from Phabricator. This is being worked on in T763.

Calls to action

 * Tasks needing triaging appear at the top of lists (including search results) and in personal dashboards of the members of the affected projects.

Permissions

 * We can decide who can create projects, rather than only let a few administrators do so.
 * We can have specific teams with specific permissions whose members could add new members, rather than relying on admins only.

Known issues
Below you can find a selection of known issues. You can help by providing feedback and code.

Missing features
Bugzilla users might miss some features that Phabricator still lacks, or is not pursuing. For more details, check T22: Identify features Bugzilla users would miss in Phabricator and related blocking tasks.

Some missing features or convenience functionality that has been brought up includes: We are maintaining a list of Phabricator tasks to be upstreamed. You can also follow directly the tasks we have upstreamed.
 * T763: The wikibugs-l mailing list does not yet receive update notification emails from Phabricator. This is being worked on in T763.
 * T45: Suggestions of possible duplicates when creating a new bug/task (upstream)
 * T54: Adding dependencies directly while creating a (sub-)task (upstream)
 * T96: Dependency tree view of tasks (upstream)
 * T28: Metrics for key Wikimedia projects software in Maniphest. Phabricator reports are very limited, and it will take a while until this is fixed upstream.
 * T109: Phabricator is assistive technology–unfriendly (upstream, some improvements have taken place in upstream)
 * T67: RSS/Atom feeds (upstream, not in Phabricator's plans)
 * Phabricator doesn't have an equivalent to Bugzilla's "Users Watching". However, the possibility of subscribing to projects and tasks directly makes this feature almost unnecessary.
 * T53: Phabricator does not offer the same wiki markup syntax. Note that any wiki markup syntax support in Bugzilla was custom code.
 * Bugzilla had support for turning certain expressions automatically into links (see the list in ). Most of this functionality is not implemented (yet) in Phabricator; the migration leaves most of these as plain text. For related tasks (also about converting Bugzilla and RT ticket numbers into the corresponding Phabricator task numbers), see T687, T850, T873, T874, T875.

Missing data
Bugzilla and Phabricator have different elements in tasks, and the migration of data is done through the Bugzilla and Phabricator APIs, which don't support the export/import of all possible types of data.

For all these reasons, the data available in Phabricator will not be exactly the same than in Bugzilla, although we have put a lot of effort keeping everything that matters. The main changes are:


 * Bug numbers. We cannot assign to Phabricator tasks the same number as their Bugzilla equivalents. Instead, automatic redirects will link old Bugzilla URLs with their corresponding new Phabricator tasks. Phabricator already has >1200 tasks with numbers taken. The migration needs to be done by batches of bugs instead of sequentially, which makes the mapping of numbers more complicated. Still, smaller numbers will correspond to older bugs, and we are doing our best to improve the sorting.
 * History of changes. We are migrating the current metadata including the last modification date, but we cannot migrate the whole history of changes (e.g. previous statuses or product/components assigned, whether it was marked as duplicate and then reverted, changes to the priority value, previous summaries, attachments marked as obsolete and superseded by newer attachments...)
 * Votes. Phabricator doesn't have votes, although it has tokens. The Bugzilla API doesn't help extracting this data, and this is why we cannot automatically subscribe (CC) voters on tickets when migrating. Users will be able to access their votes in the archived Bugzilla after the migration.
 * Saved searches. Search parameters are fundamentally different in Bugzilla and Phabricator. Users will be able to access their saved searches in the archived Bugzilla after the migration. Phabricator also allows users to save search queries; the old queries can be recreated manually and saved.
 * Default CC list. If you are on the "default CC" list of some Bugzilla components you will need to "watch" the corresponding project in Phabricator to receive notifications automatically.

Also note that Severity, Version, URL, Whiteboard, and other fields will not be available in Phabricator, because we have decided to discontinue them. The related data available in Bugzilla reports will be added to the description of the task.

For an exhaustive list of changes, check the Bugzilla_data_migrated table below.

Considerations
Aspects that are not missing, but have a different approach or alternative solutions.


 * Selecting projects for tasks is done like tagging, using type-ahead. This is a very different paradigm than Bugzilla's tree selection. We need to monitor whether users find their way. Then again, in case of doubt tasks can be submitted without specifying any project. See Add projects list when creating new task in Maniphest. There is also a page listing all projects in Phabricator, but it is not linked from the task creation form.
 * Search can be improved. Phabricator's search is powered by Elasticsearch (the same library used by MediaWiki's CirrusSearch), but we are still fine tuning it (i.e.. searching for "delete" should also find "deleting" and "deletion") -- see T679. In general, the search user interface is simpler and does not offer all the advanced options (regex etc.) that Bugzilla's search offers.
 * Born in a corporate intranet context, Phabricator puts a lot of trust on users. We need to think how to prevent vandalism and how to react when it happens. In Bugzilla we also had to find social and technical solutions that would not be provided out of the box. More in T84.
 * Offering a simpler UI, some advanced actions require an extra click, e.g. Phabricator should let you add dependencies both ways (depending and blocking) and No link to advanced search for tasks.
 * Files are added to tasks via drag&drop, while users requiring a classical upload form (mobile, certain desktop browsers/configurations) have to use a separate upload form (T1026).

Migration
We will migrate all the public Bugzilla reports, including their public, non-obsolete attachments. We announced the date of the Bugzilla migration several weeks in advance (via a banner on top of Bugzilla, on wikitech-l, English Wikipedia Village Pump).

Test instance
A test instance was available for review for two weeks, showing a sample of 10% of all the Bugzilla reports, automatically migrated. Known issues could be found as tasks of the Bugzilla-Preview project. The feedback of the community helped to find and fix further issues.

Bugzilla in read-only mode
Bugzilla will be switched to read-only mode as soon as the actual migration starts. We will keep it available as is during an indeterminate period of time, just in case.

Bugzilla data migrated
All public Bugzilla reports will be migrated, but it is not possible (neither desirable) to make a 1:1 mapping between Bugzilla reports and Phabricator tasks. All the details about the migration script can be found at T259. Below you have an exhaustive list of Bugzilla fields and their future in Phabricator:

Status (and related Resolution) options have been revised (T212):

Priority options have been revised (T268):

Bugzilla URLs and their redirects

 * URLs of Bugzilla reports will redirect to their equivalent Phabricator tasks (T40).
 * Links to a specific comment of a Bugzilla report will redirect to the corresponding task, but not to the specific comment in that task (T1358).
 * The (frozen, outdated) Wikimedia Bugzilla will still be available under https://old-bugzilla.wikimedia.org (T366) for a while. You will still be able to log into Bugzilla to run a search query or access your list of votes but you will not be able to make any changes. In the long run, Bugzilla might get replaced by static HTML so we do not have to maintain Bugzilla security upgrades.
 * Only temporarily: Do not be surprised if your browser warns you about https://old-bugzilla.wikimedia.org being insecure or that "your connection is not private" while we get the certificate to work.
 * URLs for (saved) searches in Bugzilla will be redirected to read-only Bugzilla under https://old-bugzilla.wikimedia.org to allow you seeing and editing your search parameters. A banner on top of Bugzilla (T1234) warns that Bugzilla is read-only and outdated. We do not redirect search results to Phabricator as Phabricator does not offer passing URL parameters in a similar way to Bugzilla (also see T40).

Redirected URLs after Bugzilla migration

 * Direct links to single bug reports:
 * http://bugzilla.wikimedia.org/show_bug.cgi?id=50120 → https://phabricator.wikimedia.org/T52120
 * https://bugzilla.wikimedia.org/show_bug.cgi?id=50120 → https://phabricator.wikimedia.org/T52120
 * http://bugzilla.wikimedia.org/50120 → https://phabricator.wikimedia.org/T52120
 * https://bugzilla.wikimedia.org/50120 → https://phabricator.wikimedia.org/T52120
 * http://bugs.wikimedia.org/show_bug.cgi?id=50120 → https://phabricator.wikimedia.org/T52120
 * https://bugs.wikimedia.org/show_bug.cgi?id=50120 → https://phabricator.wikimedia.org/T52120
 * http://bugs.wikimedia.org/50120 → https://phabricator.wikimedia.org/T52120
 * https://bugs.wikimedia.org/50120 → https://phabricator.wikimedia.org/T52120
 * To Phabricator frontpage:
 * https://bugzilla.wikimedia.org/ → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/chart.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/describecomponents.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/enter_bug.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/index.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/page.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/query.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/report.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/reports.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/showdependencygraph.cgi → https://phabricator.wikimedia.org/
 * https://bugzilla.wikimedia.org/showdependencytree.cgi → https://phabricator.wikimedia.org/
 * Search results and attachments:
 * https://bug-attachment.wikimedia.org/attachment.cgi?id=16500 → (will not be redirected)
 * https://bugzilla.wikimedia.org/attachment.cgi?id=12345 → https://old-bugzilla.wikimedia.org/attachment.cgi?id=12345 → https://bug-attachment.wikimedia.org/attachment.cgi?id=12345
 * https://bugzilla.wikimedia.org/buglist.cgi?bug_id=1,2,3,4 → https://old-bugzilla.wikimedia.org/buglist.cgi?bug_id=1,2,3,4
 * https://bugzilla.wikimedia.org/buglist.cgi?bug_status=REOPENED&keywords=easy&product=Pywikibot → https://old-bugzilla.wikimedia.org/buglist.cgi?bug_status=REOPENED&keywords=easy&product=Pywikibot

Interwiki links and templates
Existing Bugzilla interwiki links and templates on wiki pages will still work, because those links will redirect to the equivalent Phabricator tasks.

Phabricator/Help

Automatic linking
Wikimedia Bugzilla had many automatic linking behaviors (see the list in ), supporting magic words, wiki markup to some extent, and turning bug 99999 (and variations, see ) into a link to bug report number 99999. Most of this functionality is not implemented (yet); the migration leaves most of these as plain text. For related tasks (also about converting Bugzilla and RT ticket numbers into the corresponding Phabricator task numbers), see T687, T850, T873, T874, T875).

Phabricator has its own great TNNNN short syntax for linking to Tasks, with other letter prefixes to link to Mockups, Pastebins, etc. (also see T53).

Bots

 * An IRC bot will report Phabricator activity just like Wikibugs does with Bugzilla (T131)
 * Gerrit Notification Bot will notify Phabricator tasks about related patches (T169)