Phabricator/Trello to Phabricator

SPage (WMF) (talk) migrated Flow backlog tasks using trello_create.py script in 179057.

Other teams remain on Trello, e.g. T772 and T832, but more due to lack of features in Phabricator than unimplemented features of the migration script.

Notes on conversion
General discussion in https://secure.phabricator.com/T3179

Export board from Trello

 * You can do it manually from Trello board menu Menu > Share, Print, and Export > Export to JSON
 * WMF has an enterprise account, just go to https://trello.com/wikimediafoundation/export

Both produce JSON. Both have problems :(


 * The export from Trello board menu does not include all comments! For example https://trello.com/c/oJHPeRvq/9-split-topic-old-design-ideas the JSON export only has "final-comment", does not have "Issue: where does sidebar go?"!! and does not include the split_topic_normal.svg attachment on the card, perhaps because it's SVG!


 * The enterprise export does not have shortUrl for each card (it's only in the actions history)

Formatting JSON
It's a single line of JSON, so it helps a lot to format it. The export JSON is probably too big to copy and paste into http://jsonlint.com, so instead use still converts stuff to Unicode escapes, e.g. "name": "Flow — backlog" to  "name": "Flow \u2014 backlog"

Tip: Use  to explore the JSON.

Parsing Trello board JSON with cburroughs' export-trello.py
It seems the format of enterprise export change since cburroughs wrote a script? Some cards in Flow_backlog enterprise export don't have a "labels" field, and even in those that have "labels" it's an empty array. Instead, cards have "idLabels" field that references elements in the "labels" array. (Similar to "checklists"? Board export doesn't have this, just "idChecklists" in each card linking to per board "checklists" array. Enterprise export has both?)

cburroughs export_trello.py generates "init-custom": { "std:maniphest:addthis:import-trello-column": "In Development" }, doesn't mean anything to our installation.

FIXED: The enterprise export does not identify the current column of a card It records the changes to cards including updateCard that changes the list of a card, but not where it is currently.

cburroughs export-trello does not do anything with attachments, the comment is    # [u'attachments',  Never used on board nor does it handle type=addAttachmentToCard or the "main" image for a card. Not fixed

Doesn't properly handle checklists, comment is   #  u'idChecklists']   # a bunch of ids for checklists? Fixed, checklists appear in the Description.

Ways to import to Phabricator
trello_create.py is similar to bugzilla_create.py but imports in a single step without DB manipulation.


 * choice A) Use PHP
 * The same cburroughs did:
 * P1403 to import-attachments.php
 * P1404 import-users.php (not relevant, we just need a mapping, but won't hurt if users have same e-mail in Phabricator as they did in Trello?)
 * P1405 to import-tickets.php (nothing specifically for Trello...)


 * Choice B) HTTP requests
 * basic createtask form at https://phab-01.wmflabs.org/conduit/method/maniphest.createtask/
 * is it a problem the creator will be the person doing the import?
 * have to find other APIs to add comments, attachments


 * Choice C) bugzilla code
 * https://github.com/wikimedia/phabricator-tools/blob/master/README, https://github.com/wikimedia/phabricator-tools/blob/master/bugzilla_create.py
 * Python API requests plus DB manipulation


 * Choice D) Gilles' mingleterminator import script
 * https://gerrit.wikimedia.org/r/#/c/179091/
 * Python API requests

Notes on conduit (choice B)
The /conduit/method/maniphest.createtask conduit creates a task.

An early version of the trello_create.py script provided the title and description ready to copy-paste into this Conduit form. It appends a link to the Trello card to the description, see sample. You can still generate this using the --conduit command-line argument of the latest script.

All fields in the conduit form are JSON, thus strings in double quotes and if field is plural, inside  array. No way to assign column in workboard in this.
 * copy-paste title of card into Title field inside
 * edit description, copy-paste markup, paste into Description field inside
 * projectPHIds is , more if you want in multiple projects (see ).
 * Priority is 50 for Normal (from create task)
 * not sure about ownerPHID, editPolicy, etc.

Getting projectPHIDs
Instructions to get PHIDs for Conduit
 * projectPHIDs for "Core-Features" on is
 * projectPHIDs for"Flow (test)" on phab-01.wmflabs.org is

User PHIDs?
Didn't do this, but could:

Could copy and paste Core-Features members into  to get "phid"    : "PHID-USER-37czexl6ekrvcmp2tyqq", "userName" : "DannyH"

Steps
See detailed comments in trello_create.py

Tricky bits

 * Checkboxes. I don't see special support for them in cburroughs import-tickets.php
 * Attachments.

Markup
Trello's markdown is different from Phabricator's markup
 * Italics in Trello are, in Phabricator
 * A link in Trello is, in Phabricator it is

Handling the Flow backlog board
In 2014 the Collaboration team migrated the Trello cards in Next sprint/story column to Phabricator using the conduit web form. Some go straight into Sprint M column, others go into backlog.

On 2014-12-03 we archived several columns in Trello Flow backlog board: (Note that if you visit a card in one of these columns, there's no indication that the card is archived (a bug in Trello).)
 * Bugs to prioritize -- these were all just copy and paste from BZ, which are now cards
 * QA -- Sellello generated these
 * Mobile - TODO list from Jon Robson
 * Testing - another TODO list from Jon Robson

Danny created a "Send to Phabricator - Collaboration-Team board" in the Flow backlog board, and on 2014-02-19 we finally migrated these to Phabricator using the trello_create.py script in 179057, see details in /Flow backlog import