Parsoid/Changing Parsoid HTML Output

To consider as you make/propose a change

What follows are some conderations for developers when changing Parsoid's HTML output.

• Parsoid's output is spec'd. Has the spec been updated? Does the change require version bump? Generally we have been following semver conventions but haven't been bumping for patch level fixes.
• Does the content roundtrip? With or without normalizations? How heavily are we relyion selective serialization?
• How does the change affect read views?
• How does it affect editing clients?
• Does it consider when if the content is templated?
• Blah blah blah ...

QA process before deploying a change

(These are informal notes from CSA, copied from discussion on https://gerrit.wikimedia.org/r/593016, not yet any sort of formal process.)

I will say that we generally have different processes for:

1) changes which are not expected to affect pages at all except in very unusual edge cases or where pages are broken.

2) changes which are not expected to affect rendering/visual appearance but change the DOM tree in a way which might affect downstream gadgets & tools

3) changes which affect rendering/visual appearance

For case #1 we just show (via visual diff testing, rt testing, random sample of pages, etc) to some level of confidence that the change doesn't affect pages. I think this patch is in this category.

For case #2 we seem to do an RFC process and/or outreach to the dev community about the pending change, via Tech News, etc. (Could be formalized further.) Using the <figure> tag for media (phab:T118517) is in this category.

For case #3 we do a more aggressive community outreach, including working with editors to de-lint affected pages, and generally don't merge the change until every community has had a chance to fix their pages. (The criterion isn't that the number of remaining affected pages is zero, but we've generally at least reviewed the remaining pages.) The transition from old-tidy to Remex was in this category.

First order of business is usually to figure out what category a proposed change is in. Visual diff testing will help determine whether there are significant rendering changes "in the wild" (the hallmark of case #3); usually examining the proposed HTML alone is enough to determine whether DOM structure changes could affect 3rd party gadgets/tools (case #2).

Our survey tools aren't perfect, however. At some point after we've done what due diligence we can, we'll probably have to merge and be ready with a quick-revert if it turns out there are significant reader/tool impacts. In general we should be ready with a plan to revert, and avoid making changes that can't be backed out easily. Usually the mechanism is to patch downstream users/tools to handle both the "before" and "after" HTML, so that we can generate either one during the transition period w/o breaking anything, and only remove support for parsing "before" HTML after we are sure we the proposed change is going to "stick". But we also have versioning and upgrade/downgrade mechanisms built into our APIs; hopefully the code for both upgrade and downgrade lands at the same time so that everything is in place in case we need to revert.