Parsoid/Deployments

From mediawiki.org
Jump to navigation Jump to search

Planned deployments, linked from wikitech:Deployments. For a list of past deployments, look for 'parsoid' in wikitech:Server Admin Log.

For the current train deploy schedule, see MediaWiki 1.36/Roadmap#Schedule for the deployments.

See wikitech:Parsoid#Deploying changes to learn how to deploy a new version of Parsoid.

Oct 26: In progress Deploying v0.13.0-a13 as a cherry-pick to 1.36.0-wmf.14[edit]

  • T266285: Revert "Don't generate 6-element DSRs yet"

Oct 20 - 22: Yes Deployed v0.13.0-a12 as part of 1.36.0-wmf.14[edit]

  • Don't generate 6-element DSRs yet (Deploy hack for wmf.14, will be reverted in wmf.15 a backport to wmf.14)
  • T263502: Move whitespace heuristics for trimmable whitespace nodes to buildSep
  • T51538: Added cite error checks for numeric digit in name; added cite error checks for invalid text direction
  • T240642: Fix broken nested <noinclude> handling
  • T260959: Rest: Use upstreamed ResponseException
  • SerializerState: Bug fix + code refactoring in current line handling
  • T262408: Record length of trimmed whitespace in additional DSR fields
  • Always wrap follows content (avoids crashers when there's a valid follow with some other error)
    • Backwards incompat for invalid follows will result in the contents of the ref being dropped; selser will save us.
  • Fix invalid indexing of null which triggers notices in PHP 7.4
  • HTML->WT metrics: Tweak html2wt timing metrics
  • T254501: Introduce preprocessing in the HTML -> WT direction
  • T230861: Release fragments once they're no longer needed
  • T221790, T179082, T217705: One document to rule them all (do entire parse in a single document context)

Oct 13: Yes Deployed v0.13.0-a11 as part of 1.36.0-wmf.11[edit]

  • Move SelSer fallback code to the parent wikitext handler
  • Move applyPageBundle to PageBundle class
  • Remove unused Parsoid::html2html
  • Convert offsets in pb2pb after data attribs are loaded
  • Make ParsoidHandler::updateRedLinks() consistent with ::languageConversion
  • phab:T260960, phab:T263502: Avoid double dip whitespace reuse when getting trailing space
  • Minor: Remove node and npm references in a couple of comments
  • phab:T255007: Handle mw:DisplaySpace when serializing mw:Nowiki content
  • phab:T254502, phab:T260960: Avoid modified-wrapper on mw:DisplaySpace

Sep 22 - Sep 24: Yes Deployed v0.13.0-a10 as part of 1.36.0-wmf.10[edit]

  • T249743: Fix PHP log notice when origHref is or null
  • T214241: Preserve typeof on extension first encap wrapper nodes
  • T254502: Remove mw:Placeholder from mw:DisplaySpace
  • TableFixups: s/$node/$cell for code clarity
  • T261711: Fix internal links without short URLs set up
  • Highlight when we have a valid follow
  • T262986: Removed parsoid cite extension follow ref id's
  • T262452: Handle exceptional conditions in pb2pb endpoint pageconfig creation
  • T262637, T263025: Fix crashers in TableFixups handler + document unhandled scenarios
  • TableFixups: Simplify combinable cell checks
  • T262409: Refine source reusability check for nested list items
  • Restructure [[File:...|link=...] handling to avoid complex if-condition

Sep 14 - Sep 16: Yes Deployed v0.13.0-a9 as part of 1.36.0-wmf.9[edit]

  • Fix integrated phan testing when optional langconv component is not present
  • T51538: Adding "follow" functionality to the Cite extension
  • T262636: Coalescing null rel to empty string
  • Remove data-mw->attribs hack in template wrapper
  • Drop html5shiv
  • "\n", not '\n': Fix regression introduced by 18a4fc70
  • T262454: html2wt: Catch RevisionAccessException & return HTTP 404 instead
  • T108504: Don't generate |link=... in image markup when href is encoded

Sep 8 - Sep 10: Yes Deployed v0.13.0-a8 as part of 1.36.0-wmf.8[edit]

  • Handle media-in-link scenario
  • Bail on wikilink-in-wikilink scenarios
  • AddMediaInfo need not be idempotent since it runs before unpacking
  • Implement DOMCompat::replaceChildren()
  • Move AddMediaInfo earlier in pipeline
  • ExtensionHandler: Process extension metadata on DOM instead of HTML
  • T179082, T217705, T221790: Remove special case for the html extension when unpacking
  • Convert error Response returns to throws of HttpException
  • TableFixups: <th> & <td> cells can combine, not just <td> & <td>
  • Get rid of modulescripts from MediaWiki library/API invocations
  • T178927: Fix unhandled reparse table attribute scenario
  • T261044: Avoid array_key_first() for compatibility with PHP 7.2
  • T260583: Title with leading colon is invalid
  • Use DOMCompat:normalize() when normalizing
  • Make `revid` an int as well as `oldid` and distinguish between null and 0

Aug 25 - Aug 27: Yes Deployed v0.13.0-a7 as part of 1.36.0-wmf.6[edit]

  • phab:T259855 PageConfigFactory: don't use latest revision if getRevisionById() fails
  • phab:T94603 Use entity list from RemexHtml
  • Make DataAccess::fetchTemplateSource() use ParserOptions::getTemplateCallback()
  • Clean up IE-compatibility code from Sanitizer::normalizeCss which was removed upstream
  • Rename DataAccess::fetchPageContent to ::fetchTemplateSource
  • Remove $revid parameter from DataAccess::fetchPageContent
  • phab:T236812 Clarify uses of legacy Parser objects

Other changes to test infrastructure:

  • Round-trip testing fixes
  • Add parser test for phab:T236866
  • The subtree of mw:Placeholder should be uneditable during selser parser tests
  • ParserTests: Move section wrapping tests to its own file
  • ParserTests: Create knownFailures json file if one doesn't exist

Aug 18 - Aug 20: Yes Deployed v0.13.0-a6 as part of 1.36.0-wmf.5[edit]

  • T236866: WrapSections::getDSR: Don't assume all non-elements are text nodes!!
  • Ensure Parsoid doesn't throw when <ref> is used w/o Cite installed
  • T259676: Fix for missing content check where ..body->extsrc is undefined
  • Add an API method to determine if the optional langconv library is available
  • Clean up some of the block tag set notions in ParagraphWrapper
  • Fully implement 'page' parameter for paged media; mock it as well
  • Extension config option: Rename sealFragment to unpackOutput
  • T234932: Match ordering of core's srcset attribute; add mock API support

Aug 11 - Aug 12: Yes Deployed v0.13.0-a5 as part of 1.36.0-wmf.4[edit]

  • phab:T259677 Fix O(N^2) behavior parsing nested broken templates
  • Use expanded attr when fragments have been expanded to DOM
  • Allow $wgParsoidSettings to override nativeGallery and thumbsize
  • Changes that don't (shouldn't) impact production:
    • ParserTests improvements and refactoring
    • phab:T258767 Improved TimedMediaHandler test harness (should not have any effect on production output)

Aug 4 - Aug 6: Yes Deployed v0.13.0-a4 as part of 1.36.0-wmf.3[edit]

  • Add lang attribute to ResourceLoader url to get correct ltr/rtl CSS
  • phab:T259311: Fix incorrect computation of 'inTpl' predicate in HandleLinkNeighbours
  • phab:T259063: Error handling when json_decoding in ParsoidHandler::getParsedBody()
  • phab:T257504: Protect against invalid start/end offsets in ParsoidExtensionAPI::renderMedia
  • Improvements to MWSiteConfig, MWDataAccess, MWPageConfig (divergence from bundled VE extension)
  • Changes that don't (shouldn't) impact production:
    • Factor out (parser test) TestFileReader that can be used by core; move known failures into Test object
    • phab:T251422 SiteConfig unit tests

Note: due to phab:T259832, parsoid v0.13.0-a4 didn't actually get rolled out with the train; the train deploys this week included parsoid v0.13.0-a3 due to an oversight. This was discovered on Aug 6 after 1.36.0-wmf.3 had rolled out to all wikis, and parsoid v0.13.0-a4 was backport-deployed to all groups.

July 31 - Aug 3: N Created v0.13.0-a3 for 1.36.0-wmf.2 Not taken, see T257970[edit]

The train for 1.36.0-wmf.2 was rolled back to group 1 due to phab:T259311. New deploy-20200731 branch for Parsoid rooted at v0.13.0-a2. The following patch was cherry-picked, and tagged v0.13.0-a3.

  • phab:T259311: Fix incorrect computation of 'inTpl' predicate in HandleLinkNeighbours

July 28 - July 30: Yes Deployed v0.13.0-a2 as part of 1.36.0-wmf.2[edit]

  • T255190 Rewrite sep indent pre suppressing in terms of block scope
  • Cleanup workaround added for TimedMediaTransformOutput::getAPIData()
  • (Extension API) Wrap extension token so that it won't be exposed
  • T88495 Add the start of a design doc
  • T110772,T255584: Fix incorrect handling of tpl-span-wrapped text in HandleLinkNeighbors
  • Test for valid data-mw as in DOMDataUtils::validDataMw()
  • Refactor ParserTests/Test to allow it to be used by mediawiki-core

July 21 - July 23: Yes Deployed v0.13.0-a1 as part of 1.36.0-wmf.1[edit]

  • ParsoidExtensionAPI: Fix bugs in migrateChildrenBetweenDocs
  • T255190: Get rid of WikitextConstants::$BlockScope(Open|Close)Tags
  • T255190: Get rid of WikitextConstants::$SolSpaceSensitiveTags
  • T234549, T233736: Add a test for serializing when a revision isn't found
  • T234549: Unify revision content checks when starting from wikitext
  • T255190: Get rid of WikitextConstants::$HTML['HTML4BlockTags']
  • T255190: Be precise about why list item ends are implied
  • Remove some porting notes
  • Remove unported JS->PHP file
  • Combine doBlockLevel tags to form a notional wikitextBlockElems
  • T255190: Move doBlockLevel constants to WikitextConstants

July 14 - July 16: Yes Deployed v.0.12.0-a23 as part of 1.35.0-wmf.41[edit]

  • T252448: Make wikimedia/langconv optional
  • The Parsoid ConfigRegistry appears to be unused
  • Add missing MediaWiki\Rest\Handler::getParamSettings() implementations
  • T255190: Use $onlyInlineElements from Remex when DOM p-wrapping
  • T248343: DRY out default ParsoidSettings into a single location
  • Remove parseExtensionHTML
  • Don't reallocate ExtensionTagHandler while processing ext-tags
  • Fix typo-bug in ParsoidExtensionAPI
  • Collapse heading handlers (gen ids, dedupe ids) into a single pass
  • Reorder DOM post processing passes for a more coherent ordering

July 7 - July 9: Yes Deployed v0.12.0-a20 as part of 1.35.0-wmf.40[edit]

  • T51538: Match core error key for self-closed ref without name
  • Rename "cite_error_ref_no_text" to "cite_error_references_no_text"
  • T51538: Add Cite error for named refs that attempt to redefine the content
  • Remove $nestedRefsHTML
  • Move setting data-mw on autogenerated references to createReferences

June 30 - July 1: Yes Deployed v0.12.0-a19 as part of 1.35.0-wmf.39[edit]

  • Pull out a renderMedia method from the gallery extension
  • Extension API: Use generic 'context' option instead of 'inlineContext'
  • T254051: Match upstream change for margin-inline-start
  • T51538: Adding error handling for cite refs with name but no content
  • Remove HandlePres dom pass
  • Clarify when content is missing in cite
  • T255746: Preserving leading whitespace in indent-pre suppressing contexts
  • Replace assertElt use with phan-var declarations
  • T251920: DOMNormalizer: Don't convert the odd visual newline to <br/>

June 23 - June 25: Yes Deployed v0.12.0-a18 as part of 1.35.0-wmf.38[edit]

  • Fix deprecated uses of ParserOptions::newCanonical()
  • T255500: Compute DSR info for mw:DisplaySpace
  • T223194, T229740: Replace some instances of DOMUtils::assertElt

June 16 - June 17: Yes Deployed v0.12.0-a17 as part of 1.35.0-wmf.37[edit]

  • Fix deprecated use of getCurrentRevisionCallback()
  • HTML5TreeBuilder: Revert now unneeded Remex-bug workarounds + emit transclusion shadow meta at the end of a run of text nodes
  • Fix extensions to use Ext\DOMUtils instead of Utils\DOMUtils
  • Fix wrong var used in PWrap::hasBlockTag
  • T51538: Whitespace only content in <ref> tags is no content
  • T210647: Remove now unneeded workarounds for handling empty p tags
  • T245206: html2wt: Newly inserted elements shouldn't disrupt whitespace heuristics
  • T254804, T254646: Renamed terms in comments, code, and filenames.

June 9 - June 11 : Yes Deployed v0.12.0-a16 as part of 1.35.0-wmf.36[edit]

  • T222560, T222770, T222774, T247110: Simplifications and cleanup of several PEG tokenizer rules
  • T197879: Move armoring French spaces to a DOM post-processing pass
  • T233815: html2wt in selser mode: Don't crash on bad DSR
  • T133320: Hook up ExtensionRegistry with Parsoid and related fixes

June 2 - June 4 : Yes Deployed v0.12.0-a15 as part of 1.35.0-wmf.35[edit]

  • T222561: Don't start autolink matching at "/"
  • Remove nodeName check in meta handler
  • T253703: <*include*> tags don't need newlines before/after
  • T51538: Fix Cite extension <ref> no name and no content error handling
  • T210647: Add caption to always suppressing
  • Use DOMUtils::hasTypeOf/matchTypeOf/addTypeOf consistently
  • More phpcs related cleanups

May 26 - May 28 : Yes Deployed v0.12.0-a14 as part of 1.35.0-wmf.34[edit]

  • T252648: DOMNormalizer: Fix method signature causing production crashers
  • Remove unnecessary LanguageVariantHandler::set/::has//::delete handers
  • Various phpcs related cleanups

May 12 - May 18 : Yes Deployed v0.12.0-a13 as part of 1.35.0-wmf.32[edit]

  • T249958: Don't add unneeded extra newlines before/after existing lists

May 5 - May 11 : Yes Deployed v0.12.0-a12 as part of 1.35.0-wmf.31[edit]

  • All extension DOM processors should extend Ext\DOMProcessor
  • Add extension registration mechanism to SiteConfig
  • T249740: Bug fix in complex mixed-attr-content multi-template scenarios
  • T250935: Bump version of Remex to v2.2.0 (bump zest and alea as well)
  • T231568: SiteConfig: DRY out common computation of various config properties across subclasses
  • T250888: Move DOMDataUtils::addAttributes to DOMUtils
  • Remove dead code in SiteConfig.php
  • Add accessor methods for trace / dump flags

April 28 - May 4 : Yes Deployed v0.12.0-a11 as part of 1.35.0-wmf.30[edit]

  • T242746: Refactoring and cleanup of extension API, registration, and extension code (multiple patches)
  • T250629: Fix crasher in ParsoidExtensionAPI
  • T192913: html2wt: Fix link regexp to handle parser functions
  • T225849: Don't apply display hack at sol
  • T250111: Don't include .phan/ directory in composer library
  • T247093: Use PHPUtils::unreachable() instead of assert(false)
  • Use PHPUtils::jsonEncode consistently
  • AttributeExpander: Code cleanup
  • Fix tokenizer to properly encode attributes needed by QuoteTransformer
  • Enable /page/lint/... endpoint
  • Allow composer/semver ^2.0.0

April 14 - April 16 : Yes Deployed v0.12.0-a10 as part of 1.35.0-wmf.28[edit]

  • T242746: Refactoring and cleanup of extension API and extension code
  • Improve debuggability of non-canonical-DOM assertions
  • Use MediaWikiServices::getBadFileLookup
  • phan-related and tracing-related code cleanup

April 7 - April 9 : Yes Deployed v0.12.0-a9 as part of 1.35.0-wmf.27[edit]

  • T221989: Fix edge case misnested-tag lint detection
  • T242746: Refactoring and cleanup of extension API and extension code (many patches)
  • T246701: Fix (JS -> PHP) porting bug in interwiki computation

March 31 - April 2: Yes Deployed v0.12.0-a8 as part of 1.35.0-wmf.26[edit]

  • T242746: Refactoring and cleanup of extension API and extension code (many patches)
  • T235307: Remove use of Env in the REST API code in extension/* (many patches)
  • T248121: Drop unnecessary style modules in parsoid output

March 23 - March 26: Yes Deployed v0.12.0-a7 as part of 1.35.0-wmf.25[edit]

  • T247910: Allow users to set tabindex=0 on elements
  • Use MediaWikiServices::getRepoGroup()
  • T247212: Allow wikimedia/ip-utils 2.0.0
  • T242746: Refactoring extensions interface
  • T247353: Move DataParsoid.php

March 18 - March 19: Yes Deployed v0.12.0-a5 as part of 1.35.0-wmf.24[edit]

  • T238385: Make id attributes not include ascii whitespace per spec
  • T238385: Escape % sign if from valid percent-encoding in fragment identifiers
  • T240055: Ensure Parsoid on scandium executes from git checkout
  • T245627: TemplateData: Handle multibyte unicode characters correctly
  • T237462: Port JSON content-model extension
  • Remove use of Env parameter in Poem extension
  • Use expanded href to test for xmlish tags in wikilink title position
  • T242746: Remove direct access to Sanitizer from extension code
  • Fix minor bugs in AddMediaInfo.php, Gallery, and Sanitizer.php
  • Make extension tags optional in ext-config
  • Bump dependency versions for wikipeg, psr/log, wikimedia/assert, mediawiki/mediawiki-codesniffer, composer/semver, mediawiki-phan-config, ockcyp/covers-validator, mediawiki/minus-x. wikimedia/langconv
  • T242746: Remove more Parsoid internals knowledge from Cite
  • T241164: Sync with Cite
  • T239642: Ensure tests pass in /deploy

Monday, Feb. 10 around 1:33 pm PT: Yes Deployed 612106d[edit]

  • T242746: No need to explicitly pass 'inTemplate' flag from extension code
  • T235273: Remove PHPUtils::jsSort call from TemplateHandler and correct tests
  • T235307: Remove Env use from content version resolution functionality
  • Restore return 406 for an incorrect offset type
  • T238845: Fix for request with revID that has no content
  • T204618: Whitelist `aria-hidden` attribute in Sanitizer
  • T240054: Move all code from Parsoid to Wikimedia\Parsoid namespace
  • Update langconv package to 0.3.3
  • T244412, T244413: Corrected PAGE_UNAVAILABLE check for invalid RevID
  • Fix notice when tracing selser
  • T242746: Start untangling Parsoid internals from extensions
  • T242746: Use extension config option for html2wt formatting of extension tags

Wednesday, Feb. 5 around 1:15 pm PT: 74730a3 to be deployed Reverted due to T244413[edit]

  • T242746: No need to explicitly pass 'inTemplate' flag from extension code
  • T235273: Remove PHPUtils::jsSort call from TemplateHandler and correct tests
  • T235307: Remove Env use from content version resolution functionality
  • Restore return 406 for an incorrect offset type
  • T238845: Fix for request with revID that has no content
  • T204618: Whitelist `aria-hidden` attribute in Sanitizer
  • T240054: Move all code from Parsoid to Wikimedia\Parsoid namespace
  • Update langconv package to 0.3.3

Wednesday, Jan. 22 around 1:18 pm PT: Yes Deployed 7390988[edit]

  • T242513: Serialize reference tags by themselves on a line
  • Rename DOM handling methods toDOM/fromDOM to reflect reality
  • T243008: Fix PHP Notice when using pb2pb endpoint
  • Refactor Parsoid::html2html into Parsoid::pb2pb
  • T241146: Use DOMDataUtils::getNodeData in MachineLanguageGuesser
  • Remove backward-compatibility code for old-style DOMTraverser handler

Thursday, Jan. 16 around 10:15 am PT: 02f0066 to be deployed Reverted due to T243008[edit]

  • Refactor Parsoid::html2html into Parsoid::pb2pb
  • T241146: Use DOMDataUtils::getNodeData in MachineLanguageGuesser
  • Remove backward-compatibility code for old-style DOMTraverser handler

Monday, Jan. 13 around 1:32 pm PT: Yes Deployed 5d37da1[edit]

  • Clean up DOMTraverser handlers to use new calling convention

Wednesday, Jan. 8 around 1:46 pm PT: Yes Deployed f963e51[edit]

  • Fix 'source variant' functionality; add missing static types to src/Language
  • Cleanup extension arg normalization code
  • html2wt: Account for missing $dsr
  • T238934: html2wt selser: Return http 409 if the previous revision is not found
  • Ensure that SyncTransformManager has correct frame
  • T237318: Ensure Sanitizer::sanitizeToken uses correct frame source text
  • T238022: Fix autolink url parsing code
  • Minor efficiency tweak to Util::lastUniChar()
  • Ensure LiFixups::handleLIHack uses correct frame source text
  • T228217: Add --maxdepth option to bin/parse.php
  • ParsoidHandler: Fix throwing ValidationException for invalid domain
  • Make Monolog version match MediaWiki core
  • Fix offset type handling for Lint API requests

Deployments from previous years[edit]