Parsoid/Deployments

Planned deployments, linked from Deployments. For a list of past deployments, look for 'parsoid' in Server Admin Log. See also Parsoid/Releases for non-alpha versions.

For the current train deploy schedule, see MediaWiki 1.42/Roadmap.

See Parsoid to learn how to deploy a new version of Parsoid.

Nov 28 - Nov 30:

 * Track extension uses on all namespaces on all wikis, T351230
 * Rename Token::getAttribute to ::getAttributeV
 * Use DOMCompat::getAttribute instead of Element::getAttribute
 * [DOMUtils] Add 'prepend' option to DOMUtils::addValueToMultivalAttr
 * [DOMUtils] Make *MultivalAttr methods private
 * Add DOMCompat::getAttribute for compatibility with DOM spec
 * Remove kk lang conversion tests: kk converter no longer exists, T268143
 * Avoid one sprintf that's not strictly needed
 * Sync parsertests with core
 * [cleanup] Use PHPUtils::iterator_to_array where possible
 * Allow &lt;sub&gt; tags in TOC entries
 * Document an issue with WTUtils::findFirstEncapsulationWrapperNode
 * Optimize XMLSerializer::encodeHtmlEntities for speed
 * Fix a bunch of easy to fix PHPCS exceptions
 * Reduce total number of DOMCompat::nodeName calls
 * Add missing closure to an expensive log call

Nov 14 - Nov 16: ✅

 * Avoid problematic empty/isset when they aren't needed
 * Drop obsolete PHP 7.2 "arrayToObject" workaround, T254519
 * Ensure synthetic TOC tags don't break template encapsulation, T350625
 * Rewrite DOMUtils::nodeDepth for speed
 * Avoid using gettype where it is not needed
 * Make use of the ??= operator where it makes sense
 * Sync parserTests with core
 * Separate and remove DOMUtils::pathToAncestor helper
 * ParserTests: update language variant options to new option names
 * ParserTests: parser test option names are always lowercased
 * TestUtils: Replace trivial preg_match with str_starts_with
 * Drop a few duplicate implementations of DOMUtils methods
 * Serialize media option value from html in data-mw if isset, T206940

Nov 7 - Nov 9 ✅

 * Fix broken WTUtils::NON_EXTENSION_ENCAP_REGEXP
 * Add fast helper method DOMUtils::hasClass
 * Move LinkTarget from core to Parsoid, part 2/2, T296023
 * Move LinkTarget from core to Parsoid, part 1/2, T296023
 * Fix mistake in $pat(c)h variable name
 * Optimize DOMUtils::addValueToMultivalAttr implementation
 * Simplify redundant hasAttribute/getAttribute calls
 * Replace [^\s] with identical \S in regular expressions
 * Don't expect parsoid endpoints to redirect, T350359
 * Emit relative redirects., T349001, T350219, T311867
 * Revert "Temporarily disable broken API-testing test that broke with core changes"
 * Optimize very slow SiteConfigTest
 * Optimize hot preg_match in ParsoidLogger
 * Don't use count when the actual count is not needed
 * Rewrite SiteConfig::interwikiMapNoNamespaces implementation
 * Replace some ?? with identical isset
 * Optimize hot SiteConfig::namespaceId
 * Minimize code duplication in TDHandler::before
 * Remove accidental duplicate assignment
 * Simplify ScriptUtils::booleanOption implementation
 * Update various PHPDoc blocks
 * Temporarily disable broken API-testing test that broke with core changes

Oct 31 - Nov 2: ✅

 * Annotations in template targets, T349078
 * DOMUtils::addHttpEquivHeaders: Documentation comment improvements
 * Remove variant-metadata-hack; add helper to add http-equiv meta tags
 * Exit regression script if no url or titles are provided
 * Reuse LinkHandlerUtils::figureHandler in gallery, T252246, T214603, T333907

Oct 24 - Oct 26: ✅

 * TOCData/SectionMetadata implements JsonCodecable, T327439
 * Mark ContentMetadataCollector::setIndicator as taint sink for phan
 * Fix running Parsoid tools on scandium
 * Fix data-parsoid json in a http api test
 * Disable json content http api tests, T349098

Oct 17 - Oct 19: ✅

 * Add wikimedia/json-codec 2.2.1 to Parsoid dependencies, T346829,  T339927
 * Handle multiple annotation end without interrupting processing, T346805
 * Migrate rt-testing to testreduce1002, T345220
 * Remove ?useparsoid=1 handler from Parsoid
 * Deprecate any place accepting string-valued mediawiki-internal lang codes
 * Rename 'source' and 'target' variants to 'wikitext' and 'html' variants
 * maybeConvert: Short-circuit early if language conversion is disabled

Oct 10: v0.19.0-a1 to kick off 1.42 release cycle

 * No changes from 0.18.0-a29

Oct 10: v0.18.0-a29 for 1.41.0 release candidate

 * Add a Parsoid option to skip the language conversion pass
 * Add variant-metadata-hack pb2pb transformer to support core
 * Rename DataI18n to DataMwI18n

Oct 10 - Oct 12: ✅

 * Remove tbody tags from new annotation parser tests, T347180
 * Add optional data-transcodekey to media sources, T305210
 * Remove (short)?title from derivatives in mocks
 * Fix calling Timing::start without $metrics, T339365
 * Proper class type for DataMw, T226428
 * Include fostered annotation tags in fostered transclusions, T347180
 * bin/start-rt-test.sh: Fix overagressive quoting

Oct 3: ✅

 * Fix some bash shell scripts based on shellcheck warnings
 * Bring Language::isValidInternalCode closer to core
 * Add missing strict_types declarations on PHP files
 * Allow wikimedia/bcp-47-code 2.0.0
 * Followup on 4e31735a: Some more PageBundle cleanup + documentation

Oct 3 - Oct 5: ✅

 * Use PageBundle object in DataBag as well
 * Parsoid pb2pb: Preserve pb->parsoid->counter + document brokenness
 * Readability cleanup: Use switch in Parsoid::pb2pb
 * build: Update wikimedia/ip-utils to 5.0.0
 * Parsoid.php: Add 'convertoffsets' pb2pb transformation
 * Putting fostered end annotation markers after the table, T347180
 * Remove attributes deprecated in TMH, T199129, T305210, T313875
 * Update api-testing after features were disabled upstream, T347426
 * Sync parserTests with TMH
 * Magic words aliases aren't necessarily unique to a magic word, T345026
 * ImageMap: Remove standalone knownFailures test failures file
 * Strip fragment when checking for ? when serializing interwiki target, T347418
 * Remove ImageMap implementation from Parsoid repo
 * Stop creating dynamic properties in SerializerState, T237463
 * Avoid using deprecated ParserOutput::getCategories, T305161
 * ParserTestRunner: fix handling of multiple property/extension options
 * Don't lint missing wikitext table ends when followed by deletable-table-tag, T295197
 * Disable Zh Language converter, T346657
 * Update namespace for class ParsoidFormatHelper
 * Introduce stripping of annotation in non-wikitext extension content, T341009
 * Fail to stringify if quote found in non-wikitext media option, T335538
 * Lint missing wikitext syntax table ends, T295197

Sep 26 - Sep 28: ✅

 * Update integrated parsoid result after core commit
 * Add durationhint to timed media, T305210
 * Add missing TMH modules, T346703
 * Add magnify links on thumbs when not linking to file desc page, T329413,  T273013
 * Get rid of debugApi support

Sep 19 - Sep 21: ✅

 * Instrument dom2wikitext entrypoint based on HtmlToContentTransform metrics, T339365
 * Remove infinite loop in AnnotationDOMRangeBuilder, take 2, T325575

Sep 12 - Sep 14: ✅

 * mw:DOMFragments shouldn't copy inTransclusion flag from first encap node, T345705
 * Remove PageConfig::hasLintableContentModel
 * Log calls to the MockMetrics
 * Instrument wikitext2html entrypoint based on ParsoidHandler metrics, T339365
 * TableFixups: Generalize how two adjacent cells combine, T343874
 * Don't report lints for fostered section tags, T216350
 * Satisfy neovim: Add missing return types
 * Satisfy neovim: Fix incorrect capitalizations
 * Make use of PWrap::pWrapOptional in MarkFosteredContent pass
 * Minor: Fix capitalization of class name
 * Document DOM processors need to process attribute-embedded-HTML, T214994
 * Run `npm audit fix`
 * Update sitematrix
 * Assert that a pass with applyToAttributeEmbeddedHTML is a traverser

August 15 - August 17: ✅

 * Followup to d66d2b3b: Abort table fixups without a valid DSR
 * Add a DOMUtils::isMetaDataTag
 * Deal with unsupported table combining scenario, T343874
 * TableFixups: Handle NO_ATTRS flag correctly
 * Rename and tweak WTUtils::hasParsoidAboutId
 * Ignore metadata tags in textContentFromCaption, T339237
 * Update DOMCompat tests for CI since getElementById is fixed

August 8 - August 10: wmf.21 branch was never cut and these changes went out with wmf.22

 * Linter: Deal with empty tables in large-tables lint
 * Follow up to 64fea319c: Fix up integrated test failures
 * Give extensions an option to customize the data-mw, T341735
 * Remove backward-compatibility methods using MediaWiki-internal language codes
 * LinterTest: s/parseWT/wtToLint/g
 * Linter: Fix performance of large-tables lint and re-enable it, T336528
 * Match core NO_TOC logic, T332243

August 1 - August 3: ✅

 * Update ContentMetadataCollectorCompat to allow RT testing
 * Use composer-runtime-api to determine the running version of Parsoid
 * Export appendOutputStrings to ContentMetadataCollector
 * DOMPostProcessor: Remove a potential future gotcha

July 28: v0.18.0-a19 for a Parsoid API change

 * [BREAKING CHANGE to Parsoid Extension API] DOMPostProcess: Don't run extension dom processors on nested pipelines
 * Revert "Export appendOutputString to ContentMetadataCollector"
 * Set synthetic DSR on the synthetic TOC meta tag
 * Export appendOutputString to ContentMetadataCollector
 * DOMPostProcessor: Explicitly set skipNested property for DOM passes
 * Minor: Update DOMPostProcessor docs

July 25 - July 27: ✅

 * Add option to disable code updates to testreduce1001
 * MigrateTemplateMarkerMetas: Update fostered markers for metas, T334957
 * Make DOMTraverser a generic utility that can be used with DOMs
 * TreeBuilder: Foster all metas by default instead of the other way round
 * ParserTests: indicator tests is a sync target with core

July 18 - July 20: ✅

 * Sync parserTests with core
 * Indicators: Fix broken fostering of indicator meta tags
 * Cite: Add missing resource modules, T328695
 * Indicators: Fix crashers + test invalid indicators + new tests, T214241
 * ParserTests: Migrate indicator tests to its own file
 * Add support for indicators, T107332
 * Omit wrapperTag in references call to extTagToDOM
 * ParsoidExtensionAPI: Add no-wrapper option to extTagToDOM

July 11 - July 13: ✅

 * PipelineUtils: Clear auxiliary tsr info in addition to tsr itself
 * Followup on 82eecb2c: Update knownFailures & parserTests.json files
 * Sync parserTests with core
 * Add tests for linting stripped tags in nested content, T338325, T214994
 * Enable processing of attribute-embedded-HTML in some DOM passes, T214994
 * Extract useparsoid processing into a static method for use in exts
 * Rename processAttributeEmbeddedHTML in DOMTraverser
 * Split combined cleanup DOM pass: save data parsoid on its own
 * Fix retokenizing when bailing on links, T340192
 * Capture flattened extlink url rather than doing it twice
 * Turn flattenStringlist comment into type assertion
 * DOMTraverser: Add option to process attribute embedded HTML, T214994

June 27 - June 29: ✅

 * Fix WTUtilsTest to properly use load / store data attribs
 * Rename docId to nodeId in DataBag and friends
 * Rename extName to extTagName
 * Perf tweak: Short-circuit processAttributeEmbeddedHTML
 * Sync parserTests with TMH
 * Add a utility to parse extension tag name from typeof
 * Address FIXME in ContentUtils class
 * Dedupe code to process HTML embedded in data-mw attributes
 * ParsoidExtensionAPI: Add method to process HTML embedded in attributes

June 12 - June 14: ✅

 * Linter: Stop calling Large table linting rule, T336528
 * ParsoidExtensionAPI: Rename Extension API method
 * Lazily construct the ParsoidExtensionAPI
 * Remove return from lintHandler's defaultHandler callable
 * Replace deprecated MWException, T328220
 * Script to process RC stream for dirty TOC edits, T336101
 * Remove PageConfig::getRedirectTarget
 * Clarify comments about redirects

June 6 - June 8: ✅

 * Bug fix in TraceProxy.php: Followup to 44df7071
 * Make block_line always return an array, T334707
 * Remove infinite loop in AnnotationDOMRangeBuilder, T325575
 * Port changes to add description links, T329364
 * Sync parserTests with core

May 30 - June 1: ✅

 * Bump Parsoid HTML version to 2.8.0
 * Sanitizer: Permit the `aria-level` HTML attribute in wikitext
 * Sync parserTests with core+extensions, T333907

May 23 - May 25: ✅

 * Minor JS tweaks: use const & let instead of var
 * Assume strings for dataMw->body properties in ref handlers, T337036
 * Punt on trying to resolve subpages on "/", T335847
 * Add a "isPreview" method to ParsoidExtensionAPI
 * Remove "Error:" prefix from I18n pass
 * No need to resolve subpages when resolving lonely fragments, T335847
 * Update WMF site matrix, T335674, T336675
 * Sync parserTests with Cite

May 16 - May 18: ✅

 * Make the free disk space check more conservative
 * Bump Parsoid HTML version to 2.7.1
 * Update the authors section of the composer.json
 * Add classes on elements inside the media structure, T270150, T314097
 * Sync parserTests with core and Cite
 * Remove special handling for DISPLAYTITLE & DEFAULTSORT, T294621

May 11: v0.18.0-a9 for linter testing on beta

 * Linter: Detect unclosed tags in article headings, T308398

May 9 - May 11: ✅

 * Create linting rule for large tables, T334528
 * build: Updating eslint-config-wikimedia to 0.25.0
 * Use DOMUtils::isHeading instead of matching node names
 * Sync parserTests with core

May 2 - May 4: ✅

 * Add synthetic TOC if necessary, T92999, T333023
 * Sync parserTests with ImageMap
 * Sync parserTests with core + extensions
 * doc: Declare some properties as nullable

Apr 25 - Apr 27: ✅

 * Experimentally enable Parsoid Read Views with ?useparsoid=1 query string, T335157
 * TokenStreamPatcher: Cleanup dead code (followup to 6f457e34)
 * Ensure the correct dimensions are applied to gallery lines, T333907, T305628
 * Minor cleanup in TemplateHandler
 * build: Switch phan to special library mode in standalone mode, T334492
 * Suppress media formats when used on a gallery line, T333907, T214601
 * Allow self-closed galleries to be edited
 * Fix linting multi-colon-escape
 * Handle null tag data

Apr 18 - Apr 20: ✅

 * Add $key for getTempNodeData
 * Fix regenerating old html from cli, T234549
 * ParserTests: Move all TOC tests from parserTests.txt to toc.txt
 * ParserTests: Fix bug (re)setting test defaults
 * ParserTests: Bring parserTests-knownFailures.json up to date
 * ParserTests: Move more magicword tests to magicWords.txt
 * Fix passing wikitext to bin/parse.php in integrated mode

Apr 4 - Apr 6: ✅

 * Html2wt: method naming tweaks + log when old html is not available
 * Get rid of nativeGalleryEnabled config
 * ParsoidExtensionAPI: Simplify extTagToDOM
 * Sync parserTests with core
 * Add more commit footer variations to gen_deploy_log script

Mar 28 - Mar 30: ✅

 * Alert if disk space needs freeing when running start-rt-test.sh
 * Title.php: bring closer to MediaWikiTitleCodec; allow colons in Talk titles, T332903, T329690
 * Revert "Make first argument to DOMUtils::parseHTMLToFragment optional"
 * Title.php: Handle user name equal to interwiki prefix in TitleParser, T329690
 * Title.php: Use str_contains/str_starts_with/str_ends_with
 * Title.php: Check for null return of preg_replace, T313663
 * Title.php: remove unreachable parts of title invalid regexp, T297578
 * Title.php: slightly improve efficiency of bidi character stripping

Mar 21 - Mar 23: ✅

 * Cite CSS: Update generated CSS
 * Use parseHTMLToFragment in TemplateHandler
 * baseconfig: Remove formatversion=1 configurations, T252448
 * Reuse source in galleries, T329662
 * Move isDiffMarker to DiffUtils
 * Remove $env from DiffUtils
 * Be consistent about PHPUtils uses in Ext/
 * Make first argument to DOMUtils::parseHTMLToFragment optional
 * Fix crasher when parser function name is not a valid title, T332242
 * Re-apply "Restore proper ETag handling"", T331629,  T310464
 * Add comments about maintaining language link metadata, T329303,  T34189,  T310512,  T296019
 * Deduplicate markNode with addDiffMark
 * Use constants for diff marks
 * Opt in to early warning comments during CI
 * Update copyright year

Mar 14: v0.18.0-a1 to kick off MediaWiki 1.41 release cycle

 * build: Updating eslint-config-wikimedia to 0.24.0
 * ScriptUtils: Don't hardcode parse.php script name

Mar 14 - Mar 16: ✅

 * Parser tests sync with all targets
 * TestRunner: support !!metadata sections and most metadata-related options, T270312
 * Parser tests: Fix Parsoid integrated mode breakage when using 'nohtml' option, T331871
 * build: Updating mediawiki/mediawiki-codesniffer to 41.0.0
 * Sync parserTests with core
 * Sync parserTests with core
 * Add AllowDynamicProperties to Wikimedia\Parsoid\DOM\Document, T314099

Mar 10: v0.17.0-a20 for dependencies in core on latest Parsoid code

 * TOCData Computation: Eliminate some egregious diffs with legacy parser, T331483
 * TOCData Computation: Handle top-level properly
 * Re-enable TOC data, T329740
 * Remove backward-compatibility methods for SectionMetadata::$byteOffset
 * Update wikimedia/{alea,remex-html,zest-css} dependencies, T320245
 * Move default file namespace to gallery
 * Deal with parser functions generating headings, T329740
 * Tweak SectionMetadata::prettyPrint for improved debuggability
 * [VIM tooling]: Update ptest.vim
 * StubMetadataCollector: handle numeric string category names correctly, T331084
 * TableFixups: Use entity wt, not rendered text, while fixing cells, T331486
 * Get rid of Cite-specific SiteConfig method, T268777
 * Report Parsoid section offsets in codepoints, not bytes, T319141
 * Restore proper ETag handling (reverted)

Mar 7 - Mar 9: ✅

 * Sync parserTests with core
 * Ignore gallery extsrc if native gallery is enabled, T329662
 * Roundtrip file prefix in galleries, T151367,  T211895,  T252246
 * Move redlink removal from "after normalization" to "canonicalization"
 * Cite CSS: Address unresolved FIXME to fix bhwiki CSS output

Feb 28 - Mar 2: ✅

 * Remove IE6 compatibility, T232563
 * Cite CSS: Add missing rule for data-mw-group refs with custom CSS
 * Cite CSS: Followup to 0aa4397e: Additional fix for mw-ref-linkback
 * Tweak SectionMetadata::prettyPrint to handle weird characters better
 * Sync parserTests with core
 * build: Pin PHPUnit to 9.5.28
 * Sync parserTests with core
 * Cite CSS: Deal with one more source of language configuration!
 * Use alt as link content for broken media, T273014
 * Cite CSS: Followup to 29df0a85: Fix init for non-decimal counters
 * Make start-rt-test.sh fail early, T330107
 * SiteConfig: Transition away from hardcoded Cite info, T268777

Feb 21 - Feb 24: ✅

 * Cite CSS: Suppress error-test group labels
 * Cite CSS: Bugfix for ref linkbacks using custom counters
 * Cite CSS: Generate :: for pseudo-elements
 * Add methods to enable "arbitrary language" for i18n, T263762
 * Sync parserTests with core
 * Sync parserTests with TMH
 * Temporarily suppress TOC data, T329740
 * ParserTests/TestRunner: we always use '{}' to represent the empty array
 * Sync parserTests with core
 * Tooling: make the gen_deploy_log script aware of more annotations

Feb 14 - Feb 16: ✅

 * Temporarily suppress TOC data, T329740

The following were part of v0.17.0-a15, which was superceded by -a16 for the wmf.23 deployment:
 * Remove outdated comment about lack of a PMID interwiki
 * SectionMetadata: Rename byteOffset to codepointOffset to reflect reality, T319141
 * Generate section metadata for TOC, T270199
 * TOCData/SectionMetadata: add ::prettyPrint method for parser tests
 * Sync parserTests with core + extensions
 * Restore removed empty alt attribute from html/parsoid section
 * Sync parserTests with core + ImageMap extension
 * Cite CSS: Use more W3C predefined counter styles where possible
 * Add explicit wgParserEnableLegacyMediaDOM config to parserTests, T314318
 * Update scripts to fix some bugs in baseline CSS in Cite extension

Feb 7 - Feb 10: ✅

 * Reconcile Parsoid's Sanitizer with upstream's., T247804
 * tools/sync-parserTests.js: clarify documentation
 * Sync parserTests with core
 * Add TOCData::getExtensionData and SectionMetadata::getExtensionData

Jan 31 - Feb 2: ✅

 * Add TOCData::setExtensionData
 * Fix LanguageConverter::implementsLanguageConversion; use Bcp47Code, T318401
 * Minor cleanup of the grammar file
 * build: Updating cookiejar to 2.1.4
 * Update start-rt-test.sh to php 7.4
 * Fix multiple ref nestings from the tag parser function, T326521
 * [Read Views Migration Tooling]: Scripts to generate cite CSS, T156350, T156351, T308094
 * Section wrapping should accept HTML h-tags as well, T289191
 * Remove the block_tag rule
 * build: Updating eslint to 8.31.0

Jan 23 - Jan 25: ✅

 * Remove the paragraph rule
 * Rename xmlish_tag_opened to just xmlish_tag
 * Permit old tvar syntax in all places where annotation tags are present
 * Avoid matching xmlish_tag unnecessarily
 * Move ScriptUtils to a namespace that makes it available in vendor
 * Sync parserTests with core
 * Treat behaviour switches as sol transparent in grammar, T327045
 * gen_deploy_log.sh: Git rid of stale comments that are no long relevant
 * Add "git fetch" to the regression-testing script

Jan 17 - Jan 19: ✅

 * Add documentation about localization required in section metadata
 * Add ContentMetadataCollector::setTOCData
 * SectionMetadata: Add ::setExtensionData to let extensions set custom data, T326677
 * TOCData: workaround for non-recursive JsonCodec, T312589
 * Sync parserTests with core
 * Adjust some whitespace in the grammar
 * Add a getter for i18n attribute names, T309024
 * Address FIXMEs in WTUtils A-tag helpers
 * Sync parserTests with core
 * Cleanup: Rename WTUtils helpers to better reflect expectations
 * Serialize empty twoway key in language variant, T302678
 * Add more output to parserTests for "unexpected pass" and "changes"
 * Use Parsoid's UrlUtils::parseUrl to parse wikilinks, T325329
 * Adding tests for copy-paste of table cells in rows, T319143

Jan 9 - Jan 11: ✅

 * [VIM tooling]: Syntax highlighting for parser tests
 * Sync parserTests with core
 * composer.json: Allow wikimedia/object-factory ^5.0.0
 * Sanitizer: Minor code style / perf tweak to sanitizeTitleURI
 * Mocha test for fetching old revision content, T324801
 * Followup to e121f4fc: Update hLevel property in SectionMetadata
 * Fix behavior of AddRedLinks when parse_url returns false, T325329
 * Don't reuse separator between newly inserted cell and cell, T319143
 * NodeData: Allow dynamic properties on PHP 8.2, T314099
 * DOM\Document: Add body parameter for caching, T324902
 * Lint away 'pxpx', T207032
 * Redlink cleanups and new tests, T309024