Parsoid/Deployments

Planned deployments, linked from wikitech:Deployments. For a list of past deployments, look for 'parsoid' in wikitech:Server Admin Log.

See wikitech:Parsoid#Deploying changes to learn how to deploy a new version of Parsoid.

Thursday, Aug. 16, 2018 around 11:47 am PT: Y Deployed dbbad6a (deploy-2018-08-16 branch)

• T201115: Prepare for MCR

Monday, Aug. 6, 2018 around 1:15 pm PT: Y Deployed e02124b

• Post-tidy-replacement: Add detection for new lint
• LanguageConverter: switch to byte-oriented state machines
• Improve byte machine format
• T199849: Improve templatedata spec compliance wrt leading and trailing newlines
• T198400, T199577, T199509: Update sitematrix.json
• T200403: Add a fast path to avoid unnecessarily retokenizing the extlink href
• Use .getAttribute instead of Util.lookup where possible
• T200403: Test for a valid protocol before attempting to tokenize extlink content
• Convert error classes in ApiRequest to proper ES6 classes
• Don't permit < in html attribute positions
• T201054: Ignore solidus (/) in various attribute tokenizing states

Thursday, July 26, 2018 around 10:59 am PT: Y Deployed cdf8ace

• Remove pre_indent rule from the grammar
• Update domino and jsdoc-wmf-theme
• Use DOM-based p-wrapping of text found on same wikitext line as block nodes
  • T194806: Pare down some things in paragraph wrapping
  • p-wrapping code on DOM
  • T110004: Move to DOM based wrapping of bare text
  • Remove meta markers before handling empty elements
  • Close paragraphs when entering indent pre
  • Account for mw:(Start|End)Tags in isRenderingTransparentNode
  • Account for about attributes on empty paragraphs from templates
  • Don't force paragraphs inside blockquotes to serialize on a new line
  • Don't migrate trailing nls from fostered content under any condition
  • Update edge-case handling in migrateTemplateMarkerMetas
  • Quote transformer is tripped up on html table cells
  • Make migrateTemplateMarkerMetas more robust
  • Drop auto-inserted empty elements before p-wrapping
  • Migrate template marker metas past auto-inserted start/end tags
  • pwrap: Update auto-inserted start/end flags during tree splits
  • Remove auto-inserted empty elements after meta marker cleanup
• T156099, T188478: Updated extension interface + native Parsoid impl of Poem
• T194083: Followup on de1d7bc: Refine that original check

Monday, July 23, 2018 around 1:22 pm PT: Y Deployed a1e851c (deploy-2018-07-23 branch)

• T199808: Update foundationwiki url
• T194083: Dump debugging info for nested inserted dom-diff flags error

Monday, June 25, 2018 around 11:20 am PT: Y Deployed b068bb51

• Implement pb2pb endpoint for language variant conversion
• T197949: Ensure Parsoid doesn't crash on unimplemented/invalid language variant conversions
• T197702: Correct Vary header in pb2pb endpoint for language variant conversion
• T196799: Crasher fix: Set inTemplate for pipelines where it was missing, add further protection from missing data-mw parts
• Cleanups:
  • Move _findHeaders to DOMUtils
  • Split ExtensionHandler from TemplateHandler
  • Get rid of unused _startDocumentPipeline and recordArgDict state
  • Convert DU.addRedLinks and TemplateRequest.setPageSrcInfo to Promise.async

Monday, June 11, 2018 around 1:49 pm PT: Y Deployed 06b74d2

• Set last nltk when encountering a block while pre collecting
• Stop shimming array includes
• T191843: TRY #2: Fix inefficiency in SyncTokenTransformManager
• Get rid of pointless DU.hasNodeName helper

Thu, June 7, 2018, around 11:10 am PT: Y Deployed 7819c9e7

• T183706, T192726, T194879, T196357, T196360: Update sitematrix to activate newer wikis
• Suppress p-wrapping of fostered content to match the php parser
• T43716: Basic implementation of Serbian/Kurdish/Pig Latin LanguageConverter support
  • This functionality is not yet fully-compliant with PHP (or supported); it is being deployed in order to unblock the services team work on phab:T159985.

Monday, June 4, 2018 around 1:34 pm PT: Y Deployed bd5a840

wt2html

• T157418: Trim whitespace in wikitext headings, lists, tables
• <span typeof="mw:FallbackId"> is a rendering transparent node
• Fix incorrect pipeline options in AttributeExpander (subtle old bug)
• T194777: Migrate trailing categories out of dt/dd tags as well
• T195317: Update migrateTrailingCategories pass to handle comments
• Add rudimentary tracing to QuoteTransformer
• Set DSR on mw:FallbackId spans
• T195174: Protect indented table syntax from indent pre parsing
• T195174: Protect the pipe variable as well from indent pre
• T194659: Fix off by one in attribute expander
• T195227: Check for own properties when using object as a dict
• T195171: Fix confusion when attribute expander is used for template targets

html2wt

• T194763: Whitespace scrubbing should be sensitive to SOL-sensitive markup
• Ensure diff markers aren't lost during normalizations
• T194658: Empty td and th cells are serialized with a single whitespace char
• Followup to eae8f0c0: Delete spurious normalization change
• T195414: html2wt: Strip mw:FallbackId spans before DOM diffs
• Skip deletion markers while counting children
• T195486: Minimize whitespace dirty diffs in existing headings, tables, lists
• T195486, T195414: WS-minimization heuristics only apply to comments & text nodes
• T195414: Suppress whitespace heuristics for HTML versions < 1.7.0
• Redo emitting of optional trailing space for <td>s and <th>s
• html2wt: Get rid of useless mw:FallbackId checks

Other

• Bump HTML version to 1.7.0
• Upgrade request for node security advisory 664
• Remove some pre-node-v6 compatibility code

Tuesday, May 29, 2018 around 11:04 am PT: Y Reverted to fd49ab4

Again, because of T195414: Whitespace normalized outside of modified tree

Tuesday, May 29, 2018 around 10:36 am PT: Y Deployed bf3a2fd2 Reverted as above

wt2html

• T157418: Trim whitespace in wikitext headings, lists, tables
• <span typeof="mw:FallbackId"> is a rendering transparent node
• Fix incorrect pipeline options in AttributeExpander (subtle old bug)
• T194777: Migrate trailing categories out of dt/dd tags as well
• T195317: Update migrateTrailingCategories pass to handle comments
• Add rudimentary tracing to QuoteTransformer
• Set DSR on mw:FallbackId spans
• T195174: Protect indented table syntax from indent pre parsing
• T195174: Protect the pipe variable as well from indent pre
• T194659: Fix off by one in attribute expander
• T195227: Check for own properties when using object as a dict
• T195171: Fix confusion when attribute expander is used for template targets

html2wt

• T194763: Whitespace scrubbing should be sensitive to SOL-sensitive markup
• Ensure diff markers aren't lost during normalizations
• T194658: Empty td and th cells are serialized with a single whitespace char
• Followup to eae8f0c0: Delete spurious normalization change
• T195414: html2wt: Strip mw:FallbackId spans before DOM diffs
• Skip deletion markers while counting children
• T195486: Minimize whitespace dirty diffs in existing headings, tables, lists

Other

• Upgrade request for node security advisory 664
• Remove some pre-node-v6 compatibility code

Wednesday, May 23, 2018 around 1:56 pm PT: Y Reverted to fd49ab4

Reverted because of T195414: Heading whitespace normalized outside of modified tree

Wednesday, May 23, 2018 around 1:18 pm PT: Y Deployed dccfeafd Reverted as above

wt2html

• T157418: Trim whitespace in wikitext headings, lists, tables
• <span typeof="mw:FallbackId"> is a rendering transparent node
• Fix incorrect pipeline options in AttributeExpander (subtle old bug)
• T194777: Migrate trailing categories out of dt/dd tags as well
• T195317: Update migrateTrailingCategories pass to handle comments
• Add rudimentary tracing to QuoteTransformer
• Set DSR on mw:FallbackId spans
• T195174: Protect indented table syntax from indent pre parsing

html2wt

• T194763: Whitespace scrubbing should be sensitive to SOL-sensitive markup
• Ensure diff markers aren't lost during normalizations
• T194658: Empty td and th cells are serialized with a single whitespace char
• Followup to eae8f0c0: Delete spurious normalization change

Other

• Upgrade request for node security advisory 664
• Remove some pre-node-v6 compatibility code

Thursday, May 17, 2018 around 10:15 am PT: Y Deployed fd49ab4 (deploy-2018-05-16 branch)

• T194821: Whitelist wikibase-lexeme content model
• T194687: s/doNotIgnoreMissingTitles/includeMissingTitles/ in TemplateDataRequests

Monday, May 14, 2018 around 1:54 pm PT: Y Deployed 945ed23

• T194082: TRY #2: normalizeDOM: normalize a node after it is merged with a sibling
• T194083: Distinguish between inserted & deleted diff markers
• T194084: More fun with non-string template targets
• Code cleanup and refactoring:
  • Refactor & simplify couple DU helpers
  • Simplify emitsSolTransparentSingleLineWT DOM helper
  • Add DU.isRenderingTransparentNode helper
  • Cleanup normalizeSiblingPair for readability

Wednesday, May 9, 2018 around 1:17 pm PT: Y Deployed 5ce2608

• T194081: Fix crasher in moveFormatTagOutsideATag
• T188118: Suppress autoInsertedEnd flags where not required
• Update some dependencies

Monday, May 7, 2018 around 1:41 pm PT: Y Deployed 6e38948

• T192909: Unconditionally strip section tags
• Cleanup section tags
• The `html5-legacy` mode of ID generation has been removed from core
• Return NlTk instead of "\n" in tokenizer almost everywhere
• PEG: Inline table_line rule & rename table_lines to table_line
• Parse more block constructs in media captions

Monday, April 30, 2018 around 1:21 pm PT: Y Deployed 50b0588

• normalizeDOM: normalize a node after it is merged with a sibling
• T186358: Increase the default height mw:Audio to 32px
• Fix newline handling in p-wrapper when # newlines > 2
• T191700: Avoid piping links for formatting when possible
• T192909: Sections that expand tpl ranges lose roundtrip data when dropped
• Update sitematrix

Thursday, April 12, 2018 around 11:02 am PT: Y Deployed 322b6e8

• Allow DOMTraverser handlers to specify a context (w/o an explicit bind)
• Sync addExtLinkClasses with new PHP strictly-syntactic rules
• Stop adding valign classes to block media
• T191839: Fix crasher in normalizeDOM.js
• Revert "Fix inefficiency in SyncTokenTransformManager"

Monday, Apr. 9, 2018 around 1:32 pm PT: Y Deployed edeeb60

• Fix inefficiency in SyncTokenTransformManager
• T191281: Serialize gallery image captions in single line context
• Avoid cloning result token arrays
• T187386: Protect against data-mw w/o parts when section wrapping
• T185266: Prevent crashing on foundationwiki pages
• Make the decodeURI / decodeURIComponent distinction more apparent
• Code cleanup (without changes to functionality):
  • Rename Util.getArgInfo to Util.getExtArgInfo
  • Get rid of immediately invoked function
  • Make the Sanitizer "static" and decouple it from the parsing pipeline

Wednesday, Apr. 4, 2018 around 1:18 pm PT: Y Deployed d887aff

• Get rid of nested_block_line rule
• Account for failure to make a major version bump for sections
• Handle indented tables with content following it
• Don't unwrap p-wrapping around [http://foo.com]-style wikitext
• html2wt:
  • T177102: Add new lines before and after lists in wikitext
  • T189474: Image captions ignore paragraph tags
  • Extra newline now suppressed after a final table tag

Monday, Mar. 12, 2018 around 1:37 pm PT: Y Deployed 16ced34

• T188670: Fix even further crashes from non-string template targets
• T90902: Remove &nbsp; and similar characters from section IDs
• Minor: html2wt: Emit <br /> when we need to emit that HTML tag

Monday, Mar. 5, 2018 around 1:25 pm PT: Y Deployed d115592

• The `normalize` post processor isn't needed any more
• Remove type from _startTokenPipeline
• Suppress bogus warning about "variant" parameter to action=parse
• T188591: xmlish tags in wikilink title position are invalid
• Use the same string for the href in linkToMedia that got us there
• BODY is not a block tag

Wednesday, Feb. 28, 2018 around 1:26 pm PT: Y Deployed 1415a2a

• Bump patch level of content version to 1.6.1
• Bug fix in Async Token Transform Manager
• T58756: external links class= now setting free, text and autonumber
• DOM Passes: add option to enable/disable it on nested DOMs
• Unbreak time tracing on nested DOM passes
• T169006: Redirect to the right path
• Handle exception from title-invalid-talk-namespace
• Don't process token attributes unnecessarily
• Don't allocate attribute arrays if not required

Monday, Feb. 26, 2018 around 1:42 pm PT: Y Deployed 24c783c

• Make Token "a real base class instead of some weird hackish object"
• Use an ES6 class for wt2html/tt/TokenHandler
• Use Promise.async/yield in env#getParserEnv / env#switchToConfig
• Clean up:
  • Minor cleanup in AttributeExpander
  • Eliminate unnecessary property from state in TemplateHandler
  • Fix verifyTokensIntegrity
  • Simplify accumChain code in Async TTM
  • Upgrade eslint
  • Improve jsdoc

Tuesday, Feb. 20, 2018 around 10:34 am PT: Y Deployed e5e8113

• Update some dependencies
• Clean up:
  • remove Batcher#onLegacyResponse
  • Move escapeRegExp from Util to JSUtil
  • Remove wrapperStyles dead code
  • Remove old defensive code
• Speed up fast path (no logging) of Logger
• Use Promise.async/yield in lib/ext/Cite
• Use Promise.async/yield in lib/ext/Nowiki

Thursday, Feb. 15, 2018 around 10:42 am PT: Y Deployed 0650195

• LinkHandler: Inline the invocation in #getWikiLinkHandler
• WTS: Use Promise.async/yield
• wt2html/tt/LinkHandler: get rid of setupDataMW
• Promisify AttributeExpander transformation
• Bump to domino@2.0.1
• Fix performance regression caused by 96203db

Wednesday, Feb. 14, 2018 around 1:30 pm PT: Y Deployed caee2ed

• Hoist Page out of the MWParserEnvironment class
• Get rid of the handleUnbalancedTables DOM pass
• Replace `.bind(this)` and `var self = this;` with arrow functions

Thursday, Feb. 8, 2018 around 10:27 am PT: Y Deployed 961a5cf

• T186630: Don't try to bump parser resource use if setPageSrcInfo is throwing exceptions
• AttributeTransformManager#process now returns a Promise
• Replace callbacks with Promises in Logger
• Remove remaining uses of async module

Tuesday, Feb. 6, 2018 around 10:15 am PT: Y Deployed 8a0ff6c

• T183515: Permit extension tags in xmlish attribute values
• Consolidate bailing out from ext/wiki links
• T129372: Match core's parsing of gallery dimensions
• T181408: Rethink responsive references wrappers
• tsr end is only valid when bailing out if there's no content
• Use autoGenerated property when deciding on selser reuse
• Use Promise.async/yield in various places
• Have expandValuesToDOM return a Promise
• Add `TokenTransformManager.addTransformP` helper
• Bunch of minor cleanup in Frame.expand & TTM
• Allow Frame#expand to return a Promise
• Fix implementation of padleft/padright

Monday, Jan. 29, 2018 around 1:23 pm PT: Y Deployed 91854ff (deploy-2018-01-29 branch)

• T185643: Strip everything before the first pf token
• T185346: Stop clearing tplInfo if don't find a table for fostered content
• T185385: Fix crasher from rendering self-closed html pre tags
• T185267: Fix infinite recursion from linting named ref cycles
• Fix a bug linting content from templated extension tags with lint handlers

Thursday, Jan. 18, 2018 around 1:03 pm PT: Y Config update

• Enable useWorker

Thursday, Jan. 18, 2018 around 11:17 am PT: Y Deployed af06386

• T45094: Replace <span> with <sup> for references
• Deep clone expression result before modifying
• Fix crasher from tokenizing headings

Tuesday, Jan. 16, 2018 around 10:30 am PT: Y Deployed 231bfff

• Update sitematrix
• Assert supported options for the parser pipeline cache key
• Restrict nested refs not coming from templates
• Give some time for async loggers to deliver fatal messages
• Add a function to create the references node uniformly
• Improve handling of section-wrapping and extcontent boundary conflicts
• Linter:
  • Fix crasher in Cite - don't assume well-formedness

Monday, Jan. 8, 2018 around 1:22 pm PT: Y Deployed e133312

• T182349: Account for SOL transparent templates in p-wrapping
• Only use max depth detection since we aren't doing template expansion
• Move addRedLinks to DOMPostProcessor; allow async processors
• T43716, T159985: Create skeleton of language variant support in Parsoid API
• Add node-pn package
• Linter:
  • Simplify logic a bit for readability + fix file comment
  • T183893: Linter: Suppress multiple-unclosed-formatting-tags false positives
  • Don't crash while detecting fostered content 
  • Don't emit missing-end-tag warnings for builder-inserted tags

Tuesday, Jan. 2, 2018 around 10:21 am PT: Y Deployed 28d7734

• T143378: Template syntax w/ empty link target shouldn't be considered broken
• Use a Set for trace/debug/dump flags
• Log api warnings from batch requests
• Pass the page name when parsing extension content
• Cleanup and refactor so mw api warnings are always logged
• T110961: Use pure compute workers for the request processing (disabled by default in the config)
• T123446: Bring back request timeouts (disabled by default in the config)
• Linter:
  • Suppress false positives for multiline-table-in-list
  • Additional updates to multiline-html-table-in-list
  • Suppress false positives for multiple-unclosed-formatting-tags
  • Move linting pass to the very end
  • Make linter its own self-contained DOM pass
  • Use extension mechanism to provide ext-specific lint handers 

Deployments from previous years

• Parsoid/Deployments/2013
• Parsoid/Deployments/2014
• Parsoid/Deployments/2015
• Parsoid/Deployments/2016
• Parsoid/Deployments/2017