Parsoid/Deployments

From MediaWiki.org
Jump to navigation Jump to search

Planned deployments, linked from wikitech:Deployments. For a list of past deployments, look for 'parsoid' in wikitech:Server Admin Log.

See wikitech:Parsoid#Deploying changes to learn how to deploy a new version of Parsoid.

Contents

Monday, Oct. 15, 2018 around 1:32 pm PT: YesY Deployed 8f3ff40[edit]

  • T205642: Handle callers of parseWikitextToDOM with non-string attribute keys
  • T206003: Set request agent after updating uri with mwApiServer
  • Cite leakage
    • Remove more Cite leakage from html2wt code
    • Rename parsing pipeline options to something more semantic
    • Remove another source of Cite leaking in html2wt code
  • T187848: Fix token transformer return types
    • No special token array handling in AsyncTTM
    • Clean up token handling in SyncTTM
    • Ensure that we always use {tokens:[...]} form in TTM.
    • Ensure ParserFunctions flattens its token arrays.
    • Ensure LinkHandler flattens its token arrays.
    • Always return `tokens` array in token transformers.
    • Ensure LanguageConverter markup flattens token arrays in attributes
    • No need to check for extinct `{token:...}` form in TTM
  • bin/transformTests.js
    • Count failures during transformTests and exit with non-zero status if failed
    • T205455: Emit tests only when the token is transformed
    • Updates to keep up with TTM changes
    • T205743: Replaced token type 'COMMENT' with 'CommentTk'
    • Make bin/transformTests.js executable

Tuesday, Oct. 2, 2018 around 11:01 am PT: YesY Deployed 65d6f82[edit]

  • T163438: Fix the logic (and comments) when skipping the funnel into mwApiServer
  • ConfigRequest doesn't need a separate proxy argument
  • Remove the possibility of proxy being set on request options
  • T205674: Don't migrate template start markers past auto-inserted table ends
  • T205673: Pass through inTemplate pipeline options when handling language variant

Monday, Oct. 1, 2018 around 1:45 pm PT: YesY Deployed 224ecde[edit]

  • Bump content version to 2.0.0 + add downgrades to 1.8.0
  • Set strictAcceptCheck in prod
  • T198504, T133673: Use <audio> elements for rendering audio files
  • HTTP API:
    • T202666: Downgrade original content when serializing where possible
    • Add some metrics around downgrading

Thursday, Sep. 27, 2018 around 1:09 pm PT: YesY Deployed ff6ffb5 (deploy-2018-09-27 branch)[edit]

  • Forward compatibility for audio tags
  • Add some mocha tests around serializing 2.0.0 content

Thursday, Sep. 27, 2018 around 11:04 am PT: YesY Deployed af3a920 (deploy-2018-09-27 branch)[edit]

  • Update domino to 2.1.0
  • Reversion the data-mw split as 999.0.0
  • T198511: Use `resource` attribute for Media:.... links
  • T163438: Respect the proxy overriding even when going through the mwApiServer
  • Rename config / file names to clarify wmf-specific configurations
  • Make QuoteTransformer a proper ES6 class
  • Avoid using JS Function#bind() in QuoteTransformer
  • Avoid the use of Array.prototype.METHOD
  • Domino 2.0.3+ supports case-insensitive matches in querySelector
  • Remove one source of Cite leakage in generic html2wt code
  • T108776: Update sitematrix and remove special casing for labswiki
  • T205334: Remove unused token toString and associated unused (tracing) code
  • HTTP API:
    • Updates should return the original version
    • Separate updates from reparse / up/downgrades
    • Update the version in the head when downgrading
    • T114413: Ensure that Content-Language and Vary headers are not "undefined"
    • Avoid string concat for function names
    • Use uppercase for HTTP header names

Wednesday, Sep. 5, 2018 around 1:19 pm PT: YesY Deployed 740b3a4[edit]

  • Effectively revert 50056f6
  • Only add trailing ws to table cell if preceded by another cell on the row
  • Rewrite include_limits in terms of xmlish_tag
  • T198400, T202819: Update sitematrix.json
  • Collect configs for extensions in a single object in WikiConfig
  • Tweak extension config a bit more

Tuesday, Aug. 28, 2018 around 11:01 am PT: YesY Deployed 61086f6[edit]

  • Bump content version to 1.8.0
  • T199246: Expose content inside <includeonly> to editors via data-mw attribute
  • Tweak extension DOM config for extensibility
  • T198221: Support directionality for reference

Monday, Aug. 20, 2018 around 1:26 pm PT: YesY Deployed 129d71f[edit]

  • Updates to the Parsoid Extension API
    • Cite: Get rid of tokenHandler that relies on Parsoid internals (and associated bug fixes)
    • T130224: Enable extensions to distinguish between self-closed and empty refs
    • Set data-mw.body to undefined instead of null for extension tags
  • Get rid of unused extTagId parser option
  • Remove Cite-specific flag from parser pipeline options
  • Consolidate meta type regexp in AE and stop matching "Extension"
  • Rename wrapTemplates to expandTemplates to clarify its use
  • T199926: html -> wt: Parsoid sometimes trips up on | chars in hrefs
  • Move uri sanitizing utils to the Sanitizer
  • URI decode interwiki titles before sanitizing

Thursday, Aug. 16, 2018 around 11:47 am PT: YesY Deployed dbbad6a (deploy-2018-08-16 branch)[edit]

Monday, Aug. 6, 2018 around 1:15 pm PT: YesY Deployed e02124b[edit]

  • Post-tidy-replacement: Add detection for new lint
  • LanguageConverter: switch to byte-oriented state machines
  • Improve byte machine format
  • T199849: Improve templatedata spec compliance wrt leading and trailing newlines
  • T198400, T199577, T199509: Update sitematrix.json
  • T200403: Add a fast path to avoid unnecessarily retokenizing the extlink href
  • Use .getAttribute instead of Util.lookup where possible
  • T200403: Test for a valid protocol before attempting to tokenize extlink content
  • Convert error classes in ApiRequest to proper ES6 classes
  • Don't permit < in html attribute positions
  • T201054: Ignore solidus (/) in various attribute tokenizing states

Thursday, July 26, 2018 around 10:59 am PT: YesY Deployed cdf8ace[edit]

  • Remove pre_indent rule from the grammar
  • Update domino and jsdoc-wmf-theme
  • Use DOM-based p-wrapping of text found on same wikitext line as block nodes
    • T194806: Pare down some things in paragraph wrapping
    • p-wrapping code on DOM
    • T110004: Move to DOM based wrapping of bare text
    • Remove meta markers before handling empty elements
    • Close paragraphs when entering indent pre
    • Account for mw:(Start|End)Tags in isRenderingTransparentNode
    • Account for about attributes on empty paragraphs from templates
    • Don't force paragraphs inside blockquotes to serialize on a new line
    • Don't migrate trailing nls from fostered content under any condition
    • Update edge-case handling in migrateTemplateMarkerMetas
    • Quote transformer is tripped up on html table cells
    • Make migrateTemplateMarkerMetas more robust
    • Drop auto-inserted empty elements before p-wrapping
    • Migrate template marker metas past auto-inserted start/end tags
    • pwrap: Update auto-inserted start/end flags during tree splits
    • Remove auto-inserted empty elements after meta marker cleanup
  • T156099, T188478: Updated extension interface + native Parsoid impl of Poem
  • T194083: Followup on de1d7bc: Refine that original check

Monday, July 23, 2018 around 1:22 pm PT: YesY Deployed a1e851c (deploy-2018-07-23 branch)[edit]

  • T199808: Update foundationwiki url
  • T194083: Dump debugging info for nested inserted dom-diff flags error

Monday, June 25, 2018 around 11:20 am PT: YesY Deployed b068bb51[edit]

  • Implement pb2pb endpoint for language variant conversion
  • T197949: Ensure Parsoid doesn't crash on unimplemented/invalid language variant conversions
  • T197702: Correct Vary header in pb2pb endpoint for language variant conversion
  • T196799: Crasher fix: Set inTemplate for pipelines where it was missing, add further protection from missing data-mw parts
  • Cleanups:
    • Move _findHeaders to DOMUtils
    • Split ExtensionHandler from TemplateHandler
    • Get rid of unused _startDocumentPipeline and recordArgDict state
    • Convert DU.addRedLinks and TemplateRequest.setPageSrcInfo to Promise.async

Monday, June 11, 2018 around 1:49 pm PT: YesY Deployed 06b74d2[edit]

  • Set last nltk when encountering a block while pre collecting
  • Stop shimming array includes
  • T191843: TRY #2: Fix inefficiency in SyncTokenTransformManager
  • Get rid of pointless DU.hasNodeName helper

Thu, June 7, 2018, around 11:10 am PT: YesY Deployed 7819c9e7[edit]

  • T183706, T192726, T194879, T196357, T196360: Update sitematrix to activate newer wikis
  • Suppress p-wrapping of fostered content to match the php parser
  • T43716: Basic implementation of Serbian/Kurdish/Pig Latin LanguageConverter support
    • This functionality is not yet fully-compliant with PHP (or supported); it is being deployed in order to unblock the services team work on phab:T159985.

Monday, June 4, 2018 around 1:34 pm PT: YesY Deployed bd5a840[edit]

wt2html

  • T157418: Trim whitespace in wikitext headings, lists, tables
  • <span typeof="mw:FallbackId"> is a rendering transparent node
  • Fix incorrect pipeline options in AttributeExpander (subtle old bug)
  • T194777: Migrate trailing categories out of dt/dd tags as well
  • T195317: Update migrateTrailingCategories pass to handle comments
  • Add rudimentary tracing to QuoteTransformer
  • Set DSR on mw:FallbackId spans
  • T195174: Protect indented table syntax from indent pre parsing
  • T195174: Protect the pipe variable as well from indent pre
  • T194659: Fix off by one in attribute expander
  • T195227: Check for own properties when using object as a dict
  • T195171: Fix confusion when attribute expander is used for template targets

html2wt

  • T194763: Whitespace scrubbing should be sensitive to SOL-sensitive markup
  • Ensure diff markers aren't lost during normalizations
  • T194658: Empty td and th cells are serialized with a single whitespace char
  • Followup to eae8f0c0: Delete spurious normalization change
  • T195414: html2wt: Strip mw:FallbackId spans before DOM diffs
  • Skip deletion markers while counting children
  • T195486: Minimize whitespace dirty diffs in existing headings, tables, lists
  • T195486, T195414: WS-minimization heuristics only apply to comments & text nodes
  • T195414: Suppress whitespace heuristics for HTML versions < 1.7.0
  • Redo emitting of optional trailing space for <td>s and <th>s
  • html2wt: Get rid of useless mw:FallbackId checks

Other

  • Bump HTML version to 1.7.0
  • Upgrade request for node security advisory 664
  • Remove some pre-node-v6 compatibility code

Tuesday, May 29, 2018 around 11:04 am PT: YesY Reverted to fd49ab4[edit]

Again, because of T195414: Whitespace normalized outside of modified tree

Tuesday, May 29, 2018 around 10:36 am PT: YesY Deployed bf3a2fd2 Reverted as above[edit]

wt2html

  • T157418: Trim whitespace in wikitext headings, lists, tables
  • <span typeof="mw:FallbackId"> is a rendering transparent node
  • Fix incorrect pipeline options in AttributeExpander (subtle old bug)
  • T194777: Migrate trailing categories out of dt/dd tags as well
  • T195317: Update migrateTrailingCategories pass to handle comments
  • Add rudimentary tracing to QuoteTransformer
  • Set DSR on mw:FallbackId spans
  • T195174: Protect indented table syntax from indent pre parsing
  • T195174: Protect the pipe variable as well from indent pre
  • T194659: Fix off by one in attribute expander
  • T195227: Check for own properties when using object as a dict
  • T195171: Fix confusion when attribute expander is used for template targets

html2wt

  • T194763: Whitespace scrubbing should be sensitive to SOL-sensitive markup
  • Ensure diff markers aren't lost during normalizations
  • T194658: Empty td and th cells are serialized with a single whitespace char
  • Followup to eae8f0c0: Delete spurious normalization change
  • T195414: html2wt: Strip mw:FallbackId spans before DOM diffs
  • Skip deletion markers while counting children
  • T195486: Minimize whitespace dirty diffs in existing headings, tables, lists

Other

  • Upgrade request for node security advisory 664
  • Remove some pre-node-v6 compatibility code

Wednesday, May 23, 2018 around 1:56 pm PT: YesY Reverted to fd49ab4[edit]

Reverted because of T195414: Heading whitespace normalized outside of modified tree

Wednesday, May 23, 2018 around 1:18 pm PT: YesY Deployed dccfeafd Reverted as above[edit]

wt2html

  • T157418: Trim whitespace in wikitext headings, lists, tables
  • <span typeof="mw:FallbackId"> is a rendering transparent node
  • Fix incorrect pipeline options in AttributeExpander (subtle old bug)
  • T194777: Migrate trailing categories out of dt/dd tags as well
  • T195317: Update migrateTrailingCategories pass to handle comments
  • Add rudimentary tracing to QuoteTransformer
  • Set DSR on mw:FallbackId spans
  • T195174: Protect indented table syntax from indent pre parsing

html2wt

  • T194763: Whitespace scrubbing should be sensitive to SOL-sensitive markup
  • Ensure diff markers aren't lost during normalizations
  • T194658: Empty td and th cells are serialized with a single whitespace char
  • Followup to eae8f0c0: Delete spurious normalization change

Other

  • Upgrade request for node security advisory 664
  • Remove some pre-node-v6 compatibility code

Thursday, May 17, 2018 around 10:15 am PT: YesY Deployed fd49ab4 (deploy-2018-05-16 branch)[edit]

  • T194821: Whitelist wikibase-lexeme content model
  • T194687: s/doNotIgnoreMissingTitles/includeMissingTitles/ in TemplateDataRequests

Monday, May 14, 2018 around 1:54 pm PT: YesY Deployed 945ed23[edit]

  • T194082: TRY #2: normalizeDOM: normalize a node after it is merged with a sibling
  • T194083: Distinguish between inserted & deleted diff markers
  • T194084: More fun with non-string template targets
  • Code cleanup and refactoring:
    • Refactor & simplify couple DU helpers
    • Simplify emitsSolTransparentSingleLineWT DOM helper
    • Add DU.isRenderingTransparentNode helper
    • Cleanup normalizeSiblingPair for readability

Wednesday, May 9, 2018 around 1:17 pm PT: YesY Deployed 5ce2608[edit]

  • T194081: Fix crasher in moveFormatTagOutsideATag
  • T188118: Suppress autoInsertedEnd flags where not required
  • Update some dependencies

Monday, May 7, 2018 around 1:41 pm PT: YesY Deployed 6e38948[edit]

  • T192909: Unconditionally strip section tags
  • Cleanup section tags
  • The `html5-legacy` mode of ID generation has been removed from core
  • Return NlTk instead of "\n" in tokenizer almost everywhere
  • PEG: Inline table_line rule & rename table_lines to table_line
  • Parse more block constructs in media captions

Monday, April 30, 2018 around 1:21 pm PT: YesY Deployed 50b0588[edit]

  • normalizeDOM: normalize a node after it is merged with a sibling
  • T186358: Increase the default height mw:Audio to 32px
  • Fix newline handling in p-wrapper when # newlines > 2
  • T191700: Avoid piping links for formatting when possible
  • T192909: Sections that expand tpl ranges lose roundtrip data when dropped
  • Update sitematrix

Thursday, April 12, 2018 around 11:02 am PT: YesY Deployed 322b6e8[edit]

  • Allow DOMTraverser handlers to specify a context (w/o an explicit bind)
  • Sync addExtLinkClasses with new PHP strictly-syntactic rules
  • Stop adding valign classes to block media
  • T191839: Fix crasher in normalizeDOM.js
  • Revert "Fix inefficiency in SyncTokenTransformManager"

Monday, Apr. 9, 2018 around 1:32 pm PT: YesY Deployed edeeb60[edit]

  • Fix inefficiency in SyncTokenTransformManager
  • T191281: Serialize gallery image captions in single line context
  • Avoid cloning result token arrays
  • T187386: Protect against data-mw w/o parts when section wrapping
  • T185266: Prevent crashing on foundationwiki pages
  • Make the decodeURI / decodeURIComponent distinction more apparent
  • Code cleanup (without changes to functionality):
    • Rename Util.getArgInfo to Util.getExtArgInfo
    • Get rid of immediately invoked function
    • Make the Sanitizer "static" and decouple it from the parsing pipeline

Wednesday, Apr. 4, 2018 around 1:18 pm PT: YesY Deployed d887aff[edit]

  • Get rid of nested_block_line rule
  • Account for failure to make a major version bump for sections
  • Handle indented tables with content following it
  • Don't unwrap p-wrapping around [http://foo.com]-style wikitext
  • html2wt:
    • T177102: Add new lines before and after lists in wikitext
    • T189474: Image captions ignore paragraph tags
    • Extra newline now suppressed after a final table tag

Monday, Mar. 12, 2018 around 1:37 pm PT: YesY Deployed 16ced34[edit]

  • T188670: Fix even further crashes from non-string template targets
  • T90902: Remove &nbsp; and similar characters from section IDs
  • Minor: html2wt: Emit <br /> when we need to emit that HTML tag

Monday, Mar. 5, 2018 around 1:25 pm PT: YesY Deployed d115592[edit]

  • The `normalize` post processor isn't needed any more
  • Remove type from _startTokenPipeline
  • Suppress bogus warning about "variant" parameter to action=parse
  • T188591: xmlish tags in wikilink title position are invalid
  • Use the same string for the href in linkToMedia that got us there
  • BODY is not a block tag

Wednesday, Feb. 28, 2018 around 1:26 pm PT: YesY Deployed 1415a2a[edit]

  • Bump patch level of content version to 1.6.1
  • Bug fix in Async Token Transform Manager
  • T58756: external links class= now setting free, text and autonumber
  • DOM Passes: add option to enable/disable it on nested DOMs
  • Unbreak time tracing on nested DOM passes
  • T169006: Redirect to the right path
  • Handle exception from title-invalid-talk-namespace
  • Don't process token attributes unnecessarily
  • Don't allocate attribute arrays if not required

Monday, Feb. 26, 2018 around 1:42 pm PT: YesY Deployed 24c783c[edit]

  • Make Token "a real base class instead of some weird hackish object"
  • Use an ES6 class for wt2html/tt/TokenHandler
  • Use Promise.async/yield in env#getParserEnv / env#switchToConfig
  • Clean up:
    • Minor cleanup in AttributeExpander
    • Eliminate unnecessary property from state in TemplateHandler
    • Fix verifyTokensIntegrity
    • Simplify accumChain code in Async TTM
    • Upgrade eslint
    • Improve jsdoc

Tuesday, Feb. 20, 2018 around 10:34 am PT: YesY Deployed e5e8113[edit]

  • Update some dependencies
  • Clean up:
    • remove Batcher#onLegacyResponse
    • Move escapeRegExp from Util to JSUtil
    • Remove wrapperStyles dead code
    • Remove old defensive code
  • Speed up fast path (no logging) of Logger
  • Use Promise.async/yield in lib/ext/Cite
  • Use Promise.async/yield in lib/ext/Nowiki

Thursday, Feb. 15, 2018 around 10:42 am PT: YesY Deployed 0650195[edit]

  • LinkHandler: Inline the invocation in #getWikiLinkHandler
  • WTS: Use Promise.async/yield
  • wt2html/tt/LinkHandler: get rid of setupDataMW
  • Promisify AttributeExpander transformation
  • Bump to domino@2.0.1
  • Fix performance regression caused by 96203db

Wednesday, Feb. 14, 2018 around 1:30 pm PT: YesY Deployed caee2ed[edit]

  • Hoist Page out of the MWParserEnvironment class
  • Get rid of the handleUnbalancedTables DOM pass
  • Replace `.bind(this)` and `var self = this;` with arrow functions

Thursday, Feb. 8, 2018 around 10:27 am PT: YesY Deployed 961a5cf[edit]

  • T186630: Don't try to bump parser resource use if setPageSrcInfo is throwing exceptions
  • AttributeTransformManager#process now returns a Promise
  • Replace callbacks with Promises in Logger
  • Remove remaining uses of async module

Tuesday, Feb. 6, 2018 around 10:15 am PT: YesY Deployed 8a0ff6c[edit]

  • T183515: Permit extension tags in xmlish attribute values
  • Consolidate bailing out from ext/wiki links
  • T129372: Match core's parsing of gallery dimensions
  • T181408: Rethink responsive references wrappers
  • tsr end is only valid when bailing out if there's no content
  • Use autoGenerated property when deciding on selser reuse
  • Use Promise.async/yield in various places
  • Have expandValuesToDOM return a Promise
  • Add `TokenTransformManager.addTransformP` helper
  • Bunch of minor cleanup in Frame.expand & TTM
  • Allow Frame#expand to return a Promise
  • Fix implementation of padleft/padright

Monday, Jan. 29, 2018 around 1:23 pm PT: YesY Deployed 91854ff (deploy-2018-01-29 branch)[edit]

  • T185643: Strip everything before the first pf token
  • T185346: Stop clearing tplInfo if don't find a table for fostered content
  • T185385: Fix crasher from rendering self-closed html pre tags
  • T185267: Fix infinite recursion from linting named ref cycles
  • Fix a bug linting content from templated extension tags with lint handlers

Thursday, Jan. 18, 2018 around 1:03 pm PT: YesY Config update[edit]

  • Enable useWorker

Thursday, Jan. 18, 2018 around 11:17 am PT: YesY Deployed af06386[edit]

  • T45094: Replace <span> with <sup> for references
  • Deep clone expression result before modifying
  • Fix crasher from tokenizing headings

Tuesday, Jan. 16, 2018 around 10:30 am PT: YesY Deployed 231bfff[edit]

  • Update sitematrix
  • Assert supported options for the parser pipeline cache key
  • Restrict nested refs not coming from templates
  • Give some time for async loggers to deliver fatal messages
  • Add a function to create the references node uniformly
  • Improve handling of section-wrapping and extcontent boundary conflicts
  • Linter:
    • Fix crasher in Cite - don't assume well-formedness

Monday, Jan. 8, 2018 around 1:22 pm PT: YesY Deployed e133312[edit]

  • T182349: Account for SOL transparent templates in p-wrapping
  • Only use max depth detection since we aren't doing template expansion
  • Move addRedLinks to DOMPostProcessor; allow async processors
  • T43716, T159985: Create skeleton of language variant support in Parsoid API
  • Add node-pn package
  • Linter:
    • Simplify logic a bit for readability + fix file comment
    • T183893: Linter: Suppress multiple-unclosed-formatting-tags false positives
    • Don't crash while detecting fostered content 
    • Don't emit missing-end-tag warnings for builder-inserted tags

Tuesday, Jan. 2, 2018 around 10:21 am PT: YesY Deployed 28d7734[edit]

  • T143378: Template syntax w/ empty link target shouldn't be considered broken
  • Use a Set for trace/debug/dump flags
  • Log api warnings from batch requests
  • Pass the page name when parsing extension content
  • Cleanup and refactor so mw api warnings are always logged
  • T110961: Use pure compute workers for the request processing (disabled by default in the config)
  • T123446: Bring back request timeouts (disabled by default in the config)
  • Linter:
    • Suppress false positives for multiline-table-in-list
    • Additional updates to multiline-html-table-in-list
    • Suppress false positives for multiple-unclosed-formatting-tags
    • Move linting pass to the very end
    • Make linter its own self-contained DOM pass
    • Use extension mechanism to provide ext-specific lint handers 

Deployments from previous years[edit]