Parsoid/Deployments/2017

Thursday, Dec. 14, 2017 around 11:41 am PT: ✅

 * : Handle extension <-> section boundary conflicts
 * Make the log levels configurable for the initial logger
 * Add some metrics to the http api to count the original version
 * Restore location data to start/complete parse logs
 * : Pseudo-section marker takes precedence over uneditable marker

Tuesday, Dec. 12, 2017 around 11:45 am PT: ✅

 * Bump Parsoid HTML version to 1.6.0
 * Section wrappers:
 * : Add wrappers to Parsoid output
 * : Don't emit wrappers in body only mode
 * Headings:
 * : Properly handle short headings
 * , : Update Parsoid to generate modern HTML5 IDs w/ legacy fallback
 * : Use mw:WikiLink/Interwiki for interwiki links
 * : Use figure-inline instead of span for inline media
 * : Stop adding packing wrappers for last nodes
 * : Abort on invalid chars in interwiki links
 * : Use a map for namespaceIds
 * Prefer wiki.mainpage to defaultPageName
 * Linter:
 * : Linter: Detect multiple-unclosed-formatting-tags lint errors
 * Two new tidy replacement issues
 * Enable missing-end-tag detection for quotes
 * HTTP API:
 * Slowly start to wean us off using parser env in http api routes
 * Reject lint requests if linting isn't enabled
 * Remove unreachable page name clear
 * : Make strictSSL configurable per wiki as well
 * Fix accepting a posted original title
 * : Preserve request method when redirecting to oldid
 * Remove the `bodyOnly` spelling for the API parameter

Wednesday, Dec. 6, 2017 around 1:34 pm PT: ✅ (deploy-2017-12-06 branch)

 * : Permit table rows from templates in figure captions
 * Stop rewriting html escaping everywhere
 * Move expansion reuse to the env so cpu workers can partake
 * : Remove top level dependence on jsDiff
 * Strip  from parser tests options in the parser
 * Use lib/parse.js in api/routes.js
 * lib/parse.js: Move env initializations to relevant parsing paths
 * lib/parse.js: Make wt2html and html2wt methods do what they say
 * Move out internal routes
 * : Selser shouldn't reuse orig sep for autoinserted tags
 * : Preserve original transclusion's parameter order
 * Improve parameter aliases handling when ordering template parameters

Tuesday, Nov. 14, 2017 around 10:15 am PT: ✅

 * Bump to mediawiki-title@0.6.5
 * Sync parserTests with core + code fixes to pass updated tests
 * : Handle pipe ending table attributes in figure captions
 * Rename stx_v data-parsoid flag to stx

Thursday, Nov. 9, 2017 around 10:28 am PT: ✅

 * Separate command line arg parsing from job execution
 * Git rid of unused brokenHTMLTag data-parsoid attribute
 * : Parse nested tables in figure captions
 * : Update sitematrix
 * Deps:
 * : Bump to domino@2.0.0 (improves DOM performance and expect improvements in parse latencies in production).
 * Bump to service-runner@2.4.0
 * html -> wt:
 * : Broken wikitext for link inside square brackets
 * No trailing newline required between and
 * Linter:
 * : Handle optional end tags in missing-end-tag category
 * : Fix false positive for tidy-font-bug
 * Turn off non-actionable mixed-content-templates category

Monday, Nov. 6, 2017 around 1:42 pm PT: ✅

 * Fix page title normalization when oldid is supplied
 * : Fix nowiki in LC markup tokenizing
 * : Fix infinite template name expanding
 * Shift newly parsed KVs by the old attribute offset
 * Linter: Image captions are not affected by tidy-font-bug
 * Refactoring:
 * Use scriptpath instead of wgScriptPath
 * Rename property s/str/html/ in xmlserializer return value
 * Be explicit about returning contentmodel
 * Move some env properties to the prototype

Monday, Oct. 30, 2017 around 1:26 pm PT: ✅

 * : Ref marker in caption in data-mw
 * Remove unnecessary serialize/parse of references content
 * Avoid serialize/parse of mw:dom-fragment-token content
 * Code cleanup:
 * There's only one refInRefProcessor
 * Refactor and document seperator serialization

Thursday, Oct. 26, 2017 around 10:30 am PT: ✅

 * Bump to domino@1.0.30
 * Performance optimizations based on DOM access:
 * Add DOMUtils.hasNChildren for quickly testing # of child nodes
 * Only use our hacked Node#normalize if domino is old
 * Remove unnecessary calls to Node#childNodes
 * Remove unnecessary calls to Node#childNodes.length
 * : Use replaceChild instead of insertBefore

Tuesday, Oct. 24, 2017 around 10:30 am PT: ✅

 * : Remove further access to src when not in selserMode
 * : Update reverse interwiki map to prefer language prefixes over others
 * Consolidate separator handling when emitting text
 * , : Linter: Add tidy-font-bug category

Monday, Oct. 23, 2017 around 1:25 pm PT: ✅

 * New run of fetch-sitematrix
 * Improve perf by removing calls to Date.now when not profiling
 * Limit core-js shimming to what we need
 * Use resource loader path for html5shiv
 * Avoid creating an array slice for every token in _getTransforms
 * Linter: Provide accurate DSR offsets for issues in extension content

Wednesday, Oct 11, 2017 around 1:18 pm PT: ✅

 * , : Use improved format specifier for TemplateData
 * Useless p-wrapper stripping shouldn't get tripped by tpl meta tags
 * Linter: Suppress more false positives for html5-misnesting
 * Linter: Suppress html5-misnesting false positives due to Tidy weirdness

Friday, Oct 6, 2017 around 11:25 pm PT: ✅ (deploy-2017-10-06 branch)

 * Linter: Suppress html5-misnesting false positives due to Tidy weirdness

Wednesday, Oct 4, 2017 around 2:03 pm PT: ✅

 * : P-wrapper: Generalize removal of useless p-wrappers
 * Linter: Improve detection of html5-misnesting issues to remove false positives
 * html2wt: Code cleanup and tweaks to handling of inter-element separator around text nodes

Thursday, September 28, 2017 around 10:45 am PT: ✅

 * Updates to several node module dependencies based on security advisories
 * Suppress logspam in production logs
 * Pass revid to parse request
 * Fix further crashes from non-string template targets
 * Linter:
 * : Flag misnested tags with different behavior in HTML5 vs Tidy
 * Lint multiple colon escaped links
 * Tweak tidy-whitespace-bug detection to handle br,hr,wbrs
 * : Handle linting in the presence of fostering and templating
 * : Don't emit template info in lint for extension tags
 * Turn off ignored-table-attr output
 * Move linter config properties to the linter config object
 * Fill in defaults for linting configs

Monday, September 25, 2017 around 1:30 pm PT: ✅

 * : Transfer about group to reference wrapper
 * : Fetch sitematrix for hiwikivoyage

Tuesday, September 19, 2017 around 11:00 am PT: ✅

 * : Support HTML5 elements in older browsers
 * : Remove excessive HTML entity encoding
 * : Improve handling of tokens in pf targets
 * : Caption splitting on pipe entity
 * : Add mw-empty-elt class to empty li items that can't be deleted
 * : Pass revid to batcher for preprocessing
 * : Add support for Cite's `responsive` parameter
 * Add 'references' class to reflist node

Wednesday, September 6, 2017 around 1:27 pm PT: ✅

 * Remove inline styling for vertical alignment in traditional galleries
 * : Render missing gallery images the same as plain
 * Accept figure-inline wrapper for inline images

Thursday, August 24, 2017 around 10:33 am PT: ✅

 * Linter: Tweak tidy-whitespace-bug output
 * All media should scale in gallery
 * Rename bidir/unidir => twoway/oneway in LC markup.

Monday, August 21, 2017 around 2:00 pm PT: ✅

 * Update sitematrix for bawikibooks

Wednesday, August 16, 2017 around 1:45 pm PT: ✅

 * Match php parser's attribute sanitizer
 * Support gradual migration to oneway/twoway in LC markup.
 * Linter: Add tidy-whitespace-bug linter category

Thursday, August 3, 2017 around 11:52 am PT: ✅

 * ,, : Add hiwikiversity, techconductwiki and wikimania2018wiki

Thursday, August 3, 2017 around 10:51 am PT: ✅

 * : Fix parse of empty param in link in template param
 * : Allow attribute names to use any Unicode "Letter" or "Number"
 * : Accept url in wikilink target position while tokenizing

Monday, July 31, 2017 around 1:00 pm PT: ✅

 * : Piped link with unclosed extlink-like text
 * : Aliases get the same sort order as paramOrder
 * : LanguageConverter support in Parsoid

Thursday, July 20, 2017 around 11:48 am PT: ✅

 * Make multiple colons escaping interlanguage links invalid
 * Make sanitizer independent of the manager (step toward )
 * Set the modified flag consistently for attrs w/o shadow info
 * : Colon prefixed media links still point to media
 * Simplify wikilink validation logic

Thursday, July 13, 2017 around 11:23 am PT: ✅

 * : Media links should point to url not thumb
 * Update sitematrix.json for Thuɔŋjäŋ and Kabɩyɛ

Thursday, June 29, 2017 around 10:21 am PT: ✅

 * , : More permissive attribute name parsing
 * : Permit empty param in final position of template arg
 * Bump mediawiki-title to 0.6.3
 * Add a workaround to try all user namespace aliases
 * : Linter: Log error when DSR is missing

Monday, June 26, 2017 around 11:20 am PT: ✅

 * Bump HTML version to 1.5.0
 * Bump mediawiki-title to 0.6.1
 * , Implement rendering of redlinks, etc. as a post-processor
 * Followup on : Don't buffer redirect text in start of file posn
 * Require some form of whitespace after serializing redirects

Wednesday, June 21, 2017 around 1:15 pm PT: ✅

 * Parse block line syntax on the same line as redirects
 * : Hoist redirect to the top of the wikitext
 * Use case-insensitive regexps for detecting interwiki prefixes in link hrefs
 * : Use mediawiki-title to resolve the template namespace
 * : Add atjwiki
 * Suppress warnings for "proofread-index" contentmodel

Tuesday, June 20, 2017 around 10:30 am PT: e2e2b5f6 to be deployed Cancelled because of a problem noticed in canaries during deploy

 * Bump HTML version to 1.5.0
 * , Implement rendering of redlinks, etc. as a post-processor
 * Parse block line syntax on the same line as redirects
 * : Hoist redirect to the top of the wikitext
 * Use case-insensitive regexps for detecting interwiki prefixes in link hrefs
 * : Use mediawiki-title to resolve the template namespace
 * : Add atjwiki
 * Suppress warnings for "proofread-index" contentmodel

Thursday, June 8, 2017 around 10:30 am PT: ✅

 * : Handle VE-style interwiki shortcuts
 * : Use non-greedy comment-stripping regexp in the sanitizer
 * Use wikilinkHandler to escape wikilink fragments.

Monday, June 5, 2017 around 1:11 pm PT: ✅

 * Fix html2wt of new transclusion/extension added after a list
 * : Use generalized concept of lintable content models

Tuesday, May 30, 2017 around 10:17 am PT: ✅

 * Moving setting env properties closer to where they're used
 * Don't modify passed in apiConf
 * : Use already resolved href when available
 * : Handle interwiki shortcuts
 * WikitextSerializer: Handle new TemplateData API option
 * Stop using usePHPPreProcessor as a proxy for an existing mw api to parse extensions

Thursday, May 25, 2017 around 2:15 pm PT: ✅

 * Revert "T161151: Linter: Emit fully resolved template names"

Monday, May 22, 2017 around 1:15 pm PT: ✅

 * Remove dependence on native parser functions
 * Only skip to the phase end in native template expansion
 * Preprocessor precedence: rightmost-opening/"broken-link"/"broken-template"
 * : Prevent wrapping of action=parse output

Monday, May 15, 2017 around 1:12pm PT: ✅
wt2html:
 * : Fix logic when unpacking dom fragments
 * , : Add mw-parser-output class to Parsoid's output
 * Refactor the pipetrick handler to avoid using `text`

html2wt:
 * : Fix unhandled detection of modified link content
 * Fix bugs serializing modified content in simple wikilink form
 * html2wt: Reconfigure checks in getRoundTripData for links
 * Fix typo introduced in f051b262
 * Return to previous modified state

Linter:
 * , : Add an API endpoint to get lint errors for wikitext
 * : Linter: Emit fully resolved template names
 * : Linter: Add a flag to templateInfo for multi-part templated blocks
 * : Only lint pages that have wikitext contentmodel

API:
 * Assert valid get formats in http api
 * Assert valid transformations between formats in the http api

Tuesday, May 9, 2017 around 11:22 am PT: ✅

 * Remove support for pb2html in the http api
 * : Render [[Media:..]] wikitext as media links

Monday, May 8, 2017 around 1:15 pm PT: 9d8badc8 to be deployed Postponed due to etcd failures

 * Remove support for pb2html in the http api
 * : Render [[Media:..]] wikitext as media links

Wednesday, April 26, 2017 around 1:55 pm PT: ✅

 * : Remove .mw-body from Parsoid content
 * Bump the content version to 1.4.0 for a/v
 * Remove non-actionable warnings about stripped empty elements
 * Remove advanced PDoc usage from the jsapi
 * Fetch LanguageConverter status as part of wiki configuration
 * : Support video and audio content
 * : Set default dimensions for audio files

Tuesday, April 25, 2017 around 10:17 am PT: ✅

 * Added fallback for
 * Update site matrix for dtywiki
 * : Handle templated template names
 * Don't recognize invalid template targets
 * Factor out common encapsulation pattern
 * : Fix regression from
 * File handling:
 * Fetch imageinfo for both the file and manualthumb
 * Pull out size extraction from image handler
 * : Read thumb sizes from siteinfo
 * : Honour the "page" option for files
 * Linter:
 * Provide name of stripped tags
 * : Don't lint fostered rendering-transparent nodes
 * Linter detection for

Wednesday, April 12, 2017 around 1:12 pm PT: ✅

 * Lint misnested tags for better accuracy
 * Address edge case in DSR code that affects lint output in some cases
 * Add linter detection for

Thursday, April 6, 2017 around 10:19 am PT: ✅

 * Upgrade service-runner, and various other deps

Wednesday, April 5, 2017 around 1:18 pm PT: ✅

 * : Handle anchors without hrefs
 * Add additional options to slideshow gallery
 * Slideshow does not support per row option
 * : Apply all the valid gallery attributes

Wednesday, Mar 29, 2017 around 1:21 pm PT: ✅

 * : Only send lints for full parse
 * Fixup redirect-detecting regular expressions in multiple places
 * : Fix serializing multi-line indent-pre w/ sol wt syntax
 * : Lower heap limit to 600m

Monday, Mar 27, 2017 around 1:21 pm PT: ✅

 * : Ignore thumbtime linter errors temporarily
 * Add follow_redirects query param to the wt2html endpoint in dev API
 * : Apply defined classes to galleries
 * : Escape extlink content when containing ] anywhere

Wednesday, Mar 8, 2017 around 1:19 pm PT: ✅

 * Enable linting in production
 * Automatically detect whether Linter extension is installed
 * : Implement ; fix language code in HTML
 * ParserTests
 * Add an option to force strong normalization of parser tests
 * Fix bin/parserTests.js --help
 * Clean up parserTests output
 * Collect and print summary information at end of parser tests

Wednesday, Mar 1, 2017 around 1:45 pm PT: ✅

 * Fix namespace bug exposed by
 * Update CSS modules added to header
 * Use videoinfo if it's available
 * useVideoInfo is only for legacy requests
 * : Set mw:Error and provide error info in data-mw for ext/tpl
 * Bug fix in Async Token Transform Manager
 * Refactor in link handler
 * Refactor file handling a bit
 * Break img element handling out to its own func
 * Refactor figure html2wt to be a little more elt agnostic
 * Remove unused PrefixOptionsReverseMap
 * Pass dataMw to the media handlers
 * Fetch link info from data-mw even when no link is present
 * ParserTests, and bunch of fixes and refactors, including:
 * Pull out a parserTests.utils.js
 * Use sync-parserTests.js with other targets
 * : Run tests against the Cite parserTests
 * Move parsoid's ref tests to citeParserTests.txt
 * Avoid max callstack size exceeded in parserTests
 * Add TimedMediaHandler parserTests.txt
 * Get rid of PARSOID_MOCKAPI_URL env var
 * : Make tests php only
 * : Update baseconfigs, this time including enwiki

Tuesday, Feb 14, 2017 around 10:15 am PT: ✅

 * , : Treat html pre and nowiki as extension tags

Tuesday, Feb 7, 2017 around 10:15 am PT: ✅

 * Code refactoring in TemplateHandler
 * Don't count known images as errors
 * : Remove implicit_table_data_tag rule
 * Remove special-case non-void semantics for

Tuesday, Jan 31, 2017 around 10:50 am PT: ✅

 * : Accept entities in extlink href and url links
 * Stop normalizing hrefs by stripping "./" in html/parsoid sections
 * : Set relativeLinkPrefix before setting titlePageURI
 * Fallback to mw:Error for missing images
 * Get wfIsBadImage information in API requests
 * Shadow href w/ the original value, not the current one
 * API improvements needed for ProofreadPage extension
 * Set src as the original value when using title as href
 * Use onAnchor for sanitizing link href's as well
 * Bump to domino@1.0.28
 * Fix to resolveTemplateTarget to better handle magic word aliases

Tuesday, Jan 24, 2017 around 10:45 am PT: ✅

 * : Port sanitizer changes from core commit feb23b46
 * Switch to npm@3
 * Bump service-runner to 2.1.13 and various deps minor versions
 * Frame args is expected to be an array
 * Don't accept pipe unconditionally in extlink
 * : Serialize gallery without attrs in data-mw
 * : Fix crasher from ConstrainedText
 * flow-board is a known contentmodel

Wednesday, Jan 18, 2017 - Thursday, Jan 19, 2017: ✅

 * : Upgrading nodejs to v6

Thursday, Jan 5, 2017 around 10:16 am PT: ✅
wt2html tokenizer Other wt2html Linter (not yet enabled in production) Code cleanup
 * Make the tplarg rule resemble template
 * tplargs don't have k=v pairs, only values
 * Give a chance to break on | in element attribute name in template
 * Suppress breaking on table pipe in tags
 * Suppress breaking on tableCellArg in block tags too
 * Remove unreachable action
 * : Extensions take precedence over templates
 * Get rid of the generic_tag rule
 * Escape cite ids with Sanitizer.escapeId
 * : Fix cite hrefs to render properly
 * Percent-encode | char in the stylesheet link tag in
 * Percent-encode the entire thing
 * Get rid of BehaviorSwitchPreprocessor
 * : Complete templatearg representation in spec
 * Fix obsolete tags linting + ignore BIG
 * Cleanup and add tests for multi-template
 * Code cleanup in MWParserEnvironment.js
 * ApiRequest.js.request: Add comment to clarify intent