Parsoid/Deployments/2017

From mediawiki.org

Thursday, Dec. 14, 2017 around 11:41 am PT: Yes Deployed ca20680[edit]

  • task T182793: Handle extension <-> section boundary conflicts
  • Make the log levels configurable for the initial logger
  • Add some metrics to the http api to count the original version
  • Restore location data to start/complete parse logs
  • task T182774: Pseudo-section marker takes precedence over uneditable marker

Tuesday, Dec. 12, 2017 around 11:45 am PT: Yes Deployed 741fc5d[edit]

  • Bump Parsoid HTML version to 1.6.0
  • Section wrappers:
  • Headings:
  • task T97093: Use mw:WikiLink/Interwiki for interwiki links
  • task T118520: Use figure-inline instead of span for inline media
  • task T181229: Stop adding packing wrappers for last nodes
  • task T182338: Abort on invalid chars in interwiki links
  • task T65370: Use a map for namespaceIds
  • Prefer wiki.mainpage to defaultPageName
  • Linter:
    • task T182170: Linter: Detect multiple-unclosed-formatting-tags lint errors
    • Two new tidy replacement issues
    • Enable missing-end-tag detection for quotes
  • HTTP API:
    • Slowly start to wean us off using parser env in http api routes
    • Reject lint requests if linting isn't enabled
    • Remove unreachable page name clear
    • task T66003: Make strictSSL configurable per wiki as well
    • Fix accepting a posted original title
    • task T169006: Preserve request method when redirecting to oldid
    • Remove the `bodyOnly` spelling for the API parameter

Wednesday, Dec. 6, 2017 around 1:34 pm PT: Yes Deployed 01c1fc3 (deploy-2017-12-06 branch)[edit]

  • task T178253: Permit table rows from templates in figure captions
  • Stop rewriting html escaping everywhere
  • Move expansion reuse to the env so cpu workers can partake
  • task T61840: Remove top level dependence on jsDiff
  • Strip [[ ]] from parser tests options in the parser
  • Use lib/parse.js in api/routes.js
    • lib/parse.js: Move env initializations to relevant parsing paths
    • lib/parse.js: Make wt2html and html2wt methods do what they say
  • Move out internal routes
  • task T180930: Selser shouldn't reuse orig sep for autoinserted tags
  • task T179259: Preserve original transclusion's parameter order
  • Improve parameter aliases handling when ordering template parameters

Tuesday, Nov. 14, 2017 around 10:15 am PT: Yes Deployed e71937d0[edit]

  • Bump to mediawiki-title@0.6.5
  • Sync parserTests with core + code fixes to pass updated tests
  • task T178253: Handle pipe ending table attributes in figure captions
  • Rename stx_v data-parsoid flag to stx

Thursday, Nov. 9, 2017 around 10:28 am PT: Yes Deployed 2887b5ad[edit]

  • Separate command line arg parsing from job execution
  • Git rid of unused brokenHTMLTag data-parsoid attribute
  • task T178253: Parse nested tables in figure captions
  • task T173643: Update sitematrix
  • Deps:
    • task T176728: Bump to domino@2.0.0 (improves DOM performance and expect improvements in parse latencies in production).
    • Bump to service-runner@2.4.0
  • html -> wt:
    • task T180010: Broken wikitext for link inside square brackets
    • No trailing newline required between <p> and <body>
  • Linter:
    • task T171381: Handle optional end tags in missing-end-tag category
    • task T179757: Fix false positive for tidy-font-bug
    • Turn off non-actionable mixed-content-templates category

Monday, Nov. 6, 2017 around 1:42 pm PT: Yes Deployed 6cb77104[edit]

  • Fix page title normalization when oldid is supplied
  • task T179579: Fix nowiki in LC markup tokenizing
  • task T175792: Fix infinite template name expanding
  • Shift newly parsed KVs by the old attribute offset
  • Linter: Image captions are not affected by tidy-font-bug
  • Refactoring:
    • Use scriptpath instead of wgScriptPath
    • Rename property s/str/html/ in xmlserializer return value
    • Be explicit about returning contentmodel
    • Move some env properties to the prototype

Monday, Oct. 30, 2017 around 1:26 pm PT: Yes Deployed 292633c4[edit]

  • task T133334: Ref marker in caption in data-mw
  • Remove unnecessary serialize/parse of references content
  • Avoid serialize/parse of mw:dom-fragment-token content
  • Code cleanup:
    • There's only one refInRefProcessor
    • Refactor and document seperator serialization

Thursday, Oct. 26, 2017 around 10:30 am PT: Yes Deployed 8e99708a[edit]

  • Bump to domino@1.0.30
  • Performance optimizations based on DOM access:
    • Add DOMUtils.hasNChildren() for quickly testing # of child nodes
    • Only use our hacked Node#normalize if domino is old
    • Remove unnecessary calls to Node#childNodes
    • Remove unnecessary calls to Node#childNodes.length
    • task T176728: Use replaceChild instead of insertBefore

Tuesday, Oct. 24, 2017 around 10:30 am PT: Yes Deployed a3be9cfc[edit]

  • task T96923: Remove further access to src when not in selserMode
  • task T177784: Update reverse interwiki map to prefer language prefixes over others
  • Consolidate separator handling when emitting text
  • task T176568, task T25467: Linter: Add tidy-font-bug category

Monday, Oct. 23, 2017 around 1:25 pm PT: Yes Deployed 1cefef12[edit]

  • New run of fetch-sitematrix
  • Improve perf by removing calls to Date.now() when not profiling
  • Limit core-js shimming to what we need
  • Use resource loader path for html5shiv
  • Avoid creating an array slice for every token in _getTransforms
  • Linter: Provide accurate DSR offsets for issues in extension content

Wednesday, Oct 11, 2017 around 1:18 pm PT: Yes Deployed ddf7b293[edit]

  • task T138492, task T135667: Use improved format specifier for TemplateData
  • task T177612 Useless p-wrapper stripping shouldn't get tripped by tpl meta tags
  • Linter: Suppress more false positives for html5-misnesting
  • Linter: Suppress html5-misnesting false positives due to Tidy weirdness

Friday, Oct 6, 2017 around 11:25 pm PT: Yes Deployed 772e11bf (deploy-2017-10-06 branch)[edit]

  • Linter: Suppress html5-misnesting false positives due to Tidy weirdness

Wednesday, Oct 4, 2017 around 2:03 pm PT: Yes Deployed 477942c3[edit]

  • task T177115: P-wrapper: Generalize removal of useless p-wrappers
  • Linter: Improve detection of html5-misnesting issues to remove false positives
  • html2wt: Code cleanup and tweaks to handling of inter-element separator around text nodes

Thursday, September 28, 2017 around 10:45 am PT: Yes Deployed 2f4b9a8c[edit]

  • Updates to several node module dependencies based on security advisories
  • Suppress logspam in production logs
  • Pass revid to parse request
  • Fix further crashes from non-string template targets
  • Linter:
    • task T176363: Flag misnested tags with different behavior in HTML5 vs Tidy
    • Lint multiple colon escaped links
    • Tweak tidy-whitespace-bug detection to handle br,hr,wbrs
    • task T176151: Handle linting in the presence of fostering and templating
    • task T170832: Don't emit template info in lint for extension tags
    • Turn off ignored-table-attr output
    • Move linter config properties to the linter config object
    • Fill in defaults for linting configs

Monday, September 25, 2017 around 1:30 pm PT: Yes Deployed 4230c27b[edit]

Tuesday, September 19, 2017 around 11:00 am PT: Yes Deployed 05a0965[edit]

  • task T122965: Support HTML5 elements in older browsers
  • task T175101: Remove excessive HTML entity encoding
  • task T173384: Improve handling of tokens in pf targets
  • task T173061: Caption splitting on pipe entity
  • task T172896: Add mw-empty-elt class to empty li items that can't be deleted
  • task T174977: Pass revid to batcher for preprocessing
  • task T159894: Add support for Cite's `responsive` parameter
  • Add 'references' class to reflist node

Wednesday, September 6, 2017 around 1:27 pm PT: Yes Deployed f9d367ea[edit]

  • Remove inline styling for vertical alignment in traditional galleries
  • task T169342: Render missing gallery images the same as plain
  • Accept figure-inline wrapper for inline images

Thursday, August 24, 2017 around 10:33 am PT: Yes Deployed 538dad7f[edit]

  • Linter: Tweak tidy-whitespace-bug output
  • All media should scale in gallery
  • Rename bidir/unidir => twoway/oneway in LC markup.

Monday, August 21, 2017 around 2:00 pm PT: Yes Deployed 28a9a22b[edit]

  • Update sitematrix for bawikibooks

Wednesday, August 16, 2017 around 1:45 pm PT: Yes Deployed 1832a78e[edit]

  • Match php parser's attribute sanitizer
  • Support gradual migration to oneway/twoway in LC markup.
  • Linter: Add tidy-whitespace-bug linter category

Thursday, August 3, 2017 around 11:52 am PT: Yes Deployed 6e1a20d5[edit]

Thursday, August 3, 2017 around 10:51 am PT: Yes Deployed 651f12c2[edit]

  • task T119802: Fix parse of empty param in link in template param
  • task T73386: Allow attribute names to use any Unicode "Letter" or "Number"
  • task T170289: Accept url in wikilink target position while tokenizing

Monday, July 31, 2017 around 1:00 pm PT: Yes Deployed 08114f35[edit]

Thursday, July 20, 2017 around 11:48 am PT: Yes Deployed a89a9cc4[edit]

  • Make multiple colons escaping interlanguage links invalid
  • Make sanitizer independent of the manager (step toward task T54941)
  • Set the modified flag consistently for attrs w/o shadow info
  • task T169293: Colon prefixed media links still point to media
  • Simplify wikilink validation logic

Thursday, July 13, 2017 around 11:23 am PT: Yes Deployed 71c07681[edit]

  • task T169293: Media links should point to url not thumb
  • Update sitematrix.json for Thuɔŋjäŋ and Kabɩyɛ

Thursday, June 29, 2017 around 10:21 am PT: Yes Deployed b4187f18[edit]

  • task T168404, task T153203: More permissive attribute name parsing
  • task T168675: Permit empty param in final position of template arg
  • Bump mediawiki-title to 0.6.3
  • Add a workaround to try all user namespace aliases
  • task T168900: Linter: Log error when DSR is missing

Monday, June 26, 2017 around 11:20 am PT: Yes Deployed b59045f2[edit]

  • Bump HTML version to 1.5.0
  • Bump mediawiki-title to 0.6.1
  • task T39902, task T149794 Implement rendering of redlinks, etc. as a post-processor
  • Followup on f171985: Don't buffer redirect text in start of file posn
  • Require some form of whitespace after serializing redirects

Wednesday, June 21, 2017 around 1:15 pm PT: Yes Deployed 881ade32[edit]

  • Parse block line syntax on the same line as redirects
  • task T127421: Hoist redirect to the top of the wikitext
  • Use case-insensitive regexps for detecting interwiki prefixes in link hrefs
  • task T167933: Use mediawiki-title to resolve the template namespace
  • task T167714: Add atjwiki
  • Suppress warnings for "proofread-index" contentmodel

Tuesday, June 20, 2017 around 10:30 am PT: e2e2b5f6 to be deployed Cancelled because of a problem noticed in canaries during deploy[edit]

  • Bump HTML version to 1.5.0
  • task T39902, task T149794 Implement rendering of redlinks, etc. as a post-processor
  • Parse block line syntax on the same line as redirects
  • task T127421: Hoist redirect to the top of the wikitext
  • Use case-insensitive regexps for detecting interwiki prefixes in link hrefs
  • task T167933: Use mediawiki-title to resolve the template namespace
  • task T167714: Add atjwiki
  • Suppress warnings for "proofread-index" contentmodel

Thursday, June 8, 2017 around 10:30 am PT: Yes Deployed 108eed81[edit]

  • task T136653: Handle VE-style interwiki shortcuts
  • task T167081: Use non-greedy comment-stripping regexp in the sanitizer
  • Use wikilinkHandler to escape wikilink fragments.

Monday, June 5, 2017 around 1:11 pm PT: Yes Deployed 141fc07d[edit]

  • Fix html2wt of new transclusion/extension added after a list
  • task T166655: Use generalized concept of lintable content models

Tuesday, May 30, 2017 around 10:17 am PT: Yes Deployed d07dfe1a[edit]

  • Moving setting env properties closer to where they're used
  • Don't modify passed in apiConf
  • task T161151: Use already resolved href when available
  • task T136653: Handle interwiki shortcuts
  • WikitextSerializer: Handle new TemplateData API option
  • Stop using usePHPPreProcessor as a proxy for an existing mw api to parse extensions

Thursday, May 25, 2017 around 2:15 pm PT: Yes Deployed 5b52d07b[edit]

  • Revert "T161151: Linter: Emit fully resolved template names"

Monday, May 22, 2017 around 1:15 pm PT: Yes Deployed ebac1890[edit]

  • Remove dependence on native parser functions
  • Only skip to the phase end in native template expansion
  • Preprocessor precedence: rightmost-opening/"broken-link"/"broken-template"
  • task T165139: Prevent <div> wrapping of action=parse output

Monday, May 15, 2017 around 1:12pm PT: Yes Deployed a182c227f[edit]

wt2html:

  • task T141226: Fix logic when unpacking dom fragments
  • task T164792, task T37247: Add mw-parser-output class to Parsoid's output
  • Refactor the pipetrick handler to avoid using `text()`

html2wt:

  • task T153107: Fix unhandled detection of modified link content
  • Fix bugs serializing modified content in simple wikilink form
  • html2wt: Reconfigure checks in getRoundTripData for links
  • Fix typo introduced in f051b262
  • Return to previous modified state

Linter:

API:

  • Assert valid get formats in http api
  • Assert valid transformations between formats in the http api

Tuesday, May 9, 2017 around 11:22 am PT: Yes Deployed 9d8badc8[edit]

  • Remove support for pb2html in the http api
  • task T151277: Render [[Media:..]] wikitext as media links

Monday, May 8, 2017 around 1:15 pm PT: 9d8badc8 to be deployed Postponed due to etcd failures[edit]

  • Remove support for pb2html in the http api
  • task T151277: Render [[Media:..]] wikitext as media links

Wednesday, April 26, 2017 around 1:55 pm PT: Yes Deployed 4949857a[edit]

  • task T116508: Remove .mw-body from Parsoid content
  • Bump the content version to 1.4.0 for a/v
  • Remove non-actionable warnings about stripped empty elements
  • Remove advanced PDoc usage from the jsapi
  • Fetch LanguageConverter status as part of wiki configuration
  • task T64270: Support video and audio content
  • task T133673: Set default dimensions for audio files

Tuesday, April 25, 2017 around 10:17 am PT: Yes Deployed 55b90511[edit]

  • Added fallback for general.legaltitlechars
  • Update site matrix for dtywiki
  • task T153885: Handle templated template names
  • Don't recognize invalid template targets
  • Factor out common encapsulation pattern
  • task T163330: Fix regression from 906375b
  • File handling:
    • Fetch imageinfo for both the file and manualthumb
    • Pull out size extraction from image handler
    • task T89262: Read thumb sizes from siteinfo
    • task T154709: Honour the "page" option for files
  • Linter:
    • Provide name of stripped tags
    • task T162919: Don't lint fostered rendering-transparent nodes
    • Linter detection for task T161306

Wednesday, April 12, 2017 around 1:12 pm PT: Yes Deployed 75debae3[edit]

  • Lint misnested tags for better accuracy
  • Address edge case in DSR code that affects lint output in some cases
  • Add linter detection for task T161341

Thursday, April 6, 2017 around 10:19 am PT: Yes Deployed 56ae82bb[edit]

  • Upgrade service-runner, and various other deps

Wednesday, April 5, 2017 around 1:18 pm PT: Yes Deployed 32b7c677[edit]

  • task T112043: Handle anchors without hrefs
  • Add additional options to slideshow gallery
  • Slideshow does not support per row option
  • task T161936: Apply all the valid gallery attributes

Wednesday, Mar 29, 2017 around 1:21 pm PT: Yes Deployed b1b27146[edit]

  • task T161558: Only send lints for full parse
  • Fixup redirect-detecting regular expressions in multiple places
  • task T160207: Fix serializing multi-line indent-pre w/ sol wt syntax
  • task T153798: Lower heap limit to 600m

Monday, Mar 27, 2017 around 1:21 pm PT: Yes Deployed 6eaad376[edit]

  • task T160599: Ignore thumbtime linter errors temporarily
  • Add follow_redirects query param to the wt2html endpoint in dev API
  • task T161178: Apply defined classes to galleries
  • task T133267: Escape extlink content when containing ] anywhere

Wednesday, Mar 8, 2017 around 1:19 pm PT: Yes Deployed dec47257[edit]

  • Enable linting in production
  • Automatically detect whether Linter extension is installed
  • task T59603: Implement {{PAGELANGUAGE}}; fix language code in HTML <body>
  • ParserTests
    • Add an option to force strong normalization of parser tests
    • Fix bin/parserTests.js --help
    • Clean up parserTests output
    • Collect and print summary information at end of parser tests

Wednesday, Mar 1, 2017 around 1:45 pm PT: Yes Deployed 9f96b2a0[edit]

  • Fix namespace bug exposed by ae90e7cd
  • Update CSS modules added to header
  • Use videoinfo if it's available
  • useVideoInfo is only for legacy requests
  • task T50900: Set mw:Error and provide error info in data-mw for ext/tpl
  • Bug fix in Async Token Transform Manager
  • Refactor in link handler
    • Refactor file handling a bit
    • Break img element handling out to its own func
    • Refactor figure html2wt to be a little more elt agnostic
    • Remove unused PrefixOptionsReverseMap
    • Pass dataMw to the media handlers
    • Fetch link info from data-mw even when no link is present
  • ParserTests, and bunch of fixes and refactors, including:
    • Pull out a parserTests.utils.js
    • Use sync-parserTests.js with other targets
    • task T114256: Run tests against the Cite parserTests
    • Move parsoid's ref tests to citeParserTests.txt
    • Avoid max callstack size exceeded in parserTests
    • Add TimedMediaHandler parserTests.txt
    • Get rid of PARSOID_MOCKAPI_URL env var
    • task T156295: Make <indicator> tests php only
    • task T156296: Update baseconfigs, this time including enwiki

Tuesday, Feb 14, 2017 around 10:15 am PT: Yes Deployed 79ccfb93[edit]

Tuesday, Feb 7, 2017 around 10:15 am PT: Yes Deployed f0732260[edit]

  • Code refactoring in TemplateHandler
  • Don't count known images as errors
  • task T109897: Remove implicit_table_data_tag rule
  • Remove special-case non-void semantics for <source>

Tuesday, Jan 31, 2017 around 10:50 am PT: Yes Deployed 734dc996[edit]

  • task T98960: Accept entities in extlink href and url links
  • Stop normalizing hrefs by stripping "./" in html/parsoid sections
  • task T155070: Set relativeLinkPrefix before setting titlePageURI
  • Fallback to mw:Error for missing images
  • Get wfIsBadImage() information in API requests
  • Shadow href w/ the original value, not the current one
  • API improvements needed for ProofreadPage extension
  • Set src as the original value when using title as href
  • Use onAnchor for sanitizing link href's as well
  • Bump to domino@1.0.28
  • Fix to resolveTemplateTarget to better handle magic word aliases

Tuesday, Jan 24, 2017 around 10:45 am PT: Yes Deployed d000fdb4[edit]

  • task T58846: Port sanitizer changes from core commit feb23b46
  • Switch to npm@3
  • Bump service-runner to 2.1.13 and various deps minor versions
  • Frame args is expected to be an array
  • Don't accept pipe unconditionally in extlink
  • task T154804: Serialize gallery without attrs in data-mw
  • task T152633: Fix crasher from ConstrainedText
  • flow-board is a known contentmodel

Wednesday, Jan 18, 2017 - Thursday, Jan 19, 2017: Yes Upgrading Parsoid cluster to node v6.9.1[edit]

Thursday, Jan 5, 2017 around 10:16 am PT: Yes Deployed 974dd5b3[edit]

wt2html tokenizer

  • Make the tplarg rule resemble template
  • tplargs don't have k=v pairs, only values
  • Give a chance to break on | in element attribute name in template
  • Suppress breaking on table pipe in tags
  • Suppress breaking on tableCellArg in block tags too
  • Remove unreachable action
  • task T143183: Extensions take precedence over templates
  • Get rid of the generic_tag rule

Other wt2html

  • Escape cite ids with Sanitizer.escapeId
  • task T102134: Fix cite hrefs to render properly
  • Percent-encode | char in the stylesheet link tag in <head>
  • Percent-encode the entire thing
  • Get rid of BehaviorSwitchPreprocessor
  • task T113044: Complete templatearg representation in spec

Linter (not yet enabled in production)

  • Fix obsolete tags linting + ignore BIG
  • Cleanup and add tests for multi-template

Code cleanup

  • Code cleanup in MWParserEnvironment.js
  • ApiRequest.js.request: Add comment to clarify intent
Retrieved from "https://www.mediawiki.org/w/index.php?title=Parsoid/Deployments/2017&oldid=4326216"