Scrum of scrums/2017-11-15

= 2017-11-15 =

Callouts

 * Biggest fundraising campaign of the year starts in less than two weeks - please double-check with fr-tech if you're about to do anything that might affect CentralNotice or donatewiki.
 * #wikimedia-fundraising on IRC, say the word "fr-tech"
 * If anyone hears anything about incorrect LanguageConverter behavior, please contact Bawolff (https://phabricator.wikimedia.org/T119158)
 * Global Collab is working on a patch to apply change tags to all edits that have automatic edit summaries (including all rollbacks), if you think that's a terrible idea please speak up: https://gerrit.wikimedia.org/r/#/c/390224/

iOS native app

 * Blocked by:
 * Blocking:
 * Updates:
 * Continuing work on 5.8 - https://phabricator.wikimedia.org/project/view/2913/ (faster article loads, synced reading lists)

Web

 * Still working on chromium service
 * Working on limiting resources used by the Chromium-based PDF renderer service 
 * Tested the service with the Debian Stretch Chromium package  and it worked just fine! This unblocks the conversation with Ops around how to deploy the service.

Multimedia

 * Neither blocked nor blocking
 * Finishing up some cleanup work on MediaInfo, nearly ready to go.
 * Will begin prototyping features once a working, stable API is merged and deployed to a labs^Wcloud site
 * Making progress on 3D legal issues - patents, dangerous items, items illegal in various jurisdictions, etc. - also nearly ready to go.

Reading Infrastructure

 * Blocked by:
 * Blocking:
 * Updates:
 * Continuing with Reading Lists
 * Continuing with summary, media, references endpoints
 * Considering turning off trending-edits endpoint

Discovery

 * Blocked by:
 * Blocking:
 * Updates:
 * Got a Jenkins job running that builds a new version of Wikimedia portals weekly

Parsing

 * section-tag (and other patches bundled with parsoid html version 1.6.0) patches not going out this week after all since we don't have time to test thoroughly. Plan to merge patches tomorrow and spend this and next week testing and have it deployed week after thanksgiving.
 * Heads up to VE, MCS, CX, Flow folks
 * section tag patch is https://gerrit.wikimedia.org/r/#/c/364933/; figure-inline tag patch is https://gerrit.wikimedia.org/r/#/c/370227/; html5-id patch work in progress by cscott.
 * Have been running a linter reparse script on terbium that lints every single page on every wiki -- this is useful / required to initialize linter issues for all pages on all wikis (and get rid of some false positives based on fixes over the last couple months). This hits Parsoid cluster on eqiad. This is a python script with 160 threads. The parsoid eqiad cluster load is < 10% with this req rate so no cause for alarm. But, just a heads for Ops (I added a SAL entry last night for this) and it is safe to kill the script at any point if there are any issues.

UI Standardization

 * Blocked: icons overhaul, pls see below. New iteration got out end of last week
 * OOUI no release this week
 * Ongoing:
 * OOUI & based products:
 * icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide https://phabricator.wikimedia.org/T177432
 * Make Special:Preferences use OOUI – related UI/UX amendments https://phabricator.wikimedia.org/T180538
 * Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867

Global Collaboration

 * Blocked: by no one
 * Blocking: ops for Flow dumps, Roan didn't contact Ariel like he said he would last week, sorry
 * Updates:
 * RCFilters going to all remaining wikis (was not yet on FlaggedRevs wikis) at the top of the hour
 * Also deploying a UBN fix for RCFilters
 * Working on properly stripping tags from Parsoid HTML (T179978); anyone know why we wouldn't/couldn't use PHP built-in strip_tags?
 * Never mind, Tim pointed out some bad behavior in strip_tags
 * We are working on a patch to apply change tags to all edits that have automatic edit summaries (including all rollbacks), if you think that's a terrible idea please speak up: https://gerrit.wikimedia.org/r/#/c/390224/

Community Tech

 * Mostly busy with our Wishlist Survey

Security

 * New MediaWiki Security release 1.29.2 1.28.3 1.27.4
 * Working on a phan plugin - very experimental https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/tools/phan/SecurityCheckPlugin
 * LanguageConverter patch released (https://phabricator.wikimedia.org/T119158 )- If you hear anything about it behaving weirdly please forward to bawolff

Services

 * Not attending
 * Blockers: none
 * Updates:
 * RecordLintJob transferred to kafka-based queue
 * Please upgrade services to node-service-template v0.5.3

Scoring Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * Long slog of moving to the new cluster continues.
 * Thresholds will be available in mw-vagrant.

Analytics

 * Worked on cergen: generic certificate management lib to use in puppet
 * Improving mediawiki history reconstruction after data vetting
 * Worked on cluster jobs to automatically load EventLogging data into Druid/Pivot
 * Migrated EL databases to new hosts db1107 (master) and db1108 (replica); managed to finish EL purging script. Data still in old hosts.
 * More wikistats2 work, refactoring for better breakdowns and designing map visualization.

Technical Operations

 * Blocked
 * By collaboration on flow dumps
 * Blocking
 * None
 * Updates
 * Ongoing trendingedits issues due to an outdated kafka-node version see https://phabricator.wikimedia.org/T179786
 * Asia DC: All hardware physically installed, procuring connectivity, acquiring address space
 * An interesting bug with webscockets (affecting etherpad) in varnish https://gerrit.wikimedia.org/r/#/c/389794/

Search Platform

 * Blocked by: none
 * Blocking: none
 * Updates:
 * Improving completion suggester interaction with namespaced prefix search https://phabricator.wikimedia.org/T178474
 * Preparing for next relevancy survey https://phabricator.wikimedia.org/T176428
 * Working on fixing 500’s on search.wikimedia.org https://phabricator.wikimedia.org/T179266
 * Working on porting Selenium tests from Ruby to JS
 * Working on upgrade to Elastic 5.5
 * Working on indexing Wikidata descriptions and adding them to fulltext search

Fundraising Tech

 * Going into lockdown mode
 * More CiviCRM UI improvements
 * Investigated possible link betweeen mobile rendering slowdown and CentralNotice, CN didn't look like the cause: https://phabricator.wikimedia.org/T180049
 * Security patches for fundraising projects
 * Still working on sending more stats to prometheus
 * Talking with analytics about better ways to get banner impression and donatewiki hit stats

Wikidata

 * Work on JSON-LD support in the Purtle library used for exports: https://gerrit.wikimedia.org/r/379669
 * Start on tracking description usages separately: https://phabricator.wikimedia.org/T176417
 * Had to remove PSR-4 class loading for a cleaner Wikidata build: https://phabricator.wikimedia.org/T180067
 * Which kind of makes this a blocker for us: https://gerrit.wikimedia.org/r/373626