Scrum of scrums/2018-08-29

= 2018-08-29 =

Callouts
Switchover/Switchback dates reminder. It's in 2 Weeks.
 * SRE:

Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC

Switchback:

Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC

IMPORTANT: For the duration of the switchover (1 month), please refrain from large db schema changes and please avoid deploying any kind of new feature that requires creation of tables. IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct 8th. [nuria] Dashiki just uses the files that are now not rendering from meta, it is really a seprate system that consumes those files
 * Fundraising campaigns https://meta.wikimedia.org/wiki/CentralNotice/Calendar
 * From RI: help wanted with https://phabricator.wikimedia.org/T203029 from those who know about Dashiki
 * wp10 models in ORES API are renamed to articlequality, if you're using this model, please change
 * Wikidata would appreciate input/decision on way forward with two patches in MW core, related to testing (thanks to Anotine and Timo for input so far)
 * https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/455176/
 * https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/446543/

Community Tech

 * Blocked by:
 * Blocking:
 * Updates:

Anti-Harassment Tools

 * Blocked by:
 * Blocking:
 * Updates:
 * Continue work on Partial Blocking (Goal is to have a MVP ready by the end of the sprint)
 * Analytics on Harassment cases on A/NI

Editing

 * Blocked by:
 * Blocking:
 * Updates:
 * Improving event logging for better data analysis
 * Mobile bug fixes to improve editing experience

Growth

 * Blocked by:
 * Blocking:
 * Updates:

Language

 * Blocked by: None
 * Blocking: None
 * Updates:
 * Work on ContentTranslation version 2 continues.

iOS native app

 * Blocked by:
 * Blocking:
 * Updates:

Android native app

 * Blocked by:
 * Blocking:
 * Updates:
 * Finished polling Echo notifications (limited to certain types of notifications initially)
 * Waiting on user testing of navigation prototypes.

Readers Web

 * Blocked by:
 * Blocking:
 * Updates:

Readers Infrastructure

 * Blocked by:
 * Blocking:
 * Updates:
 * Bernd out rest of week, no PCS updates
 * Maps:
 * Working on setting up map tile invalidation on resource change (via ChangeProp)
 * maps-test cluster to be decommissioned; production maps cluster to be updated to Stretch
 * JsonConfig bug affecting Dashiki config pages on meta.wikimedia.org, should be fixed but isn't. Help wanted!  https://phabricator.wikimedia.org/T203029
 * https://gerrit.wikimedia.org/r/c/mediawiki/extensions/JsonConfig/+/455868 fixed the issue on zero.wikimedia.org (https://phabricator.wikimedia.org/T203006 ), but not for Dashiki config pages

Parsing
Continuing porting TokenTansformers subsystem to PHP as part of overall php port
 * Blocked by: None
 * Blocking: None, coordinating with VE on several updates such as audio/video elements
 * Updates: Offsite upcoming on 9/10-9/15

Multimedia

 * Blocked by:
 * Blocking:
 * Updates
 * SDoC: multi-lingual captions display and edit on File page
 * SDoC: prototyping depicts search almost complete
 * SDoC: design artwork

UI Standardization

 * Blocked by:
 * Blocking:
 * Updates:

Analytics

 * Blocked by:
 * Blocking:
 * Updates:
 * Updating privacy policy of wikistats2 and other sites like http://analytics.wikimedia.org to the new non-wiki policy per Legal’s request.
 * Working with performance on making eventlogging a lot more limber so it can be included in every page: https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/EventLogging/+/453457/
 * Changed all random identifiers in mediawiki codebase to have more entropy, from 64 to 80 bits. This benefits eventlogging but also it is used widely on mediawiki client code.: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/454460/
 * Integrating archiva (java libraries dependency management depot) with LDAP.
 * Moving all our analytics infrastructure to Debian Stretch. Along the way, we will be moving (superset|turnilo|yarn|hue).wikimedia.org to new VMs and upgrading them.

Cloud Services

 * Blocked by:
 * Blocking:
 * Updates:

Fundraising Tech

 * Blocked by:
 * Blocking:
 * Updates:
 * More testing and bug fixes on main CC processor's new API integration
 * CiviCRM UI enhancements
 * Donor data export/erase bug fixes
 * Testing new pipeline to get banner and landing page impressions into fundraising DBs
 * Testing stretch and PHP7 to unblock payments mediawiki LTS upgrade
 * Getting started on hiring another fr-tech

MediaWiki Core Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * Platform Evolution: summarizing stakeholder input to cluster themes for Tech Conf
 * Requirements gathering and discussion for storage component
 * More work on deploying the phan security check (aka phan-taint-check) to more extensions
 * Porting poolcounterd tests to Python (gerrit:455385)
 * LinksUpdate failure on replag (phab:T201482)
 * EU copyright landing page

Performance

 * Blocked by:
 * Blocking:
 * Updates:
 * mcrouter broadcast now live (deletes go to both data centers)
 * Thought we had made the world much faster by removing a ResourceLoader round trip, but then realized that we'd accidentally gamed our numbers. Still evaluating the actual improvement.
 * Gilles is back from vacation, and cranking through Thumbor bugs
 * Testing "hero images" timings.
 * Gilles is back from vacation, and cranking through Thumbor bugs
 * Testing "hero images" timings.

Release Engineering

 * Blocked by:
 * Noise from https://phabricator.wikimedia.org/T201082 during Train deployment (not really blocked but distracted)
 * Block the CI migration to Quibble / off of Nodepool
 * MediaWiki team: "[TrustedXFF] on fresh install trusted-xff.cdb is missing and User::getDefaultOptions cause a fatal" https://phabricator.wikimedia.org/T198120
 * MediaWiki team: "[ReadingLists] No registered builder available for ReadingLists." https://phabricator.wikimedia.org/T196567
 * WMDE or a Qunit/JavaScript guru: ArticlePlaceholder should use MediaWiki qunit runner https://phabricator.wikimedia.org/T180171


 * Blocking:
 * Updates:
 * Two post mortems this week about recent train problems
 * Train: no major problems, 1.32.0-wmf.19 at group 0 https://phabricator.wikimedia.org/T191065 https://tools.wmflabs.org/versions/
 * Log spam: Unknown modifier 'R': [/^page\-User\:BeneBot.+/RfD\-open/text$/] in /srv/mediawiki/php-1.32.0-wmf.16/extensions/Translate/stringmangler/StringMatcher.php https://phabricator.wikimedia.org/T202058
 * Log spam: Unknown modifier 'R': [/^page\-User\:BeneBot.+/RfD\-open/text$/] in /srv/mediawiki/php-1.32.0-wmf.16/extensions/Translate/stringmangler/StringMatcher.php https://phabricator.wikimedia.org/T202058

Research

 * Blocked by: None
 * Blocking: None
 * Updates:
 * Sent out emails to collect more data (2nd stage of the experiment) for the Elicit new editor interests research: https://meta.wikimedia.org/wiki/Research:Voice_and_exit_in_a_voluntary_work_environment/Elicit_new_editor_interests#Stats_and_study_condition
 * Will enable collecting data for the CitationUsage schema this Thursday: https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/454854/
 * Working towards productionizing the recommendation API. Talking about storage with Services/Analytics: https://phabricator.wikimedia.org/T203039

Scoring Platform

 * Blocked by: RelEng on these two: https://phabricator.wikimedia.org/T192042 and https://gerrit.wikimedia.org/r/c/All-Projects/+/455629
 * Blocking: None
 * Updates:
 * The work for using PoolCounter in ores continues, support for multiple nodes is almost there
 * Renaming wp10 models to articlequality, if you're using this model, please change
 * Reducing size of ores setup in labs, to reduce the cloud service team maintenance workload
 * working on JADE table schemes

Search Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * Enabled indexing of all wikidata statements referencing items: https://phabricator.wikimedia.org/T199884 newly edits have it enabled, old items pending reindex
 * Added collecting stats for data processing daemons: https://phabricator.wikimedia.org/T201948
 * Fixed errors with large offsets on search: https://phabricator.wikimedia.org/T201360
 * Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654
 * Working on searching for statement values without additional keywords: https://phabricator.wikimedia.org/T163642
 * Working on ES 6.3 upgrade: https://phabricator.wikimedia.org/T197960
 * Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108

Security

 * Blocked by:
 * Blocking:
 * Updates:
 * Starting work into using Content-Security-Policy header
 * Security review of chromimum-render https://phabricator.wikimedia.org/T177765 and EU copyright once ready

Services

 * Blocked by:
 * Blocking:
 * Updates:

Site Reliability Engineering
Switchover/Switchback dates reminder
 * Blocked by:
 * None
 * Blocking:
 * None
 * Updates:

Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC

Switchback:

Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC

IMPORTANT: For the duration of the switchover (1 month), please refrain from large db schema changes and please avoid deploying any kind of new feature that requires creation of tables. IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct 8th.

Wikidata

 * Blocked by:
 * Blocked is a big word, but we'd appreciate input/decision on way forward with two patches in MW core, related to testing (thanks to Anotine and Timo for input so far)
 * https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/455176/
 * https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/446543/
 * Blocking:
 * Updates:
 * Work in Lexicographical data continues:
 * adding senses
 * improving the editor workflow
 * https://phabricator.wikimedia.org/project/view/2292/
 * Experiment about not using secondary SQL table for formatting item links being gradually deployed
 * https://phabricator.wikimedia.org/T201831

German Technical Wishlist

 * Blocked by:
 * Blocking:
 * Updates:

Multi-Content Revisions

 * Blocked by:
 * Blocking:
 * Updates:
 * almost complete phase 2 tasks (today?), planning to finish phase 3 tasks by end of week
 * deployment impacted by CI test failures - remediation in progress

SoS Meeting Bookkeeping

 * Updates: