Scrum of scrums/2018-08-22

= 2018-08-22 =

Callouts
Switchover/Switchback dates finalized
 * SRE:

Services: Tuesday, September 11th 2018 14:30 UTC Media storage/Swift: Tuesday, September 11th 2018 15:00 UTC Traffic: Tuesday, September 11th 2018 19:00 UTC MediaWiki: Wednesday, September 12th 2018: 14:00 UTC

Switchback:

Traffic: Wednesday, October 10th 2018 09:00 UTC MediaWiki: Wednesday, October 10th 2018: 14:00 UTC Services: Thursday, October 11th 2018 14:30 UTC Media storage/Swift: Thursday, October 11th 2018 15:00 UTC

IMPORTANT: For the duration of the switchover (1 month), please refrain from large db schema changes and please avoid deploying any kind of new feature that requires creation of tables. IMPORTANT: There will be a train freeze in the week of Sept 10th and Oct 8th.


 * Release Engineering blocked by: Feedback needed (how problems could have been prevented) from many people/teams on a recent MediaWiki train related incident report.
 * 1.32.0-wmf.13, 9 blockers, feedback needed for 8 of them: https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
 * Aaron Schulz (Performance), Adam Wight (Scoring Platform), Bartosz Dziewoński (Contributors), Brad Jorsch (MediaWiki Platform), C. Scott Ananian (Contributors), Daniel Kinzler (Wikimedia Deutschland), Timo Tijhof (Performance), Prateek Saxena (Audiences Design)
 * Release Engineering looking for more participants to join the Code Health Metrics working group. This group's purpose is to define and later implement a set of core metrics that we will use to asses the health of our code base. More info: https://www.mediawiki.org/wiki/Code_Health_Group/projects/Code_Health_Metrics

iOS native app

 * Blocked by: -
 * Blocking: -
 * Updates:
 * 6.0.0 released ( https://phabricator.wikimedia.org/tag/ios-app-v6.0-walrus-on-a-unicycle/ )
 * 6.0.1 to be released next week ( https://phabricator.wikimedia.org/tag/ios-app-v6.0.1-walrus-on-a-golf-cart/ )
 * 6.1 up next, release date TBD ( https://phabricator.wikimedia.org/tag/ios-app-v6.1-narwhal-on-a-bumper-car/ )

Android native app

 * Blocked by:
 * Blocking:
 * Updates:
 * Finishing up polling Echo notifications (limited to certain types of notifications initially)

Readers Web

 * Blocked by:
 * Blocking:
 * Updates:
 * Mobile website (MinervaNeue / MobileFrontend):
 * Page issues UI and instrumentation: T191528 T201124 T197932 T196150
 * Invest in the MobileFrontend & MinervaNeue frontend architecture: T188937
 * Avoid abrupt and jarring white flash in media viewer: T197110
 * Other fixes and hygiene: T197499 T199000 T197497 T202021 T201956

Readers Infrastructure

 * Blocked by:
 * Blocking:
 * Updates:
 * PCS:
 * Separating pagelib CSS from base CSS this week
 * Adding simple header to mobile-html (title, description, page image)
 * Maps:
 * Nothing to report.

Parsing

 * Blocked by:
 * Blocking:
 * Updates:
 * Nothing to report

Multimedia

 * Blocked by:
 * Blocking:
 * Updates
 * SDoC: Discussion with search team about searching for related statements
 * SDoC: prototyping depicts search
 * SDoC: design artwork
 * No longer blocked by MCR

Community Tech

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

Anti-Harassment Tools

 * Blocked by:
 * Blocking:
 * Updates:
 * Continuing work on Partial Blocks: https://phabricator.wikimedia.org/T2674
 * Updated the TechComm RFC to include Multiblocks: https://phabricator.wikimedia.org/T199917

Editing

 * Blocked by:
 * Blocking:
 * Updates:
 * Improved/bug fixes: collapsed tables in visual diffs; insertion of horizontal rules; pages with lots of tables in WTE 2017; switching between visual editor and source mode; VE linking in IE11; regressions with icon displays; etc.
 * Compact multiple transactions into one:

Growth

 * Blocked by:
 * SRE: Query about plausibility of measuring e-mail traffic to our no-reply addresses https://phabricator.wikimedia.org/T202329
 * Blocking:
 * Updates:
 * No update.

Language

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

Audiences Design

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

UI Standardization

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

Analytics

 * Blocked by:
 * Blocking:
 * Updates:
 * Changes to hash and salt (every 90 days) appinstallIDs are on, these will be retained (hashed) forever
 * Working with performance in making eventlogging javascript module lighter so it can be loaded on 1st page load.
 * Working on computing “top contributors” metric per project so it is available in wikistats2, this is one of the most coveted reports by our community.
 * Spark available now in Jupyter notebooks.
 * Meetings around modern event data platform to define what parts will we start developing next quarter.

Cloud Services

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

Fundraising Tech

 * Blocked by:
 * Blocking:
 * Updates:
 * Put off upgrade of mediawiki on payments servers
 * testing payment notification listener on debian stretch and php 7.0
 * Bugfixes and UI tweaks for new card processor API integration
 * Nearly done new system for ingesting EventLogging data about banners and FR landing pages
 * Activated EventLogging for all banner campaigns, sampled at 1% on the front end.
 * Working on email Opt-In page
 * CiviCRM:
 * reviewing final group of patches for donor data export and deletion
 * bugfixes on imports
 * Looking at performance of donation imports

Core Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * LoadBalancer overload investigation (T180918)
 * TitleValue performance (T201801)
 * Awaiting review: Security review for Guzzle 6.3.3 (T202143)
 * In testing: Create GuzzleHttpRequest class as new default for HttpRequestFactory (T202110)
 * Ready for testing: Convert MultiHttpClient to use Guzzle (T202352)
 * Platform Evolution: developing sessions for TechConf
 * REST mobile endpoints removed from projects other then wikipedia and wikivoyage
 * New event type coming to EventBus on this train: mediawiki.revision-tags-change

Performance

 * Blocked by:
 * Blocking:
 * Updates:
 * Deferred jQuery loading is rolling out to all wikis on this week's train
 * All wikis using mcrouter
 * nutcracker writes have been disabled
 * Mcrouter broadcast operations enabled for all wikis
 * patched a bug in Html::inlineScript
 * Removed use of StartProfiler.php from mw-init

Release Engineering

 * Blocked by:
 * Feedback needed (how problems could have been prevented) from many people/teams on a recent MediaWiki train related incident report.
 * 1.32.0-wmf.13, 9 blockers, feedback needed for 8 of them: https://wikitech.wikimedia.org/wiki/Incident_documentation/20180717-Train
 * Aaron Schulz (Performance), Adam Wight (Scoring Platform), Bartosz Dziewoński (Contributors), Brad Jorsch (MediaWiki Platform), C. Scott Ananian (Contributors), Daniel Kinzler (Wikimedia Deutschland), Timo Tijhof (Performance), Prateek Saxena (Audiences Design)
 * Blocking:
 * MediaWiki 1.29 final release and EOL; was due in June: https://phabricator.wikimedia.org/T197669 (w/ Security)
 * Updates
 * New general purpose CI job that builds and runs test containers via Blubber/Docker based on config provided in each project (think `.travis.yml` file)
 * Read more about Blubber here: https://wikitech.wikimedia.org/wiki/Blubber
 * See recent builds at https://integration.wikimedia.org/ci/blue/organizations/jenkins/blubber-test/activity
 * Gives developers one major benefit of the CD pipeline work now, having control over their pre-merge and gating tests without having to mess with integration/config
 * Only scheduled to run for a few repos at the moment, but will eventually be expanded to many more projects (we need to tune CI infra around it first)
 * Looking for more participants to join the Code Health Metrics working group. This group's purpose is to define and later implement a set of core metrics that we will use to asses the health of our code base.  More info:  https://www.mediawiki.org/wiki/Code_Health_Group/projects/Code_Health_Metrics

Research

 * Blocked by: None
 * Blocking: None
 * Updates:
 * Deployed a new UnsourcedStatements campaign (en, fr, it): https://labels.wmflabs.org
 * Finished the first phase of the "Elicit new editor interests" research. Starting the second phase soon.
 * Looking into using Cassandra as a storage engine of Recommendation JS API.

Scoring Platform

 * Blocked by: SRE, RelEng
 * Blocking: None
 * Updates:
 * All SP repos have been moved to wikimedia organization in github.
 * Git LFS support for ores models is going live, some help from RelEng would be great: https://phabricator.wikimedia.org/T192042
 * JADE scalability discussion continues, TechCom meeting is scheduled for today.
 * PoolCounter support to limit number of external connections to ORES has started. SRE input would be appericiated: https://phabricator.wikimedia.org/T201824
 * Wikilabels robustness is being worked on, some puppet changes will need review. The node is being reimaged to stretch

Search Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * Published blog post about search tokenization: https://wikimediafoundation.org/2018/08/07/anatomy-search-token-affection/
 * Added support for haswbstatement without value (any statement with given property) https://phabricator.wikimedia.org/T199886
 * Re-indexed wikis in Malay, Indonesian, and Polish, enabling Malay stemming and fixing errors in Polish stemming (https://phabricator.wikimedia.org/T196780, https://phabricator.wikimedia.org/T186046)
 * Created ElasticSearch plugin for Esperanto analysis: https://phabricator.wikimedia.org/T200099
 * Migrated ElasticSearch cluster to RAID0 and upgraded to Debian stretch: https://phabricator.wikimedia.org/T198391
 * Working on migrating communications from analytics to search cluster to Kafka: https://phabricator.wikimedia.org/T198490
 * Working on running multiple Elastic instances on the same hardware: https://phabricator.wikimedia.org/T193654
 * Working on ES 6.3 upgrade: https://phabricator.wikimedia.org/T197960
 * Working on query parsing refactoring: https://phabricator.wikimedia.org/T185108

Security

 * Blocked by:
 * Blocking:
 * MediaWiki 1.29 final release and EOL; was due in June:https://phabricator.wikimedia.org/T197669 (w/ RelEng)
 * Updates:

Site Reliability Engineering

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

Wikidata

 * Blocked by:
 * none
 * Blocking:
 * none
 * Updates:
 * Work in Lexicographical data continues:
 * adding senses
 * improving the editor workflow
 * https://phabricator.wikimedia.org/project/view/2292/
 * Experiment about not using secondary SQL table for formatting item links being gradually deployed starting Thu Aug, 23rd
 * https://phabricator.wikimedia.org/T201831

German Technical Wishlist

 * Blocked by:
 * Blocking:
 * Updates:
 * No update.

Multi-Content Revisions

 * Blocked by:
 * Blocking:
 * Updates:
 * MCR schema migrations are in progress
 * Finished running populateContentTables.php except for a few small wikis
 * Filed T202032: duplicate ar_rev_id values causing populateContentTables.php to fail
 * Phase 2 (file caption support) development tasks nearing completion
 * Phase 3 (statement support) development tasks commencing

SoS Meeting Bookkeeping

 * Updates: