Scrum of scrums/2019-08-28

= 2019-08-28 =

Callouts

 * Release Engineering
 * REMINDER: We're at 1.34.0-wmf.20 this week. The last branch for this release will be wmf.25 on 30 September. Teams who want to ship things for MW 1.34 should land them now.
 * Search Platform blocked (by Core Platform?): RecentChange support for SDC: https://phabricator.wikimedia.org/T230862

iOS native app

 * Updates:
 * Development on 6.4 release https://phabricator.wikimedia.org/tag/ios-app-v6.4-seamonkey-on-a-hoverboard/
 * iOS13/iPadOS updates
 * New loading screen
 * Bug fixes
 * History & Diffs pushed off to the next release

Android native app

 * Updates:
 * Minor release now in production: optimized page loading speed and efficiency.
 * Starting work on Suggested Edits v3 (design updates to user contributions screen).
 * Continuing to profile and test integration with mobile-html endpoint.

Readers Web

 * Updates:
 * Summary: continuing advanced mode.
 * Responsive website (MinervaNeue / MobileFrontend / Portals):
 * Desktop refresh:
 * Light technical planning.
 * Advanced mode:
 * File '/srv/mediawiki/php-1.34.0-wmf.20/skins/MinervaNeue/resources/resources/skins.minerva.personalMenu.icons/userSandbox.svg' does not exist
 * ServiceContainer.php: Circular dependency when creating MobileFrontend service "AMC.UserMode > AMC.Manager > FeaturesManager > UserModes > AMC.UserMode"
 * Contributions icon is missing from main menu in non-AMC mode for logged in users
 * Touch targets not large enough
 * Browser tests failing now AMC Outreach drawer is enabled
 * AMC Outreach Drawer encouraging me to turn on advanced mode sends me from Special:Homepage to my user page
 * Cleanup AMC feature flagging
 * Create, and deploy working MobileWebUIActionsTracking schema
 * Reduce spacing of icons in mobile header
 * Fix icon sizing by updating all 24x24 icons to 20x20 icons
 * Miscellaneous:
 * Excessive paragraph padding in MF editor save dialog
 * Mobile VE doesn't use DB title for RESTBase load, resulting in a 301 redirect
 * Separate View code from SearchOverlay
 * Bug: Section edit links should be hidden for blocked users

Product Infrastructure

 * Updates:
 * Pagelib
 * CSS now minimized. Deployed yesterday
 * Experimenting with pagelib to improve mobile-html pageload https://phabricator.wikimedia.org/T231201
 * Mobile-html
 * Fixed srcset in media-list endpoint
 * WIP: updating footer style
 * Maps
 * Investigated OSM DB bug, but results inconclusive
 * Machine vision middleware
 * Could be (optimistically) code-complete by EOW for DBA, Performance pre-deployment reviews
 * Planning JS frontend integration
 * Data Engineering
 * Specified identifiers for event platform clients
 * Specified sampling functions for event platform clients

Structured Data

 * Updates:
 * working on getting structured data in campaigns ready for WLM

Parsing

 * Updates:
 * Parsoid/PHP testing:
 * Parser tests almost 100% green now (barring some insignificant key-value ordering differences in JSON blobs which we need to normalize to reduce this false positives).
 * We have got our first runs of Parsoid/PHP round trip tests running scandium. We still have some crashers to fix and more bugs to fix from that initial test run.
 * Performance-wise, after Tim's initial work tweaking some things in Parsoid, here is a rough performance update for test runs on scandium. We need real perf. benchmarking to get a more robust sense of where we are.
 * Parsoid/PHP w2html:
 * On large pages (like en:Barack_Obama), 1.5x slower than Parsoid/JS. But, most of that seems to be GC issues (both PHP7.2 and Parsoid/PHP). See https://phabricator.wikimedia.org/T230861 for details
 * On medium and smaller pages, as fast as or upto 2x faster than Parsoid/JS in some cases which is a pretty good result.
 * Parsoid/PHP html2wt (impacts VE and CX):
 * On large DOMs (like en:Barack_Obama), >2x slower than Parsoid/JS. But, on slightly smaller DOMs (like en:Berlin), < 10% slower. So, we seem to be getting non-linear slowdown as the pages get larger.
 * Needs additional testing on medium and smaller sized pages.
 * Note that in production usage, we will be using a more efficient version of html -> wt. But, this slowdown is still a reliable indicator of where we need to focus our performance work.
 * Other Parsoid/PHP updates:
 * Working with core platform team on RESTBase - Parsoid/PHP integration testing.

Analytics

 * Blocking:
 * Search Platform - https://phabricator.wikimedia.org/T229882

Fundraising Tech

 * Updates:
 * Large-scale testing of India payments form
 * Prepping new CiviCRM point upgrade with more of our local hacks upstreamed or turned into proper extensions: https://phabricator.wikimedia.org/T228826
 * Importing more donor data into CiviCRM: https://phabricator.wikimedia.org/T228715
 * Fixes for one-time->recurring conversion ask: https://phabricator.wikimedia.org/T228162
 * Deploying new Eventlogging-based pipeline for landing page and CentralNotice data
 * Fixing issues with sending events from both sources: https://phabricator.wikimedia.org/T231406, https://phabricator.wikimedia.org/T231404
 * Payments form layout and text changes: https://phabricator.wikimedia.org/T169795, https://phabricator.wikimedia.org/T224063
 * Almost ready to deploy python3 upgrade to fundraising python tools: https://phabricator.wikimedia.org/T197805

Core Platform

 * Blocked by:
 * Site Reliability Engineering: Investigate Kask request latency https://phabricator.wikimedia.org/T229697
 * Blocking:
 * Search Platform: RecentChange support for SDC: https://phabricator.wikimedia.org/T230862
 * Release Engineering: MediaWiki installer silently ignores invalid extensions https://phabricator.wikimedia.org/T225512
 * Performance: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/394430/ https://phabricator.wikimedia.org/T230800 (triaged by Daniel and Brad a week ago?)
 * Updates:
 * Kask performance issue
 * Parsoid/PHP last routes
 * Core REST API routes in MediaWiki
 * Action API integration testing

Performance

 * Blocked by:
 * WMDE/Wikidata: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/497537/
 * Core Platform: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/394430/ https://phabricator.wikimedia.org/T230800 (triaged by Daniel and Brad a week ago?)
 * Updates:
 * resourceloader: New Grafana dashboard providing insight into JS startup cost for extensions. – https://grafana.wikimedia.org/d/BvWJlaDWk/startup-module-size

Release Engineering

 * Blocked by:
 * Core Platform Team: MediaWiki installer silently ignores invalid extensions https://phabricator.wikimedia.org/T225512
 * Updates:
 * REMINDER: We're at 1.34.0-wmf.20 this week. The last branch for this release will be wmf.25 on 30 September. Teams who want to ship things for MW 1.34 should land them now.
 * Train Health
 * Last week: 1.34.0-wmf.19 - https://phabricator.wikimedia.org/T220744 - some problems, but deployed on time
 * This week: 1.34.0-wmf.20 - https://phabricator.wikimedia.org/T220745
 * Next week: 1.34.0-wmf.21 - https://phabricator.wikimedia.org/T220746

Search Platform

 * Blocked by:
 * Core Platform: RecentChange support for SDC: https://phabricator.wikimedia.org/T230862
 * Analytics: https://phabricator.wikimedia.org/T229882
 * Updates:
 * Support for Wikidata Query Service will be limited for the foreseeable future; hiring is in progress
 * Cloudelastic import still going on, causing some queues to grow quicker than expected https://phabricator.wikimedia.org/T231194
 * Still reindexing some wikis to enable the new create_timestamp sort criteria https://phabricator.wikimedia.org/T230990
 * Adapted search/analytics data pipelines to use swift to deploy data (did you mean suggestions and pageview ranking signal) to production https://phabricator.wikimedia.org/T227364
 * We evaluated druid+turnilo+superset for search metrics going forward and found it to be reasonable; built a superset dashboard for search satisfaction 'did you mean' metrics https://phabricator.wikimedia.org/T229268
 * Worked on documenting the limitations of the current search syntax https://phabricator.wikimedia.org/T228925
 * Added support to haswbstatement:* syntax: https://phabricator.wikimedia.org/T230175
 * Fixed issue with case sensitivity of deepcat searches: https://phabricator.wikimedia.org/T228633
 * SDC query test server: http://sdcquery.wmflabs.org/ - not updated yet!
 * wdqs1009 restored to its role as auto-deploy test server: https://phabricator.wikimedia.org/T230244
 * WDQS Updater performance improvements code complete, starting testing: https://phabricator.wikimedia.org/T212826
 * Working on fixing categories delete issue: https://phabricator.wikimedia.org/T228348

Security

 * Updates:
 * Backlog
 * Security Review for MediaWiki REST API, assigned to Sam, https://phabricator.wikimedia.org/T230140
 * John to explore funding of 3rd party audits this quarter, stalled - https://phabricator.wikimedia.org/T155537, https://phabricator.wikimedia.org/T156960, https://phabricator.wikimedia.org/T148246, https://phabricator.wikimedia.org/T187846
 * Active
 * Security review of Ex:DoubleWiki, in-progress - https://phabricator.wikimedia.org/T131199
 * Parsoid-PHP, moved to active, additional review by Sam, in-progress - https://phabricator.wikimedia.org/T227209
 * Security review of preact 8.4.2, Scott did a very simple assessment, unassigned, stalled https://phabricator.wikimedia.org/T227726
 * Page Content Service route /page/mobile-html, starting - https://phabricator.wikimedia.org/T227114
 * Security review of WebAuthn library dependencies, in-progress - https://phabricator.wikimedia.org/T227244
 * Planet wikimedia - assigned to Michal Anna, stalled - https://phabricator.wikimedia.org/T207246
 * Waiting
 * Labs db/sanitarium and maintain-views.yaml audits, assigned to James F, stalled - https://phabricator.wikimedia.org/T169097, https://phabricator.wikimedia.org/T103011
 * Frozen (delayed indefinitely)
 * Audiences growth team emails concept review, stalled - Jen contacted to close - https://phabricator.wikimedia.org/T220242
 * Banner preview, stalled - https://phabricator.wikimedia.org/T230176
 * Closing Soon / Closed
 * Doublewiki (old), Scott to resolve this week, in-progress - can Jen contact to close? - https://phabricator.wikimedia.org/T131199

Site Reliability Engineering

 * Blocking:
 * Core Platform Team, Investigate Kask request latency https://phabricator.wikimedia.org/T229697

Wikidata

 * Blocking:
 * Performance - WMDE/Wikidata: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/497537/

SoS Meeting Bookkeeping

 * Updates:
 * Don't forget to copy "blocked" section from your team to the team that's blocking you "blocking" section