Scrum of scrums/2017-12-06

= 2017-12-06 =

Callouts

 * Operations blocked on https://phabricator.wikimedia.org/T172025 (Flow)
 * Operations DBAs s8 master switchover programmed for 9th January
 * Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th.
 * MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers, phabricator tag is #mw-1.30-release
 * Wikidata welcomes secondary review from somebody with knowledge about the recent "section editing" change: https://phabricator.wikimedia.org/T181807
 * Have a look at newly initiated https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG. Feedback welcome!

iOS native app

 * Blocked by:
 * Blocking:
 * Updates:
 * Releasing 5.7.2 (high priority fix for https://phabricator.wikimedia.org/T69015 ) ( https://phabricator.wikimedia.org/tag/ios-app-v5.7.2/ )
 * Continuing work on 5.7.3 (Faster article loading, other minor enhancements) for release before the end of the year ( https://phabricator.wikimedia.org/project/view/2913/ )
 * Continuing work on 5.8 (Reading Lists) for release next year ( https://phabricator.wikimedia.org/project/view/3131/ )

Android native app

 * Blocked by:
 * Blocking:
 * Updates: Releasing beta 2.6.207

Reading Web

 * Blocked by:
 * Blocking:
 * Updates:

Reading Infrastructure

 * Blocked by:
 * Blocking:
 * Updates:
 * ReadingLists performance improvements
 * Wrapping up media + summary endpoints
 * References needs a bit more design work

Multimedia

 * Blocked by: N/A
 * Blocking: N/A
 * Updates: 3D nearly ready to go, probably before holidays, else shortly thereafter. Also working on prototyping for first feature of Structured Data on Commons.

Discovery

 * Blocked by:
 * Blocking:
 * Updates:

Maps

 * Blocked by: N/A
 * Blocking: N/A
 * Updates:
 * Maps mediawiki integration (Kartographer) improvements #1 on community wishlist: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey/Tracking
 * Redeploying Kartotherian
 * Tracking down conflicting versions + NPM

Community Tech

 * Blocked by:
 * Blocking:
 * Updates:

Editing

 * Blocked by:
 * Blocking:
 * Updates:

Parsing

 * Blocked by: Services (because of Cassandra3 migration) on deployment of HTML version bump (for section, figure-inline, html5-id, interwiki-link changes). This is just an FYI since Services & us have already synced up about this.
 * Blocking: Reading Infra on section tag markup
 * Updates:
 * itwiki, dewiki, and 170 small wikis got switched from Tidy to RemexHtml yesterday - itwiki have flagged a bunch of new issues that hadn't been caught by linting so far. Investigation ongoing and might introduce one or two newer categories to aid editors -- we expect these categories to be sparsely populated
 * Templatedata related fixes being deployed today -- For pre-existing transclusions, this will prevent Parsoid from normalizing parameter order to templatedata format

Global Collaboration

 * Blocked by: MW core (or Parsing?) for code review on https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits
 * Blocking: ops for Flow dumps
 * Updates:
 * New tags for https://phabricator.wikimedia.org/T167656 are on this week's train

UI Standardization

 * No OOUI release this and upcoming weeks
 * Ongoing:
 * OOUI & based products:
 * icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide https://phabricator.wikimedia.org/T177432 finishing up
 * Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867
 * Continuous work and per-project SVGO based optimizations, also initiated https://www.mediawiki.org/wiki/Manual:Coding_conventions/SVG

Analytics

 * Blocked by:
 * Blocking:
 * Updates:
 * The prometheus druid exporter (to report druid metrics to graphana via prometheus) got some attention and others (outside WMF) will be using it
 * First productionized version of EventLogging backend on hadoop launched this week. This means SQL-friendly tables on hadoop with eventlogging data, and for the first time, ability to join to all mediawiki databases in one query
 * Wikistats 2 APIs (like Pageview API, but for edit data), alpha launch: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats
 * Wikistats 2 UI, alpha launch: https://stats.wikimedia.org/v2/
 * Decommissioning old DB hosts for EventLogging done, remember that eventlogging DB is now only on analytics-slave, not on analytics-store anymore
 * Work on kafka jumbo cluster
 * Rebooting one druid host for maintenance brought up unexpected issues with druid and zookeeper, working on that

Cloud Services

 * Blocked by:
 * Blocking:
 * Updates:
 * Annual Toolforge survey closed. 141 responses received (11% response rate).
 * Team in Austin this week doing an offsite + attending KubeCon conference
 * Will upgrade labpuppetmaster* to v4 puppet packages after the offsite
 * Presuming everything in eqiad goes well next week

Fundraising Tech

 * Blocked by: None
 * Blocking: None
 * Updates:
 * New CiviCRM Import tool for Major Gifts
 * Paying down some tech debt in Central Notice
 * Fixes for fundraising vagrant role
 * Continued improvements to Fundraising Dashboard
 * Continuing to support FR-non-tech during the fundraiser

MediaWiki Platform

 * Blocked by: N/A
 * Blocking: Global Collaboration for code review on https://gerrit.wikimedia.org/r/#/c/392990/ and surrounding stack of commits
 * Updates:
 * MediaWiki 1.30 Release Candidate 0 released
 * MCR and Actor table reviews ongoing
 * Comment table: Tried to enable WRITE_BOTH mode on testwikis, but had to revert because it broke CentralAuth and ForeignDBViaLBRepo.
 * Will likely wait until the schema change is done everywhere before we move anything to WRITE_BOTH.
 * TemplateStyles: ParserOutput stateless transforms got merged. Some Wikibase issues that are being dealt with.
 * Firejail: will be deployed for Score
 * New PSR-4 autoloader patchset https://gerrit.wikimedia.org/r/373626 ready for review
 * PoolCounter for ORES client https://gerrit.wikimedia.org/r/394407 ready for testing
 * MP3 transcoding deployment
 * Discussions with third party MediaWiki developers regarding extension deployment/dependency management (composer)

Performance

 * Blocked by: N/A
 * Blocking: N/A
 * Updates:
 * Significant new performance testing documentation in progress (aimed at developers), expected to be delivered before end of 2017
 * New varnish slow log configuration in final testing
 * perf testing of cross-data center database writes
 * Q3 Goals planning: if you think you might need anything from the perf team, hit me up.
 * docker-based MW dev environment (in collab with WMDE)

Release Engineering

 * Blocking:
 * Scoring platform, https://phabricator.wikimedia.org/T181661
 * New scap release planned for this week that will hopefully address this issue.
 * Blocked by:
 * none
 * Updates:
 * [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th.
 * #mw-1.30-release MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten

Research

 * Blocked by:
 * Blocking:
 * Updates:

Scoring Platform

 * Blocked by:
 * Parallel scap SSH issue https://phabricator.wikimedia.org/T181661
 * Blocking:
 * Updates:
 * Outage on Nov 28, incident report is going out today.
 * SimpleWiki deployed to ORES
 * FYI: Global Collab, please announce and update RC Filters. The extension is only configured on the beta cluster.
 * testwiki (RevIdScorer) includes thresholds now, which makes extension development easier, against the real server or in either of our vagrant roles.
 * ORES Ext. Refactoring
 * Incoming basic support of Icelandic (No RC Filters yet)

Search Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * Working on improvements to LTR training
 * Reindexed wikis to enable improved katakana/hiragana mapping https://phabricator.wikimedia.org/T179945
 * Improving completion suggester interaction with namespaced prefix search https://phabricator.wikimedia.org/T178474
 * Wikidata descriptions indexed, working on fulltext search
 * Working on porting Selenium tests from Ruby to JS
 * Logstash upgrade to ElasticSearch 5.5 on Thu, completing the 5.5 upgrade

Security

 * Blocked by:
 * Blocking:
 * Updates:
 * Lots of development on Phan security plugin by Bawolff
 * Maintanenace work on security alert configuration (Github, NSP)
 * Reviews:
 * Ex:WikipediaExtracts
 * Next set of security reviews will be scheduled this week

Services

 * Blocked by: none
 * Blocking: Parsing till early next week
 * Updates:
 * Round of Cassandra 3 bootstraps is complete
 * htmlCacheUpdate for wiktionaries are on kafka queue

Technical Operations

 * Blocked by:
 * Global Collaboration on Flow dumps https://phabricator.wikimedia.org/T172025
 * Blocking:
 * None
 * Updates:
 * Q3 goal planning started, if teams have dependencies on Ops for next quarter, reach out
 * Part of DC ops in Singapore for eqsin (new caching DC's name) rollout
 * s8 (wikidata) work ongoing, work ontrack.
 * Operations DBAs s8 master switchover programmed for 9th January

Wikidata

 * Finally got rid of the manual "Wikidata" build process, Wikidata is part of the weekly deployment now: https://phabricator.wikimedia.org/T173818
 * Introduced a custom, "compact" entity diff serialization in the wb_changes table. Please look out for side-effects: https://phabricator.wikimedia.org/T113468
 * First patches demo the possibility to persist statements on sub-entities utilizing Wikibase's standard wbsetclaim API (namely Forms on Lexemes): https://phabricator.wikimedia.org/T163724
 * Core changed the way it handles section editing. Wikibase hooks into this feature, and now behaves unexpected. Review welcome: https://phabricator.wikimedia.org/T181807

German Technical Wishlist

 * Blocked by:
 * Blocking:
 * Updates: