Scrum of scrums/2017-11-08

= 2017-11-08 =

Callouts

 * No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week).
 * RelEng: Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained). task
 * Tidy replacement: if you are an editor on some wiki, please get the word out to your wikis to fix the high-priority linter issues. We want to get Tidy removed from the cluster sooner than later.A bunch of wikis (including mediawiki) no longer have Tidy, and a few 100 small wikis can get Tidy replaced based on stats about linter issues ( https://www.mediawiki.org/wiki/Parsing/Replacing_Tidy/Linter/Stats/2017-11-06#Wikis_with_zero_high-priority_errors ). But would be good to get some largish wikis moved over so we can identify any lingering problems sooner than later.

iOS native app

 * Blocked by:
 * Blocking:
 * Updates:
 * 5.7.1 in the app store today (minor bug fixes) https://phabricator.wikimedia.org/project/view/3047/
 * Continuing work on 5.8 - https://phabricator.wikimedia.org/project/view/2913/ (faster article loads, synced reading lists)

Android native app

 * Blocked by:
 * Blocking:
 * Updates:
 * Released version 2.6.206 to production.
 * Continuing to test / integrate with Reading List service.

Reading Web

 * Blocked by:
 * Ops&Release engineering - How to use Chromium in Puppeteer +https://phabricator.wikimedia.org/T178189
 * Currently investigating on path forward in https://phabricator.wikimedia.org/T180037.
 * Blocking:
 * Updates:
 * Working on the Chromium-based PDF renderer service.
 * Starting performance testing within two weeks.
 * Researching behaviour around the "Download to PDF" button https://phabricator.wikimedia.org/T177969#3742056
 * Working product and technical documentation for Marvin

Reading Infrastructure

 * Blocked by:
 * Blocking:
 * Updates:
 * Continuing work on MCS summary, media (gallery) endpoints.
 * Continuing with Reading Lists

Multimedia

 * Blocked by:
 * Blocking:
 * Updates:

Discovery

 * Blocked by:
 * Blocking:
 * Updates:
 * continuing work on portal automation. going well.

Maps

 * Blocked by: N/A
 * Blocking: N/A
 * Updates: None

Editing

 * Blocked by:
 * Blocking:
 * Updates:

Parsing

 * section + figure-inline + html5-ids changes aren't going out this week after all -- reviews not yet finalized + we need thorough testing in beta cluster. We are still hoping to get it out next week.
 * Tidy replacement: if you are an editor on some wiki, please get the word out to your wikis to fix the high-priority linter issues. We want to get Tidy removed from the cluster sooner than later. A bunch of wikis (including mediawiki) no longer have Tidy, and a few 100 small wikis can get Tidy replaced based on stats about linter issues

Global Collaboration

 * Blocked by: nobody
 * Blocking: ops maybe? There's the Flow dumps issue but not sure how much of a blocker that is
 * Updates:
 * Going to roll out RCFilters to remaining (FlaggedRevs) wikis on Nov 15
 * Early work on using wikitext instead of HTML for storing Flow content, need something like pcache except 1) it's Parsoid HTML not PHP HTML 2) Flow comments aren't pages

UI Standardization

 * OOUI v0.24.1 & v0.24.2 released (missed last week's Scrum of Scrums), https://phabricator.wikimedia.org/diffusion/GOJU/browse/master/History.md;v0.24.2; among changes:
 * UI design
 * WikimediaUI theme with 3 changes: SVG markup (see below), MenuSectionOptionWidget clearer on long lists and ComboBoxInputWidget providing stronger user feedback on open
 * Ongoing:
 * OOUI & relying products:
 * icons: Work on icon set to be more harmonious and align to WikimediaUI Style Guide's guidelines https://phabricator.wikimedia.org/T177432
 * Unify SVG markup across Foundation products https://phabricator.wikimedia.org/T178867

Community Tech

 * Not blocking
 * Not blocked
 * Our annual survey is underway: https://meta.wikimedia.org/wiki/2017_Community_Wishlist_Survey
 * Fixing issues uncovered by Unicode sections rollout to Russian projects

Analytics

 * New kafka Jumbo cluster is up, for now mirror-ing topics from analytics cluster.
 * New DB hosts for eventlogging back in place, purging data out of compliance at a faster rate due to better hardware.
 * We have spark2 in cluster!
 * Still troubleshooting metric difference wikistats 1.0 on wikistats 2.0. Take a look edit metrics available now (some of them have a 5% difference, so not final yet): https://stats.wikimedia.org/v2/#/am.wikipedia.org
 * Work on refining eventlogging data in hadoop continues so hadoop can be a easy to query backend for eventlogging data

Cloud Services

 * Blocked by:
 * Blocking:
 * Updates:

Fundraising Tech

 * Blocked by:
 * Blocking:
 * Updates:
 * Deploying new English thank you letter
 * Payments form validation improvements (https://phabricator.wikimedia.org/T176501)
 * Better API error handling for backup card processor (https://phabricator.wikimedia.org/T177893)
 * More stats reporting to prometheus
 * CiviCRM UI improvements (https://phabricator.wikimedia.org/T142549, https://phabricator.wikimedia.org/T178003, https://phabricator.wikimedia.org/T177873)
 * CentralNotice: allow delaying impression reporting (https://phabricator.wikimedia.org/T176334)

MediaWiki Platform

 * Blocked by:
 * Blocking:
 * Updates:

Performance

 * Blocked by:
 * Blocking:
 * Updates:

Release Engineering

 * Blocking
 * Blocked
 * Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained).
 * See: https://phabricator.wikimedia.org/T139740
 * Notably: Global Collaboration Team, Fundraising Tech, Wikibase, and Mulitmedia
 * Updates
 * No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week).
 * [TechDebt program] First pass of the service levels for component ownership was shared with the Code Health Group last week; feedback on-going.
 * [TechDebt program] The next blog post should be posted Real Soon Now™ (done on our side).
 * [SSD Program] Working on getting the mathoid tests running on submit.
 * [SSD Program] A new release of Blubber is on the horizon.
 * [SSD Program] Aside: what it is: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Deployment_pipeline
 * [SSD Program] Bubber: https://phabricator.wikimedia.org/source/blubber/
 * [scap tech debt] Working to support both pre git-2.11 and post git 2.11 hosts (namely Trusty and Jessie (with backports)/Stretch, respectively) to let us use newer functionality (notably `--jobs`).
 * [CI] Most tox jobs are moved to Docker containers
 * [CI] Investigating why many docker containers are left behind after a SIGTERM
 * [CI] Investigating why many docker containers are left behind after a SIGTERM

Research

 * Blocked by:
 * Blocking:
 * Updates:

Scoring Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * frwiki -- ongoing issue. https://phabricator.wikimedia.org/T179711 (should be fixed by SOS)
 * Draft quality data is coming to English Wikipedia (ping Commtech and Global Collab)
 * You can use this to help patrollers sort new pages by (Spam, Attach, Vandalism/Hoax, or OK). OK == not immediately concerning but not necessarily good.
 * [Reponse from Roan: we don't plan to use draftquality directly at this point]
 * Wikilabels getting some improvements.
 * Links to on-wiki documentation for each campaign ("info" link)
 * Statistics about labelers will have usernames
 * Amir is working on Icelandic language support.

Search Platform

 * Blocked by:
 * Blocking:
 * Updates:
 * Added hiragana<->katakana translations for English wiki (not for Japanese one) https://phabricator.wikimedia.org/T176197
 * Implemented demoting disambig pages for Wikidata prefix search https://phabricator.wikimedia.org/T148411
 * Improved reusability of Searcher code https://phabricator.wikimedia.org/T178906
 * Improving completion suggester https://phabricator.wikimedia.org/T178474
 * Working on porting Selenium tests from Ruby to JS
 * Working on upgrade to Elastic 5.5
 * Working on indexing Wikidata descriptions and adding them to fulltext search

Security

 * Blocked by: None
 * Blocking: Everyone waiting for security reviews
 * Updates:
 * Reviews:
 * BeautifulSoup (done, comments will be added shortly)
 * vue.js (done, same)
 * mediawiki-services-chromium-render
 * Preparing for MW security release

Services

 * Blocked by: none
 * Blocking: none
 * Updates:
 * Parsoid storage switched to new storage model for all but wikipedias
 * Definitions storage switched to new model
 * Planning on switching RecordLintJob to kafka-based JobQueue

Technical Operations

 * Blocked by:
 * Flow isAllowed gets actual revision text before it is needed https://phabricator.wikimedia.org/T172025
 * Blocking:
 * Updates:
 * labsdb1001 storage crashed; Manuel got the dbs back up, but we are leaving in read-only - https://phabricator.wikimedia.org/T179464
 * Turning up s8 databases for wikidata https://phabricator.wikimedia.org/T177208
 * wikidiff2 1.5.1 fully rolled out, enabled for group0 wikis, additional will be enabled by Adam Shoreland over the next weeks

Wikidata

 * Blocked by:
 * Blocking:
 * Updates:
 * getting rid of the Wikidata build (to deploy Wikibase etc. as normal extensions): https://phabricator.wikimedia.org/T173818
 * changed how JS libraries are included (as submodules, instead of via composer)
 * work on Lexeme (persistence)
 * work on multi-content revisions
 * Coordinates in RDF exports are going to use xsd:double: https://phabricator.wikimedia.org/T179228

German Technical Wishlist

 * Blocked by:
 * Blocking:
 * Updates: