Scrum of scrums/2016-02-10

= 2015-02-10 =

Analytics

 * Blocking: Nobody we know of
 * Blocked: By nothing we're aware of
 * Updates:
 * organizing work to replace limn dashboards with dashiki
 * wikimetrics program global metrics deployed
 * moved wikimetrics from vagrant to docker-compose and it was a pleasant experience
 * working on cleaning up / organizing analytics files on dumps.wikimedia.org/other
 * estimating uniques with last access cookies is in the final stages of being productionized
 * Joseph's sanitization of pageview data (for long term retention and privacy) is up for review by halfak and anyone else interested

Release Engineering

 * Blocking:
 * https://phabricator.wikimedia.org/T125003 - move CI slaves in Beta to Jessie and Node 4.2
 * Blocked: none
 * Updates:
 * Train deploy rolling out wmf.13 this week
 * CI had some trouble due to lack of precise executioners on Monday, should be fixed now
 * Lots of scap changes in puppet getting lined up: provider, scap::target, service::node

Research

 * Blocking:
 * Blocked:
 * Updates:
 * Reader survey will go out (most probably, if all the technical components work smoothly) as part of the deployment train tomorrow (Thursday)
 * Working with the mobile web team
 * Survey will run for a week in enwiki, using QuickSurveys and we expect to collect between 200-500K responses
 * Sampling rates will be finalized today
 * Documentation: https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour

Security

 * Blocking:
 * Maps: Security review for PHP schema composer module - https://gerrit.wikimedia.org/r/#/c/269325/
 * Graphs: Security review for SVG sanitization - https://github.com/cure53/DOMPurify
 * Blocked: Waiting to see how SessionManger goes before we deploy a bunch of patches
 * Updates:
 * 2FA for Keystone done, working on Horizon integration next

Services

 * Blocking: ???
 * Blocked:
 * https://phabricator.wikimedia.org/T125003 - move CI slaves in Beta to Jessie and Node 4.2
 * https://phabricator.wikimedia.org/T107307 - move CXServer to SCB
 * https://phabricator.wikimedia.org/T107306 - Apertium pkgs for Jessie
 * Updates:
 * separated out the framework part of RESTBase into https://github.com/wikimedia/hyperswitch
 * Testing Brotli compression for Cassandra in staging, seeing OOMs
 * restbase1007 bootstrapping
 * rb1001 and rb1002 might run out of disk space because of it
 * separate AQS off of RESTBase - https://phabricator.wikimedia.org/T126294

Technical Operations

 * Blocking: ORES
 * Blocked: none known
 * Updates:
 * OTRS upgrade to 5.0.6
 * MultiDC work ongoing with the rest of Technology

Community Tech

 * Blocking: none
 * Blocked: none
 * Updates:
 * Continuing work on Gadgets 2.0 after recieving further feedback
 * Continuing work on PageAssessments extension

Discovery

 * Blocking: none
 * Blocked:
 * Maps: Security review for PHP schema composer module - https://gerrit.wikimedia.org/r/#/c/269325/
 * Graphs: Security review for SVG sanitization - https://github.com/cure53/DOMPurify
 * Updates:
 * Completion suggester changes done, probably not deployed yet due to SessionManager things
 * Tuning suggester & morelike performance
 * TextCat language detection merged, will work on making A/B test with it next
 * Working on next year plans/budgets
 * WDQS Blazegraph 2.0 update rolled back due to serious bug, investigating the bug
 * Maps: blocked on security to deploy Kartographer extension
 * Graphs: disabled http / https protocols, use custom protocols now
 * Data: begining data storage planning (Yuri will add task # here :)

Collaboration

 * Blocking:
 * Dry run patch for external store migration is merged. Now we need to set External Store up on Beta, then test the dry run patch there: https://phabricator.wikimedia.org/T119567
 * Blocked:
 * Flow dump generation on dumps.wikimedia.org: https://phabricator.wikimedia.org/T119511
 * Updates:
 * Human-readable name patch is almost done: https://phabricator.wikimedia.org/T121936
 * Cross-wiki notifications will soon be rolled out to "real" wikis (currently in production it is only on test and test2):
 * 'testwikidatawiki'
 * 'wikidatawiki'
 * 'mediawikiwiki'
 * 'commonswiki'
 * 'hewiki'
 * 'hewikibooks'
 * 'hewikinews'
 * 'hewikiquote'
 * 'hewikisource'
 * 'hewikivoyage'
 * 'hewiktionary'
 * 'frwiki'
 * 'frwikibooks'
 * 'frwikinews'
 * 'frwikiquote'
 * 'frwikisource'
 * 'frwikiversity'
 * 'frwikivoyage'
 * 'frwiktionary'

Language

 * Blocking:
 * I have got some pings outside SoS, but don't know if those are blocking
 * Packages for Jessie requested to speed up
 * Blocked: nope
 * Updates:

Multimedia
Blocking: Not that I know of.
 * Blocked: Metrics work currently paused pending some burning questions for halfak, MarkTraceur to sit down with him this week. Look out, Security, we're hoping to get security review on a new extension (see below) in the next few months.
 * Updates:
 * Labs instance with ImageTweaks (name WIP) going up today, email to (wikitech|commons|multimedia)-l later tonight
 * Some work on NewFiles, in particular filtering by tag, but it's not going super. MatmaRex wanted some extra eyes on it, but I think he's contacted someone already.

Parsing

 * Blocking: ???
 * Blocked: ???
 * Updates:
 * The templatedata-based serialization patch ( https://gerrit.wikimedia.org/r/#/c/264043/ ) didn't go out this week after all -- yet to be reviewed. Possible it will get reviewed this week.
 * Parsoid testing services (parsoid rt-testing; parsoid <--> php-parser html visual diff testing) fully puppetized and operational on ruthenium. Work ongoing to enable visual diff testing between production mediawiki and a non-production mediawiki (to test for example changes to parsing) using labs hardware ( https://phabricator.wikimedia.org/T120345 ).
 * Would appreciate help from anyone familiar with hiera (to configure labs hardware to use existing puppet roles, and making any additional tweaks).
 * This week, Arlo will start Parsoid-side work to separate out data-mw into its own RESTBase bucket -- deployment is still some ways away. We need to finalize REST API versioning policy ( https://phabricator.wikimedia.org/T124365 ) and also have all the various Parsoid clients updated. We will create phab tickets for dependent projects once T124365 is resolved.
 * Last week, Flow, Services, and us had a meeting and figured out a strategy for Flow since they use VE, but also talk with Parsoid (and not RESTBase).
 * Parsoid native implementation for gallery underway (heads up: CX, VE, Flow, Reading)
 * https://phabricator.wikimedia.org/P2506 is the spec for Parsoid's output if you want to provide feedback
 * https://gerrit.wikimedia.org/r/#/c/264026/ is the WIP patch
 * Tim working to collect source location information in the PHP preprocessor for errors / warnings and a way to expose them. Useful in general, but, also specifically in the context of work related to replacying Tidy.

VisualEditor

 * Blocking: ???
 * Blocked: ???
 * Updates:

Fundraising Tech
No blockers, most work is continued from last week
 * updates for new CiviCRM financial tracking
 * fixes and enhancements for backup credit card processor
 * prep for Latin America fundraising expansion
 * debugging email clickthrough tracking

Android

 * Updates: beta v2.1.140 released but will not be promoted due Wikipedia Zero incompatibilities. v2.1.141 coming soon.
 * Blocked: the options API was not designed for storing collection-like data. Gather collections don't support inter-wiki pages.

Reading Infrastructure

 * SessionManager is coming back this week. Please let Brad, Bryan, and Gergő know if you spot any problems. Preferably before Friday afternoon.