Scrum of scrums/2015-11-18

= 2015-11-18 =

Web

 * ReadMore works on mobile beta
 * Upcoming deployment ^ (Security review, etc.) soon. Only enabled in mobile web beta.
 * EventLogging for ReadMore
 * ReadMore supports CirrusSearch suggestions
 * QuickSurveys will be deployed on Tuesday with first survey (rollout to enwiki Thursday)
 * QuickSurvey disabled on Monday 16th Nov. Enough data.
 * Planning to maybe run another before December freeze.

Android

 * New v2.1.134 beta and prod published. Lots of crash fixes.
 * New donation campaign.
 * Many hygiene patches focused around our tests.
 * Bug fixes for unusual (flaky) but popular devices as identified by HockeyApp.

Universal Links

 * apple-apps-site-association file deployed on Monday, shouldn't be in use or affecting sites yet
 * next step is conditional markup rendering in (mobile?) web

Content Services

 * Thank you for srcset in Parsoid output. :)
 * The routes have changed, new ones have been added, the mobile-html route has been removed. https://phabricator.wikimedia.org/T117429.
 * Wanted to pass Accept-Language headers to backend-requests but that wouldn't work. https://phabricator.wikimedia.org/T118882. It comes with all sort of issues (Parsoid doesn't handle it, cache/storage fragmentation of various Accept-Language header values.) Waiting for discussion at Dev Summit how to deal with language variants. For now, to work around this issue, the Android app is going to avoid using the RESTBase API for zhwiki (https://phabricator.wikimedia.org/T118905).
 * Working on offloading more DOM transformations from client to service.

Reading Infrastructure

 * Block: Need security review for SessionManager, see https://phabricator.wikimedia.org/T116413 for details
 * Block: ApiSandbox is still blocked on whoever owns oojs-ui for https://phabricator.wikimedia.org/T91148
 * Could use Chris looking at https://meta.wikimedia.org/wiki/Special:OAuthManageConsumers/proposed/7e5461e6907ad6ed43684793046fd3a9 due to the security question in there.
 * On my radar :), I'll approve or email comments today.

Services

 * High-traffic REST API
 * https://phabricator.wikimedia.org/T117082 - deploy to happen tomorrow


 * EventBus
 * basic events defined
 * Otto working on the EL REST service
 * change propagation - https://phabricator.wikimedia.org/T117933


 * RESTBase
 * better schema validation of requests
 * better support for definining spec-only modules


 * AQS live and announced!
 * new version of MobileApps in production

Release Engineering

 * Blocking: (none)
 * Blocked by: (none)
 * Updates:
 * Deployment tooling implementation continues
 * https://phabricator.wikimedia.org/project/view/1449/
 * Documentation (for user centric docs) largely done
 * Still need different "Getting started" sections for those executing a deployment vs. devs setting up deploy repo initially
 * Working on python/Debian packaging
 * MW release 1.26 next week (!)
 * Please take a look at https://phabricator.wikimedia.org/tag/mw-1.26-release/ for tasks/bugs related to your projects

Technical Operations

 * Blocking: none
 * Blocked by: none
 * Updates:
 * EncryptAlltheThings goes better (MariaDB woes with OpenSSL, will encrypt traffic from varnish to apaches)
 * varnish level rate-limiting deployed
 * puppet swat way too successful (no backlog, we might consider scaling it down a bit)

Security

 * Reviews: session/aphlict done, review of AuthManager/Sessions scheduled this week
 * Flow security patch released, other secuity patches on the cluster
 * opsec talk soon

Fundraising Tech

 * Code mostly frozen for December
 * Fixing some bugs from the civicrm upgrade
 * Several minor, final pre-December updates to CentralNotice deployed last week
 * Completed migration of near-realtime banner impressions stats to Kafkatee
 * Diverse improvements in systems handling payment processors and e-mails, as usual
 * Upcoming: more of the preceeding point, looking into some bugs

Analytics

 * Pageview API released publicly ::song, dance:: :)
 * EventBus work ongoing, prototype eventlogging service going through code review now
 * Wikimetrics work to support program evaluation

Parsing

 * Parsoid output now provides srcset attribute in images (live as of last Wed)
 * A bunch of performance bugs fixed -- should reduce the # of pages that timeout
 * Parsoid native extension registration work begun (Arlo)
 * Plans to start working on improving multimedia support in Parsoid (audio / video) (Scott)
 * Tim has worked on a much improved visual-diff algorithm as a replacement for the resemble.js-based image comparson that we used so far
 * this new version can detect moves in an image and help accurately quantify rendering diffs => it can be used in mass testing of visual diffs where the numeric diff metric can be used to identify the most important diffs to address.
 * will be useful in comparing parsoid and php rendering, as well as for testing Tidy replacement with a HTML5 parser.

Collaboration

 * External store patch still needs to be coordinated with Jaime: https://gerrit.wikimedia.org/r/#/c/226544/
 * Phase 1 of artificial primary key is done. Next we'll run the script.
 * Dump work almost done.
 * Still have some followup for ptwikibooks.
 * Cross-DC preparation work has been successful so far. We are doing okay with the cache under reduced use, using slave queries instead.
 * Cross-wiki Echo notification work is progressing well.
 * We're working on Flow anti-spam integration, particulary Nuke.

Discovery
Not blocking or blocked.
 * Portal deployed now via gerrit
 * Targeting to deploy quick survey on 19th

Trinkets

 * Please collaborate on the vision of Wikipedia -- https://meta.wikimedia.org/wiki/User:Yurik/I_Dream_of_Content
 * Maps is on at Wikivoyage; working on migrating GeoHack
 * Migrating Graphs to Vega 2.0

Research

 * reader research QuickSurvey
 * ran a small survey on enwiki ondesktop last week
 * running another small survey on enwiki on mobile this week
 * Deployment for ORES
 * 13 wikis supported: dewiki, enwiki, eswiki, fawiki, frwiki, hewiki, idwiki, itwiki, nlwiki, plwiki, trwiki, viwiki, wikidatawiki
 * Backpressure system deployed that prevents overloading of our cluster
 * Still working on feedback from security review (security -- not blocked)
 * referrer policy cleared by Security and CE
 * https://phabricator.wikimedia.org/T87276 -- Extension is deployed. Configuration is not updated yet.
 * All external link clicks will send referrer info now -- domain name, but not path
 * https://meta.wikimedia.org/wiki/Research_talk:Wikimedia_referrer_policy#Origin_referrer_has_been_code-reviewed