Scrum of scrums/2016-03-09

= 2016-03-09 =

Community Tech

 * Blocking:
 * None
 * Blocked:
 * Waiting for Niklas to get back from vacation to merge patch to add pageviews tool into translatewiki.net
 * Updates:
 * Deadlink fixing bot is going well. Just built a logging interface for Internet Archives to monitor progress: http://tools.wmflabs.org/deadlinks/
 * Continuing to collaborate with MusikAnimal on Pageviews tool: http://tools.wmflabs.org/pageviews/
 * New topviews interface: http://tools.wmflabs.org/pageviews/topviews/

Discovery Engineering

 * Blocking:
 * Blocked:
 * Security: SVG sanitizer
 * Ops: Maps Hardware https://phabricator.wikimedia.org/T125126
 * Updates:
 * New portal version deployed on beta, up for general deployment soon
 * Completion suggester to be deployed on lower-volume wikis on 10th
 * Working on Relevance Forge (nee Relevance Labs)
 * WDQS GUI now has maps for queries returning coordinates
 * Deployed Maps to Wikivoyage - listening to community feedback
 * WDQS query caching problem root cause found, https://phabricator.wikimedia.org/T128813
 * Discovery data access guidelines published: https://meta.wikimedia.org/wiki/Discovery/Data_access_guidelines
 * Discovery data access guidelines published: https://meta.wikimedia.org/wiki/Discovery/Data_access_guidelines

Collaboration

 * Blocking:
 * External Store - Continuing work
 * Blocked:
 * No new blockers. Matthias put up an XSD for the Flow dumps.
 * Updates:
 * Rolling out cross-wiki notifications beta feature to all wikis tomorrow
 * There have been some issues with duplicate thank-you notifications (for edit milestones, e.g. 100th edit), which are being worked on.
 * Also working on some Echo bundling i18n issues and enhancements to the Notifications special page

Fundraising Tech

 * No blockers
 * preparing real donor test of backup worldwide credit card processor
 * Working out final details of Latin American payment processing expansion
 * Steady stream of CiviCRM enhancements for us and upstream
 * More experiments in fraud monitoring and blocking
 * Investigating firewall rules for PCI scan
 * Investigating possibility of using current PHP/MW
 * Replaced dying drive in ActiveMQ box, cursed SPOF

Android

 * Updates:
 * 2.1.142 beta publishing very soon.
 * Reading list endpoint needs - depending on timing/approach may need security review

iOS

 * Blocked:
 * Need Piwik access for Josh Minor - done. https://phabricator.wikimedia.org/T124218
 * Updates:
 * iOS v5.0 is schedule to release this Thursday March 10th
 * We need to do additional updates to the Apple site association file we host for further app enhancements (https://en.wikipedia.org/apple-app-site-association)

Web

 * Blocking:
 * Blocked:
 * Path forward needed on a references API - https://phabricator.wikimedia.org/T125329 Invitation to join conversation.
 * Updates:
 * Hopefully lazy loaded images we will begin rolling out week commencing 14th [Dependency: performance team]
 * Hopefully lazy loaded images we will begin rolling out week commencing 14th [Dependency: performance team]

Reading Infrastructure

 * AuthManager is still coming! (see last week for details)
 * load.php now logs warnings when the session is accessed in wmf.16. See https://logstash.wikimedia.org/#dashboard/temp/AVNcob7pO3D718AOt5cM

Analytics

 * Blocking:
 * Hadoop issues with encoding have been fixed, querying is totally ok now, data has been back-filled and has been ready for re-computation since earlier this week
 * Blocked: (none)
 * Updates:
 * Working on goals for next quarter, currenlty mostly revolving around transitioning Erik Zachte's work to Wikistats 2.0
 * If you're using UDFs in Hive, you need to add their jars explicitly now, but this gives more flexibility to test changes to UDFs
 * Bots following the user agent policy are now detected correctly in our pageview data pipeline
 * data on unique devices estimated by the last access cookie is now publicly available, but please let us announce it first on analytics-l before spreading word of its existence (we want to make sure people know what this is)
 * The "-" article should no longer show up in the "top articles" endpoint of the pageview API, starting with today's data

Release Engineering

 * Blocking:
 * Are we doing deployments week of Jerusalem Hackathon? [Sync with ops and will announce]
 * Blocked:
 * None
 * Updates:
 * Reminder: code freeze week of 3/21 for codfw fallover. E-mail/tech news forthcoming as well.
 * Notice: Expect an e-mail shortly, but trebuchet is now deprecated for all new service deployments going forward. New services should use scap3.

Research and Data

 * Blocking:
 * None that we know of.
 * Blocked:
 * None.
 * Updates:
 * Stopped the reader survey (at a 1:50 rate) on 2016-03-08. Research documentations are in https://meta.wikimedia.org/wiki/Research:Characterizing_Wikipedia_Reader_Behaviour/S3-English_Large_Scale
 * No major news for the coming week.

Security

 * Blocking:
 * AuthManger?
 * SVG sanitizer - is this blocking a quarterly goal?
 * Blocked:
 * Updates:
 * password hash format updated
 * Lots of security patches deployed - releng, release soon (T124940)?
 * AuthManger review starting
 * AuthManger review starting

Services

 * Blocking:
 * none
 * Blocked:
 * none
 * Updates:
 * morelike API live - https://phabricator.wikimedia.org/T125983
 * Reading: Android wants 320px thmubs, is the rest of Reading OK with it?
 * RB cluster expansion continuing
 * Cassandra
 * switched to 2.1.13, soon testing of 2.2.5
 * start of Brotli compression in staging imminent
 * rate limiting in service-runner
 * RB endpoints - now only logging excesses, soon will start enforcing them
 * API: removing some listing endpoints

Technical Operations

 * Blocking:
 * ORES - moving into a python wheel repo structure
 * Blocked:
 * None
 * Updates:
 * moved ocg to use a replicated redis
 * redis replicated and encrypted on the sessions
 * got <40 precise machines now
 * Please delete unwanted labs instances