Wikimedia Release Engineering Team/Checkin archive/20180108

= 2018-01-08 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * Jan 15 (Mon): Martin Luther King Day (All US Staff)
 * Jan 22/23: Dev Summit
 * Jan 24: Tech Management F2F
 * Jan 25/26: WMF All Hands
 * Jan 29-31: Team offsite
 * Feb 19 (Mon): President's Day (All US Staff)
 * Mar 30 (Fri): WMF Holiday

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R

Jan 1 and Jan 8

 * Train: Mukunda
 * wmf.15
 * wmf.16
 * SoS: Tyler
 * Out:
 * January 1 (Monday): Holiday, everyone

Jan 15 and Jan 22

 * Train: Tyler
 * wmf.17
 * No train: Mukunda
 * SoS:
 * Out
 * Jan 15 (Mon): Martin Luther King Day (All US Staff)

Scrum of Scrums

 * Greg to copy to etherpad after meeting: https://etherpad.wikimedia.org/p/Scrum-of-Scrums

Release Engineering

 * Blocking
 * Blocked
 * Updates
 * Updates
 * Updates

Last week

 * Blocking
 * Blocked
 * Teams that did not port Selenium tests from Ruby to Node.js (T139740): No team (1), Editing (1), Fundraising Tech (1), Global Collaboration (4), Multimedia (1), Reading Web (3), WMDE Engineering (5)
 * Updates
 * REMINDER: No deploys this week and next (18th and 25th). Take a break! [wiki][email]
 * We’re (Chad) going to spend some time this week cleaning up log-spam.
 * Our Q3 goals posted
 * Completed the security upgrade of RuboCop and Rubyzip
 * Our Q2 “SSD” (Program 6) goals are now all complete [wiki][email]
 * Antoine is migrating the CI docker image build process to using docker-pkg (from Giuseppe)
 * Follow-up continues on the ORES post-mortem follow-ups
 * Follow-up continues on the ORES post-mortem follow-ups

Puppet SWAT

 * list of patches you want to submit to Puppet SWAT

Logspam \ Last week's train updates

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

* Almost all logspam eliminated: https://i.imgur.com/1iOT9S1.png

Other Team Business

 * Offsite!
 * https://etherpad.wikimedia.org/p/releng-offsite-201801-topics


 * Create a -releng-feed channel and move bots there?
 * https://phabricator.wikimedia.org/T181582
 * already just deleted the selenium irc notifications, no one was watching them

Q3 goal/project check-in

 * All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q3

Program 1: Outcome 5: Milestone 1: Develop and migrate to a JavaScript-based browser testing stack

 * Due: End of this quarter
 * What: Specific improvements to the now canonical framework, see: task T182421, notably:
 * Upgrade webdriverIO to version 4.9
 * Investigate replacing nodemw with mwbot
 * Video recording for Selenium tests in Node.js
 * Task:

* no major updates this past week: Z's been sick over the holdiays, sadly

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure

 * Goals
 * Draft requirements for a Kubernetes based solution for CI -
 * Migrate MediaWiki PHPUnit tests to Shipyard (docker-based CI) (~40% of Nodepool usage) -


 * most npm jobs are migrated. Gotta handle the /deploy repositories that have a node_modules commited in
 * PHPUnit not started yet


 * Unify production and CI docker image build process -
 * npm from the Debian package (1.4.x) does not support running behind a proxy . Prevents building npm based Docker container on contint1001.

Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans

 * Due: End of quarter

Nothing new to report

Program 3: Outcome 2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation

 * Due: End of quarter

Nothing new to report

Program 3: Outcome 2: Objective 3: Promote and surface important technical debt topics at large gatherings of Wikimedia developers (e.g., DevSummit and Hackathon(s))

 * Due: End of next quarter

Investigating what can be done during this coming dev summit/all-hands

Program 6: Outcome 2: Objective 2: Set up a continuous integration and deployment pipeline

 * Due: End of this quarter
 * Keyword: SSD
 * phab project: https://phabricator.wikimedia.org/project/view/2453/
 * Goal:
 * Verify basic functionality of 'production' deployment and image (initially targeting mathoid):
 * Functional PoC within integration in the deployment-pipeline
 * Deploy to isolated k8s


 * had a sad meeting last week, just lack of attendance
 * dan filed tasks
 * I (tyler) filed tasks this morning
 * Child tasks of https://phabricator.wikimedia.org/T183165

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)

 * workboard


 * Need to get the scap CI situation figured out because of repeated breakage of scap in beta. Two competing needs:
 * Need a place to test scap changes
 * Other teams use scap in beta to test their own deployments

Program 1: Outcome 6: Milestone 1: Maintain Gerrit

 * We need to find out when we're doing the zuul/gerrit upgrade
 * Deb package needs to be rebuild. It is in bad shape https://phabricator.wikimedia.org/T158243

Program 1: Outcome 6: Milestone 2: Maintain Phabricator

 * Streamline logspam workflows by adding some integration with phabricator
 * Store git-lfs (and other phab uploads) in swift:

Program 1: Outcome 5: Objective 1: MW Nightlies server

 * We're building non-security nightlies! (not quite stable enough to announce to public yet)
 * \o/
 * Working with security team to figure out best way to manage patches with minimal overhead
 * any changes to weekly branch cut process coming soon?

Other work
select count(distinct ip) from webrequest where year = 2017 and month = 12 and uri_host = 'meta.wikimedia.org' and uri_path = '/WikipediaMobileFirefoxOS/index.html'; ^^^ Seems like it over-counts, given how frequently mobile IPs can change ^^^
 * Working on a Hive query to find out how many unique users are hitting WikipediaMobileFirefoxOS in a month


 * More apache docroot cleanup

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng
 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart