Wikimedia Release Engineering Team/Checkin archive/20180604

= 2018-06-04 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * May 31 (Thrusday): Željko (Holidays in Croatia - Corpus Christi)
 * June 1 (Friday): Željko vacation (git merge holiday weeked)
 * June 19: WMF Monthly Holiday
 * July 4: US Holiday
 * August 15: WMF Monthly Holiday
 * Middle of August...a few days somewhere - thcipriani

Train

 * Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R


 * May 21 - wmf.5 - Mukunda
 * May 28 - wmf.6 - Tyler
 * June 04 - wmf.7 - Tyler <-
 * June 11 - wmf.8 - ???
 * June 18 - wmf.9 - no train, SRE summit
 * June 25 - wmf.10 - Mukunda
 * July 02 - wmf.11 - Mukunda

SoS

 * May 21 - Chad
 * May 28 - Mukunda
 * June 04 - Mukunda <-
 * June 11 - Tyler
 * June 18 - Tyler
 * June 25 - ???
 * July 02 - ???

Updates

 * Chad :(


 * Train/SoS rotation proposal while we're down a person:
 * Train: Include Antoine/Zeljko into the rotation and move the train deployment time to a EU-friendly time during those weeks
 * something else.. ?
 * DECIDED: spread the pain across all (Antoine, Dan, Mukunda, Tyler, Zeljko, with JR after the first rotation)
 * DECIDED: for weeks when it's EU people doing it we'll move the train window to a EU-centric time slot
 * ACTION: Tyler to setup a branch cut hangout for this week's train (eg: tomorrow)
 * DECIDED: AUTOMATE TEH SHIT OUT OF IT

Offsite notes and action items

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Offsites/2018-05-Barcelona/Notes

Data Data Data

 * Talk with Analytics - JR
 * Talk with CE/Bitergia - JR
 * Explore Bitergia - JR
 * Identify data sources we want to collect - RelEng (who know what systems)
 * https://etherpad.wikimedia.org/p/releng-data-sources
 * TODO: Greg: email the etherpad out to the team list
 * Erik Bernhardson / Guillaume Lederrey
 * After we get our list of data sources talk to them

SWATs/Trains

 * TODO: Tyler reasses scap swat in mw-config from Mukunda
 * Look into parsing scap messages for known patterns and pulling out the data
 * Look into enabling scap start/done
 * Look into recording if mwdebug was used during the deploy (eg: 'scap stage')
 * H/Now will we get time for this?
 * Have Mukunda do a couple weeks of SWATs
 * Mukunda has a lot to say about this subject.... writeup incoming

TODO tl;dr: Tyler taskify the above and talk to Mukunda

Staging

 * Greg to talk with Deb about what to do next with talking to Victoria
 * Done
 * Greg to figure out how we can better market what we are accomplishing (eg "monthly showcase")
 * TODO
 * Get a k8s cluster from SRE for CI to deploy to.
 * Dan/Tyler discuss later today
 * potentially poke/get a committment on a timeline soon

Scrum of Scrums

 * Greg to copy to etherpad after meeting: https://etherpad.wikimedia.org/p/Scrum-of-Scrums

Release Engineering

 * Blocking
 * Blocked
 * Updates
 * Heads up: There will be more people in the normal MW Train deployment rotation (namely: Antoine, Zeljko, and Dan to start)
 * Heads up: There is a new primary deployment host, deploy1001.eqiad.wmnet (tin is gone)
 * Reminder: just use deployment.eqiad.wmnet for ssh
 * Quarterly cross-dependencies
 * Reminder: just use deployment.eqiad.wmnet for ssh
 * Quarterly cross-dependencies

Release Engineering

 * Blocking
 * Blocked
 * Updates
 * Train status: https://phabricator.wikimedia.org/T191051
 * 1.32.0-wmf.5 got held up last week by an incident that happened during the scheduled Thursday train deployment window.
 * https://wikitech.wikimedia.org/wiki/Incident_documentation/20180524-wikidata
 * We should be back on track this week, wmf.5 is clear of blockers and wmf.6 should be rolling out as usual
 * Quarterly cross-dependencies
 * We should be back on track this week, wmf.5 is clear of blockers and wmf.6 should be rolling out as usual
 * Quarterly cross-dependencies

Train status and happenings

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Past week status updates

 * All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q4

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure

 * Migrate away from Nodepool -
 * Migrate MediaWiki PHPUnit tests to Shipyard (docker-based CI) (~40% of Nodepool usage) -
 * Add Composer support to Blubber -
 * Add Python/Tox support to Blubber -
 * Add Ruby/Gem/Bundler/Rake support to Blubber -

Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans

 * Broad role out of Code Stewardship model.
 * Update Maintainers/Developers page with currently known Code Stewards.
 * Use Code Stewardship review process to address gaps.
 * Deploy dashboard of Code Stewardship Coverage


 * No activity

Program 3: Outcome2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation

 * Roll out of technical debt reduction approaches.
 * Identify early adopter engineering teams to test approaches.


 * Tech Debt engagement kickoff meeting with Platform
 * Tech Debt engagement kickoff meeting with Search Platform

Program 6: Outcome 2: Objective 2: Prove viability of testing staged service containers alongside MediaWiki extension containers

 * Add Composer support to Blubber -
 * Small, standalone, MediaWiki containers built using Blubber
 * Limited scope with: Debian Stretch, php7, composer dependencies, Mariadb, Apache (or last two with standalone containers)
 * Build MW base containers upon branch cut at master branch point from core
 * Build ext specific containers using MW image as base


 * At a bit of an impasse with the whole MW-extensions-and-composer thing
 * Still not sure why https://gerrit.wikimedia.org/r/#/c/435289/ was -2'd as it's just adding meta data
 * Pushed to https://github.com/marxarelli/mediawiki-extensions-Math for now to run experiments
 * Waiting for feedback on https://phabricator.wikimedia.org/T193824#4244594

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)

 * workboard

Program 1: Outcome 6: Milestone 2: Maintain Phabricator

 * Streamline logspam workflows by adding some integration with phabricator
 * Store git-lfs (and other phab uploads) in swift:

Other work

 * Staging Environment
 * Sent out Beta Cluster survey. 33 responses so far.

Antoine
https://phabricator.wikimedia.org/J107


 * What I plan to do this week
 * Wikibase does not play well with Quibble
 * Experimental job for gated extensions and maybe migrate them
 * Catch up with train deploy
 * What I'm blocked on
 * Lot of extensions fails due to random reason
 * Other?

Chad

 * What I plan to do this week
 * What I'm blocked on
 * Other?
 * Other?
 * Other?

Dan

 * What I plan to do this week
 * Continue experimenting with composer and MW extensions
 * It's hella slow and can't recurse when used with git repos
 * Seeing whether Satis might be utilized to keep a private MW composer repo up to date
 * Maybe update it from git repo periodically but it also supports partial updates given a specific package name and repo url (so, a CI job could update it after a merge to MW ext repo)
 * What I'm blocked on
 * https://gerrit.wikimedia.org/r/#/c/435289/ but not really, because I just forked it on (Micro$loth)GH... :)
 * Other?

Greg

 * What I plan to do this week
 * OMG Annual Reviews and such
 * Hiring Follow-up with Recruiting
 * What I'm blocked on
 * Other?
 * Other?

Jean-Rene

 * What I plan to do this week
 * Follow up with Q3 stewardship review newly assigned stewards re progress with plans.
 * Dig into Platform tech debt - followup to engagement kickoff meeting.
 * Search Platform tech debt - prioritizing approach for them.
 * Follow-up on "Data,Data,Data" tasks from offsite.
 * What I'm blocked on
 * Other?
 * Other?

Mukunda

 * What I plan to do this week
 * What I'm blocked on
 * Other?
 * Other?
 * Other?

Tyler

 * What I plan to do this week
 * Train (meetings for train)
 * Discuss ways to move-forward/unblock P6O2O2
 * Help moving deployment-deploy01
 * Peer review writing time
 * What I'm blocked on
 * Other?
 * Other?

Zeljko

 * What I plan to do this week
 * T195496 mediawiki-core-qunit-selenium-jessie sometimes fails for extensions
 * T194252 Configure the CI job that runs WikibaseLexeme's browser tests against beta wikidata
 * T167432 Run Wikibase daily browser tests on Jenkins
 * What I'm blocked on
 * T196374 language-screenshots-VisualEditor fails with `stderr: warning: failed to remove cache`
 * Other?

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng
 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart