Wikimedia Release Engineering Team/Checkin archive/2023-07-19

= 2023-07-19 =


 * Last time

🏆 Wins & Gratitude

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
 * Jul '23 edition


 * Bunch of GitLab accounts created ~200 accounts
 * Tooling for recurring account syncing
 * Default phabricator.wikimedia.org frontpage has a "I need access or permissions to..." tab panel - https://phabricator.wikimedia.org/W2984
 * Building out a deployment pipeline
 * Good reviews and talented engineers on the team
 * Desiloed
 * Did math!
 * JohnBond upgrading to puppet 7
 * Contint servers upgraded to in-warranty hardware! (Dzahn, John Bond, Jelto)—ssds! zuul-merger should be way faster
 * git::clone puppet resource updated! LDAP group sync stuff
 * Great team + smart people + good reviews
 * ^ +1 + Daniel Zahn, Eoghan, Ahmon's review
 * Meeting yesterday with Dzahn, grateful for him reaching out, having a plan for phab migration stuff
 * Mention of doing a git blame on stacktraces. Experiment in scap was https://gerrit.wikimedia.org/r/c/mediawiki/tools/scap/+/773870


 * Phatality digging
 * BuildKitd allowlist is deployed to production
 * Phab backlog cleaning
 * Boldly deleting wikipages!
 * Andre has phab-root, gerrit-root, contint-root
 * Toronto is beautiful :)
 * List of supergroupers (DX)

🌻 Open source/Upstream contributions

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Upstream


 * Antoine reviewed/fixed a stack of patchset for python-jenkins https://opendev.org/jjb/python-jenkins (used by JJB) + granted release rights to one of the participants

Code review

 * +1'd gerrit changes
 * TODO: gitlab equivalent

Gerrit Access requests

 * Gerrit access requests

GitLab Access requests

 * Accounts and auth -
 * GitLab access requests

📅 Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2023
 * https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off

July 2023

 * 03-04 Mon: U.S. July Holiday, US staff with reqs
 * 05 Wed & 06 Thu: Andre CZ public holidays
 * 06 Wed–11 Tue: Dancy out (wedding, not mine)
 * 14 Fri Antoine (Bastille Day, France national holiday)


 * 18 Tue - 21 Fri Jaime
 * 23 Fri Jun–18 Fri Aug: Jeena → Mongolia :D :D
 * 25Tue-28Fri Andre less available (conference)
 * 27 Thr - 28 Fri Dan
 * 28 Fri: Brennen probably out for RockyGrass 🪕
 * 31 Mon Jul – 21 Mon Aug Antoine

August 2023

 * 31 Mon Jul – 21 Mon Aug – Antoine
 * 23 Fri Jun–18 Fri Aug: Jeena → Mongolia :D :D :D
 * 07 Fri: Brennen flying back from NYC
 * 09 Wed: International Day of the World's Indigenous Peoples, US staff with reqs
 * 11 Fri: Brennen out for Folks Fest (?)
 * 7-11 Mon-Fri: Dan out for family vacation
 * 27 Aug–04 Sep: Brennen (🔥)

September 2023

 * 26 Aug–05 Sep: Brennen (🔥)
 * 13 Weds–17 Sun: Brennen → KS (approximate)

🔥🚂 Train

 * https://tools.wmflabs.org/versions/
 * https://train-blockers.toolforge.org/
 * https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar


 * 2 Jan - wmf.17 - Dan + Antoine (Jaime out)
 * 9 Jan - wmf.18 - Jeena + Dan (Jaime out)
 * 16 Jan - wmf.19 - Jaime + Jeena
 * 23 Jan - wmf.20 - Brennen + Jaime
 * 30 Jan - wmf.21 - Ahmon + Brennen
 * 6 Feb - wmf.22 - Chad + Ahmon
 * 13 Feb - wmf.23 – Dan + Chad
 * 20 Feb - wmf.24 – Antoine + Dan
 * 27 Feb - wmf.25 – Jaime + Antoine
 * 6 Mar – wmf.26 – Jeena + Jaime
 * 13 Mar – wmf.27 – Brennen + Jeena
 * 20 Mar – wmf.1 – Ahmon + Brennen
 * 27 Mar – wmf.2 – Chad Dan + Ahmon
 * 3 Apr – wmf.3 – Antoine + Dan
 * 10 Apr – wmf.4 – Chad + Antoine
 * 17 Apr – wmf.5 – Jaime + Chad
 * 24 Apr – wmf.6 – Jeena + Jaime
 * 1 May – wmf.7 – Brennen + Jeena
 * 8 May – wmf.8 – Antoine + Brennen (Ahmon out + Antoine Out 8th)
 * 15 May – wmf.9 – Ahmon + Antoine (Dan out + Chad out)
 * 22 May – wmf.10 – Chad + Ahmon (Dan out + Jeena out 26th)
 * 29 May – wmf.11 – Dan + Chad (Memorial Day 29th)
 * 5 Jun – wmf.12 – Jeena + Dan (Brennen out, Jaime out)
 * 12 Jun – wmf.13 – Jaime + Jeena
 * 19 Jun – wmf.15 – Cancelled for offsite
 * 26 Jun – wmf.16 – Brennen + Jaime (Jeena out)
 * 3 Jul – wmf.17 – Antoine + Brennen (3rd + 4th holidays)
 * 10 Jul – wmf.18 – Dan + Antoine (Ahmon out)


 * 17 Jul – wmf.19 – Ahmon+Dan (Brennen out Friday)
 * 24 Jul – wmf.20 – Jaime+Ahmon

Team discussions
Contint Decomm
 * Objections or other thoughts on the plan? https://phabricator.wikimedia.org/T342017

Move Phab to Phorge
 * Migrate ganetti VM to phorge, send around login information
 * Build a testing plan—happy path of workflows
 * Investigate aphlict upgrade

Phab stuff
Objective: prepare for bullseye upgrade
 * 7.4 incompatibilities
 * https://phabricator.wikimedia.org/T342007
 * Get our repo in-order vs upstream (stack patches better)
 * Stretch: pair on deploys
 * Upgrade dev-tools instance?
 * Via Jaime 1-on-1: develop environment

PipelineLib → GitLab
Objective: Migrate 1 SRE service thing
 * GitLab running out of diskspace https://phabricator.wikimedia.org/T340887
 * storage the for the registry—we have a device we can use for storage space
 * Bigger instances
 * Restarting BuildKitd on a timer
 * Figure out how to reduce the number of jobs running on WMCS runners
 * 1031 is instance-wide
 * Common patterns may not work here since there's no registry
 * WMCS vs K8s runners
 * There are pros and cons
 * Proposal: Disable WMCS runners for a week and track the impact
 * Collect data about job failures/job run time
 * General schema—sending events *somewhere* and calculating events later
 * Example: https://people.wikimedia.org/~dduvall/jenkins/2019-06-07/30-day/durations-by-builton.html
 * Let's use our prometheus/grafana...somehow :)


 * Stacked patchsets: https://phabricator.wikimedia.org/T300819
 * Push rule for GitLab: https://phabricator.wikimedia.org/T341638
 * Investigate whether push rules are just .git/hooks with a UI
 * Rendering for T\d+: https://phabricator.wikimedia.org/T337570 +1
 * Running some javascript on MR pages...somehow
 * Writing an integration—we know where the ruby code lives
 * Others?
 * Meeting tomorrow will likely yield more tasks