Wikimedia Release Engineering Team/Checkin archive/2023-07-19
2023-07-19[edit]
🏆 Wins & Gratitude[edit]
- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
- Jul '23 edition
- Bunch of GitLab accounts created ~200 accounts
- Tooling for recurring account syncing
- Default phabricator.wikimedia.org frontpage has a "I need access or permissions to..." tab panel - https://phabricator.wikimedia.org/W2984
- Building out a deployment pipeline
- Good reviews and talented engineers on the team
- Desiloed
- Did math!
- JohnBond upgrading to puppet 7
- Contint servers upgraded to in-warranty hardware! (Dzahn, John Bond, Jelto)—ssds! zuul-merger should be way faster
- git::clone puppet resource updated! LDAP group sync stuff
- Great team + smart people + good reviews
- ^ +1 + Daniel Zahn, Eoghan, Ahmon's review
- Meeting yesterday with Dzahn, grateful for him reaching out, having a plan for phab migration stuff
- Mention of doing a git blame on stacktraces. Experiment in scap was https://gerrit.wikimedia.org/r/c/mediawiki/tools/scap/+/773870
- Phatality digging
- BuildKitd allowlist is deployed to production
- Phab backlog cleaning
- Boldly deleting wikipages!
- Andre has phab-root, gerrit-root, contint-root
- Toronto is beautiful :)
- List of supergroupers (DX)
🌻 Open source/Upstream contributions[edit]
- Antoine reviewed/fixed a stack of patchset for python-jenkins https://opendev.org/jjb/python-jenkins (used by JJB) + granted release rights to one of the participants
😶 Let's keep these empty[edit]
Code review[edit]
- +1'd gerrit changes
- TODO: gitlab equivalent
Gerrit Access requests[edit]
GitLab Access requests[edit]
📅 Vacations/Important dates[edit]
- https://office.wikimedia.org/wiki/HR_Corner/Holiday_List#2023
- https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
- https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off
July 2023[edit]
- 03-04 Mon: U.S. July Holiday, US staff with reqs
- 05 Wed & 06 Thu: Andre CZ public holidays
- 06 Wed–11 Tue: Dancy out (wedding, not mine)
- 14 Fri Antoine (Bastille Day, France national holiday)
- 18 Tue - 21 Fri Jaime
- 23 Fri Jun–18 Fri Aug: Jeena → Mongolia :D :D
- 25Tue-28Fri Andre less available (conference)
- 27 Thr - 28 Fri Dan
- 28 Fri: Brennen probably out for RockyGrass 🪕
- 31 Mon Jul – 21 Mon Aug Antoine
August 2023[edit]
- 31 Mon Jul – 21 Mon Aug – Antoine
- 23 Fri Jun–18 Fri Aug: Jeena → Mongolia :D :D :D
- 07 Fri: Brennen flying back from NYC
- 09 Wed: International Day of the World's Indigenous Peoples, US staff with reqs
- 11 Fri: Brennen out for Folks Fest (?)
- 7-11 Mon-Fri: Dan out for family vacation
- 27 Aug–04 Sep: Brennen (🔥)
September 2023[edit]
- 26 Aug–05 Sep: Brennen (🔥)
- 13 Weds–17 Sun: Brennen → KS (approximate)
October 2023[edit]
🔥🚂 Train[edit]
- https://tools.wmflabs.org/versions/
- https://train-blockers.toolforge.org/
- https://wikitech.wikimedia.org/wiki/Deployments/Yearly_calendar
- 2 Jan - wmf.17 - Dan + Antoine (Jaime out)
- 9 Jan - wmf.18 - Jeena + Dan (Jaime out)
- 16 Jan - wmf.19 - Jaime + Jeena
- 23 Jan - wmf.20 - Brennen + Jaime
- 30 Jan - wmf.21 - Ahmon + Brennen
- 6 Feb - wmf.22 - Chad + Ahmon
- 13 Feb - wmf.23 – Dan + Chad
- 20 Feb - wmf.24 – Antoine + Dan
- 27 Feb - wmf.25 – Jaime + Antoine
- 6 Mar – wmf.26 – Jeena + Jaime
- 13 Mar – wmf.27 – Brennen + Jeena
- 20 Mar – wmf.1 – Ahmon + Brennen
- 27 Mar – wmf.2 – Chad Dan + Ahmon
- 3 Apr – wmf.3 – Antoine + Dan
- 10 Apr – wmf.4 – Chad + Antoine
- 17 Apr – wmf.5 – Jaime + Chad
- 24 Apr – wmf.6 – Jeena + Jaime
- 1 May – wmf.7 – Brennen + Jeena
- 8 May – wmf.8 – Antoine + Brennen (Ahmon out + Antoine Out 8th)
- 15 May – wmf.9 – Ahmon + Antoine (Dan out + Chad out)
- 22 May – wmf.10 – Chad + Ahmon (Dan out + Jeena out 26th)
- 29 May – wmf.11 – Dan + Chad (Memorial Day 29th)
- 5 Jun – wmf.12 – Jeena + Dan (Brennen out, Jaime out)
- 12 Jun – wmf.13 – Jaime + Jeena
- 19 Jun – wmf.15 – Cancelled for offsite
- 26 Jun – wmf.16 – Brennen + Jaime (Jeena out)
- 3 Jul – wmf.17 – Antoine + Brennen (3rd + 4th holidays)
- 10 Jul – wmf.18 – Dan + Antoine (Ahmon out)
- 17 Jul – wmf.19 – Ahmon+Dan (Brennen out Friday)
- 24 Jul – wmf.20 – Jaime+Ahmon
Team discussions[edit]
Contint Decomm
- Objections or other thoughts on the plan? https://phabricator.wikimedia.org/T342017
Move Phab to Phorge
- Migrate ganetti VM to phorge, send around login information
- Build a testing plan—happy path of workflows
- Investigate aphlict upgrade
Next sprint![edit]
Phab stuff[edit]
Objective: prepare for bullseye upgrade
- 7.4 incompatibilities
- Get our repo in-order vs upstream (stack patches better)
- Stretch: pair on deploys
- Upgrade dev-tools instance?
- Via Jaime 1-on-1: develop environment
PipelineLib → GitLab[edit]
Objective: Migrate 1 SRE service thing
- GitLab running out of diskspace https://phabricator.wikimedia.org/T340887
- storage the for the registry—we have a device we can use for storage space
- Bigger instances
- Restarting BuildKitd on a timer
- Figure out how to reduce the number of jobs running on WMCS runners
- 1031 is instance-wide
- Common patterns may not work here since there's no registry
- WMCS vs K8s runners
- There are pros and cons
- Proposal: Disable WMCS runners for a week and track the impact
- Collect data about job failures/job run time
- General schema—sending events *somewhere* and calculating events later
- Example: https://people.wikimedia.org/~dduvall/jenkins/2019-06-07/30-day/durations-by-builton.html
- Let's use our prometheus/grafana...somehow :)
- Stacked patchsets: https://phabricator.wikimedia.org/T300819
- Push rule for GitLab: https://phabricator.wikimedia.org/T341638
- Investigate whether push rules are just .git/hooks with a UI
- Rendering for T\d+: https://phabricator.wikimedia.org/T337570 +1
- Running some javascript on MR pages...somehow
- Writing an integration—we know where the ruby code lives
- Others?
- Meeting tomorrow will likely yield more tasks