Wikimedia Release Engineering Team/Checkin archive/20191023

= 2019-10-23 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * October 8: Željko holiday (Independence Day)
 * October 11: Željko conference (https://2019.webcampzg.org/ )
 * October 14 (Indigenous Peoples' Day): US Staff


 * October 24–27: James at WikidataCon 2019 in Berlin


 * November 1 (All Saints' Day): Željko holiday
 * November 8–10: WikiCon North America (James)
 * November 11 (Veterans' Day): US Staff
 * November 12–15: TechConf19
 * November 16–20: Offsite
 * November 21-30: Jeena
 * November 25-29: Dan
 * November 28–29 (Thanksgiving): US Staff


 * December 1-2: Jeena
 * December 6: Lars (Finnish Independence Day)
 * December 25–31 (Christmas): US Staff
 * December 25–26: Lars, Željko, Christmas


 * 1 January (New Year's Day): US Staff, Lars, Željko
 * 20 January (Martin Luther King Jr. Day): US Staff

Train

 * Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/query/s3KW8bpsXhYF/#R


 * 14 Oct - wmf.2 - Jeena + Lars


 * 21 Oct - wmf.3 - Lars + Brennen
 * 28 Oct - wmf.4 - Brennen + Mukunda
 * 4 Nov - wmf.5 - Mukunda + Antoine
 * 11 Nov - wmf.6 - No Train (Tech conf)
 * 18 Nov - wmf.7 - No Train (Off site)
 * 25 Nov - wmf.8 - Antoine + Jeena (Thanksgiving)
 * 2 Dec - wmf.9 - TBD
 * 9 Dec - wmf.10 - TBD
 * 16 Dec - wmf.11 - TBD
 * 23 Dec - wmf.12 - No Train or any other deploys (Winterval)
 * 30 Dec - wmf.13 - No Train or any other deploys (Still Winterval)
 * 6 Jan - wmf.14 - TBD
 * 13 Jan - wmf.15 - TBD

SoS

 * Aug 14 onward: Zeljko 🎸 🎷 \o/

Timespent spreadsheet

 * For the avoidance of doubt: fill out the sheet week number for the previous week


 * spreadsheet for week 42 (2019-10-14):https://docs.google.com/spreadsheets/d/1YS_2eUQWUOnygshMfxgGPu93GKGCO5t00yjUREQ5B-I/edit#gid=442607278
 * reminder: your time spent doesn't need to total to 100%

Clinic Duty Proposal

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Clinic_Duty
 * thcipriani JFDI

November Offsite/Summit

 * TechConf19: November 12–15
 * Offsite: November 16–20
 * Arrive the 15th
 * 16th is our fun day/outing
 * 17–20 are working days
 * 21st fly home
 * https://etherpad.wikimedia.org/p/EngProdTeam-Offsite-2019-11-planning
 * Dietary restrictions due TODAY!
 * New proposal on etherpad: how does EngProd want to use betterworks
 * Outing day! Vote please

Book club

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Book_club
 * Good discussion! https://etherpad.wikimedia.org/p/releng-book-club

Monthly reflection on accomplishments - October '19 edition

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Monthly_notable_accomplishments
 * Add as you have them!

October

 * Dev images are now automatically created as part of postmerge via the pipeline for:
 * Parsoid
 * Soon: RestBASE
 * Selenium documentation updated https://www.mediawiki.org/wiki/Selenium/Node.js
 * Quibble 0.0.36 released https://lists.wikimedia.org/pipermail/wikitech-l/2019-October/092658.html
 * Quibble 0.0.37 released https://lists.wikimedia.org/pipermail/wikitech-l/2019-October/092660.html
 * Quibble 0.0.38 & 0.0.39 released for mediawiki/tools/api-testing
 * Introducing Phatality - Streamlined error reporting from Kibana to Phabricator https://phabricator.wikimedia.org/phame/post/view/177/introducing_phatality/
 * HHVM removed from CI and MediaWiki.
 * Gerrit is on gerrit1001 now
 * … and so is most of the code review. ;-)

Outgoing next week

 * Blocked:
 * Site Reliability Engineering: logstash-beta.wmflabs.org does not receive any mediawiki events https://phabricator.wikimedia.org/T233134
 * Updates:
 * v3 of CI arch document for new CI wants review and feedback https://www.mediawiki.org/wiki/User:LarsWirzenius/NewCI
 * Gerrit is now on a newer (larger) server
 * There was some problem with rollout (patches looking unmerged that were actually merged)
 * Resolved now
 * All gory details on https://phabricator.wikimedia.org/T236114
 * Train Health
 * Last week: 1.35.0-wmf.2 - T233850
 * This week: 1.35.0-wmf.3 - T233851
 * Next week: 1.35.0-wmf.4 - T233852

Train status and happenings

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Standup!

 * Make sure your tasks are correct on our kanban board!
 * https://phabricator.wikimedia.org/project/view/4311/ - 2019-10

Last week

 * Best thing
 * Paired git fixage with Tyler for hmm lot of hours. Pair programming bash over Google Meet is a great experience.
 * For Wikibase.git : extracted Selenium to a standalone jobs ( https://phabricator.wikimedia.org/T232759 )
 * Quibble 0.0.39 for mediawiki/tools/api-testing (very cool project)
 * Worst thing
 * What I planned vs did
 * Switch integration/config to use jenkins-job-builder from pypi instead of our git repo
 * PostBuildScript jenkins plugin upgrade prepared
 * Nerd snipped on some weird logstash index fields limit https://phabricator.wikimedia.org/T234564
 * Nerd snipped on some weird logstash index fields limit https://phabricator.wikimedia.org/T234564


 * Unplannned
 * Gerrit post migration magic (thank you Tyler!) (thcipriani: thank you!)

This week

 * What I plan to do
 * TechConf preparation with Kosta / WMDE-Fisch


 * Blockers
 * WMCS blocked: they need CI for a golang projects https://phabricator.wikimedia.org/T236203 (dduvall: we have a docker-pkg-built golang base image; see blubber's .pipeline/blubber.yaml)
 * php-xdebug needs upgrade (slows down phpunit coverage report)

Last week

 * Best thing
 * Pairing in general, messing with sshecret.
 * Worst thing
 * Moments of panic about logs prior to deploy.
 * What I planned vs did
 * ✅ Make sure local-charts install script works on buster
 * ✅ Further TechConf input, CI session planning
 * Train pairing
 * Unplannned
 * Some time investigating broken mediawiki/core dev images
 * Some time investigating toollabs to restart dockerregistry

This week

 * What I plan to do
 * Train
 * Sort out all hands travel
 * Dev environment hackery
 * Check out k3sup?
 * Think about TechConf local dev sessions / discuss
 * Improve sshecret pull req: https://github.com/thcipriani/sshecret/pull/2
 * Blockers

Last week

 * Best thing
 * Diagramming Seakeeper subsystems
 * Worst thing
 * Fixing janky Jenkins reports
 * starting in August, 30-day reports should be accurate
 * What I planned vs did
 * Seakeeper proposal, fleshed out descriptions of CI subsystems for proposed k8s namespaces
 * Unplannned
 * Fixed problems with Jenkins reports at https://people.wikimedia.org/~dduvall/jenkins
 * Build retention differences made for inaccurate 30-day reports. Working around this by aggregating from past build archives.

This week

 * What I plan to do
 * Seakeeper proposal, gathering capacity/resource utilization stats
 * Blockers

Last week

 * Best thing
 * The TechConf program committee is great to work with, even when we are stressed
 * Worst thing
 * Time constraint on the Tuning Session slide creation (which had knock off effects of figuring out what the hell these OKRs and metrics really are)
 * Effing moving
 * What I planned vs did
 * techconf
 * tuning session
 * Unplannned

This week

 * What I plan to do
 * tech conf schedule creation (well, version 2.0 of the schedule) and post it
 * engprod logistics setup
 * Blockers
 * time

Other

 * TechConf office hours happening next week on IRC, email going out shortly announcing them
 * Desire/stretch: create a homepage for Engineering Producitivity somewhere :)

Last week

 * Best thing
 * Made progress discussing with Timo way forward for static configuration work; next step is autogenerating dblists from YAML.
 * Worst thing
 * Real jetlag for the first time in a few years. Meh.
 * What I planned vs did
 * ✅ OOUI migration to node 10 / PHP72
 * ❌ Fundraising Tech CRM image to PHP70 (could do with input from Antoine) https://gerrit.wikimedia.org/r/c/integration/config/+/540683 for https://phabricator.wikimedia.org/T230446
 * More static configuration work, a bit delayed.
 * ✅ More TechConf thinking/planning.
 * ✅ UBNs as they arise.
 * Unplannned
 * Renamed "unversioned" PHP (PHP70) images to be explicit
 * Antoine: THANK YOU
 * Helping Analytics put a "stats for this site" link in the site footer T235803

This week

 * What I plan to do
 * WikiDataCon from tomorrow onwards; back in the US next Wednesday.
 * Whatever breaks.
 * YAML/static builds of dblists.
 * Blockers

Last week

 * Best thing
 * Worst thing
 * What I planned vs did
 * Unplannned
 * What I planned vs did
 * Unplannned
 * Unplannned

This week

 * What I plan to do
 * Blockers
 * Blockers

Last week

 * Best thing
 * finished restrouter chart...hopefully \o/ \O/ \:D/
 * Worst thing
 * spent time trying to debug Visual Editor not working, unexpectedly there was an error in the browser console
 * What I planned vs did
 * ❌ timespent spreadsheet
 * ❌ work with zeljko on getting local-charts running - Željko: feel free to pospond until you're done with train
 * look at seakeeper proposal
 * ✅ train
 * ✅ update restrouter deployment-charts config for dev purposes
 * Unplannned
 * debug broken mw core image

This week

 * What I plan to do
 * tech conf session planning
 * changes to mw core image publish pipeline to log more and fail if prereq script errors
 * re-open patchsets to build restbase dev image
 * timespent spreadsheet
 * work with zeljko on getting local-charts running
 * Blockers
 * Blockers

Other

 * tiny sunflower

Last week

 * Best thing
 * Going to TechConf
 * Worst thing
 * Not going to a mountain in TN
 * What I planned vs did
 * ✅ Help Jeena with train, a little, as backup.
 * ✅ Seek feedback on v3 of CI arch doc.
 * ✅ Update engprod on PGP signing for offsite.
 * ❌ Set up Argo in a K8s cluster somewhere.
 * Needs uninterrupted time, and I've been easily distractable. Also train.
 * Unplannned
 * Was invited to TechConf, and did much arranging of travel.
 * More review of Dan's Seekeeper proposal.
 * Wrote up book club discussion on wiki.
 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Book_club/Software_Testing_Anti-Patterns
 * Failed to deploy train, didn't get to group0 even, but much excitement was had by everyone.
 * Wrote a wrapper around rsync, in case it's useful some day.
 * Registered for All Hands, started arranging travel.

This week

 * What I plan to do
 * Train, 1.35.0-wmf.3
 * Skim Go book, write a small program from scratch: fetch a web page, diff against previously fetched version.
 * Attempt to set up study groups, or possibly SIGs, for Go and Rust.
 * Blockers

Other

 * The Darkness is here until about March or April, expect SADness
 * performance has TT; releng has BB, DD, and MM; qte has no doubled initials
 * EngProd has GG :P
 * My "legal" name is Tyler Henry Turley-Cipriani, so TTish
 * My "legal" name is Tyler Henry Turley-Cipriani, so TTish

Last week

 * Best thing
 * Phab1001 is up and running
 * Worst thing
 * Didn't get around to testing the "Ferret" Fulltext search engine in phab (yet)
 * What I planned vs did
 * Mostly ✅ Finish getting branch.py ready for the wmf production branching (WIP: https://gerrit.wikimedia.org/r/#/c/mediawiki/tools/release/+/543248/ )
 * ✅ Try to build some tests for branch.py
 * ✅ More work on phabricator high-availability & disaster recovery planning.
 * phab1001 should be nearly ready to become the primary server now.
 * ❌ Test out phabricator's built-in search engine with an eye toward dropping elasticsearch if the integrated search performs as well.
 * Unplannned
 * ✅ https://phabricator.wikimedia.org/T235945

This week

 * What I plan to do
 * Test out phabricator's built-in search engine
 * Drum up interest in patch-triage (email wikitech-l, not sure what else to do)
 * Get branch.py ready for full automation in releases-jenkins
 * Deploy phatality (after https://gerrit.wikimedia.org/r/#/c/545593/ merges)
 * Deploy phabricator
 * Blockers
 * code review of https://gerrit.wikimedia.org/r/#/c/545593/

Last week

 * Best thing
 * Seeing the train pairing thing work :)
 * pairing with Antoine on Gerrit migration fallout
 * Worst thing
 * Gerrit migration fallout
 * What I planned vs did
 * ✅ connect with SRE re New CI (again)
 * ✅ Reach out to folks about being on Sr. SWE interview loop
 * ✅ announce dec code freeze
 * ✅? tuning session prep
 * ✅ Gerrit -> gerrit1001
 * Unplannned
 * Gerrit migration fallout
 * Updates to deployment-calendar scheduler -- awight noticed some issues

This week

 * What I plan to do
 * Resumé review
 * Work out clinic duty rotation
 * Next bookclub email
 * Gerrit/JVM tuning to make use of Moar Memory™
 * Blockers
 * Time
 * Dog
 * Blazey at conference

Other

 * Picking up Blazey at airport this evening, may cut out a bit early

Last week

 * Best thing
 * Email from Safeguard: "Please note: all documentation must be password protected from this point using the following (password)"
 * Worst thing
 * T236282 `npm run selenium` fails with `SyntaxError: Unexpected identifier`
 * What I planned vs did
 * T234610 ERROR webdriver: Request failed due to Error: session not created: Chrome version must be between 70 and 73
 * T199113 All repositories with Selenium tests should use wdio-mediawiki
 * Unplannned
 * Safeguard
 * T235795 Create test for GrowthExperiments to check for ability to disable homepage
 * T233820 mediawiki/core Selenium tests fail when targeting mediawiki/vagrant VM
 * T236282 `npm run selenium` fails with `SyntaxError: Unexpected identifier`

This week

 * What I plan to do
 * T234610 ERROR webdriver: Request failed due to Error: session not created: Chrome version must be between 70 and 73
 * T199113 All repositories with Selenium tests should use wdio-mediawiki
 * Blockers

Other

 * Saw Rambo: Last Blood. Not recommended.