Wikimedia Release Engineering Team/Checkin archive/20181029

= 2018-10-29 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * November 1 (Thursday) - Holiday (All Saints' Day - Željko)
 * November 1st / 2nd - Holiday (Antoine)
 * November 12th - Holiday (Veteran's Day, Observed)
 * November 22+23 - Holidays (Thanksgiving)
 * November 25-december 2nd: Mukunda vacation (in California ahead of the offsite)
 * Week of December 3rd - Team offsite
 * December 24-28 - Holidays (Christmas)

Train

 * Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R


 * Oct 08 - wmf.25 - Dan (No train due to DC switchover)
 * Oct 15 - wmf.26 - Mukunda (last 1.32 wmf.XX release, 1.33 starts the next week)
 * Oct 22 - wmf.1 - Mukunda (warning, TechConf happening, ping Greg if you need responses from anyone there...)
 * Oct 29 - wmf.2 - Tyler <
 * Nov 05 - wmf.3 - Tyler
 * Nov 12 - wmf.4 - Antoine
 * Nov 19 - wmf.5 - No Train (Thanksgiving)
 * Nov 26 - wmf.6 - Antoine
 * Dec 03 - wmf.7 - No Train (Offsite)
 * Dec 10 - wmf.8 - Zeljko
 * Dec 17 - wmf.9 - Zeljko
 * Dec 24 - wmf.10 - No Train (Holiday break)
 * Dec 31 - wmf.11 - No Train (Holiday break)
 * Jan 07 - wmf.12 - Dan
 * Jan 14 - wmf.13 - Dan
 * Jan 21 - wmf.14 - Mukunda
 * Jan 28 - wmf.15 - No Train (All Hands)
 * Feb 04 - wmf.16 - Mukunda
 * Feb 11 - wmf.17 - Tyler
 * Feb 18 - wmf.18 - Tyler
 * Feb 25 - wmf.19 - Antoine

SoS

 * Oct 10 - Zeljko
 * Oct 17 - Zeljko
 * Oct 24 - Zeljko
 * Oct 31 - Zeljko <
 * Nov 07 - Zeljko
 * Nov 14 - Zeljko
 * Nov 21 - Zeljko
 * Nov 28 - Zeljko
 * Dec 05 - Zeljko
 * Dec 12 - Zeljko
 * Dec 19 - Zeljko
 * Dec 26 - Zeljko
 * Jan 02 - Zeljko
 * Jan 09 - Zeljko
 * Jan 16 - Zeljko
 * Jan 23 - Zeljko
 * Jan 30 - Zeljko
 * Feb 06 - Zeljko
 * Feb 13 - Zeljko
 * Feb 20 - Zeljko
 * Feb 27 - Zeljko

Hiring

 * Software Engineer position open and reviewing/hiring for now
 * https://boards.greenhouse.io/wikimedia/jobs/1225258


 * update....

First Offsite
Details:
 * Week of December 3rd
 * At the Queen Mary hotel in Long Beach
 * Deb T will be facilitating

Topics!
 * https://etherpad.wikimedia.org/p/RelEng-Offsite-201811-Topics

Needs attention

 * gerrit security release 2018-10-08
 * https://groups.google.com/forum/m/#!topic/repo-discuss/eH0iLt2XawU
 * jGit update, we are unaffected
 * may want to hold off until next week: https://bugs.chromium.org/p/gerrit/issues/detail?id=9836
 * 2018-10-15 -- paladox tells me they're working on a fix and should have a 2.15.6 tagged Soon™
 * 2018-10-22 -- jGit updated to fix leaks https://gerrit-review.googlesource.com/c/gerrit/+/201273
 * 2018-10-29 -- 2.15.6 released: https://groups.google.com/forum/?hl=en#!topic/repo-discuss/9EUYI2eyIZM
 * thcipriani: Will send email today to update on...Wednesday? Anyone wanna work on this with me?
 * Antoine to pair, and be point next time


 * deploy1001:/srv/mediawiki out of date?
 * https://phabricator.wikimedia.org/T207602
 * Found because the Security team noticed that a previously deployed security patch was no longer deployed, should sync up with them this week about that (Reedy or Brian)
 * See: https://phabricator.wikimedia.org/T207600
 * 2018-10-22: no idea, thcipriani will look, I guess
 * 2018-10-29: scap updated, needs release this week


 * Jenkins: Security advisory pre-announcement 2018-10-29
 * https://phabricator.wikimedia.org/T207828
 * The two plugins have been upgraded


 * deployment-prep region migration
 * See email with same subject on releng@lists
 * Question: incrementally or not?
 * looks like "however Andrew wants to do it"
 * REMINDER: send an email update to wikitech-l@/qa@ with the planned timeline/outage
 * Tyler to reply saying "take it away, andrew, and when are you going to do it?"

Scrum of Scrums

 * Greg to copy to etherpad after meeting: https://etherpad.wikimedia.org/p/Scrum-of-Scrums

Incoming from last week

 * Blocking:
 * Fundraising Tech: CRM tests still regularly failing due to full mysql partition on integration hosts. Possible fix noted by Eileen on https://phabricator.wikimedia.org/T205950
 * ACTION: Tyler to comment on the task with his proposal

Release Engineering

 * Blocked by:
 * Blocking:
 * Updates:
 * Train Health:
 * Last week: 1.33.0-wmf.1 deployment blockers https://phabricator.wikimedia.org/T206655
 * Six blockers closed, one opened on friday after conclusion of the train
 * Group1 was delayed until Thursday and Group2 finally happened late Thursday evening.
 * Otherwise it was a mostly uneventful train. The blockers were perhaps more worrisome than destructive, no outages occurred.
 * This week: 1.33.0-wmf.2 deployment blockers https://phabricator.wikimedia.org/T206656
 * Next week:
 * Log Health:
 * Code Health:
 * Log Health:
 * Code Health:

Callouts

 * Release Engineering

Train status and happenings

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor


 * Six blockers closed, one opened on friday after conclusion of the train
 * Group1 was delayed until Thursday and Group2 finally happened late Thursday evening.
 * Otherwise it was a mostly uneventful train. The blockers were perhaps more worrisome than destructive, no outages occurred.
 * Mukunda: TODO: simplified log version of incident report


 * OMG discussion on how to do this better
 * TODO: Greg to file a task

TEC1 (Maint): Outcome 1 / Output 1.1

 * GOAL: Determine the procedure and requirements for an automated MediaWiki branch cut.
 * WHO: Mukunda, Tyler, Antoine

- Filed task for figuring out job storage for releases-jenkins

TEC3 (Pipeline): Outcome 1 / Output 1.2

 * GOAL: Formalize the collection of CI infrastructure and tooling metrics
 * WHO: Dan, Antoine

TEC3 (Pipeline): Outcome 2 / Output 2.3

 * GOAL: Develop set of metrics to assess incident reports/post mortems -
 * WHO: Greg, Zeljko

https://docs.google.com/spreadsheets/d/1AUqMgzThBHNL7DgI8C9PO_YQ1oD5CSd0iWvcVbowzdg/edit#gid=1154483822

TEC3 (Pipeline): Outcome 3 / Output 3.1

 * GOALS:
 * Adopt more services into Deployment pipeline -
 * Migrate graphoid to the Deployment pipeline
 * Deploy zotero v2 to the Deployment pipeline
 * Deploy blubberoid
 * WHO: Dan, Tyler, Lars

TEC12 (DevProd): Outcome 2 / Output 2.1

 * GOAL: The Annual Developer Productivity Survey results are synthesized and shared, creating a first year baseline.
 * WHO: Mukunda, Greg


 * A draft survey has been created in google forms: https://docs.google.com/forms/d/e/1FAIpQLSfXGpjUIO3ARqxPHOYPwI2Dw-jEg1xMeLi_HpZ_HcU-_i_Arw/viewform
 * This is, I believe, very nearly ready to send out to the mailing lists.

TEC13 (Code Health): Outcome 1 / Output 1.1

 * GOAL: Update/refresh review queue (review process for initial code deployment)
 * WHO: JR

Reviewed existing review queue process

TEC13 (Code Health): Outcome 2 / Output 2.2

 * GOAL: 5 of the 15 prioritized repositories have at least 1 end-to-end test -
 * WHO: Zeljko

TEC13 (Code Health): Outcome 2 / Output 2.3

 * GOAL: Assess Platform unit test practices and define improvement plan
 * WHO: JR, Core Platform Team

No progress

TEC13 (Code Health): Outcome 3 / Output 3.2

 * GOAL: Core Platform and Search Platform teams are using TDM PoC
 * WHO: JR, Core Platform Team

No progress

TEC13 (Code Health): Outcome 3 / Output 3.4

 * GOALs:
 * Identify key Tech Debt areas
 * Put in place Tech Debt management process for PEP
 * WHO: JR, Core Platform Team

Discussion with Editing team re Code Health and Tech Debt.

TEC13 (Code Health): Outcome 4 / Output 4.1

 * GOAL: Metrics defined and deployed for all 4 Code Health areas.
 * WHO: JR, Code Health Metrics Working Group

WG worked on spike task which is focused on getting a single metric running in our CI environment using a single tool (Sonarqube). Antoine deployed phpmetrics into CI ( https://integration.wikimedia.org/ci/job/mediawiki-core-phpmetrics-docker/3/console, https://doc.wikimedia.org/mediawiki-core/master/phpmetrics/, and https://doc.wikimedia.org/mediawiki-core/master/phpmetrics/violations.html)

Gerrit

 * Upgrade this week barring anything strange from the mailing lists
 * Need to test out upgrade on local instance

Phabricator

 * I've implemented a prototype extension that can map urls like https://phabricator.wikimedia.org/train/1.33.0-wmf.1/ to the corresponding train blockers task.
 * This will also support /train/current to map to the currently active train.
 * This can also support other task-series schemes such as /swat/2018-10-31.1/

Antoine

 * What I plan to do this week
 * Complete Wikibase tests comparison and migrate to Docker
 * Progress on mediawiki extensions dependencies doc
 * What I'm blocked on
 * No feedback from fundraising team for DonationInterface, will probably just switch it
 * Other?
 * Not there 11/1st 11/2nd (holiday + relocating)

Dan

 * What I plan to do this week
 * Continue with Jenkins/Prometheus
 * What I'm blocked on
 * Nada
 * Other?

Greg

 * What I plan to do this week
 * take the rest of today and most of tomorrow off, I lost a weekend and a half with the travel/work last week
 * catch up on l10nupdate follow-ups
 * follow-up from TechConf program committee (cleaning/sanitzing notes and posting to wiki mostly)
 * follow-up from TechConf hallway session asks
 * Phabricator meeting on Wednesday
 * What I'm blocked on
 * sickness (just your basic cold from hanging around people from around the world)
 * Other?
 * I won't make you all do 8 days straight of work without a break somewhere in there, ftr

Jean-Rene

 * What I plan to do this week
 * Continue work on Code Health Metrics
 * Continue work on Review Queue/RoO
 * What I'm blocked on
 * Other?
 * Other?

Lars

 * What I plan to do this week
 * Try to understand how Blubberoid works.
 * See if I can find a way to run it locally.
 * If I can run it locally, sketch out the beginnings of a user guide.
 * Start a sketch of a very high level architecture diagram of how the deployment pipeline works.
 * Browse the various team Kanban boards to be comfortable with them.
 * What I'm blocked on
 * Nada
 * Other?
 * Zilch

Mukunda

 * What I plan to do this week
 * Dev Productivity Survey is ready to go out this week
 * Pairing with Tyler on keyholder and scap stuff
 * Continue work on `scap swat` and swat-in-phab stuff
 * Phabricator improvements meeting wednesday
 * Train incident report
 * Work on paring down my list of open tasks
 * What I'm blocked on
 * Other?
 * Other?

Tyler

 * What I plan to do this week
 * Gerrit
 * Scap
 * Train
 * Finish keyholder
 * What I'm blocked on
 * Other?
 * Other?

Zeljko

 * What I plan to do this week
 * T199133 Find top 15 target projects that could use Selenium tests to prevent incidents - almost there(tm) I have all the data, working on a report (one big sheet to rule them all) - need help from Greg and/or JR with picking 5 repos once I have all the data
 * T207046 Code health metrics spike - need help from Antoine
 * What I'm blocked on
 * Other?
 * Other?

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng
 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart