Wikimedia Release Engineering Team/Checkin archive/20171211

= 2017-12-11 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * December Fridays - Greg is taking off
 * December 20-22: Dan - holiday
 * December 25 (Monday): Željko - holiday (Christmas Day)
 * December 26 (Tuesday): Željko - holiday (St Stephen's Day)
 * Dec 25-Jan 1 - End of year/new year holidays
 * January 1 (Monday): Željko - holiday (New Year's Day)

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R

Dec 4 and Dec 11

 * Train: Chad
 * wmf.11
 * wmf.12
 * SoS: Mukunda
 * Out
 * Dec 4th (Monday): Antoine

Dec 18 and Dec 25

 * Train: NO ONE!
 * SoS: Chad
 * Out:
 * Dec 25-Jan 1 - End of year/new year holidays
 * December 25 (Monday): Željko - holiday (Christmas Day)
 * December 26 (Tuesday): Željko - holiday (St Stephen's Day)
 * Antoine - not sure yet of the exact dates

Jan 1 and Jan 8

 * Train: Mukunda
 * wmf.15
 * wmf.16
 * SoS: Tyler
 * Out:
 * January 1 (Monday): Holiday, everyone

Release Engineering

 * Blocking
 * Blocked
 * MW Platform team: https://phabricator.wikimedia.org/T182344 - "Blocker must be a local user or a name that cannot be a local user"
 * Updates
 * [MW Train] Reminder! This is your last week of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th.
 * [Meta] Q3 goal planning
 * [Security/Ruby] T180878 Upgrade RuboCop and Rubyzip (Ruby)
 * All done except for Minerva (needs a +2) and mediawiki/debian (unsure if needed).
 * [Security/Jenkins] Upgraded a bunch of Jenkins plugins Monday morning EU time (after a bunch of security releases).
 * [Phabricator] Exploring the use of Selenium tests for search quality regressions
 * [TechDebt] Getting further on the definition of “steward”. We will be talking with Victoria and Toby “soon”.
 * [SSD] One more merge (https://gerrit.wikimedia.org/r/#/c/395570/) and our quarterly goals are complete.
 * [PostMortem] ORES post-mortem completed last week: https://etherpad.wikimedia.org/p/Post-Mortem-T181006
 * [PostMortem] ORES post-mortem completed last week: https://etherpad.wikimedia.org/p/Post-Mortem-T181006

Last week

 * Blocking
 * Blocked
 * Updates
 * [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. [wiki][email]
 * [MW Train] The post mortem for T181006 (Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki) is scheduled for December 7th.
 * #mw-1.30-release MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten
 * [MW Train] The post mortem for T181006 (Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki) is scheduled for December 7th.
 * #mw-1.30-release MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten

Puppet SWAT

 * list of patches you want to submit to Puppet SWAT

Logspam \ Last week's train updates

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor


 * https://phabricator.wikimedia.org/T182344

Q3 goals! :) :)

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q2
 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q3

Program 1: webdriver.io

 * https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Technology#Program_1:_Availability.2C_performance.2C_and_maintenance


 * T182421 Selenium framework improvements
 * Things to do in the next quarter.

Program 3: Tech Debt

 * https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Technology#Program_3:_Addressing_technical_debt

Outcome 1
 * Objective 2: Identify and find stewards for high-priority/high use code segment orphans - task T174091
 * Objective 3: Define and steward a light-weight process for adopting or orphaning/sunsetting products and infrastructure.
 * This is done by the Sunsetting working group

Outcome 2
 * Objective 2: Define and implement a process to regularly address technical debt across the Foundation
 * Objective 3: Promote and surface important technical debt topics at large gatherings of Wikimedia developers (e.g., DevSummit and Hackathon(s))

Program 6: SSD

 * https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Technology#Program_6._Streamlined_service_delivery


 * The verify stage of the pipeline
 * https://www.mediawiki.org/wiki/File:Containerized_continuous_delivery_2017_concept.png
 * https://wikitech.wikimedia.org/wiki/Streamlined_Service_Delivery_Design#Step_4:_Isolated_deploy_and_e2e_tests
 * When to make new builds
 * Pushing into a k8s cluster (not staging)
 * In the interim do something with GCE for basis end-to-end tests


 * Pre-Dependency on Ops:
 * CI k8s cluster for the Verify pipeline stage

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)
=
 * workboard

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure
=
 * Goal: A generalized POC for a docker-based CI.
 * https://phabricator.wikimedia.org/project/view/3008/ (shipyard workboard)


 * Draft requirements for a CI k8s cluster in preparation for Ops to create it in the following quarter(ish)
 * Nodepool deprecation: npm and php jobs, but there will be a long tail of corner case jobs

Program 1: Outcome 5: Objective 1: MW Nightlies server

 * chad point on this, bulk of time


 * Sam Reed and Chad will work on setting it up this week

Q2 goal/project check-in

 * All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q2

Program 1: Outcome 5: Milestone 1: Migrate majority of developers to JavaScript based browser test framework (webdriver.io)

 * Due: End of this quarter
 * Quarter Goal Task: Port Selenium tests from Ruby to Node.js -


 * T139740 Port Selenium tests from Ruby to Node.js
 * Progress. Flow deleted tests. :|


 * T180777 Move portal deployment artifacts into their own repo
 * wikimedia/portals/deploy has Selenium tests!


 * T182160 Develop tests for phabricator search to detect regressions / search quality issues
 * Selenium tests for Phabricator search

Ruby

 * T180878 Upgrade RuboCop and Rubyzip (Ruby)
 * Waiting for +2 for MinervaNeue https://gerrit.wikimedia.org/r/#/c/395514/
 * T182401 not sure how to update mediawiki/debian

Program 3: Outcome 1: Objective 1: Define a set of code stewardship levels (from high to low expectations)

 * Due: End of this quarter
 * Quatertly Goal task: -


 * Had additional discussions regarding the definition of stewards vs owners.
 * Final updates will be complete this week.
 * Worked on discussion points for meeting with Toby and Victoria
 * Path to completion is review with Toby and Victoria and then broader discussion with engineering managers and leagues.

Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans

 * Due: End of next quarter
 * Quaterly Goal task -


 * no progress this past week.

Program 3: Outcome 2: Objective 1: Define a “Technical Debt Project Manager” role that regularly communicates with all Foundation engineering teams regarding their technical debt

 * Due: End of this quarter


 * done

Program 3: Outcome 2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation

 * Due: End of next quarter


 * behind on the blog post completion. Finishing up with week.

==== Program 6: Outcome 2: Objective 2: Set up a continuous integration and deployment pipeline to publish new versions of an application to production via testing and staging environments that reliably reproduce production ====
 * Due: End of this quarter
 * Keyword: SSD
 * Complete build phase of release pipeline


 * Build test variant
 * Run test entrypoint w/developer feedback - services dependency
 * Build production variant w/developer feedback - services dependency
 * Tag production container
 * Push to production docker registry - ops dependency - staging namespace
 * Tracking: https://phabricator.wikimedia.org/T157469
 * current status: https://phabricator.wikimedia.org/project/view/2453/


 * mathoid blubber config merged! https://gerrit.wikimedia.org/r/#/c/395050/
 * now ready to merge jjb change to make service-pipeline primary test for mathoid and push to prod on post-merge! (quarterly goal complete) https://gerrit.wikimedia.org/r/#/c/395570/

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)

 * workboard

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure

 * Goal: A generalized POC for a docker-based CI.
 * https://phabricator.wikimedia.org/project/view/3008/ (shipyard workboard)


 * Upgraded a bunch of Jenkins plugins (security updates)

Other work

 * Post Mortem - T181006 - Watchlist and RecentChanges failure due to ORES failures
 * had good discussion about ORES failures
 * https://etherpad.wikimedia.org/p/Post-Mortem-T181006

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng


 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart