Wikimedia Release Engineering Team/Checkin archive/20171113

= 2017-11-13 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * Nov 14th - 16th: Antoine half there (schedule madness)
 * November 17 (Friday) Željko - conference (Coderetreat)
 * Nov 17th: Mukunda vacation
 * Nov 20th - Dec 1st: Greg vacation
 * Nov 20th - 22nd: Mukunda vacation
 * Nov 21th - 22nd: Dan vacation
 * Nov 23+24 - Thanksgiving
 * Nov 23 morning: Antoine
 * Dec 4th (Monday): Antoine
 * December 25 (Monday): Željko - holiday (Christmas Day)
 * December 26 (Tuesday): Željko - holiday (St Stephen's Day)
 * Dec 25-Jan 1 - End of year/new year holidays
 * January 1 (Monday): Željko - holiday (New Year's Day)

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R

Nov 6 and Nov 13

 * Train Chad
 * wmf.7
 * wmf.8
 * SOS: Tyler
 * Out
 * Nov 10 (Fri) - Veteran's Day
 * November 17 (Friday) Željko - conference (Coderetreat)

Nov 20 and Nov 27

 * Train Chad
 * No train week of the 20th
 * [tbc wmf.10]
 * SOS: Mukunda
 * Out
 * Nov 20th - Dec 1st: Greg vacation
 * Nov 20th - 22nd: Mukunda vacation
 * Nov 23+24 - Thanksgiving

Actions from last meeting

 * ACTION: git-lfs in Gerrit

Release Engineering

 * Blocking
 * Blocked
 * Updates
 * Reminder: no MW train week of Thanksgiving
 * [TechDebt Program] Currently investigating best practices on commit review guidelines/prerequisites
 * [TechDebt Program] Defining the PM role more explicitly: https://www.mediawiki.org/wiki/Technical_Debt_Program/Program_Management
 * [Scap sprint] New scap release 3.7.2 pushed based on https://phabricator.wikimedia.org/rMSCA2561b82
 * Helping with the upcoming security release of MediaWiki.
 * [Scap sprint] New scap release 3.7.2 pushed based on https://phabricator.wikimedia.org/rMSCA2561b82
 * Helping with the upcoming security release of MediaWiki.

Release Engineering

 * Blocking
 * Blocked
 * Please port your browser tests to the nodejs framework. 7 repositories have not started yet (still in ruby, which is no longer maintained).
 * See: https://phabricator.wikimedia.org/T139740
 * Notably: Global Collaboration Team, Fundraising Tech, Wikibase, and Mulitmedia
 * Updates
 * No MW Train the week of the 20th due to Thanksgiving, SWATs will be open on Mon and Tuesday (Wednesday is “Friday” that week).
 * [TechDebt program] First pass of the service levels for component ownership was shared with the Code Health Group last week; feedback on-going.
 * [TechDebt program] The next blog post should be posted Real Soon Now™ (done on our side).
 * [SSD Program] Working on getting the mathoid tests running on submit.
 * [SSD Program] A new release of Blubber is on the horizon.
 * [scap tech debt] Working to support both pre git-2.11 and post git 2.11 hosts (namely Trusty and Jessie (with backports)/Stretch, respectively) to let us use newer functionality (notably `--jobs`).
 * [CI] Most tox jobs are moved to Docker containers
 * [CI] Investigating why many docker containers are left behind after a SIGTERM
 * [CI] Investigating why many docker containers are left behind after a SIGTERM

We got pinged on https://phabricator.wikimedia.org/T178570 / https://phabricator.wikimedia.org/T178189

Puppet SWAT

 * list of patches you want to submit to Puppet SWAT

Logspam \ Last week's train updates

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Other Team Business

 * Update on 1 pager for Victoria

Q2 goal/project check-in

 * All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q2

Program 1: Outcome 5: Milestone 1: Migrate majority of developers to JavaScript based browser test framework (webdriver.io)

 * Due: End of this quarter
 * Quarter Goal Task: Port Selenium tests from Ruby to Node.js -


 * T139740 Port Selenium tests from Ruby to Node.js
 * Updated stats (per status, per team, per repository)
 * https://phabricator.wikimedia.org/T139740


 * T164721 Run WebdriverIO tests in CI for extensions
 * Merged during the meeting

Ruby

 * T167432 Run Wikibase daily browser tests on Jenkins
 * Amir whitelisted Jenkins IPs on testwiki, it should fix test failures https://gerrit.wikimedia.org/r/#/c/390975/
 * JJB configuration done, waiting for review/merge https://gerrit.wikimedia.org/r/#/c/390394/
 * job deployed https://integration.wikimedia.org/ci/view/Selenium/job/selenium-Wikibase-chrome/

Program 3: Outcome 1: Objective 1: Define a set of code stewardship levels (from high to low expectations)

 * Due: End of this quarter
 * Quatertly Goal task: -


 * Investigated what we currently do regarding commit/review prerequisites.

Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans

 * Due: End of next quarter
 * Quaterly Goal task -


 * no activity

Program 3: Outcome 2: Objective 1: Define a “Technical Debt Project Manager” role that regularly communicates with all Foundation engineering teams regarding their technical debt

 * Due: End of this quarter


 * First pass role responsability defined : https://www.mediawiki.org/wiki/Technical_Debt_Program/Program_Management

Program 3: Outcome 2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation

 * Due: End of next quarter


 * Education/Alignment:Tech Debt Blog post in queue

==== Program 6: Outcome 2: Objective 2: Set up a continuous integration and deployment pipeline to publish new versions of an application to production via testing and staging environments that reliably reproduce production ====
 * Due: End of this quarter
 * Keyword: SSD
 * Complete build phase of release pipeline


 * Build test variant
 * Run test entrypoint w/developer feedback - services dependency
 * Build production variant w/developer feedback - services dependency
 * Tag production container
 * Push to production docker registry - ops dependency - staging namespace
 * Tracking: https://phabricator.wikimedia.org/T157469
 * current status: https://phabricator.wikimedia.org/project/view/2453/


 * new blubber revision in process https://phabricator.wikimedia.org/D875
 * question for marxarelli: do we need to add dependencies? Also: how do we add dependencies ;)
 * docker-pkg all the things in process (maybe fits in docker for CI work)

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)

 * workboard


 * New scap release 3.7.2 pushed based on https://phabricator.wikimedia.org/rMSCA2561b82
 * Fillippo said he'll get it uploaded first thing this week.
 * https://phabricator.wikimedia.org/D877
 * whoops :-/
 * We should update the How to deploy code/Scap wikipage, I have been ignoring that change in tagging, but gbp changes it automagically.
 * ✅ https://wikitech.wikimedia.org/w/index.php?title=How_to_deploy_code/Scap&diff=1775303&oldid=1761230

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure

 * Goal: A generalized POC for a docker-based CI.
 * https://phabricator.wikimedia.org/project/view/3008/ (shipyard workboard)

trap 'docker stop <(cat container.id)' SIGTERM docker run --cidfile container.id & wait
 * Process left behind after Jenkins SIGTERM "docker run". Some work in progress to relay signals, shutdown cleanly.
 * Ultimately, mightl want to do something like:


 * TODO: write a blogpost about docker run and signals


 * How to ship dependencies ( libXXX-dev )

Program 1: Outcome 6: Milestone 1: Maintain Gerrit

 * Volunteers are archiving mediawiki/extensions in Gerrit (marked read-only + [ARCHIVED] in description) and in CI (Zuul triggers a "failed-archived" job that always vote Verified -1). Usually each is tracked by a task.

Program 1: Outcome 6: Milestone 2: Maintain Phabricator

 * Nothing new to report

Program 1: Outcome 5: Objective 1: MW Nightlies server

 * security

Other work

 * Scap plugin: "Task Series" for maintaining the deployment blocker tasks: https://gerrit.wikimedia.org/r/#/c/390429/
 * This can be used for creating other series' of tasks in phabricator, if we have a need for any others
 * Documentation:
 * https://wikitech.wikimedia.org/wiki/Deployments/Blocking_Tasks
 * https://wikitech.wikimedia.org/wiki/Deployments/Holding_the_train#Train_.22blocker_tasks.22

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng


 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart