Wikimedia Release Engineering Team/Checkin archive/20171204

= 2017-12-04 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * Dec 4th (Monday): Antoine - no train were available from Paris to my city!
 * Dec 11-13th - Chad. Maybe? Friend might be flying into town and I need to play tour guide if so
 * December Fridays - Greg is taking off
 * December 25 (Monday): Željko - holiday (Christmas Day)
 * December 26 (Tuesday): Željko - holiday (St Stephen's Day)
 * Dec 25-Jan 1 - End of year/new year holidays
 * January 1 (Monday): Željko - holiday (New Year's Day)

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R

Dec 4 and Dec 11

 * Train: Chad
 * wmf.11
 * wmf.12
 * SoS: Mukunda
 * Out
 * Dec 4th (Monday): Antoine

Dec 18 and Dec 25

 * Train: NO ONE!
 * SoS: Chad
 * Out:
 * Dec 25-Jan 1 - End of year/new year holidays
 * December 25 (Monday): Željko - holiday (Christmas Day)
 * December 26 (Tuesday): Željko - holiday (St Stephen's Day)
 * Antoine - not sure yet of the exact dates

Jan 1 and Jan 8

 * Train: Mukunda
 * wmf.15
 * wmf.16
 * SoS: Tyler
 * Out:
 * January 1 (Monday): Holiday, everyone

Actions from last meeting

 * ACTION: git-lfs in Gerrit
 * a test repo was created
 * question: git-lfs supports objects that are <= 2GB ... is that a problem?
 * phab side: still need to enable it and figure out storage backend
 * ACTION: see if the 2 gig limit is real :)
 * there's a config thing

Release Engineering

 * Blocking
 * Blocked
 * Updates
 * [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. [wiki][email]
 * [MW Train] The post mortem for T181006 (Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki) is scheduled for December 7th.
 * #mw-1.30-release MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten
 * [MW Train] The post mortem for T181006 (Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki) is scheduled for December 7th.
 * #mw-1.30-release MediaWiki 1.30 is about to be released in the next few days, so last call for any blockers people might have...you know...forgotten

Last week

 * Blocking
 * Blocked
 * [SSD] blocked on ops updating nodejs-devel base image: 
 * Updates
 * [MW Train] Reminder! This is your last few weeks of deployments for the year/quarter! No non-emergency deploys starts the week of December 18th. [wiki][email]
 * [MW Train] The post mortem for T181006 (Watchlist and RecentChanges failure due to ORES on frwiki and ruwiki) is scheduled for December 7th.
 * [Sunsetting WG] Meeting with Victoria and Toby tomorrow (Wednesday).
 * [nodejs browser tests] CirrusSearch and Mobile are active with migrations.
 * [nodejs browser tests] Investigation of using mwbot instead of nodemw (javascript api interfaces to MW). CirrusSearch uses mwbot. We use nodemw (maintained by a Wikia engineer).
 * [ruby] We will upgrade rubocop (Ruby linter/static analyzer) across the corpus of extensions due to a security issue (pointed out by the new Github security issue notification service). We (WMF production and CI) are not affected by the issue AND we have officially deprecated our ruby browser test stack (where 99% of all rubocup uses/dependencies are) BUT we’ll do it :) [wiki][email]
 * [techdebt] Wider conversations regarding SLAs/stewardship of components/software: https://www.mediawiki.org/wiki/Talk:Development_policy/Code_Stewardship
 * [techdebt] 2nd of the 3 tech debt series of blog posts (after the 0th introduction one on Code Health) is in drafting.
 * [CI] We’ve migrated almost all of the tox (python) jobs to the new container based CI infra.
 * [CI] We’ve migrated almost all of the tox (python) jobs to the new container based CI infra.

Puppet SWAT

 * list of patches you want to submit to Puppet SWAT

Logspam \ Last week's train updates

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor


 * Krinkle sent this: https://lists.wikimedia.org/pipermail/engineering/2017-December/000493.html
 * thanks Krinkle :)

Other Team Business

 * Team offsite:
 * Schedule: https://docs.google.com/document/d/1Ku2-djYs11U1CAlkTZmdt0YUbZq82aakp-74D6Gdhv4/edit?
 * Should look familiar :P
 * Saturday logistics: we have vehicles
 * topic generation: https://etherpad.wikimedia.org/p/releng-offsite-201801-topics

Q3 goals! :) :)

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q2
 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q3

Program 1: webdriver.io

 * https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Technology#Program_1:_Availability.2C_performance.2C_and_maintenance


 * Jon's list https://www.mediawiki.org/wiki/User:Jdlrobson/Experiences_porting_Ruby_browser_tests_to_NodeJs
 * TODO list at https://phabricator.wikimedia.org/T139740


 * update to recent version of webdriver.io
 * video recordings
 * cucumber support
 * cleanup of some hardcoded urls


 * composer support (instead of vendor)
 * Jan is adding wdio/selenium tests to wikimedia/portals

Program 3: Tech Debt

 * https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Technology#Program_3:_Addressing_technical_debt

Outcome 1
 * Objective 2: Identify and find stewards for high-priority/high use code segment orphans - task T174091
 * Objective 3: Define and steward a light-weight process for adopting or orphaning/sunsetting products and infrastructure.
 * This is done by the Sunsetting working group

Outcome 2
 * Objective 2: Define and implement a process to regularly address technical debt across the Foundation
 * Objective 3: Promote and surface important technical debt topics at large gatherings of Wikimedia developers (e.g., DevSummit and Hackathon(s))

Program 6: SSD

 * https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/Final/Programs/Technology#Program_6._Streamlined_service_delivery

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)
=
 * workboard


 * Progress so far:
 * 4 closed: https://phabricator.wikimedia.org/maniphest/query/4lWgir9kgulF/#R
 * https://phabricator.wikimedia.org/T126306 could probably be marked as resolved?
 * 10 remain open: https://phabricator.wikimedia.org/maniphest/query/EMX2HewvHUAN/#R

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure
=
 * Goal: A generalized POC for a docker-based CI.
 * https://phabricator.wikimedia.org/project/view/3008/ (shipyard workboard)

Program 1: Outcome 5: Objective 1: MW Nightlies server

 * chad point on this, bulk of time

Q2 goal/project check-in

 * All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q2

Program 1: Outcome 5: Milestone 1: Migrate majority of developers to JavaScript based browser test framework (webdriver.io)

 * Due: End of this quarter
 * Quarter Goal Task: Port Selenium tests from Ruby to Node.js -


 * T139740 Port Selenium tests from Ruby to Node.js


 * T174591 Flow: Migrate browser tests from Ruby to node.js
 * Deleting tests


 * T171848 Port Echo Selenium tests from Ruby to Node.js
 * Probably deleting tests too


 * Selenium tests for wikimedia/portals/deploy!
 * https://gerrit.wikimedia.org/r/#/c/394539/


 * T181285 Replace jshint with eslint in nodemw
 * Done. Merged upstream.


 * T180144 Upgrade WebdriverIO to 4.9
 * Found the upstream commit that broke our tests. Next step is reporting upstream.

Ruby

 * T180878 Upgrade RuboCop and Rubyzip across the extension fleet
 * In progress. A couple of repos already causing trouble. :|

Program 3: Outcome 1: Objective 1: Define a set of code stewardship levels (from high to low expectations)

 * Due: End of this quarter
 * Quatertly Goal task: -


 * Continued discussion of what Stewardship means. Some of the feedback suggests that "stewardship" may be too soft of a word, perhaps "ownership" is better.

Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans

 * Due: End of next quarter
 * Quaterly Goal task -


 * Restarted clean-up/gap filling for developers/maintainers page.

Program 3: Outcome 2: Objective 1: Define a “Technical Debt Project Manager” role that regularly communicates with all Foundation engineering teams regarding their technical debt

 * Due: End of this quarter


 * done

Program 3: Outcome 2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation

 * Due: End of next quarter


 * Blog post 2 of 3 almost done.

==== Program 6: Outcome 2: Objective 2: Set up a continuous integration and deployment pipeline to publish new versions of an application to production via testing and staging environments that reliably reproduce production ====
 * Due: End of this quarter
 * Keyword: SSD
 * Complete build phase of release pipeline


 * Build test variant
 * Run test entrypoint w/developer feedback - services dependency
 * Build production variant w/developer feedback - services dependency
 * Tag production container
 * Push to production docker registry - ops dependency - staging namespace
 * Tracking: https://phabricator.wikimedia.org/T157469
 * current status: https://phabricator.wikimedia.org/project/view/2453/


 * npm is bologna

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)

 * workboard

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure

 * Goal: A generalized POC for a docker-based CI.
 * https://phabricator.wikimedia.org/project/view/3008/ (shipyard workboard)
 * DONE


 * being generalized
 * more infra work needed such as:
 * gotta move toward kubernetes
 * docker-pkg and work closely with SSD / ops

Other work

 * TestED
 * Zeljko and JR are working to develope some testing education material primarily focused on helping developers test for effectively.
 * Zeljko, Cormac, and JR are exploring ways to introduce TDD to WMF teams.
 * General ED on TDD -> MediaWiki specific support (tech sessions)

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng


 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart