Wikimedia Release Engineering Team/Checkin archive/20171030

= 2017-10-30 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * November 1 (Wednesday) Željko - holiday (All Saints' Day)
 * Nov 10 (Fri) - Veteran's Day
 * November 17 (Friday) Željko - conference (Coderetreat)
 * Nov 20th - Dec 1st: Greg vacation
 * Nov 23+24 - Thanksgiving
 * December 25 (Monday): Željko - holiday (Christmas Day)
 * December 26 (Tuesday): Željko - holiday (St Stephen's Day)
 * Dec 25-Jan 1 - End of year/new year holidays
 * January 1 (Monday): Željko - holiday (New Year's Day)

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/maniphest/?project=PHID-PROJ-fmcvjrkfvvzz3gxavs3a&statuses=open%28%29&group=none&order=newest#R

Oct 23 and Oct 30

 * Train Chad
 * wmf.5
 * wmf.6
 * SOS: Mukunda
 * Out
 * Week of Oct 23: Tyler
 * Oct 25: Zeljko
 * Nov 1: Zeljko

This week

 * Blocking
 * Blocked
 * Updates
 * We had to pause the MediaWiki train last week due to a hard to diagnose issue in production: https://phabricator.wikimedia.org/T179156
 * T171852 Tech talk: Selenium tests in Node.js https://phabricator.wikimedia.org/phame/post/view/78/tech_talk_selenium_tests_in_node.js/
 * T173488 Selenium Ruby framework deprecated https://phabricator.wikimedia.org/phame/post/view/79/selenium_ruby_framework_deprecation_october/
 * CI: Antoine was sick last week. But, he setup a simple package manager caching system for the docker based CI (a port of “castor” in our nodepool based CI). He will migrate some of the CI config related tox jobs to it.
 * T173488 Selenium Ruby framework deprecated https://phabricator.wikimedia.org/phame/post/view/79/selenium_ruby_framework_deprecation_october/
 * CI: Antoine was sick last week. But, he setup a simple package manager caching system for the docker based CI (a port of “castor” in our nodepool based CI). He will migrate some of the CI config related tox jobs to it.

Last week

 * Blocking
 * Blocked
 * Updates
 * Deployment logspam is mostly quiet, except one non-deployment related log from the tidy migration (Parsing/MW Platform know)
 * Zeljko paired with Elena T. to get some Echo notification browser tests written (nodejs/mocha framework).
 * Zeljko scheduled a Tech Talk on Oct 31st to discuss/teach the nodejs browser testing framework.
 * Scap tech-debt project well underway, merged many improvements last week
 * new/docker based CI: the phan job is now migrated
 * Gerrit was upgraded last week (minor version)
 * Working on git-lfs (large file store, for big binary blobs) support in Gerrit for ORES and Reading teams
 * Redesigned (simplified) the Phabricator login screen , should be deployed soon.
 * Working on git-lfs (large file store, for big binary blobs) support in Gerrit for ORES and Reading teams
 * Redesigned (simplified) the Phabricator login screen , should be deployed soon.

Logspam \ Last week's train updates

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor


 * 503s: https://phabricator.wikimedia.org/T179156
 * syncing up with bblack re train

Other Team Business

 * Annual Plan proposals: https://docs.google.com/document/d/1mOC_vBDmQCGoPYZug4cd2DwAbZ1K0qR7370F2VL0ttM/edit#

Q2 goal/project check-in

 * All of it in table form: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201718Q2

Program 1: Outcome 5: Milestone 1: Migrate majority of developers to JavaScript based browser test framework (webdriver.io)

 * Due: End of this quarter
 * Quarter Goal Task: Port Selenium tests from Ruby to Node.js -


 * T139740 Port Selenium tests from Ruby to Node.js
 * New tables: stats and status.
 * TLDR: out of 23 repos, not started 7 (30%), stalled 2 (9%), in progress 6 (26%), done (ruby->deleted) 6 (26%), done (ruby->node) 2 (9%)


 * T171852 Tech talk: Selenium tests in Node.js
 * Tomorrow!


 * T173488 Announce Selenium Ruby framework deprecation on appropriate mailing lists (QA, engineering, wikitech-l)
 * Resolved.
 * Selenium Ruby framework deprecation (October) https://phabricator.wikimedia.org/phame/post/view/79/selenium_ruby_framework_deprecation_october/


 * T179157 Did something change in beta cluster configuration around September 16 2017?
 * Resolved: a page was deleted and it caused the tests to fail. :|


 * T173505 Add createAccount method to nodemw
 * Implemented, waiting for review/merge upstream.


 * T116027 Make selenium users use botflags at beta-cluster
 * Blocked: I still do not know what to do.


 * TODO
 * T179190 Run Cucumber+Selenium+Node.js in CI
 * T179188 Video recording for Selenium tests in Node.js

Ruby

 * T167432 Run Wikibase daily browser tests on Jenkins
 * TODO: convert the job to JJB

Program 3: Outcome 1: Objective 1: Define a set of code stewardship levels (from high to low expectations)

 * Due: End of this quarter
 * Quatertly Goal task: -

Program 3: Outcome 1: Objective 2: Identify and find stewards for high-priority/high use code segment orphans

 * Due: End of next quarter
 * Quaterly Goal task -

Program 3: Outcome 2: Objective 1: Define a “Technical Debt Project Manager” role that regularly communicates with all Foundation engineering teams regarding their technical debt

 * Due: End of this quarter

Program 3: Outcome 2: Objective 2: Define and implement a process to regularly address technical debt across the Foundation

 * Due: End of next quarter

==== Program 6: Outcome 2: Objective 2: Set up a continuous integration and deployment pipeline to publish new versions of an application to production via testing and staging environments that reliably reproduce production ====
 * Due: End of this quarter
 * Keyword: SSD
 * Complete build phase of release pipeline


 * Build test variant
 * Run test entrypoint w/developer feedback - services dependency
 * Build production variant w/developer feedback - services dependency
 * Tag production container
 * Push to production docker registry - ops dependency - staging namespace

Tracking: https://phabricator.wikimedia.org/T157469
 * current status: https://phabricator.wikimedia.org/project/view/2453/
 * Made this thing: https://www.mediawiki.org/wiki/Continuous_integration/Docker/Dockerfiles
 * Documented the sh*t out of Blubber packages, types, functions https://phabricator.wikimedia.org/T168000
 * Working on improving YAML parsing errors but it's prooving rather difficult (gives line numbers at least)
 * Implementing config validation (post parsing) https://phabricator.wikimedia.org/T175186
 * Found nice library for it https://gopkg.in/go-playground/validator.v9

Program 1: Outcome 1: Objective 1: Scap (Tech Debt Sprint FY201718-Q2)

 * workboard


 * need to ping moritz re backporting git 2.11+ to trusty for scap needs

Program 1: Outcome 5: Objective 1: Maintain existing shared Continuous Integration infrastructure

 * Goal: A generalized POC for a docker-based CI.
 * https://phabricator.wikimedia.org/project/view/3008/ (shipyard workboard)


 * Antoine's been sick :(
 * Antoine is porting castor to docker (cache of dependencies for eg npm)
 * start with migrating some tox jobs (starting with ci meta repos)

Team Kanban Board Review and Triage

 * closed and touched in the 7 days
 * No update for 4 weeks
 * No update for 3 weeks
 * No update for 2 weeks
 * No update for 1 week
 * All Open
 * Review To Triage column of #releng


 * Assigned
 * Unassigned

Once / month-ish review of backlog(s)

 * releng Review To Triage column of #releng
 * releng-kanban Review unassigned in kanban
 * releng-kanban Review 'backlog' colum of -kanban
 * releng-next - Review for things we need to put on our kanban backlog
 * releng-backlog - oh my, the huge backlog of things...

Kanban stats

 * Burnup chart