Wikimedia Release Engineering Team/Checkin archive/20170306

= 2017-03-06 =

Vacations/Important dates

 * https://office.wikimedia.org/wiki/HR_Corner/Holiday_List
 * How to do it


 * March 13: Monthly day off
 * March 23-28: Chad in Miami
 * March 30, 31: Tyler in Wichita
 * March 30-Apr 3: Greg in Berlin (Wikimedia Conference)
 * April 10-17 (ish): Dan out
 * April 14: Monthly day off
 * May 15-18: Team Offsite in Vienna
 * May 19-21: EU Hackathon

Time spent spreadsheet

 * week 9: https://docs.google.com/spreadsheets/d/1OZzvtlc116UIsFFbic1zX34LXgAJddPTXMGVE6uMsJw/edit#gid=30533694

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

Mar 6 and Mar 13

 * Train: Mukunda
 * wmf.15
 * wmf.16
 * SoS: Tyler
 * Out:
 * Mon Mar 13th: Monthly day off

Mar 20 and Mar 27

 * Train: Tyler (plus Chad for goup2 on March 30th)
 * [ wmf.17]
 * [ wmf.18]
 * SoS: Chad
 * Out:
 * March 23-28: Chad in Miami
 * March 30, 31: Tyler in Wichita
 * Thu March 30 - Mon Apr 3: Greg in Berlin (Wikimedia Conference)

Actions from last meeting

 * TODO: Sign up for the Product and Technology consultation if you have any slight amount of interest: https://office.wikimedia.org/wiki/Product_and_Tech_consultation
 * Mukunda and Antoine sync up

Scrum of Scrums

 * https://phabricator.wikimedia.org/project/board/64/
 * Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

 * Blocking
 * Blocked
 * T139740 Port Selenium tests from Ruby to Node.js, the patch is almost ready, please review https://gerrit.wikimedia.org/r/#/c/328191/
 * Updates
 * Updates

Last week

 * Blocking
 * https://phabricator.wikimedia.org/T153038
 * Blocked
 * none
 * Updates

Keyhold blogpost

 * https://etherpad.wikimedia.org/p/keyholder-blog
 * thcipriani: fiddled over the weekend
 * Checked in with Ori, he is in favor :) said we ought to mention some limitations
 * bd808 corrently mentions history/shoutouts ("Proposal to change scap ssh-agent usage" on ops-l 2014-02-25/Shared ssh key (was Re:LocalisationUpdate == useless Tin) 2015-10-07)
 * This is amazing: https://etherpad.wikimedia.org/p/DeploymentSystemRequirements
 * https://meta.wikimedia.org/wiki/Wikimedia_Blog/Guidelines

Logspam

 * Updated https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Technology-wide planning

 * E_TOOMANYGDOCS


 * New gdoc: https://docs.google.com/document/d/1NmFC17xYyoeHk4KhBGWD_FgZjlLZWrFe8D1SfPHIpew/edit#heading=h.rm0hezkjjvzb

Q3 goal/project check-in

 * https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q3

Continuous Integration Infrastructure modernization

 * Gather requirements for new CI system -


 * We made us a Jenkins to test some plugins
 * Jenkins plugin https://github.com/pearsontechnology/deployment-pipeline-jenkins-plugin
 * We haven't tested plugins
 * First step is how to build a base image suitable for k8s
 * How to run some arbritary command in it, ideally defined in the source repository

Build fanout support in scap3

 * https://phabricator.wikimedia.org/T121276

Phabricator

 * Search engine overhaul
 * Another week, another round of code review
 * https://secure.phabricator.com/D17384
 * Upstream just implemented task types:
 * The task upstream: https://secure.phabricator.com/T12314
 * Downstream: https://phabricator.wikimedia.org/T92708

Browser tests

 * T158074 Update Ruby tests to Selenium 3
 * patch for Ruby gem merged, gem released, patch in mediawiki/core merged, a few patches in extensions merged, the vast majority pending
 * T139740 Port Selenium tests from Ruby to Node.js
 * Updating documentation
 * Please review! :) https://gerrit.wikimedia.org/r/#/c/328191/
 * Timo provided good feedback, implemented all requested changes. Waiting for more feedback or -1/+1 or even +2!

Beta Cluster

 * Beta cluster scap job ( beta-scap-eqiad ) fails due to puppet erasing /etc/ssh/ssh_known_hosts
 * TIL puppetdb on beta! Allow use of PuppetDB in labs for ssh_known_hosts
 * This is awesome...but it's breaking
 * Happens since Tuesday 28 Feb roughly (apparently). No low-hanging fruit root cause candidate)
 * 3D extension (render some kind of 3D objects file format to png). Marktraceur looks for guidance to deploy the requisities (a nodejs CLI wrapper and some .deb)

Last week

 * composer autoloader breaks php 5.5 lint on mediawiki/vendor
 * Wikidata build is broken
 * gotta upgrade composer as well
 * All done. Composer bumped to 1.1.0


 * Jenkins under systemd
 * All done. deployed on Friday night.
 * Prepare to upgrade Jenkins on contint2001 (copy data, upgrade, see what happens)
 * Some preliminary work done such as clearing jenkins home


 * as time allow:
 * migrate PHP 5.3 jobs to PHP 5.5
 * Think about getting more Jenkins instances (beta, one for browser tests, maybe tools-labs)

This week

 * Qunit thing from SoS
 * ✅ Chromium upgrade solved it. https://phabricator.wikimedia.org/T153038
 * Reviews of Javascript/Selenium for MediaWiki
 * Prepare to upgrade Jenkins on contint2001 (copy data, upgrade, see what happens)


 * As time allow (since 02/27)
 * migrate PHP 5.3 jobs to PHP 5.5 (due end of march)
 * Think about getting more Jenkins instances (beta, one for browser tests, maybe tools-labs)

This week

 * Scap 3
 * Quick gerrit downtime (LDAP change)

Last week

 * Research build manifest standardization
 * Hack on scap fanout PoC for realsies

This week

 * Help break out CI/CD build PoC tasks (set up registry, image config/builder, etc.)
 * Play with Pearson Jenkins plugin
 * Grab a task and start hacking

Last week

 * quality interviews
 * Scheduled additional interviews
 * Building a picture of test activities/processes
 * investigate test results reporting
 * get development environment up and running
 * issues getting xdebug working
 * otherwise, all good
 * CI infrastructure tooling rampup
 * Docker
 * K8s

This week

 * quality interviews
 * test reporting dashboard
 * requirements/highlevel design

Last week

 * Phabricator stuff
 * Scrum of scrums
 * keyholder blog post?
 * dust off the phab milestone scap plugin
 * look at making it update the deployment schedule (in addition to the deployment blocker tasks)

This week

 * Deployment Train
 * leftover from last week
 * dust off the phab milestone scap plugin
 * look at making it update the deployment schedule (in addition to the deployment blocker tasks)
 * this probably won't happen this week
 * Search engine overhaul
 * Another week, another round of code review

Last week

 * ✅ Blog post, actually write it
 * Scap3 stuff, actually do it
 * ❌ Finish work on submodule space issue
 * ✅ New release? Possible this week. (3.5.3-1)
 * ❌ CI stuff, find an outcome for it (started poking, build an image)

This week

 * Polish up blog post
 * scap3 fanout stuff w/chad
 * CI digging deep

Last week

 * T158074 Update Ruby tests to Selenium 3
 * T139740 Port Selenium tests from Ruby to Node.js
 * T159317 [Bug] Special:SetLabel browser tests failing, but unable to run tests locally

This week

 * T152963 Increase in failures caused by Saucelabs
 * T139740 Port Selenium tests from Ruby to Node.js