Wikimedia Release Engineering Team/Checkin archive/20170306

2017-03-06

Vacations/Important dates

https://office.wikimedia.org/wiki/HR_Corner/Holiday_List

How to do it

• March 13: Monthly day off
• March 23-28: Chad in Miami
• March 30, 31: Tyler in Wichita
• March 30-Apr 3: Greg in Berlin (Wikimedia Conference)
• April 10-17 (ish): Dan out
• April 14: Monthly day off
• May 15-18: Team Offsite in Vienna
• May 19-21: EU Hackathon

Team Business

Time spent spreadsheet

• week 9: https://docs.google.com/spreadsheets/d/1OZzvtlc116UIsFFbic1zX34LXgAJddPTXMGVE6uMsJw/edit#gid=30533694

Rotating positions and absences

Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

Mar 6 and Mar 13

• Train: Mukunda
  • wmf.15
  • wmf.16
• SoS: Tyler
• Out:
  • Mon Mar 13th: Monthly day off

Mar 20 and Mar 27

• Train: Tyler (plus Chad for goup2 on March 30th)
  • [ wmf.17]
  • [ wmf.18]
• SoS: Chad
• Out:
  • March 23-28: Chad in Miami
  • March 30, 31: Tyler in Wichita
  • Thu March 30 - Mon Apr 3: Greg in Berlin (Wikimedia Conference)

Actions from last meeting

• TODO: Sign up for the Product and Technology consultation if you have any slight amount of interest: https://office.wikimedia.org/wiki/Product_and_Tech_consultation
  • Mukunda and Antoine sync up

Scrum of Scrums

https://phabricator.wikimedia.org/project/board/64/

Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

• Blocking
• Blocked
  • T139740 Port Selenium tests from Ruby to Node.js, the patch is almost ready, please review https://gerrit.wikimedia.org/r/#/c/328191/
• Updates

Last week

• Blocking
  • https://phabricator.wikimedia.org/T153038
• Blocked
  • none
• Updates

Other Team Business

Keyhold blogpost

• https://etherpad.wikimedia.org/p/keyholder-blog
  • thcipriani: fiddled over the weekend
  • Checked in with Ori, he is in favor :) said we ought to mention some limitations
  • bd808 corrently mentions history/shoutouts ("Proposal to change scap ssh-agent usage" on ops-l 2014-02-25/Shared ssh key (was Re:LocalisationUpdate == useless Tin) 2015-10-07)
    • This is amazing: https://etherpad.wikimedia.org/p/DeploymentSystemRequirements
  • https://meta.wikimedia.org/wiki/Wikimedia_Blog/Guidelines

Logspam

• Updated https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Roles#Train_Conductor

Annual Planning

Technology-wide planning

E_TOOMANYGDOCS

• New gdoc: https://docs.google.com/document/d/1NmFC17xYyoeHk4KhBGWD_FgZjlLZWrFe8D1SfPHIpew/edit#heading=h.rm0hezkjjvzb

Q3 goal/project check-in

https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q3

Continuous Integration Infrastructure modernization

Gather requirements for new CI system - T153270

• We made us a Jenkins to test some plugins
  • Jenkins plugin https://github.com/pearsontechnology/deployment-pipeline-jenkins-plugin
• We haven't tested plugins
• First step is how to build a base image suitable for k8s
• How to run some arbritary command in it, ideally defined in the source repository

Migrate MediaWiki deploys to scap3 (part 2 — This time, it's personal; A.K.A Scappy Strikes Back)

Build fanout support in scap3

https://phabricator.wikimedia.org/T121276

Phabricator

• Search engine overhaul
  • Another week, another round of code review
  • https://secure.phabricator.com/D17384
• Upstream just implemented task types:
  • The task upstream: https://secure.phabricator.com/T12314
  • Downstream: https://phabricator.wikimedia.org/T92708

Non-Quarterly goal work

Browser tests

• T158074 Update Ruby tests to Selenium 3
  • patch for Ruby gem merged, gem released, patch in mediawiki/core merged, a few patches in extensions merged, the vast majority pending
• T139740 Port Selenium tests from Ruby to Node.js
  • Updating documentation
  • Please review! :) https://gerrit.wikimedia.org/r/#/c/328191/
  • Timo provided good feedback, implemented all requested changes. Waiting for more feedback or -1/+1 or even +2!

Beta Cluster

• Beta cluster scap job ( beta-scap-eqiad ) fails due to puppet erasing /etc/ssh/ssh_known_hosts
  • TIL puppetdb on beta! Allow use of PuppetDB in labs for ssh_known_hosts
  • This is awesome...but it's breaking
  • Happens since Tuesday 28 Feb roughly (apparently). No low-hanging fruit root cause candidate)
• 3D extension (render some kind of 3D objects file format to png). Marktraceur looks for guidance to deploy the requisities (a nodejs CLI wrapper and some .deb)

People status updates

Antoine

Last week

• composer autoloader breaks php 5.5 lint on mediawiki/vendor
  • Wikidata build is broken
  • gotta upgrade composer as well
    • All done. Composer bumped to 1.1.0

• Jenkins under systemd
  • All done. deployed on Friday night.
• Prepare to upgrade Jenkins on contint2001 (copy data, upgrade, see what happens)
  • Some preliminary work done such as clearing jenkins home

• as time allow:
  • migrate PHP 5.3 jobs to PHP 5.5
  • Think about getting more Jenkins instances (beta, one for browser tests, maybe tools-labs)

This week

• Qunit thing from SoS
  • Done Chromium upgrade solved it. https://phabricator.wikimedia.org/T153038
• Reviews of Javascript/Selenium for MediaWiki
• Prepare to upgrade Jenkins on contint2001 (copy data, upgrade, see what happens)

• As time allow (since 02/27)
  • migrate PHP 5.3 jobs to PHP 5.5 (due end of march)
  • Think about getting more Jenkins instances (beta, one for browser tests, maybe tools-labs)

Chad

This week

• Scap 3
• Quick gerrit downtime (LDAP change)

Dan

Last week

• Research build manifest standardization
• Hack on scap fanout PoC for realsies

This week

• Help break out CI/CD build PoC tasks (set up registry, image config/builder, etc.)
• Play with Pearson Jenkins plugin
• Grab a task and start hacking

JR

Last week

• quality interviews
  • Scheduled additional interviews
  • Building a picture of test activities/processes
• investigate test results reporting
• get development environment up and running
  • issues getting xdebug working
  • otherwise, all good
• CI infrastructure tooling rampup
  • Docker
  • K8s

This week

• quality interviews
• test reporting dashboard
  • requirements/highlevel design

Mukunda

Last week

• Phabricator stuff
• Scrum of scrums
• keyholder blog post?
• dust off the phab milestone scap plugin
  • look at making it update the deployment schedule (in addition to the deployment blocker tasks)

This week

• Deployment Train
• leftover from last week
  • dust off the phab milestone scap plugin
    • look at making it update the deployment schedule (in addition to the deployment blocker tasks)
    • this probably won't happen this week
• Search engine overhaul
  • Another week, another round of code review

Tyler

Last week

• Done Blog post, actually write it
• Scap3 stuff, actually do it
  • N Not done Finish work on submodule space issue
  • Done New release? Possible this week. (3.5.3-1)
• N Not done CI stuff, find an outcome for it (started poking, build an image)

This week

• Polish up blog post
• scap3 fanout stuff w/chad
• CI digging deep

Željko

Last week

• T158074 Update Ruby tests to Selenium 3
• T139740 Port Selenium tests from Ruby to Node.js
• T159317 [Bug] Special:SetLabel browser tests failing, but unable to run tests locally

This week

• T152963 Increase in failures caused by Saucelabs
• T139740 Port Selenium tests from Ruby to Node.js