Wikimedia Release Engineering Team/Checkin archive/20190506



Vacations/Important dates[edit]
How to do it
  • May 8th - Antoine, 1945 victory
  • May 10th - thcipriani, wichita graduation
  • May 15 (Wednesday) - Željko vacation
  • May 13–15: James working from London
  • May 16-20 - Wikimedia Hackathon 2019 (Prague, Czechia)
    • Attending: Greg, JR, Zeljko, James, and Jeena
  • May 17th: Mukunda day off - Concert.
  • May 20-31 - Jeena Vacation
  • May 21: James travelling back to SF
  • May 27 (Memorial Day) - US Staff
  • May 28th-31st - thcipriani - family in town
  • May 30th - Lars, Ascension
  • May 30th-31th - Antoine, Feast of the Ascension
  • June 10th - Antoine, Pentecost -- see for Antoine/France Easter holidays
  • June 6-7 - Brennen, Apogaea
  • June 10-? - Dan leave (4-6 weeks, then additional leave later)
  • June 19 (Juneteenth) - US Staff
  • July 4 (US Independence Day) - US Staff
  • July 22 - August 9 - Željko vacation
  • July 22 - Lars, Midsummer
  • August 7–9 - James off
  • August 12 (Glorious Twelfth) - US Staff
  • August 14–18 - Wikimania
    • Attending: James, ? …
  • August 25 - September 4 - Brennen vacation
  • September 2 (Labor Day) - US Staff
  • October 14 (Indigenous Peoples' Day) - US Staff
  • November 11 (Veterans' Day) - US Staff
  • November 28–29 (Thanksgiving) - US Staff
  • December 6 - Lars, Finnish Independence Day
  • December 25–31 (Christmas) - US Staff
  • December 25-26 - Lars, Christmas
  • 2020 January 1 (New Year's Day) - US Staff, Lars

Rotating positions[edit]


Maniphest query for deployment blocker tasks:
  • Apr 29 - wmf.3 - Tyler
  • May 06 - wmf.4 - Tyler
  • May 13 - wmf.5 - Antoine
  • May 20 - wmf.6 - Antoine
  • May 27 - wmf.7 - Zeljko
  • June 03 - wmf.8 - Zeljko
  • June 10 - wmf.9 - Mukunda
  • June 17 - wmf.10 - Mukunda


  • Zeljko 4eva! :)

Team Business[edit]

Timespent spreadsheet[edit]

For the avoidance of doubt: fill out the sheet week number for the previous week

Book club[edit]

Spring Offsite[edit]


  • Greg: Email mark re gerrit/phab hosting discussion
    • DONE: scheduled for Wed May 8th.
    • Attending: Mark, mutante, Alex, Joe, Lars, Antoine, Dan, Mukunda, Tyler, Me
    • Not just about gerrit/phab, also "the pipeline transition period and what that means for Beta Cluster" (hence all the people)
  • Greg: email mark about capex request for next year for pipeline
    • I'm actually not sure what this is about/what the ask is, help?!
    • "staging" pipeline?
    • Production access?
  • Mukunda: talk with Timo and Fillipo about our prioritized of feature requests for LMM
    • Note: Gergo confirmed that SRE is going to work on Sentry in Q1/Q2 (from a conversation with Faidon and Filippo)
  • Greg: announce that RelEng is backup only for SWAT (removal of person’s names from getting pinged everytime on IRC) and we’ll start working on automating the train
    • Still need to do Q4 goals...table this “doing” until Q1?
    • Greg will send a signed email if someone writes it up ;)
      • Željko will write the e-mail this week

Monthly reflection on accomplishments - April '19 edition[edit]
Add as you have them!
  • Phabricator vandalism rollback tool completed 🎉 (blog post? 😉)
  • Upgrade Zuul to 2.5.1-wmf6 (which unblocks the Gerrit upgrade to 2.16) -
  • Team offsite in Chicago

Annual Planning[edit]

Budget ask is in. We will see what's next. Pan-Technology prioritisation happening now.

Annual Reviews[edit]




  • Starting now: You and I discuss who your peer reviewers should be
  • April 26th: Enter your peer reviewers into Namely (please run them by me first)
  • May 17th: Deadline to complete self-reviews, peer reviews, and reviews of your manager.
  • May 20th: I start reviewing the peer reviews and writing my feedback on you.

Non SafeGuard (aka US Employees):

  • June 14th: Deadline for managers to complete all 1:1 meetings with direct reports and provide written feedback in Namely.


  • June 14th - Managers of those employed by Safeguard submit their reviews to HR for submission to Safeguard
  • July 12th - Deadline to have a 1:1 and share final manager review with direct report in Namely

Incoming/Needs attention[edit]

Scrum of Scrums[edit]

Incoming from last week[edit]

  • Blocking:

Outgoing this week (wrong section heading level is on purpose for copy/pasting into Scrum of Scrums etherpad[edit]

Release Engineering[edit]


  • Release Engineering

Train status and happenings[edit]
  • Need to fix scap clean :\
    • thcipriani has a crappy fix in mind until http tokens in gerrit are back
      • Any idea when HTTP tokens will come back? Weeks? Months? Never? :-(
      • ~Weeks
    • 2019-05-06: cleaned up stuff last week on deploy hosts, just not the gerrit branches
  • 1.33 branch cut for extensions is blocked (except tarball ones, which James did manually)
    • 2019-05-06: Mukunda to do it this week
    • Greg: email Cindy re process of this release
  • IDEA: automatica UBN! and comment of train blockers

Quarterly Goals for Q4[edit]

TEC1 (Maint): Outcome 1 / Output 1.1[edit]

GOAL: Undeploy the CodeReview extension.
WHO: James, need help from CPT
  • James will ping CPT about this this week (April 8th)
    • … and again w/c 15 April.
    • … and again w/c 6 May (in SoS).

TEC1 (Maint): Outcome 1 / Output 1.1[edit]

GOAL: Setup 1-3 of the CI WG options (Zuul v3, Argo, GitLab)
  • Focus on a couple noteworthy repos: e.g.,
    • core
    • extensions
    • ops/puppet
  • Maybe setup in serial, i.e., a week per evaluation
  • Questions:
    • RelEng/Extended working group?
      • At least in the WG eval it was good to have non-familiar people
      • But maybe with the setup of options it might be beneficial to have experienced with current setup people.
      • Folks outside the original working group to join-in to setup options; people TBD
    • Do we need a rubric before we do this prototyping? (yes)
      • DONE lars to work on rubric week of 2019-04-01
        • See email 2019-04-08
  • 2019-05-06: Feedback from Android. Working on an arch document. Do in Q1?

TEC3 (Pipeline): Outcome 1 / Output 1.2[edit]

GOAL: Instrument Quibble for data collection
WHO: Mukunda, Antoine
  • Still no progress / nowhere to store this data and other tasks taking priority

TEC3 (Pipeline): Outcome 1 / Output 1.2[edit]

GOAL: Create a graph where time is spent and make a prioritized list for improvements.
WHO: Mukunda, Antoine

TEC3 (Pipeline): Outcome 1 / Output 1.2[edit]

GOAL: Prepare the Deployment Pipeline for changes to our CI tooling.
WHO: ???, ???
  • Blocked by not having new CI tooling yet

TEC3 (Pipeline): Outcome 3 / Output 3.1[edit]

GOAL: Create a .pipeline/config.yaml standard to give users more control over how their tests are run in the pipeline and allow the easy saving of artifacts at pipeline completion. (RelEng)
WHO: Dan, Tyler, ???

TEC3 (Pipeline): Outcome 3 / Output 3.1[edit]

Adopt more services into Deployment pipeline - task T212801
Wikidata Termbox SSR, Kask for Session Storage Service, cpjobqueue (stretch), ORES (stretch)
WHO: Dan, Tyler, Lars

There are tasks:

  • changeprop

  • Wikidata Termbox SSR
  • Kask for Session Storage Service
  • cpjobqueue (stretch)

TEC12 (DevProd): Outcome 1 / Output 1.1[edit]

GOAL: Provide an "Official" Docker base image for local development of MediaWiki based on the production tooling.
WHO: Jeena, Brennen

Various relevant tasks:

TEC13 (Code Health): Outcome 1 / Outcome 3[edit]

GOALs: Presentation/session(s) at the Wikimedia Hackathon on the current state of Code Health projects (technical debt and code stewardship)

Met and discussed Hackathon session with Code Health Metrics WG. Daniel will also be having a related session on Cycle Dependencies.

  • T216630 Present Code Health Metrics at the Hackathon

TEC13 (Code Health): Outcome 1 / Output 1.1[edit]

Publish a re-imagination of the Review Queue process.
Develop and implement metrics around task and code-review responsiveness
WHO: Greg, JR (and Andre)

No activity

= TEC13 (Code Health): Outcome 4 / Output 4.2[edit]

Expand SonarQube reporting into CI infrastructure
Perform SonarQube analysis on all extensions
Engage user communities in direct feedback solicitation
WHO: JR, Zeljko, Code Health Metrics

Kosta working with Tyler to finalize CI integration. Goal is to have this working by Hackathon.

Other non-goal work[edit]

Release MW 1.33[edit]

  • ETA: end of May
  • "just" producing the tarballs, Cindy is on point for what's in/what's out.
  • Can I get a volunteer?
  • Mukunda can build tarballs
  • Greg: email Cindy asking for status


T220755 [Spike] Have a discussion around Minerva selenium browser test architecture - talked with the team. Several sessions planned for hackathon.




QA/Code Health[edit]

Next Code Health Office hours slated to be Performance Testing.

Met with Daniel and Guillaume re: DevEd and developer software design practices.




  • What I plan to do this week
    • "Push buttons very hard"
  • What I'm blocked on
  • Other?


  • What I plan to do this week
    • T222494: Move dev-images PHP + Apache image from mod_php to php-fpm
    • blubber.yaml for MediaWiki dev image
      • Consult Antoine / others on where this should live
    • T218360: Add dev-images use to local-charts
    • T217868: Get some amount of testing in place before going much further with local-charts / dev-images
    • Quick re-skim of book chapters
    • Read Lars's CI rubric / arch document
  • What I'm blocked on
  • Other?



  • What I plan to do this week
    • Responding to budget and annual plan follow-up
    • Offsite/travel paperwork/reimbursement
    • TechConf planning
    • write out phab process/do skeleton on the labs phab instance
    • figure out the MW release open questions
    • Start writing peer reviews
  • What I'm blocked on
  • Other?


  • What I plan to do this week
    • MW static configuration concept
    • Pipeline documentation work with Martyav.
    • Apparently JSONlint has broken, so I'll probably be working on the planned replacement of that. :-(
    • Helping ServiceOps with the HHVM -> PHP72 migration, probably.
    • Possibly doing a cluster analysis of how we use our configuration options
  • What I'm blocked on
    • WikipediaZero removal blocked on SRE having time (low urgency); CodeReview removal similarly blocked on CPT.
  • Other?
    • Working from London early next week, and then at the Hackathon in Prague.


  • What I plan to do this week
    • Work on Code Review WG
    • Prep for Hackathon
    • Peer feedback
  • What I'm blocked on
  • Other?


  • What I plan to do this week
    • Review Writing
    • Prepare local-charts for hackathon
    • images of parsoid and restbase?
  • What I'm blocked on
  • Other?


  • What I plan to do this week
    • finish and publish first draft of architecture doc for future CI
    • prepare for and attend RelEng/SRE meeting on Phabricator maintenance on Wednesday
    • prepare for and attend book club meeting on Friday
    • learn how to write reviews and write first sketches, possibly attend related workshop on Friday
    • give a talk on PGP to the team if there's interest (see email)
  • What I'm blocked on
    • lack of brain (still recovering from jet lag)
  • Other?


  • What I plan to do this week
    • Sync up with Daniel about migrating Phabricator to the new server
    • Work on the branching script in make-release (with tyler on wednesday)
    • Figure out a time to pair with Jeena on local dev environment stuff
    • Write some annual performance reviews
    • Read a book
  • What I'm blocked on
  • Other?



  • What I plan to do this week
    • Submit several Selenium related sessions for the hackathon
    • Several Selenium related reviews and tasks
  • What I'm blocked on
  • Other?


Team Kanban Board Review and Triage[edit]

Once / month-ish review of backlog(s)[edit]

Kanban stats[edit]

Burnup chart