Jump to content

Wikimedia Release Engineering Team/Checkin archive/20160829

From mediawiki.org

2016-08-29

[edit]

Vacations/Important dates

[edit]

How to do it: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off

  • Sept 02: Q2 goals draft published, Dan out
  • Sept 05: US Holiday (Labor day)
  • Sept 23: Q2 goals finalized
  • Oct 01: Start of Q2
  • October 10: US Holiday (Indigenous People's Day)
  • October 17-21: Offsite in Washington D.C.
  • October 31: Mukunda
  • October 28 - Nov 2 (ish) - Chad
  • November 24: US Holiday (Thanksgiving)
  • January 9-11: Dev Summit
  • January 12-13: All Hands

Team Business

[edit]

Time spent spreadsheet

[edit]

Rotating positions and absences

[edit]

Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

weeks of Aug 22 and Aug 29

[edit]

weeks of Sep 05 and Sep 12

[edit]

Actions from last meeting

[edit]

Scrum of Scrums

[edit]
https://phabricator.wikimedia.org/project/board/64/
Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

[edit]

Last week

[edit]


Other Team Business

[edit]

Offsite

[edit]

"Upgrade all mw* servers to debian jessie"

[edit]

Q2 (Oct - Dec) Goals

[edit]
Previously listed goals
[edit]
  • Differential
    • fix Jenkins tests, maybe
    • migrate android
    • not a goal
  • Malu
    • pause
  • LLB + MW + Extension deploys to scap3 ?
    • not a goal
New goal proposals
[edit]
  • Python software deployment via scap3 (Zuul + Nodepool)
    • think more on it (Tyler and Antoine), not a goal for now
  • CI Tech Debt
    • Determine long term plan for Nodepool
      • This needs to be more specific
    • Anything else?
    • think about how we use queues
      • split queue per branch (eg: security releases hitting multiple branches, 600 jobs), can make more run parallel
      • SWAT deploys: make the wmf branch go through as fast as it can
      • ""Review and adjust CI queues for more parallel operations"
  • MW deploy tech debt (Experiment/Stretch)
    • scap swat
    • ability to have multiple checks on MW deploys (in addition to logstash), eg swagger spec for MW (node endpoints checking)



Q1 goal/project check-in

[edit]

Phase out Ubuntu Precise

[edit]

Replace primary production Continuous Integration host (gallium) - task T95757

[edit]


Upgrade Phabricator database servers to Maria10/Jessie - task T138460

[edit]
  • Yes Done

Upgrade Beta Cluster database servers to Maria10/Jessie - task T138778

[edit]
  • Chad will review Dan's patches
  • Dan will coord with Jaime for $whenever_works_for_them

Move Gerrit off of ytterbium - task T125018

[edit]
  • Yes Done

Reduce Technical Debt

[edit]

Perform a technical debt analysis of software and services maintained by WMF Release Engineering - task T138225

Next steps:

  • Greg get the documentation documented and call it done (for this goal for this quarter)


Streamline deployments (long-lived branches)

[edit]

keyresult task:

  • Convert our production deployment strategy to use long-lived branches - task T89945

project view: https://phabricator.wikimedia.org/project/view/2117/

  • Tooling will probably be done
  • static asset conclusion might not be
  • scap swat coming along nicely
    • use gerrit rest api (has need features not avail over ssh)
      • will need some sort of shared account (with frequent credential rotation, potentially each deploy)
    • can use a .netrc right now
    • scraping Deployment calendar page is crappy

Non-Quarterly goal work

[edit]

SWAT deploy changes

[edit]
  • European SWAT deploys (task T137970
  • Future changes?
    • requiring a task associated with each change being pushed out?
    • Add all swatters to each swat window, stop segmenting based on their availability (worst case they get a ping when they're not online)

CI Scaling/Nodepool

[edit]

Browser tests

[edit]

Beta Cluster

[edit]
  • Long lived cherry pick stuff popping up again
  • Antoine got one out thanks to Brandon

Other

[edit]

DB Inconsistencies

[edit]

https://phabricator.wikimedia.org/T132416 and https://phabricator.wikimedia.org/T104459 (see also: https://www.mediawiki.org/wiki/Development_policy#Database_patches )


People status updates

[edit]

Antoine

[edit]

Last week

[edit]
  • Catch up on Nodepool incident - DONE
  • Migrate jobs back to Nodepool instance - Week of Aug 29
    • Ideally get quota raised
  • Figure out contint1001 network with ops / Tyler
  • done: clear out 3 weeks worth of mails
  • personal: learn how to play https://www.youtube.com/watch?v=d9i_zXmULyk
  • pet project: rake / rspec on puppet.git and tox for operations/software.git

This week

[edit]
  • Migrate jobs back to Nodepool instance. Chase to monitor wmflabs as we progressively switch back. Starting on Tuesday Aug 30th
  • Figure out contint1001 network with ops / Tyler
    • Haven't pushed for it. Faidon in vacations this week. -- solved with mark --> public IP
  • personal: working on Ukulele major chords. C, D, F, G done. Todo: A B E. Probably gonna buy a guitar.
  • Branch cut / train deploy - done

Chad

[edit]

Last week

[edit]
  • MW release today (finally)
  • Finally going to do DB consistency script -- per our 1:1 this shouldn't be so hard
  • Long lived branches (long may they ilve)

This week

[edit]
  • Diving into the DB consistency script. Doable, but hard :)
  • More long lived branches

Dan

[edit]

Last week

[edit]

This week

[edit]

Mukunda

[edit]

Last week

[edit]
  • Finish the `scap swat` tool which is taking shape nicely.
  • Propose Improvements to the scap remote execution api to make it easy to use from scap plugins
    • This could facilitate development of arbitrary scap checks which can be ran separately from deployments
    • Will discuss with Tyler during the deployments meeting and go from there.

This week

[edit]
  • Still finishing up scap swat
    • Hopefully, resolve the screen scraping debate
  • Start on `scap merge` branch management tooling

Tyler

[edit]

Last week

[edit]
  • Bugfix scap update
  • nodepool things

This week

[edit]

Željko

[edit]

Last week

[edit]

This week

[edit]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Release_Engineering_Team/Checkin_archive/20160829&oldid=3615572"