Wikimedia Release Engineering Team/Checkin archive/20160822

= 2016-08-22 =

Vacations/Important dates
How to do it: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off
 * Sept 02: Q2 goals draft published
 * Sept 05: US Holiday (Labor day) - Greg
 * Sept 23: Q2 goals finalized
 * Oct 01: Start of Q2
 * October 10: US Holiday (Indigenous People's Day)
 * October 17-21: Offsite in Washington D.C.
 * October 31: Mukunda maybe?
 * October 28 - Nov 2 (ish) - Chad
 * November 24: US Holiday (Thanksgiving)
 * January 9-11: Dev Summit
 * January 12-13: All Hands

Time spent spreadsheet

 * Week 33 - https://docs.google.com/spreadsheets/d/1IrwGPdTDZ6H8x9Mf5dmCYlkK4hZ8sbUSLODEM4cFc4g/edit#gid=734323691

Rotating positions and absences
Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

weeks of Aug 8 and Aug 15

 * Train: Mukunda
 * wmf.14
 * wmf.15
 * SoS: Chad
 * https://phabricator.wikimedia.org/E155/19
 * https://phabricator.wikimedia.org/E155/20
 * Out:
 * Zeljko: July 25 - Aug 15
 * Antoine: July 30 - Aug 21

weeks of Aug 22 and Aug 29

 * Train: Antoine
 * wmf.16
 * wmf.17
 * SoS: Tyler
 * https://phabricator.wikimedia.org/E155/21
 * https://phabricator.wikimedia.org/E155/22
 * Out:

weeks of Sep 05 and Sep 12

 * Train: Chad
 * wmf.18
 * wmf.19
 * SoS: Mukunda
 * https://phabricator.wikimedia.org/E155/23
 * https://phabricator.wikimedia.org/E155/24
 * Out:
 * Sept 05 (Monday): US Holiday (Labor day)

Actions from last meeting

 * TODO: Chad - lay out ideation on the LongLivedBranches task to then get Timo to review ( https://phabricator.wikimedia.org/T140921 )

Scrum of Scrums

 * https://phabricator.wikimedia.org/project/board/64/
 * Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

 * Blocking
 * Blocked
 * Contint networking: https://phabricator.wikimedia.org/T140257#2553490
 * Updates
 * Bugfix release of scap
 * Bugfix release of scap

Last week ... Ooops, didn't happen

 * Blocking
 * Blocked
 * Consolidate, remove, and/or downsize Beta Cluster instances to help with Purge_2016 - https://phabricator.wikimedia.org/T142288
 * specifically: -conftool (Ops?/Joe?), -conf3, (Analytics-ops?/Ottomata?), -kafka (Analytics-ops?), OCG (-pdf1 and -pdf2, but only 1 seems to be used?)
 * Updates
 * New SWAT window schedule starting Aug 22nd
 * See: https://wikitech.wikimedia.org/wiki/Deployments#Week_of_August_22nd
 * And: https://wikitech.wikimedia.org/wiki/SWAT_deploys#The_team
 * https://phabricator.wikimedia.org/T137970
 * https://phabricator.wikimedia.org/T137970

Offsite

 * Rachel is working on venue options still; some good options so far
 * what do you want to talk about? Fill this out/vote on ideas:
 * https://etherpad.wikimedia.org/p/releng-offsite201610-proposedtopics

Q2 (Oct - Dec) Goals

 * That time again!
 * Out of date: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q2
 * Differential?
 * fix Jenkins tests, maybe
 * migrate android
 * not a goal
 * MALU
 * pause
 * LLB + MW + Extension deploys to scap3 ?
 * not a goal
 * Python software deployment via scap3 (Zuul + Nodepool)
 * think more on it (Tyler and Antoine), not a goal for now


 * CI Tech Debt (meta goal)
 * #1
 * #2
 * Experiment/Stretch: MW deploy tech debt:
 * scap swat
 * ability to check something???

Replace primary production Continuous Integration host -

 * NEXT: https://phabricator.wikimedia.org/T139771 - "Identify metric (or metrics) that gives a useful indication of user-perceived (Wikimedia developer) service of CI"

Reduce Technical Debt
Perform a technical debt analysis of software and services maintained by WMF Release Engineering -

Hot spots:
 * MW 3rd party release, l10nupdate, (and probably swat) tooling
 * Nodepool

Next steps?

Streamline deployments (long-lived branches)
keyresult task: project view: https://phabricator.wikimedia.org/project/view/2117/
 * Convert our production deployment strategy to use long-lived branches -

SWAT deploy changes

 * European SWAT deploys (
 * NEXT: this week :)
 * Future changes?

CI Scaling/Nodepool

 * Catch Antoine up on Nodepool state

Browser tests

 * https://phabricator.wikimedia.org/T142600 "Various browser tests failing with MediawikiApi::LoginError"
 * Gergo is asking for help figuring out next steps
 * Zeljko is looking into it

DB Inconsistencies

 * Figure out how to help Jaime with the DB schema inconsistencies issue:
 * https://phabricator.wikimedia.org/T132416 and https://phabricator.wikimedia.org/T104459 (see also: https://www.mediawiki.org/wiki/Development_policy#Database_patches )
 * What can we do in CI to help prevent, mostly?
 * Question: Does Chad need any help (other than time)?

Last week

 * Vacation

This week

 * Catch up on Nodepool incident
 * Migrate jobs back to Nodepool instance
 * Ideally get quota raised
 * Figure out contint1001 network with ops / Tyler
 * done: clear out 3 weeks worth of mails
 * personal: learn how to play https://www.youtube.com/watch?v=d9i_zXmULyk

Last week

 * DB consistencies ....
 * Long lived branches
 * CI outage incident report

This week

 * MW release today (finally)
 * Finally going to do DB consistency script -- per our 1:1 this shouldn't be so hard
 * Long lived branches (long may they ilve)

Last week

 * Start poking at MW-Vagrant jessie base image https://phabricator.wikimedia.org/T136429
 * Migrate deployment-prep to jessie https://phabricator.wikimedia.org/T138778
 * Follow up on mw-selenium/browser tests dependency updates re https://phabricator.wikimedia.org/T129483

This week

 * Start poking at MW-Vagrant jessie base image https://phabricator.wikimedia.org/T136429
 * Migrate deployment-prep to jessie https://phabricator.wikimedia.org/T138778

Last week

 * Mediawiki train: 1.28.0-wmf.15
 * https://phabricator.wikimedia.org/T140971
 * LLB
 * Continue working on the `scap swat` tool, more experimentation with gerrit api
 * https://phabricator.wikimedia.org/T142880
 * Made good progress on this: https://phabricator.wikimedia.org/P3854
 * Deploy release-tools repo with scap3 (if time allows)
 * https://phabricator.wikimedia.org/T142588
 * `scap swat` is going to live in mediawiki-config repo, so this is no longer needed.

This week

 * Finish the `scap swat` tool which is taking shape nicely.
 * Propose Improvements to the scap remote execution api to make it easy to use from scap plugins
 * This could facilitate development of arbitrary scap checks which can be ran separately from deployments
 * Will discuss with Tyler during the deployments meeting and go from there.

Last week

 * Bugfix scap update
 * Try to stay on top of gallium things
 * Incident report for CI outage!

This week

 * Bugfix scap update
 * nodepool things

Last week

 * Catching up

This week

 * https://phabricator.wikimedia.org/T142600 Various browser tests failing with MediawikiApi::LoginError (halp plz)
 * https://phabricator.wikimedia.org/T139613 Run language screenshots script for VisualEditor in Jenkins
 * https://phabricator.wikimedia.org/T143563 Upload language screenshot files using commons_upload ruby gem (halp plz)
 * https://phabricator.wikimedia.org/T139247 Analyze (and share analysis of) the browser testing feedback survey