Wikimedia Release Engineering Team/Checkin archive/20160229

= 2016-02-29 =

Vacations/Important dates
How to do it: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off
 * March 4th - strategic narrative annual plan due
 * March 11th - draft Q4 (April 1st - June 30th) goals due
 * March 11th - Željko - probably working half day, traveling to a conference in the afternoon
 * March 25th Friday - Tyler
 * March 28th - Antoine && Željko - local holiday (Easter Monday)
 * March 31st - April 3rd : Hackathon in Israel
 * April 1st - Q4 goals published
 * April - Antoine: holidays one of the two first weeks
 * May 6th Friday - Antoine
 * May 9-Mid June-ish?: Greg - paternity leave - exact dates TBD
 * May 17-(?): Dan - paternity leave :D
 * Late May - draft Q1 (July 1st - Sept 30th) due
 * May 30: US HOLIDAY - Memorial Day
 * June (mid): Chad - EDC
 * June 22nd - 28th : Wikimania in Italy
 * July 1st - Q1 goals published
 * July 1st – Annual Plan, Budget, Risks Document and FAQ are posted
 * August: France holiday - because french. :)
 * January 2017 : Dev Summit + All Hands (presumably)

Train conductor
Week of ...
 * Feb 29: Chad
 * Mar 7: Mukunda
 * Mar 14: Mukunda
 * Mar 21: Tyler - Code freeze, due to the eqiad -> codfw switch over (announcement coming)`
 * So we need to make sure Mar 14th week is super stable.

Scrum of Scrums representative
(bad time for EU folks) Dan, Tyler, Chad, Mukunda Week of ...
 * Feb 29: Tyler
 * Mar 7: Chad
 * Mar 14: Chad
 * Mar 21: Mukunda

CI point person

 * reassess later

Actions from last meeting

 * TODO - No One Yet: investigate carbon aggregation of stats >1 month old behavior
 * ACTION: Antoine to create a task
 * Overdue

New vs Maint time spent

 * Q3: https://docs.google.com/spreadsheets/d/1LJDc5W2Mlpzc0L1i7WyPwWU8AgWMn0fXRuNEEmg1EMU/edit#gid=0
 * Lots of "Nonsense stuff" past few weeks...

Scrum of Scrums

 * https://phabricator.wikimedia.org/project/board/64/
 * Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

https://phabricator.wikimedia.org/T111259 closed
 * Went under the radar. Turns out the root cause got fixed by @Maxsem in MediaWiki core a few days/weeks before ;-)


 * Blocked on ops (sorta):
 * Refactor phabricator module in puppet to remove git tag pinning behavior https://phabricator.wikimedia.org/T125851
 * (resolved) scap::target should use scap's debian package instead of trebuchet https://phabricator.wikimedia.org/T127215

Other Team Business

 * https://phabricator.wikimedia.org/phame/blog/view/1/ - if anyone feels like writing a blag about something.

Annual Planning

 * Spreadsheet (team only) - https://docs.google.com/spreadsheets/d/1GBokh9zeO5vflAAZLjMuagV4FeFQHCFrApjs_KXNZ7o/edit#gid=0
 * Planning worksheet: https://docs.google.com/spreadsheets/d/1ZsB0RCoZD3a6qKsX-qkCpA3HK81mNrZYI3GXeiuzzI0/edit#gid=0

Reduce CI Wait time

 * KPI: https://grafana.wikimedia.org/dashboard/db/releng-kpis?panelId=2&fullscreen
 * Migrate remaining CI jobs to Nodepool -
 * php composer (Zend and HHVM) -
 * as many miscellaneous jobs as possible -
 * Migrate Jenkins to Jessie -


 * mediawiki-core-npm migrated to mediawiki-core-npm-node4.3 (ie Nodepool)
 * mediawiki services jobs being migrated. Mathoid pending on devs, Graphoid on some puppet work
 * Nodepool needs a few cherry picks, hopefully this week
 * Nodepool upgrade blocked on python-shade

Consolidate deploy tools

 * Migrate MediaWiki to scap3 -
 * Q2 Quarterly Goal hold over: Migrate all Service team owned services and MW deploys to scap3 - https://phabricator.wikimedia.org/T109926

HTTP server tests for scap
 * probably have a winner now
 * smart git http server in python, forked and modified
 * good performance over the other ones we tested
 * https://github.com/thcipriani/gpack/pull/1 - lets get this repo into phabricator.
 * Should we just merge it directly into scap? it's MIT licensed and very self-contained.
 * Tyler: I'd vote merge into scap unless there are folks with specific objections

Updated package with bugfixes for AQS (permissions issues mostly)
 * AQS is ✅
 * Marko was on vacation last week, will check in on what's next with him today

Working on a packaging pipeline with Antoine
 * using Jenkins etc (http://jenkins-debian-glue.org/ shell wrappers around cowbuilder / git build-package ) + aptly for publishing
 * working on setting up Beta to automatically deploy packages from a repo

Goal: make it reusable. Will be an intensive for ops to migrate the .deb repositories to Differential.

Differential Migration

 * https://etherpad.wikimedia.org/p/diffuerential-weekly
 * Integrate Differential with our Continuous Integration infrastructure -
 * Shepherd the RFC -
 * Discuss at WikiDev16 -
 * Garner early adopter projects (goal: 1 project per WMF "team")

Browser tests cleanup of red tests
Going well, the red line is going down without decreasing the green line much, see https://integration.wikimedia.org/ci/view/BrowserTests/view/-Dashboard/ The creation of jobs is now configured in the repos themselves via a yaml file (!!!), mostly there

beta cluster
Clean up job queue (200k stall jobs) https://phabricator.wikimedia.org/T128357 Maybe need a second jobrunner instance or run more jobs on the existing one