Wikimedia Release Engineering Team/Deployment pipeline/2019-04-11

Last Time

 * 2019-03-28
 * Archive

General

 * Q4 Tracking tasks
 * Thank you Alexandros for creating

TODOs from last time

 * ✅ TODO merge/deploy changeprop patch for zuul https://gerrit.wikimedia.org/r/#/c/integration/config/+/496387/


 * ❌ TODO various attack vectors document to start


 * TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
 * https://wikitech.wikimedia.org/wiki/Deployment_pipeline now exists, https://wikitech.wikimedia.org/wiki/Continuous_Delivery has been deleted.
 * reached out to technical writer, in very preliminary stages for next quarter
 * think about other documentation needs beyone Deployment_pipeline main page

send email about email
 * TODO: Joe & James_F to work on eventual 2019-04-01 email
 * Beware: announces on 04/01 can be considered an April's fool [and James is out on 1 April itself. <-- this is an april fool's joke for sure]
 * postponed until there are docs ~2019-05-xx
 * Draft: https://etherpad.wikimedia.org/p/LZKuNSFpM4iKbuHkdPRf
 * Shall we send one soon, more a "hey, what do you think?" e-mail?
 * a preliminary email for people to be able to comment on work so far and how that jives with their needs
 * wikitech-l, engineering-l, services@lists.wm.org
 * what is the call to action?
 * Is there documentation you expect to see?
 * Poke around this project?
 * Overview
 * Who cares about this email?

RelEng

 * Additional NPM packages during container image build
 * Looks like optional dependencies is where this landed?
 * packages included as optionalDependencies
 * once services updated to newest servicerunner it should be automatic
 * first deployment of this happened today
 * Latencies and duration of GCs are wrong until https://phabricator.wikimedia.org/T220709 is fixed

build run publish deploy
 * Implementing `.pipeline/config.yaml`
 * https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/502917
 * https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/502918
 * a single pipeline can operate on two seperate paths working on two seperate repos
 * you can publish different docker images for different services
 * stages have a flow:
 * each stage step exports binding with the results of a step, subsequent steps can access ancestor bindings
 * Mulitple images with multiple charts with a single repo
 * config.yaml: orchestrating steps for CI
 * can specify different blubber configs to different services
 * refers to blubber variants internally
 * blubber.yaml: creates build manifests
 * Timelines: start experimenting possibly next week
 * experiment with ORES possibly(?) since they need this functionality
 * Might use for restbase possibly
 * MCS split into MCS / PCS (?)

Serviceops

 * ✅ cxserver training. Deployment done today
 * ongoing Upgrades to infrastructure ongoing, kubernetes 1.12.7 next week
 * started Started work on kask+wikidata termbox SSR
 * ongoing Need to resume work on changeprop
 * started A tech talk proposal (late june per organizers' request)

Services

 * ChangeProp is ready to go.
 * RB split is coming
 * MCS/PCS are eager to move to node10. Maybe for Q1?

= As Always =
 * Release Pipeline Workboard
 * Meeting notes