Wikimedia Release Engineering Team/SSD Sync Up/2019-02-19

Last Time: 2019-02-12

TODOs

 * ✅ TODO liw to create wikipage with questions cf: https://phabricator.wikimedia.org/T213090
 * https://wikitech.wikimedia.org/wiki/Deployment_pipeline has an initial list of questions
 * ✅ TODO thcipriani to review https://gerrit.wikimedia.org/r/c/integration/pipelinelib/+/480689
 * In production now... I think

Discussion

 * Lars: how closely should we be working with service developers when developing the pipeline?
 * We should aim at developing something the service developers will use and like
 * A good way to achieve that is to apply continuous deployment and tight iterations with developer feedback on the pipeline itself
 * Who are / should be the people who give us feedback?
 * Marko, Andrew Otto, Pchelolo, Dan
 * Andrew Otto feedback: https://phabricator.wikimedia.org/T211247#4860256
 * Lars: whatever is going into the pipeline next expands the list of users from whom we solicit feedback to eventually expand to encompass all service developers
 * Lars: do this iteratively via adding new service to the pipeline

---


 * Lars: What is the current status of the pipeline? Can we code review +2 a change to Blubberoid and have it end up in production?
 * curl -T blubberoid.json -H "Content-Type: application/json" https://blubberoid.wikimedia.org/v1/test # Blubberoid runs in some form of production, is that K8s?
 * Tyler: deployment is missing
 * At this point deployment is running scap helm upgrade
 * Lars: Should services, once hitting production, be deployed automatically?
 * Tyler: we need a K8s token on a CI machine to do the deployment automatically
 * TODO thcipriani to followup to make task for adding token for auto-deployment
 * Tyler: we should have automated deployment of at least Blubberoid by the end of this quarter

---


 * thcipriani: more .pipeline/config.yaml talk
 * https://phabricator.wikimedia.org/T209106
 * https://phabricator.wikimedia.org/T209106#4958830
 * Antoine: for CI we have standardized to a directory named "log". Exposed in jobs/containers via the LOG_DIR environment variable. It used to be set via "$WORKSPACE/log".  With the CI Docker containers it is usually "/log" (and hence we do mkdir -p log docker run --env LOG_DIR=/log" -v "$(pwd)/log":/log).
 * https://phabricator.wikimedia.org/T210267#4958266

---


 * Jeena: made some changes to local-charts for customizations, started work on RESTBase
 * Lars: where are we with local dev, can I setup something to develop Wikimedia?
 * Jeena: there are helm-charts in at https://gerrit.wikimedia.org/g/releng/local-charts this allows you to setup MediaWiki, Parsoid, and MariaDB
 * Brennen: I have scripts for pre-requisites and I'll try to get a changeset for that today
 * Lars: I will be happy to alpha/beta test instructions
 * Brennen: wait until I submit instructions and then work from there
 * Jeena: will let you know when ready

---


 * James: does a config change mean that there is a new image? (Q. from Eric E. in IRC)
 * thcipriani: currently that would mean a new image, new pipeline build
 * Jeena: isn't that part of a deployment? Would that require a new image?
 * James: there are pros and cons to both, for instance mw-config could make everything fall over
 * Lars: I think we need the same images for local/prod/staging and the way for that to work is to keep all configuration somewhere else
 * thcipriani: there is a .pipeline/config.yaml that would affect what the pipeline produces that creates a new docker image, there is other configuration in helm-charts that doesn't.
 * Antoine: need to run and e2e test when helmcharts change?