Wikimedia Release Engineering Team/Deployment pipeline/2019-02-14

From mediawiki.org

2019-02-14[edit]

Last Time[edit]

Current Quarter Goals[edit]

General[edit]

TODOs from last time[edit]

  • TODO: continuous deployment, what's missing? a k8s api token on contint1001
    • TODO thcipriani to make task
  • In progress In progress TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
    • is already a goal https://phabricator.wikimedia.org/T213090
    • blubber docs: https://wikitech.wikimedia.org/wiki/Blubber
    • liw to start a doc of questions per RelEng team meeting 
      • rename Continuous_Delivery page to Continuous_Deployment?
      • https://wikitech.wikimedia.org/wiki/Release_pipeline ?
      • it's not about making releases, though
      • Naming Things Is Hard™
      • JamesF: name the page after the program[me], not the specifc thing/instructions
        • +1
      • Joe: the idea of the pipeline has not percolated through the org, so we need to let others know and have an announce of the first version Proposal: at the end of this quarter once the docs are all in place -- set a hard date for these things -- thoughts on date? 1 April?
      • Jeena: wasn't in SF -- are we doing more than blubberoid?
      • Joe: we talked about what was still missing from the pipeline to allow more folks to use the pipeline -- user friendliness, etc
      • James_F: I agree with Joe: we should have a heads-up email and have more docs than we currently have -- we want to say in the email that we don't want people to try to migrate right away
      • Joe: although any new service for production should probably use the pipeline -- need to start pushing on this project otherwise it'll be us just moving things in the background -- email in the coming week
      • TODO: Joe & James_F to work on eventual 2019-04-01 email?
      • Dan: we haven't tested the pipeline with stuff that needs cassandra and the like for `helm test`
      • Joe: I'm worried if people need a mysql db during test, but I don't think people need that for test
      • Dan: we haven't tested that kind of setup
      • Alex: also not worried about it
      • LIW: worried about it since it is an unknown -- we should start with smallest possible thing, i.e. a microservice that is stateless and not promise new things will work until we've tried them
      • Page name: Deployment_Pipeline
      • Alex: On wikitech?
      • JamesF: it's on wikitech, let's leave it
      • Joe: it's a mess
      • JamesF: officially documentation is meant to be on mw.org, but nobody does that
      • https://wikitech.wikimedia.org/wiki/Deployment_pipeline now exists, https://wikitech.wikimedia.org/wiki/Continuous_Delivery has been deleted.
      • Release_pipeline is renamed to Continuous_Deployment

RelEng[edit]

  • Local charts for dev
    • https://gerrit.wikimedia.org/g/releng/local-charts
      • deploys mediawiki + database + parsoid
      • has sshfs working for local directory
      • SRE security concerns?
      • Joe: happy about this, want to ensure that mediawiki-docker converges with our work, there are definitely tons of things to add from prod
      • Jeena: current setup doesn't copy in files, just uses local directory. There may be opinions about namespaces, etc
      • Alex: will take a look at it, will give you some feedback this week, looks good, bikeshedding about naming
      • Jeena: have you all had issues using minikube locally? Getting it to run and getting it to start? Will this be a barrier for developers?
      • Alex: I have had a lot of problems that result from me fiddling with difficult stuff to debug for developers, we should be documenting and helping people resolving those issues. Many of these things are not technical things.
      • fselles: I have had LOTs of problems with minikube. I think it works more or less in mac, but not in linux. Charts themselves look good, but we can chat later about that. We should discuss about alignment of the docker image for prod/development.
      • Jeena: this was one of the security problems I was curious about, i.e., the base docker image is very basic
      • Alex: images with cryptominers in dockerhub have been a problem in the past
      • Dan: shared volume is a concern. Minikube sets up mount to ~ rw by default which is terrible.
      • Joe: we should use an image we've built for a base eventually that is used for prod and development. Development might have xdebug for instance. It's good if people start using MW images to use helm. There are lot of problems with how we treat configuration for mediawiki.
      • Dan: not to mention dependency management of extensions and skins
      • Joe: php extensions should be setup, but code 

Serviceops[edit]

  • Eventgate-analytics deployed in staging
    • Production + LVS next week
  • citoid changes for the pipeline merged, thanks @Marko+Tyler
  • cxserver changes up as well, need to ping Kartik
    • thcipriani: TODO will look at integration change
  • helm charts being worked on already for both

Services[edit]

As Always[edit]

Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Release_Engineering_Team/Deployment_pipeline/2019-02-14&oldid=3615673"