Wikimedia Release Engineering Team/Deployment pipeline/2017-03-14

= 2017-03-14 =

Who's here: Marko, Tyler, Dan, Antoine, JR

Last Time

 * Discussed when to build containers — as frequently as possible
 * Secrets — future discussion
 * e2e tests before things merge and are deployed to staging
 * Evaluate deployment-pipeline-jenkins plugin
 * Build testing k8s setup in ci-staging
 * Install Kubernetes Continuous Deployment Pipeline plugin on ci-staging-jenkins

k8s puppet stuff

 * Got help from Chase and Yuvi
 * On ci-staging labs project
 * Lots of dependencies using Puppet manifests (split between toolabs and prod)
 * Docker registry
 * Etcd

How it works

 * Spin up new Jenkins on k8s per project
 * gets environment from k8s create yaml file JENKINS_ADMIN_USER, JENKINS_ADMIN_PASSWORD, POD_IP, POD_NAMESPACE, POD_NAME, git repo, git private key
 * k8s yaml for jenkins service
 * This deployment uses a base container
 * There are 64 layers according to
 * Spawn a jenkins master for each project
 * Installs and pins Jenkins plugins via a shell script + curl
 * jobdsl is the big one
 * Contains scripts that create  if it doesn't exist: http://ci-staging-jenkins.wmflabs.org/ci/job/seed-job/
 * seed-job runs a bunch of groovy scripts which themselves contain a bunch of HEREDOC'd groovy scripts
 * https://github.com/thcipriani/deployment-pipeline-jenkins-plugin-setup/tree/master/jobdsl
 * seed-job makes more jobs: http://ci-staging-jenkins.wmflabs.org/ci/
 * Pulls down repo defined in k8s
 * build.bitesize, environments.bitesize, containers.bitesize
 * builds jobs based on those files + inception groovy scripts
 * builds jobs based on those files + inception groovy scripts

Random Thoughts

 * Lots of thought went into this, it's well automated
 * There is a lot going on in their Docker base container (exhumed from base container)
 * We have a lot of repos (1080 configured in CI as of March 14th). A Jenkins for every repo is a lot of Jenkinses
 * "enterprise class"
 * Some good ideas here

For next time

 * Evaluation document
 * k8s infra


 * Specifications for configuration
 * Outline for what we need from specification


 * TODO releng team meeting bring up specifications/requirements for deployment pipeline for next quarter

= As Always =
 * Release Pipeline Workboard
 * Meeting notes