Wikimedia Release Engineering Team/Deployment pipeline/2019-05-23

Jump to navigation Jump to search

Last Time[edit]


    • CI pipeline for it
      • Access to cassandra to perform tests can be simulated via sqlite since front-end will not have direct access to storage
    • actually deploying to k8s -- may not be possible due to hardware restrictions
      • Alex has benchmarking instructions and we can learn more after running those benchmarks
      • We may not have hardware to support
      • If we don't have hardware to support, it'd be next quarter
      • this front-end service should (hopefully) be pretty lightweight
      • TODO Marko to do benchmark + helm chart
    • Changeprop vs RESTBase front-end
      • Marko would like to focus on RESTBase front-end as opposed to changeprop
      • Agreed: we can't rush changeprop just to meet goals

Q1 things[edit]

  • serviceops
    • Calico this quarter, next quarter etcd v3
    • Joe more available, potentially
    • upgrading k8s itself
    • remaining q4 things
    • docker image upgrade pipeline
    • authorization model upgrade -- kube.config files vs cluster creds
    • moving scb services -- will need machines before end of q1 next year
  • services
    • changeprop
    • mcs -- maybe
    • live debugging work for services -- documentation/generalization needed (aotto has a nice wikipage)
  • Releng
    • .pipeline/config.yaml expansion
    • self-service stuff next quarter
    • lars cooking up future CI document


  • Secure publishing Jenkins
    • jobs on current Jenkins cluster trigger jobs on secure cluster
    • Limit access to Jenkins (including Read Only)
    • Minimum, secure Jenkins
  • New CI system Coming Soon™, but not soon enough, probably
    • Probably need to spec this out -- what do we need?
      • Last step of the pipeline on secure jenkins
      • docker-pkg, blubber, debs

TODOs from last time[edit]

  • Yes Done TODO what are our annual plans WRT to this project
    • Outcome: A secure and sustainable platform that empowers a thriving developer community with the ease of software-as-a-service tooling.
    • Key Deliverable: "Strengthen next generation testing and deployment pipeline to support more services, code health indicators, and local development
    • Projects
      • All applicable new and existing services (and partially MediaWiki) exist in the Deployment Pipeline
      • Actionable code health metrics are provided for code stewards
      • Provide a standardized local MediaWiki development environment
  • TODO docs for service docker container in beta




As Always[edit]