Wikimedia Technology/Annual Plans/FY2019/TEC3: Deployment Pipeline/Goals

=Program Goals and Status for FY18/19=

TEC3 Deployment Pipeline
 * Goal Owner: Greg Grossmeier
 * Program Goals for FY18/19: We will streamline and integrate the delivery of services, by building a new production platform for integrated development, testing, deployment and hosting of applications. Wikimedia developers experience a tooling parity between our Continuous Integration (CI) and production environments which enables them to release code more frequently by continuously reducing risk.
 * Annual Plan: TEC3 Deployment Pipeline
 * Primary Goal is Knowledge as a Service: Evolve our systems and structures
 * Tech Goal: Sustaining





= Q1 Goals =

Outcome 1 / Output 1.1
Continuous Integration is unified with production tooling and developer feedback is faster
 * Convert current CI builds to use the new tooling (Blubber).

Dependencies on: SRE team

Goal(s)

 * Move verify stage from Minikube to CI k8s namespace in production context

Status
July 2018

August 10, 2018
 * Discussed that work on a patch is still ongoing, need to refactor the pipeline job to the new namespace. This will be a change to the existing service but will need to be refactored when we get to the shared library.

September 14, 2018
 * This is now ✅!



=Q2 Goals =

Outcome 1 / Output 1.2
Continuous Integration is unified with production tooling and developer feedback is faster


 * Setup test execution time profiling with a report, make a prioritized list of improvements to how tests are run.

Goal

 * Formalize the collection of CI infrastructure and tooling metrics -

Status
October 2, 2018
 * This is now

November 2018
 * dduvall gave a presentation Monday looking at CI performance percentiles
 * Work continues on automating the collection of these metrics.
 * Work continues on automating the collection of these metrics.

December 2018
 * Discussed...

Outcome 2 / Output 2.3
Deployers have a better assessment of risk with each deploy


 * Improve our incident response, post-mortem, and follow-up management tooling.

Goal

 * Develop set of metrics to assess incident reports/post mortems.

Status
October 2, 2018
 * This work has not yet been started at this time

November 2018
 * This is now in-progress with Zeljko's analysis of the past year's worth of incident reports.

December 2018
 * Discussed...

Outcome 3 / Output 3.1
Deployments happen through percentage based stages (eg: canaries, 10%, 100%)


 * Migration of services currently on our "shared service cluster" into Kubernetes deployments with staged rollout

Primary teams: Service Operations, Release Engineering, Dependencies on: ___________

Goal(s)

 * Adopt more services into Deployment pipeline
 * Migrate graphoid to the Deployment pipeline
 * Deploy zotero v2 to the Deployment pipeline
 * Deploy blubberoid
 * Reprise the work on the logging infrastructure

Status
October 2, 2018
 * This is now

November 2018
 * Deploy zotero v2 to the Deployment pipeline
 * Currently living in k8s staging
 * Plan to go live next week
 * Deploy blubberoid
 * liw working on changes to internal data structuring as a prerequisite to creating OpenAPI spec required for pipeline — on track.

December 2018
 * Discussed...