Wikimedia Release Engineering Team/SSD Sync Up/2019-07-16

= 2019-07-16 =

Last Time: 2019-07-02 Goals: https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2019-20_Q1#Release_Engineering

Deployment Pipeline
Workboard


 * Last week's Tech Talk by Alexandros: https://www.youtube.com/watch?v=i0FTcG7PxzI

GOALS

 * Migrate restrouter
 * https://phabricator.wikimedia.org/T223953
 * (Stretch): MobileContentService
 * (Stretch): Preparatory MediaWiki config clean-up & static loading work
 * Moving configuration to static configuration (e.g., json files rather than a bunch of PHP)

TODOs from Last Time

 * ✅ thcipriani -- Pipeline image build cleanup for contint1001 (not that it's important anymore :))
 * ✅ contint1001 store docker images on separate partition or disk
 * thcipriani -- Base Blubber policy file for CI
 * blocked on deployment
 * TODO thcipriani to ping alexandros
 * brennen -- pipeline config validation
 * ❌ TODO Pipeline docs

GOALS

 * POCs of GitLab and Zuul3 systems; evaluate options
 * Document an implementable architecture for what we want in new CI

TODOs from Last Time

 * TODO Talk to SRE about Zuul v3 needs
 * Needed: conversation about phasing out Zuulv2
 * Meeting for zuulv2 thursday, TODO ping greg about antoine going
 * Lots of ansible to setup Zuulv3
 * 'TODO Discuss with SRE Thursday

Other Work

 * v2 of CI arch doc: https://docs.google.com/document/d/1EQuInEV-eY_5kxOZ8E1qEdLr8fb6ihwOD9V_tpVFWuU/edit
 * Only a few comments received, no significant changes. v3 someday, but not immediately.
 * No news, but s'ok


 * Lars is hacking up new components around GitLab (mostly independent of what CI engine we choose, GitLab is just the first one to be tried)
 * going slowly this week, due to train
 * All but one component done (the controller)
 * other components tested manually
 * prototype: http://git.liw.fi/wmf-ci-arch/tree/
 * gitlab-ci uploads build artifacts to the artifact store
 * Testing with http://git.liw.fi/heippa/tree/.gitlab-ci.yml -- simulates the commit stage
 * uses the ick artifact store
 * VCS Worker and Deployment Worker in api.py
 * Deployment via ansible currently (git clone)
 * Aim: get controller working, then get a volunteer -- i.e., build blubber for a prototype

GOALS

 * Migrate local-charts to deployment-charts
 * https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/522584
 * Did not handle any concurrency with 30 requests/sec
 * Didn't use too many resources
 * Blocked on: Brennen working on moving mw-base image to apache
 * Instantiate testing and linting of helm charts
 * Preliminary work on a CLI for setup/management

TODOs from Last Time

 * mediawiki/core blubber: Jam a shell script into the builder?
 * TODO: the actual jamming of shell script
 * Haven't been able to make that work
 * Job to run docker-pkg nightly to make up-to-date dev image
 * thcipriani/brennen pair Friday on working on this

Other Work

 * Porting from local-charts to deployment-charts ( https://phabricator.wikimedia.org/T224935 ):
 * Add restbase chart (port from local-charts) TODO Thursday meeting
 * Move dev-images PHP image from php -S to Apache + php-fpm
 * See also https://gerrit.wikimedia.org/r/c/integration/quibble/+/516729/