Wikimedia Release Engineering Team/SSD Sync Up/2019-08-06

= 2019-08-06 =

Last Time: 2019-07-30 Goals: https://www.mediawiki.org/wiki/Wikimedia_Technology/Goals/2019-20_Q1#Release_Engineering

Deployment Pipeline
Workboard

GOALS

 * Migrate restrouter
 * https://phabricator.wikimedia.org/T223953
 * (Stretch): MobileContentService
 * (Stretch): Preparatory MediaWiki config clean-up & static loading work
 * Moving configuration to static configuration (e.g., json files rather than a bunch of PHP)

TODOs from Last Time

 * ❌ TODO Need to fix pipelinelib's base image -- make a task
 * ❌ TODO Add a changelog/NEWS file to blubber


 * stalled brennen -- pipeline config validation
 * now unlicked

GOALS

 * POCs of GitLab, Argo, and Zuul3 systems; evaluate options
 * ✅ GitLab probably done
 * ```TODO``` Lars: write up GitLab PoC evaluation
 * ✅ Lars to set up meeting to discuss evaluation
 * Not done


 * Dan working on argo
 * parts of PoC working
 * worked with upstream to implement more features, this time for git artifacts (both merged but not released)
 * Moving with PoC running master
 * Argo "workflow" k8s resource that defines task to run -- source that from the target repo as a PoC
 * future work: transform .pipeline/config.yaml to workflow definition
 * Argo Events to report back to gerrit...maybe
 * Patched workflow controller instead of all components
 * Lars: Not necessary to run upstream releases for PoC


 * Brennen working on zuulv3
 * (Still) Highly dependent on train
 * James: is there an update about how easy it would be to migrate existing zuulv2 work to zuulv3?
 * Brennen: my sense is a lot of work but doable (based on talking to Antoine) -- I have a feeling that it is a good deal less work than migrating to another system
 * James: is the zuul migration like one we would end up building from scratch? Aka are we stacking the deck in favour of zuulv3 against argo and GitLab by doing this?
 * Tyler: We're likely to build with nodepool using a static driver to map to the current jenkins agents; if we were doing it "properly" it'd be based on k8s/whatever.
 * Dan: we should consider how difficult it is to migrate specific jobs in the evaluation
 * Brennen: Worth trying k8s with zuul / nodepool?
 * Lars: Pipelinelib written in groovy cannot be used outside of jenkins, so we should consider how difficult this will be to migrate
 * James: We have 423 JJB jobs + 8500+ of lines of layout for ~1400 specific repos -- although they're maybe not "good" -- migration is a chance to fix things
 * Dan: if pipelinelib were the new way to do things, we should evaluate how easy that is to implement in the different systems
 * Dan: if pipelinelib were the new way to do things, we should evaluate how easy that is to implement in the different systems


 * Document an implementable architecture for what we want in new CI
 * Lars: Will have new version hopefully for Thursday (didn't, still working on this)

GOALS

 * Migrate local-charts to deployment-charts
 * Instantiate testing and linting of helm charts
 * Preliminary work on a CLI for setup/management
 * Progressing slowly: https://gerrit.wikimedia.org/r/c/releng/local-charts/+/525563

TODOs from Last Time

 * mediawiki/core blubber
 * ✅ TODO add branch filter for this job to only trigger on master
 * Ready for review? https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/525972/
 * Blubber doesn't allow modification of files
 * runs: {insecurely: true}

Other Work

 * Porting from local-charts to deployment-charts
 * MediaWiki
 * ✅ Move mediawiki chart from local-charts to deployment-charts repo
 * https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/522584
 * Wasn't this blocked on? (no, we will update the chart after that work is done)
 * Move dev-images PHP image from php -S to Apache + php-fpm
 * https://gerrit.wikimedia.org/r/#/c/releng/dev-images/+/525842/
 * marked WIP
 * Update local-charts repository to use mediawiki chart from deployment-charts repo
 * Mukunda left feedback


 * Restbase
 * Move restbase chart from local-charts to deployment-charts repository
 * Add restbase chart (port from local-charts)
 * Update local-charts repository to use restbase chart from deployment-charts repo


 * Parsoid
 * ✅ Move parsoid chart from local-charts to deployment-charts repository
 * https://gerrit.wikimedia.org/r/c/releng/local-charts/+/527860 Update local-charts repository to use parsoid chart from deployment-charts repo


 * Local Dev CLI
 * Mukunda has updated patchset
 * Refactoring -- expect patchset update later today