Wikimedia Release Engineering Team/Deployment pipeline/2020-11-24
2020-11-24[edit]
Always[edit]
- [1]
- Archive
- Task tree: https://phabricator.wikimedia.org/T198901
TODOs from last time[edit]
- Thcipriani: email fandom re #mw-on-k8s + ask for presentation
- Done you all should have it as of 2020-10-14
General[edit]
- How many meetings do we have about this? (for some value of "this")?
- Manager sync meeting
- RelEng Dev meeting
- RelEng Team meeting
- Dev meeting
- This meeting
- TODO: thcipriani cancel these(deployment-pipeline meetings), dduvall fold other dev meeting into this one
RelEng[edit]
- Shellbox open question: https://phabricator.wikimedia.org/T263295
- Coordination re:Flink?
- https://phabricator.wikimedia.org/T264006
- Which repo to run the pipeline on -- blubber works
- Have not run scaffold.sh yet
- combining scaffold with apache docs -- patchset may be incoming there
- AntiPattern: replicating upstream dockerfiles with blubber
- Created super-repo: https://gerrit.wikimedia.org/g/wikidata/query/flink-rdf-streaming-updater/+/refs/heads/master
- Orchestration framework: flink; bundled with application, on top of k8s
- Costly on merge: pulls flink from the internet and verifies with GPG
- Potential race condition: change merged, CI job to publish to archiva + blubber job that uses archiva
- Trigger currently is archiva job is manual, also want blubber job manual
- TODO: more clarification needed
Serviceops[edit]
- Kubernetes staging cluster slowly being setup in codfw with all new version components (calico 3.17, kubernetes 1.16)
- Revisiting a lot of assumptions and questions for setting up a kubernetes cluster again
- Impacts CI since CI talks to staging for helm test
- service-checker support for not verifying TLS certs (should help with kask integration tests) sent out, waiting for reviews