Wikimedia Release Engineering Team/Deployment pipeline/2019-05-09
Last Time[edit]
General[edit]
—
TODOs from last time[edit]
- TODO what are our annual plans WRT to this project
- Template:Stalled TODO various attack vectors document to start
In progress TODO: support documention like the one tyler did for the portal and pipeline/helmfile and deployment
- https://wikitech.wikimedia.org/wiki/Deployment_pipeline now exists, https://wikitech.wikimedia.org/wiki/Continuous_Delivery has been deleted.
- reached out to technical writer, in very preliminary stages for next quarter
- think about other documentation needs beyone Deployment_pipeline main page
- Anything we can do here without stepping on toes?
- preliminary research happening
- work with srodlund about what the portal is
- starting on the real work of the documentation next week
- TODO docs for service docker container in beta
Done TODO: announce email
- https://lists.wikimedia.org/pipermail/wikitech-l/2019-April/091976.html
- Feedback?
- Nothing major, mostly queries about MediaWiki containerisation.
- Some talk about Migrate Beta cluster services to use Kubernetes
RelEng[edit]
- pipelinelib review meeting after this -- hopefully make progress towards .pipeline/config.yaml
- Docs are published for pipelinelib
- deployment to staging -- we deploy to ci namespace on neon, but we also run helm delete
- to deploy to staging in the general case: deploy to staging with a namespace that is the same name as the service
- problems and edgecases with this are repos with multiple services that need to be deployed; e.g., eventgate-ci
- pipelinelib could potentially allow for multiple services to be deployed from a single repository; i.e., .pipeline/config.yaml
- for helm test, consider using:
- Local development images using blubber:
Serviceops[edit]
- kask got swagger/openapi spec. Merge pending
- how do we do this for staging since kask needs cassandra?
- setup vms? we have hardware possibly...
- citoid has similar problems, but if we could pass a proxy to the helm test
- Dan: we could provide configuration to pass to helm install or helm test in the pipeline
- something to think about: things we have no plan to move to k8s (e.g., mysql) how do we deal with those in staging -- resource problems in general
- maybe possible to work around the helm test part with some hack
- e.g., restbase works around this by heavily mocking responses
- this may be harder with MediaWiki
- quibble currently works around this
- MediaWiki has this problem in the testrunner a bit i.e., unit tests fail if it tries to write to the database -- other things write to sqlite
- stress testing for projects
- staging, in theory, in the future should be able to be used for some portion of stress testing
- two problems: (1) providing servers for data (2) providing the data -- the 2nd one will be difficult
- "staging"
- is this a persistent k8s pre-production environment
- is this a environment where we could mock important services
- these two things serve two different testing strategies
- some work on making kask more easily working on minikube, courtesy of incubator/cassandra chart
- Template:Started Tech talk scheduled from July 10
- new registryha live, no complains so far
Services[edit]
—