Wikimedia Release Engineering Team/CI Evaluation Cabal/2019-09-17

Argo

 * Writeup from: https://phabricator.wikimedia.org/T229246

tl;dr: I think argo could work for us, but there are a few considerations/caveats


 * Maintenance - not super clear how many people are involved long term
 * Two contribs from Intuit
 * MinIO -- Argo works with a bunch of different artifact stores
 * Some kind of artifact store is probably necessary, stuff is ephemeral in general
 * Build logs are separate from artifact store - k8s pod logs
 * Argo Events/Argo Core


 * Knative - relies 100% on Kubernetes
 * Deeply embedded into whatever k8s it's installed inside
 * Uses CRDs -- like a resource (e.g., pods, resources) -- but your own design -- use the CRUD api and install things onto the controller to act on resources
 * CRDs use k8s workload scheduing
 * CRDs
 * Workflows
 * Listens to changes, kicks off pods to execute workflows
 * Events
 * consumes external events and kicks off processes within k8s
 * e.g., setup a gateway that exposes a webhook endpoint


 * Q: Could this be installed in its own namespace in an existing cluster?
 * Yes; it responds to things cluster-wide...
 * I think it'd work in our existing cluster.


 * Pipelinelib
 * an unknown, but a possibility
 * Argo Events to kickoff a TBD (possibly a new CRD to be definied)


 * Gateway can override anything in the spec file
 * Argo expects a bunch of inputs and outputs
 * each task operates on inputs that are outputs of previous tasks
 * Parameters == name/value pairs, operate without artifact store
 * Artifacts == artifacts in the artifact store


 * Bus Factor
 * 2 core contributors
 * Both work at Intuit
 * it's low


 * Release Engineering invests in Go is a precondition of this
 * But RelEng will probably invest in Go anyway


 * Rollout
 * Gerrit reporting
 * Merger/Dependent pipeline logic
 * Exposing workflows vs using pipelinelib
 * Porting JJB -> workflows *could be done*


 * Filters in Argo using Kafka plugin in Gerrit