Continuous integration/Architecture/Castor

Castor is an umbrella term for the caching of dependencies/package managers materials for the isolated instances.

The CI jobs start up in a fresh environment and have to retrieve dependencies over the internet and eventually, for native dependencies, compile them. The download phase can be arbitrarily long with package managers such as `maven` download a long list of dependencies, and has the risk of upstream blacklisting our network abusing bandwidth. The installation and compile phase can be quite slow as well and it does not make sense to compile again and again the same material.

We introduced a very lame system based on rsync. It copies from the instance a list of directories to a central place whenever the change succeeded in the Zuul gate-and-submit pipeline. When a job start, it first attempts to retrieve the material from the central cache, thus warming up the cache before invoking the package manager. The cache itself is namespaced by: