Codesearch/Admin

codesearch currently runs from the "codesearch" Cloud VPS project, on the codesearch6 instance. nagf is helpful to monitor the health of the instance.

Architecture
Each search profile (everything, extensions, skins, etc.) listed are backed by individual instances of hound, that are controlled by  systemd units. A flask web application proxies requests to the individual hound backends and injects in our customizations. It runs through the  systemd unit.

Everything is puppetized, so puppet will ensure all of the systemd units are running. Note that all of the backend units depend upon the proxy, so restarting the proxy will restart all of the backends.

The larger backends can take up to 15 minutes to start. The start up process can be rather memory and CPU intensive (especially on a truly cold start on a brand new instance), so the wait.py script tries to only allow one backend to start up at once.

The health check page should be helpful in seeing whether the hound backends are actually down, or just starting up still.

The configuration for each backend is written by the  systemd timer. Each backend's unit is configured to restart every 24 hours, so it picks up the new configuration.

Adding a new backend
TODO

Need to update write_config, then puppet, then app.py

Adding a new instance
TODO

Format /srv, then apply puppet role and run puppet, switch over web proxy.