Wikimedia Labs/Create a bot running infrastructure

This is a proposed architecture of bot cluster.

Bots project would consist of two nova projects: production and testing.

Home
Every bot operator would have their home directory stored on shared nfs accessible across project, there would be two projects, production and testing.

Production
Every bot operator would have own instance for their applications on production project. Configuration of instance would be done using puppet.

Testing
Every bot operator may create an instance on testing project, with full root access, to install and test various software before deploying to production. Once a software is tested, it can be deployed to production.

Web server
Production would have publicly accessible web server, each bot op would have public_html folder in their home which could be accessed using http://bots.wmflabs.org/~user_name

Private sql
Every bot op could create unlimited number of sql databases on sql server to store their own data for bots.

Production wiki's sql clone
Production and testing would have access to replicated db of all wikimedia projects.

Nagios
Every instance is monitored using nagios, bot operators can use puppet to insert various services to be monitored.