Wikimedia Labs/Create a bot running infrastructure

This is a proposed architecture of bot cluster.

Bots are important part of wikimedia :-) let's make them a more friendly platform

Bots project would consist of two nova projects: production and testing.

Home

Every bot operator would have their home directory stored on shared nfs accessible across project, there would be two projects, production and testing.

Production

Every bot operator would have own instance for their applications on production project. Configuration of instance would be done using puppet.

Testing

Every bot operator may create an instance on testing project, with full root access, to install and test various software before deploying to production. Once a software is tested, it can be deployed to production.

Web server

Production would have publicly accessible web server, each bot op would have public_html folder in their home which could be accessed using http://bots.wmflabs.org/~user_name

Private sql

Every bot op could create unlimited number of sql databases on sql server to store their own data for bots.

Production wiki's sql clone

Production and testing would have access to replicated db of all wikimedia projects.

Nagios

Every instance is monitored using nagios, bot operators can use puppet to insert various services to be monitored.