Continuous integration/Workflow 2014

The first long-term plan of our workflow (Workflow v1) is nearing completion. Time to take what we've learned, look at how the world has evolved and what new features are on the horizon. Here's what I hope we'll be doing before the end of 2014.

On submission of any patchset to any project on Gerrit by anyone:
 * Run its tests in a virtual machine
 * Gerrit notifies Zuul which distributes the job to a pool openstack of vagrant instances based on a pre-built image (these are teared down after a job and replaced with fresh ones automatically).
 * Which tasks to execute is specified in a declarative file inside the repository (e.g. a bit like, it would specify which of a handful of VM templates to use, and what commands to execute. such as   )
 * At the very least every repository runs lint checks (php, js, css, python, puppet)
 * MediaWiki core and extensions:
 * Install MediaWiki on local webserver (+ any extensions installed if relevant)
 * Run PHPUnit tests (optimised using phpunit-parallel)
 * Run QUnit tests
 * Run grunt/qunit (phantomjs)
 * Run grunt/karma (cross-browser qunit testing in saucelabs or browserstack)
 * Run selenium integration tests in saucelabs
 * Run pdiff-based job to detect any visual differences on common urls (not an assertion, purely informative to detect unintended visual side effects and to easily review intended visual differences).
 * Assert catch-all MediaWiki/php/apache error logs are empty

Tasks:
 * Install csslint
 * Start using it in core and VisualEditor
 * Merge separate phpunit jobs back into one and use phpunit-parallel
 * Implement virtual machine infrastructure
 * Set up one or more templates
 * Design and implement the travis-yml-like set up