Beta Cluster/2014-15-Q3

Measures of Success
From the WMF RelEng Jan 2015 quarterly review:
 * Green nightly builds - T426
 * Set up multiversion on Beta Cluster - T67127
 * Make Beta Cluster stable over night (no puppet updates?)
 * Select a subset of known-useful browser tests
 * Run those browser tests against the nightly build
 * Publicly report metrics (graphs) of success over time
 * Stable uptime metrics
 * Pick a set of useful data sources of uptime
 * Ensure we're recording those
 * Make a dashboard of them
 * Define alert levels and enable alerts
 * By-team test history
 * (this is enabled by a gooder Beta Cluster)
 * Group build test history by engineering team
 * Report on those to the teams and engineering wide
 * Make a dashboard of these as well (to get a quick overview of overall test health)

Related

 * 2014-15 Q3 Top Priorities
 * 2014-15 Goals