User:Tim Starling (WMF)/LoadBalancer connection metric

Take the adjusted connection count $$C_i$$ which is the number of open connections plus the number of connection failures over the last 10 seconds.

Load weights which satisfy $$\sum_{i=1}^n w_i = 1$$

The discrepancy between the desired connection count and the actual connection count is

$$\Delta C_i = C_i - w_i T$$

where $$T = \sum_{j=1}^n C_j$$

A positive number indicates that the server has too many connections. We could normalize by the total connection count:

$$w_i^\prime = w_i - q \frac{\Delta C_i}{T}$$

But that would be noisy when the number of connections is small. We could simply scale it:

$$w_i^\prime = w_i - r \Delta C_i $$

That's defensible and easy to understand. Maybe it would get noisy with high traffic. We could combine the two:

$$w_i^\prime = w_i - \frac{q \Delta C_i}{E + T}$$

or if you like, equivalently

$$w_i^\prime = w_i - \frac{s \Delta C_i}{1 + \frac{T}{E}}$$

where q, r, s and E are tunable constants.

Obviously you then have to clamp it so that $$w_i^\prime \geq 0$$, and then scale it so that the weights add up to 1 again.

With $$S = \frac{1}{s}$$ the two tunable constants S and E can be thought of as connection counts. S is the connection count discrepancy which causes the load to be increased or decreased by 100% under low-traffic conditions. E is the total connection count at which the adjustment will be scaled down by half.