Lint checks happen right away, however the unit testing:
- cloning of the mediawiki repo
- setting up several databases and installing it multiple times for different database backends
- Executing it in PHP
- Sending the QUnit test suites to all TestSwarm clients
.. all that should not happen right away due to the security risk of arbitrary code execution (both for PHP and for JS). The Swarm would function like a little bot net (JS can practically take over the browser and make as many requests for as long as it wants to whatever domain), executing arbitrary PHP code on a machine in the production cluster don't sound good either (right now Jenkins runs the unit test on the same machine as that it runs itself).
Even if we could limit the security issues (which I don't believe we can), I think it is reasonable that there should be agreement over a revision before it is tested for.