Continuous integration/Workflow

This article describes how we trigger tests on changes being submitted to a project as well as the tools integrated to fulfill the workflow.

Tools

 * Gerrit
 * it hosts the WMF git repository and provides an interface to review the code. Developers send a patchset which is then enqueued in the review queue and is not actually merged until it has been approved. Whenever a patchset is sent or is approved, Gerrit generates an event we react on.


 * Jenkins
 * This is a job control system which can be used to run unit tests, generate tarballs or whatever we can think about. Each job is a set of instructions (such as running a shell script or reporting test result) and can accept parameters to finely tune the job execution.


 * Zuul
 * Is a daemon listening for Gerrit events, according to a workflow specification file, it can trigger Jenkins jobs and pass them various parameters (ex: the Gerrit change number).

Roles
We identify several user roles:


 * Contributor
 * Anyone modifying a file hosted by the WMF. The modification can be code, translations, documentation update. Contributor send their change to Gerrit using git. Any contributor is allowed to review any change, comment on it and gives his feedback by voting the code-review label (-1 to +1).


 * Reviewer
 * Those people are the gate keepers, they have the ability to approve a change which will in turn trigger unit tests and merge the change in the repository. They can also prevent a change from being sent by voting code-review label -2.

From patch to merge
Whenever a patch is uploaded, we run basic checks to detect the most trivial errors. Whenever the checks fails, the contributor is shown a list of action to handle and invited to update a new patch.

Once the checks have been completed, Jenkins will flag the change has Verified which also mean the change is ready to be reviewed by a human being. Developers, testers or even a non coders will have a look at the code sent, test it and comment about possible issues.

Each person can vote under the Code-Review with a score varying from -2 (do not submit/merge) to +2 (change approved). Only projects owner and trusted people are able to actually approve a change. Whenever a +2 vote is casted, unit tests are run. For MediaWiki core this include attempting to install MediaWiki and running the full PHPUnit test suite. If the test suite fail, the change is verified and prevented from being submitted (Code-Review -2). If all tests are successfull, change is ready to be merged.

Basically:
 * Static analysis (linting and coding style) is done on patch submission.
 * Execution of unit tests are run once changes are reviewed (Code-Review +2).

Overview: