Continuous integration/Codehealth Pipeline

The codehealth pipeline is an initiative of Code Health Group/projects/Code Health Metrics. Currently it is enabled for a handful of MediaWiki extensions, while we collect feedback from participating projects.

tl;dr
Patches (pre-merge and post-merge) are sent to a service which analyzes the code for various health checks, then that service gives a score, and that score is reported to Gerrit. A list of repositories that have been integrated into the codehealth pipeline can be found in this Phabricator Ticket

How it works
The code health reports are generated by checking out the code for a patch (or if it's postmerge, for the master branch), then running the sonar-scanner application.

That application reads from a  file (more on that later) which tells Sonar which directories have source code, which directories have test files, which directories/files should be excluded, what type of project (PHP, Java, and so on) it is, etc.

The  application sends all of those files to a remote server. It will also send over any code coverage reports that have been generated by other scripts – sonar-scanner doesn't do that on its own. For PHP, the code coverage is generated by PHPUnit unit tests only (not integration tests), and for JavaScript it is generated with node-qunit tests which currently only exist in a handful of repositories (GrowthExperiments, Popups, MobileFrontend).

The destination for sending the analysis results could be the self-hosted version of SonarQube. But in our case we currently use the hosted version of SonarQube which is at https://sonarcloud.io.

When the analysis reaches the SonarQube application, it is stored in its database and SonarQube provides a grade for the patch. Then it issues a POST request to SonarQube Bot.

SonarQube Bot
SonarQube Bot is a Symfony application that listens for incoming POST requests from SonarQube. It looks to see if the quality gate in SonarQube for the patch was pass or fail. Then it marks a patchset as Verified +1 if the quality gate passed. It also leaves a comment with a summary of the quality gates, for both success and failure cases.

Currently the bot leaves inline comments with issues found, using Gerrit's robot comment feature. Robot comments are not well supported in Gerrit 2.15, so it is an experimental feature. If you want to have your extension in the whitelist of projects that has inline comments please let us know via the talk page.

The configuration for the bot is managed via  in Toolforge, after executing. Its logs are visible in. Currently the maintainers with SSH access are User:KHarlan (WMF) and User:GLederrey (WMF).

Feedback
Feedback via the talk page here is very welcome.