Continuous integration/Codehealth Pipeline

The codehealth pipeline is an initiative of Code Health Group/projects/Code Health Metrics. Currently it is enabled for a handful of MediaWiki extensions, while we collect feedback from participating projects.

tl;dr
Patches (pre-merge and post-merge) are sent to a service which analyzes the code for various health checks, then that service gives a score, and that score is reported to Gerrit.

How it works
The code health reports are generated by checking out the code for a patch (or if it's postmerge, for the master branch), then running the sonar-scanner application.

That application reads from a  file (more on that later) which tells Sonar which directories have source code, which directories have test files, which directories/files should be excluded, what type of project (PHP, Java, and so on) it is, etc.

The  application sends all of those files to a remote server. It will also send over any code coverage reports that have been generated by other scripts – sonar-scanner doesn't do that on its own. For PHP, the code coverage is generated by PHPUnit unit tests only (not integration tests), and for JavaScript it is generated with node-qunit tests which currently only exist in a handful of repositories (GrowthExperiments, Popups, MobileFrontend).

The destination for sending the analysis results could be the self-hosted version of SonarQube. But in our case we currently use the hosted version of SonarQube which is at https://sonarcloud.io.

When the analysis reaches the SonarQube application, it is stored in its database and SonarQube provides a grade for the patch. Then it issues a POST request to SonarQube Bot.

SonarQube Bot
SonarQube Bot is a Symfony application that listens for incoming POST requests from SonarQube. It looks to see if the quality gate in SonarQube for the patch was pass or fail. Then it marks a patchset as Verified +1 if the quality gate passed. It also leaves a comment with a summary of the quality gates, for both success and failure cases. A future version of the bot will implement inline commenting with issues found by SonarQube.

Feedback
Feedback via the talk page here is very welcome.