Continuous integration/Entry points

Jump to navigation Jump to search

We have standardized on the following tools for testing our code:

Language Command runner Linting Code style Static analysis Unit tests Documentation
PHP composer php-parallel-lint

php -l

MediaWiki-Codesniffer Phan PHPUnit Doxygen
Java maven
JavaScript npm & grunt eslint eslint eslint QUnit jsduck[1]
JSON grunt grunt-jsonlint N/A N/A N/A N/A
i18n grunt grunt-banana-checker N/A N/A N/A Localisation#Message documentation
Python tox flake8 flake8 flake8 unittest


Ruby bundler rubocop rubocop rake test

Documentation on how to configure and set up these tools can be found below.


Testing JavaScript[edit]

We are using npm test as an entry point. If your project has JavaScript files, it should at least have a package.json file and define a "test" script.

You will need the .eslintrc.json configuration file in your project (see Manual:Coding conventions/JavaScript#Linting). Look at one of the projects listed in the example section below for an example of these files.

On Linux, if npm commands fails with "node: No such file or directory", you may need to install the "nodejs-legacy" package.

Grunt task runner[edit]

If your project has complex build processes or is an extension or skin that will benefit from i18n checking and JSON file linting, the convention is to use Grunt as a task runner. Your project still has a package.json file, which has a dependency on grunt-cli and sets "test": "grunt test". In turn, a Gruntfile.js file implements grunt test, and this can run a wide variety of tools and tests:

  • eslint
  • banana-checker which checks messages in MediaWiki i18n files.
  • jsonlint which checks JSON files including composer.json and extension.json for conformance.

You can specify configuration settings for these tools in Gruntfile.js. However, it should contain little to no configuration for tools that can run outside grunt so that they operate the same when run standalone or from a text editor plugin. Always use native configuration files where possible, including .eslintrc.json mentioned above.

JavaScript documentation[edit]

Use npm run doc as the entry point. The convention is to use JSDuck. The "predoc" and "postdoc" script hooks in package.json can be used to run any additional scripts (e.g. build files for inclusion beforehand, or copy additional files for publication afterward).


Advanced setup using Grunt:

  "scripts": {
    "test": "grunt test",
    "doc": "jsduck"
  "devDependencies": {
    "grunt": "1.0.1",
    "grunt-banana-checker": "0.5.0",
    "grunt-eslint": "19.0.0",
    "eslint-config-wikimedia": "0.3.0",
    "grunt-jsonlint": "1.1.0"

Example projects:

Further reading:


Testing PHP[edit]

We are using composer test as an entry point. If your project has PHP files it should list the test framework packages it needs in composer.json under "require-dev" and list the commands to be run in the scripts.test property:

	"require-dev": {
		"jakub-onderka/php-parallel-lint": "0.9",
		"mediawiki/mediawiki-codesniffer": "0.7.2",
		"phpunit/phpunit": "4.8.*"
	"scripts": {
		"test": [
			"parallel-lint . --exclude vendor",
			"phpcs -p -s"

See composer.json of the cdb project for a good example.

Note that MediaWiki extensions are not standalone projects and cannot run their own PHPUnit test suite from composer, those repositories have a separate mediawiki-extensions job. PHPCS and PHP lint are still run via composer.json and composer test.

PHP Documentation[edit]

See: Doxygen.

Use the doxygen program to generate a Doxyfile file in the project root.

Testing Python[edit]

See Continuous integration/Tutorials/Test your python.



Use Rake to define your commands, they will be executed via Bundler.

Example Rakefile:

require 'bundler/setup'

require 'rubocop/rake_task' do |task|
  # if you use mediawiki-vagrant, rubocop will by default use it's .rubocop.yml
  # the next line makes it explicit that you want .rubocop.yml from the directory
  # where `bundle exec rake` is executed
  task.options = ['-c', '.rubocop.yml']

require 'mediawiki_selenium/rake_task'

task default: [:test]

desc 'Run all build/tests commands (CI entry point)'
task test: [:rubocop]

The above code will create following Rake targets.

$ bundle exec rake -T

rake rubocop               # Run RuboCop
rake rubocop:auto_correct  # Auto-correct RuboCop offenses
rake selenium              # Run Cucumber features
rake test                  # Run all build/tests commands (CI entry point)

The Jenkins job rake-jessie invokes test target by running bundle exec rake test.

Reference: phab:T104024

ruby debug tip[edit]

You can use the gem pry to break on error and get shown a console in the context of the failure. To your Gemfile add gem 'pry' then to break:

require 'pry'
your call that fail

You will then be in a console before the breakage that let you inspect the environment (ls). See for details.


We have a set of Jenkins jobs that run daily and execute Ruby + Selenium tests. The jobs are named selenium*.

Each repositories has only a single job defined in Jenkins. It is a multi configuration job that spawns one or more child job based on a configuration in each repositories: tests/browser/ci.yml. The main job will then spawn child jobs based on its content.

Example of a simple ci.yml is in mediawiki/core.

  - firefox

  - beta

 - Linux

As you can see, there are three variables, BROWSER, MEDIAWIKI_ENVIRONMENT and PLATFORM.

BROWSER and PLATFORM can be any valid Sauce Labs browser/OS/version combination.

MEDIAWIKI_ENVIRONMENT can have values beta, mediawiki and test, or any other environment configured in environments.yml.

For example:

  - chrome
  - firefox
  - internet_explorer 9.0
  - safari

  - beta
  - mediawiki
  - test

  - Linux
  - OS X 10.9
  - Windows 8.1

Example of a complicated ci.yml is in mediawiki/extensions/MultimediaViewer. For more information see Jenkins Yaml Axis Plugin.

Reference: phab:T128190