Manual:PHP unit testing/Running the tests

MediaWiki Flower

« Unit testing
PHPUnit testing

Contents

• Running the tests
  • PostgreSQL
  • Oracle
• Generate code coverage
• Writing testable PHP code
• Writing tests
  • ...for extensions
• Continuous integration
• Understanding build failures
• Appendix
  (how to help, resources..)

---

Tools

• PHPUnit

Entrypoints for running the tests

If you have not already, run composer update in the root of the MediaWiki core repository.

vendor/bin/phpunit

The standard way to run PHPUnit is with the entrypoint located at vendor/bin/phpunit

Terminal

$ vendor/bin/phpunit --help
PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

Usage:
  phpunit [options] UnitTest [UnitTest.php]
  phpunit [options] <directory>

Composer script

Run all the unit tests for MediaWiki core and all extensions and skins:

Terminal

$ composer phpunit:unit
> phpunit --colors=always --testsuite=core:unit,extensions:unit,skins:unit
PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

.............................................................   61 / 8977 (  0%)
.............................................................  122 / 8977 (  1%)
.............................................................  183 / 8977 (  2%)
.............................................................  244 / 8977 (  2%)
.............................................................  305 / 8977 (  3%)
.............................................................  366 / 8977 (  4%)
.............................................................  427 / 8977 (  4%)
.............................................................  488 / 8977 (  5%)
.............................................................  549 / 8977 (  6%)

Wrapper script

We use a wrapper script to run integration tests. The wrapper script phpunit.php is located in the tests/phpunit directory and is invoked using the PHP command:

$ cd tests/phpunit
$ php phpunit.php

This documentation is written assuming that you are running tests in a local development environment. Do not run tests on a production system, bad things will happen!

Select by directory or file

Running all tests from all core components and installed extensions may take a while. During development you may want to focus on just a particular test suite or directory of test suites. Pass the directory or file to the phpunit command.

For example, if you're working on a patch that modifies includes/resourceloader, with a test covering it under unit/includes/resourceloader/, then you might run the tests like so:

Terminal

$ cd tests/phpunit
$ php phpunit.php unit/includes/resourceloader/
  PHPUnit by Sebastian Bergmann.
  
...........................................................       59 / 59 (100%)

Time: 1.78 seconds, Memory: 22.00 MB

OK (59 tests, 80 assertions)

Select by test name

If you're debugging a specific test case and are adding breakpoints or var_dump statements, it may be useful to filter even more closely than by the file, so as to skip past all other invocations of the same underlying code. To do this, use the --filter option to match substrings of the test name. For example, given a test suite like the following:

class ResourceLoaderFileModuleTest extends MediaWikiIntegrationTestCase {
	protected function setUp(): void {
		parent::setUp();
		// ...
	}

	public function testLessCompiler() {
		// ...
	}

	public function testGetScriptPackageFiles() {
		// ...
	}

	public function testGetVersionHash() {
		// ...
	}
}

You can combine the directory selection with --filter to run only the second testGetScriptPackageFiles test case, like so:

Terminal

$ cd tests/phpunit
$ php phpunit.php --filter Package includes/resourceloader/ResourceLoaderFileModuleTest.php
  PHPUnit by Sebastian Bergmann.
  
...........................................................       1 / 1 (100%)

Time: 0.17 seconds, Memory: 22.00 MB

OK (1 test, 2 assertions)

Select by group

Each test case can be grouped using the @group annotation in the test case file. The group names may look obscure at first, we will describe them later on in the Writing unit tests chapter.

To only run tests in a particular group, use the --group option.

This may be slow and will be silent for a few seconds before the tests start, because it still requires PHPUnit to load and data-provide all test suites (including extensions) before the group filter is applied. For fast results, select by directory instead.

Terminal

$ cd tests/phpunit
$ php phpunit.php --group Cache
  PHPUnit by Sebastian Bergmann.
  
  ................................................IIIII........   61 / 2311 (  2%)
  ............I................................................  122 / 2311 (  5%)
  .............................................................  183 / 2311 (  7%)
  .............................................................  244 / 2311 ( 10%)
  .............................................................  305 / 2311 ( 13%)
  .............................................................  366 / 2311 ( 15%)
  .............................................................  427 / 2311 ( 18%)
  .............................................................  488 / 2311 ( 21%)
  .............................................................  549 / 2311 ( 23%)
  .....................I.......................................  610 / 2311 ( 26%)
  .............................................................  671 / 2311 ( 29%)
  .......................................................S...II  732 / 2311 ( 31%)
  IIIIIIIIIII...IIIIIIIIIIIIIII................................  793 / 2311 ( 34%)
  ............................
  
  Time: 10 seconds, Memory: 97.25Mb
  
  There were 35 incomplete tests:
[snip]
  
OK, but incomplete or skipped tests!
Tests: 821, Assertions: 78982, Incomplete: 35, Skipped: 1.

To find all available groups, run tests/phpunit/$ php phpunit.php --list-groups to print the available test groups. To run a particular test group, pass it to phpunit, for example:

tests/phpunit/$ php phpunit.php --group Editing

Output format

To see the name of each running test, uses the TAP format by passing the --tap option to PHPUnit:

Terminal

$ cd tests/phpunit
$ php phpunit.php --tap includes/libs/IPTest.php
TAP version 13
not ok 1 - Error: IPTest::testisIPAddress
ok 2 - IPTest::testisIPv6
ok 3 - IPTest::testisIPv4
ok 4 - IPTest::testValidIPs
^C
$

This format is great to filter the output for non passing test. For example with the grep command:

Terminal

$ php phpunit.php --tap includes/libs/IPTest.php | egrep '^not'
not ok 1 - Error: IPTest::testisIPAddress
$

phpunit also provide a kind of checklist that give out a great output for people not familiar with tests or shells: the testdox format:

Terminal

$ php phpunit.php --testdox includes/specials/
PHPUnit by Sebastian Bergmann.

Configuration read from /srv/trunk/tests/phpunit/suite.xml

QueryAllSpecialPages
 [x] Querypage sql query

SpecialRecentchanges
 [x] Rc ns filter
 [x] Rc ns filter inversion
 [ ] Rc ns filter association
 [x] Rc ns filter association with inversion

SpecialSearch
 [x] Profile and namespace loading

$

An unchecked box ([ ]), means the test failed, such as the one named « RC ns filter association » above.

How it works

Database

Integration tests that involve a database are automatically run against a temporary clone of your current wiki's database (the one configured in LocalSettings.php).

Because of this, it is important that the MediaWiki installation being tested has an up to date and correct database, or error messages (such as "TestUser.php: Can't create user on real database") will result.