Manual:PHP unit testing/Running the tests
Contents
- Running the tests
- Generate code coverage
- Writing testable PHP code
- Writing tests
- Continuous integration
- Understanding build failures
- Appendix
(how to help, resources..)
Tools
Entrypoints for running the tests[edit]
If you have not already, run composer update
in the root of the MediaWiki core repository.
vendor/bin/phpunit[edit]
The standard way to run PHPUnit is with the entrypoint located at vendor/bin/phpunit
$ vendor/bin/phpunit --help PHPUnit 8.5.8 by Sebastian Bergmann and contributors. Usage: phpunit [options] UnitTest [UnitTest.php] phpunit [options] <directory>
Composer script[edit]
Run all the unit tests for MediaWiki core and all extensions and skins:
$ composer phpunit:unit > phpunit --colors=always --testsuite=core:unit,extensions:unit,skins:unit PHPUnit 8.5.8 by Sebastian Bergmann and contributors. ............................................................. 61 / 8977 ( 0%) ............................................................. 122 / 8977 ( 1%) ............................................................. 183 / 8977 ( 2%) ............................................................. 244 / 8977 ( 2%) ............................................................. 305 / 8977 ( 3%) ............................................................. 366 / 8977 ( 4%) ............................................................. 427 / 8977 ( 4%) ............................................................. 488 / 8977 ( 5%) ............................................................. 549 / 8977 ( 6%)
Wrapper script[edit]
Unit tests are run from the command line as there is no graphical user interface as of January 2012. Although phpunit provide a bootstrapping facility, we use a wrapper script. It makes sure the MediaWiki framework is properly initialized using your LocalSettings.php
file, then directly call PHPUnit API. Invoking phpunit
as-is will simply not work.
The wrapper script phpunit.php
is located in the tests/phpunit
directory and is invoked using the PHP interpreter:
$ cd tests/phpunit $ php phpunit.php
The wrapper script is itself a MediaWiki maintenance script, and will pass all arguments to PHPUnit. To get a list of possible arguments have a look at phpunit --help
.
run-tests.bat
batch file
![]() | Some of the integration tests are destructive. Although they are supposed to use a copy of the database, there are most probably a few bugs in the system and may modify live data. Do not run the tests on a production wiki. |
Test database[edit]
That the test harness is used for unit and integration tests, and the latter often require a database to test against. This is done by cloning the database (the one that's referenced in LocalSettings.php) to temporary tables and running the tests against those.
Because of this, it is important that the MediaWiki installation being tested has an up to date and correct database, or error messages (such as "TestUser.php: Can't create user on real database") will result.
Using default test groups[edit]
To make it easier to newcomers, tests/phpunit in core has a makefile providing some sane targets that are easier to remember than some long shell commands.
- Entering
make target
runs sub-groups of tests. - Entering
make help
prints the list of targets and their descriptions.
You will obviously need the make utility. The sub-group names may look obscure to you right now, we will describe them later on in the writing unit tests section.
For instance, to run all tests that do not require a database connection, specify the make target databaseless
:
$ cd tests/phpunit $ make databaseless php phpunit.php --configuration /var/www/trunk/tests/phpunit/suite.xml \ --exclude-group Broken,Destructive,Database,Stub PHPUnit 3.5.13 by Sebastian Bergmann. ................................................IIIII........ 61 / 2311 ( 2%) ............I................................................ 122 / 2311 ( 5%) ............................................................. 183 / 2311 ( 7%) ............................................................. 244 / 2311 ( 10%) ............................................................. 305 / 2311 ( 13%) ............................................................. 366 / 2311 ( 15%) ............................................................. 427 / 2311 ( 18%) ............................................................. 488 / 2311 ( 21%) ............................................................. 549 / 2311 ( 23%) .....................I....................................... 610 / 2311 ( 26%) ............................................................. 671 / 2311 ( 29%) .......................................................S...II 732 / 2311 ( 31%) IIIIIIIIIII...IIIIIIIIIIIIIII................................ 793 / 2311 ( 34%) ............................ Time: 10 seconds, Memory: 97.25Mb There were 35 incomplete tests: [snip] OK, but incomplete or skipped tests! Tests: 821, Assertions: 78982, Incomplete: 35, Skipped: 1. $
As said above, a list of other targets is always available using the special help target (make help
).
Running other sets of tests[edit]
Running group of tests might be long and tedious, specifically when you are only altering a very specific part of the MediaWiki codebase. For example, you might be implementing some IPv6 functionality in the includes/libs/IP.php file. In the test suite, that file code is covered by includes/libs/IPTest.php which you can pass as an argument to our PHPUnit wrapper.
Below, "wiki" is the name of the database to use, which is required as of 1.29, but you may have to omit it in older versions of MediaWiki.
Running the individual tests from includes/libs/IPTest.php looks like:
$ cd tests/phpunit
$ php phpunit.php wiki includes/libs/IPTest.php
PHPUnit 3.5.13 by Sebastian Bergmann.
.............................
Time: 1 second, Memory: 14.75Mb
OK (29 tests, 6025 assertions)
$
To see the name of each running test, uses the format available in PHPUnit using --tap
:
$ cd tests/phpunit $ php phpunit.php --tap wiki includes/libs/IPTest.php TAP version 13 not ok 1 - Error: IPTest::testisIPAddress ok 2 - IPTest::testisIPv6 ok 3 - IPTest::testisIPv4 ok 4 - IPTest::testValidIPs ^C $
This format is great to filter the output for non passing test. For example with the grep command:
$ php phpunit.php --tap wiki includes/libs/IPTest.php | egrep '^not' not ok 1 - Error: IPTest::testisIPAddress $
phpunit also provide a kind of checklist that give out a great output for people not familiar with tests or shells: the testdox format:
$ php phpunit.php --testdox wiki includes/specials/ PHPUnit 3.6.7 by Sebastian Bergmann. Configuration read from /srv/trunk/tests/phpunit/suite.xml QueryAllSpecialPages [x] Querypage sql query SpecialRecentchanges [x] Rc ns filter [x] Rc ns filter inversion [ ] Rc ns filter association [x] Rc ns filter association with inversion SpecialSearch [x] Profile and namespace loading $
An unchecked box ([ ]
), means the test failed, such as the one named « RC ns filter association » above.
You can also run tests within a specific directory. For example, to run all of the language-related tests:
php phpunit.php languages/
Running a particular sub-group[edit]
php tests/phpunit/phpunit.php --list-groups
prints the available test groups.
To run a particular test group, pass it to phpunit, for example:
$ php tests/phpunit/phpunit.php --group Editing