Manual:PHP unit testing/Running the tests

« Unit testing
PHPUnit testing

Contents

• Running the tests
  • PostgreSQL
  • Oracle
• Generate code coverage
• Writing testable PHP code
• Writing tests
  • ...for extensions
• Continuous integration
• Understanding build failures
• Appendix
  (how to help, resources..)

---

Tools

• PHPUnit

Entrypoints for running the tests

This documentation is written assuming that you are running tests in a local development environment. Do not run tests in a production system, bad things can happen.

If you have not already, run composer update in the root of the MediaWiki core repository.

vendor/bin/phpunit

The standard way to run PHPUnit is with the entrypoint located at vendor/bin/phpunit

Terminal

$ vendor/bin/phpunit --help
PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

Usage:
  phpunit [options] UnitTest [UnitTest.php]
  phpunit [options] <directory>

Composer script

Run all the unit tests for MediaWiki core and all extensions and skins:

Terminal

$ composer phpunit:unit
> phpunit --colors=always --testsuite=core:unit,extensions:unit,skins:unit
PHPUnit 8.5.8 by Sebastian Bergmann and contributors.

.............................................................   61 / 8977 (  0%)
.............................................................  122 / 8977 (  1%)
.............................................................  183 / 8977 (  2%)
.............................................................  244 / 8977 (  2%)
.............................................................  305 / 8977 (  3%)
.............................................................  366 / 8977 (  4%)
.............................................................  427 / 8977 (  4%)
.............................................................  488 / 8977 (  5%)
.............................................................  549 / 8977 (  6%)

Wrapper script

Unit tests are run from the command line as there is no graphical user interface as of January 2012. Although phpunit provide a bootstrapping facility, we use a wrapper script. It makes sure the MediaWiki framework is properly initialized using your LocalSettings.php file, then directly call PHPUnit API. Invoking phpunit as-is will simply not work.

The wrapper script phpunit.php is located in the tests/phpunit directory and is invoked using the PHP interpreter:

$ cd tests/phpunit
$ php phpunit.php

The wrapper script is itself a MediaWiki maintenance script, and will pass all arguments to PHPUnit. To get a list of possible arguments have a look at phpunit --help.

On Windows-family operating systems, run the run-tests.bat batch file

Test database

That the test harness is used for unit and integration tests, and the latter often require a database to test against. This is done by cloning the database (the one that's referenced in LocalSettings.php) to temporary tables and running the tests against those.

Because of this, it is important that the MediaWiki installation being tested has an up to date and correct database, or error messages (such as "TestUser.php: Can't create user on real database") will result.

Using default test groups

To make it easier to newcomers, tests/phpunit in core has a makefile providing some sane targets that are easier to remember than some long shell commands.

• Entering make target runs sub-groups of tests.
• Entering make help prints the list of targets and their descriptions.

You will obviously need the make utility. The sub-group names may look obscure to you right now, we will describe them later on in the writing unit tests section.

For instance, to run all tests that do not require a database connection, specify the make target databaseless:

Terminal

$ cd tests/phpunit
$ make databaseless
  php phpunit.php --configuration /var/www/trunk/tests/phpunit/suite.xml  \
   --exclude-group Broken,Destructive,Database,Stub
  PHPUnit 3.5.13 by Sebastian Bergmann.
  
  ................................................IIIII........   61 / 2311 (  2%)
  ............I................................................  122 / 2311 (  5%)
  .............................................................  183 / 2311 (  7%)
  .............................................................  244 / 2311 ( 10%)
  .............................................................  305 / 2311 ( 13%)
  .............................................................  366 / 2311 ( 15%)
  .............................................................  427 / 2311 ( 18%)
  .............................................................  488 / 2311 ( 21%)
  .............................................................  549 / 2311 ( 23%)
  .....................I.......................................  610 / 2311 ( 26%)
  .............................................................  671 / 2311 ( 29%)
  .......................................................S...II  732 / 2311 ( 31%)
  IIIIIIIIIII...IIIIIIIIIIIIIII................................  793 / 2311 ( 34%)
  ............................
  
  Time: 10 seconds, Memory: 97.25Mb
  
There were 35 incomplete tests:
[snip]
  
OK, but incomplete or skipped tests!
Tests: 821, Assertions: 78982, Incomplete: 35, Skipped: 1.
$

As said above, a list of other targets is always available using the special help target (make help).

Running other sets of tests

Running group of tests might be long and tedious, specifically when you are only altering a very specific part of the MediaWiki codebase. For example, you might be implementing some IPv6 functionality in the includes/libs/IP.php file. In the test suite, that file code is covered by includes/libs/IPTest.php which you can pass as an argument to our PHPUnit wrapper.

Below, "wiki" is the name of the database to use, which is required as of 1.29, but you may have to omit it in older versions of MediaWiki.

Running the individual tests from includes/libs/IPTest.php looks like:

Terminal

$ cd tests/phpunit
$ php phpunit.php wiki includes/libs/IPTest.php
  
PHPUnit 3.5.13 by Sebastian Bergmann.
  
.............................
  
Time: 1 second, Memory: 14.75Mb
  
OK (29 tests, 6025 assertions)
$

To see the name of each running test, uses the format available in PHPUnit using --tap:

Terminal

$ cd tests/phpunit
$ php phpunit.php --tap wiki includes/libs/IPTest.php
TAP version 13
not ok 1 - Error: IPTest::testisIPAddress
ok 2 - IPTest::testisIPv6
ok 3 - IPTest::testisIPv4
ok 4 - IPTest::testValidIPs
^C
$

This format is great to filter the output for non passing test. For example with the grep command:

Terminal

$ php phpunit.php --tap wiki includes/libs/IPTest.php | egrep '^not'
not ok 1 - Error: IPTest::testisIPAddress
$

phpunit also provide a kind of checklist that give out a great output for people not familiar with tests or shells: the testdox format:

Terminal

$ php phpunit.php --testdox wiki includes/specials/
PHPUnit 3.6.7 by Sebastian Bergmann.

Configuration read from /srv/trunk/tests/phpunit/suite.xml

QueryAllSpecialPages
 [x] Querypage sql query

SpecialRecentchanges
 [x] Rc ns filter
 [x] Rc ns filter inversion
 [ ] Rc ns filter association
 [x] Rc ns filter association with inversion

SpecialSearch
 [x] Profile and namespace loading

$

An unchecked box ([ ]), means the test failed, such as the one named « RC ns filter association » above.

You can also run tests within a specific directory. For example, to run all of the language-related tests:

php phpunit.php languages/

Running a particular sub-group

php tests/phpunit/phpunit.php --list-groups prints the available test groups. To run a particular test group, pass it to phpunit, for example:

$ php tests/phpunit/phpunit.php --group Editing