Manual:PHP unit testing

We do unit testing for MediaWiki's PHP code base using the PHPUnit framework.

The unit tests are located in the tests/phpunit directory. Tests are organized into a directory structure that roughly matches the directory structure of the code that they are testing. For example: you can find the unit tests for file includes/IP.php in tests/phpunit/includes/IPTest.php.

Installing PHPUnit
You must have PHPUnit 3.5.0 installed to run the unit tests.

Depending on your distribution and configuration, you may have to run the following commands with.

On Unix-like operating systems, run  from the tests/phpunit directory. This will attempt to install PHPUnit using PEAR, falling back to the apt and yum package managers if PEAR is not available. The apt packages for Ubuntu 11.10 are known to be broken, so installation using PEAR is the preferred way. You may have to update the installation of PEAR itself on your machine:

When running into an error such as this one, this should help you out.

If you get an error such as below when trying to run PHPUnit(Ubuntu 11.04)

Edit /usr/share/pear/File/Iterator/Factory.php and on line 44 add require_once 'File/Iterator.php';.

If that doesn't work, or you're on Windows-family operating systems, follow the installation instructions in the PHPUnit manual.

Running the unit tests
Run the unit tests from the command line, ensuring that you are in the tests/phpunit directory.
 * On UNIX-like operating systems, run either php phpunit.php or  . To see options for running the tests, run.
 * On Windows-family operating systems, run the   batch file.

For instance, to run all tests that have no database impact:

$ cd tests/phpunit $ make databaseless php phpunit.php --configuration /var/www/trunk/tests/phpunit/suite.xml \ --exclude-group Broken,Destructive,Database,Stub PHPUnit 3.5.13 by Sebastian Bergmann. ................................................IIIII........ 61 / 2311 ( 2%)  ............I................................................ 122 / 2311 ( 5%)  .............................................................  183 / 2311 (  7%)  .............................................................  244 / 2311 ( 10%)  .............................................................  305 / 2311 ( 13%)  .............................................................  366 / 2311 ( 15%)  .............................................................  427 / 2311 ( 18%)  .............................................................  488 / 2311 ( 21%)  .............................................................  549 / 2311 ( 23%)  .....................I....................................... 610 / 2311 ( 26%) .............................................................  671 / 2311 ( 29%)  .......................................................S...II  732 / 2311 ( 31%) IIIIIIIIIII...IIIIIIIIIIIIIII................................ 793 / 2311 ( 34%) ............................  Time: 10 seconds, Memory: 97.25Mb There were 35 incomplete tests: [snip] OK, but incomplete or skipped tests! Tests: 821, Assertions: 78982, Incomplete: 35, Skipped: 1.

Running an individual test set:

$ cd tests/phpunit $ php phpunit.php includes/IPTest.php PHPUnit 3.5.13 by Sebastian Bergmann. ............................. Time: 1 second, Memory: 14.75Mb OK (29 tests, 6025 assertions)

To see the name of each running test, run:

$ cd tests/phpunit $ php phpunit.php --tap

Writing Unit Test for Extensions
The recommended bootstrapping method for extensions is to use the UnitTestsList hook. By registering your tests with this hook you can use the mediaWiki core bootstrap support for things like autoloading classes, global functions etc. Unless your code is truly a standalone library, it's NOT recommended to include a few random MediaWiki core files with custom bootstrapping and attempt to make your tests stand alone.

To exclusively run your extensions tests, duplicate the suite.xml file in tests/phpunit/suite.xml to suite.extensions.xml only include the "extensions" testsuite as a child of testsuites. Then run the test with the command:

To test a single file you can pass the test file as a parameter to

Code
Let's assume our extension is named Fruits. In Fruits/Fruits.php main file add:

In AppleTest.php add:

To make easier to run test, you could create a Makefile something like this:

Writing unit tests
The PHPUnit Manual provides good instructions for understanding and developing unit tests. Pay particularly close attention to the sections on writing and organizing tests.

Developers new to unit testing in MediaWiki should use SampleTest.php as a starting point – it contains helpful comments that will ease the process of learning to write unit tests.

Another resource is the slides from the PHPUnit Best Practices talk that Sebastian Bergmann gave at OSCON 2010.

Write Testable Code
Please try to write testable code.

MediaWiki was not written with the objective of being testable. It uses global variables all over the place and static methods in many places. This is a legacy that we have to accept, but try not to introduce these things into new code, and try to change going forward.

A good resource might be Miško Hevery's Guide to Testability. (Miško Hevery is [one of?] Google's Agile Coaches.)

Telling a story
Test output should tell a story (output options).

Number of Assertions
Only one assertion per test unless there is a good reason (expensive tests may need to be grouped).

Grouping tests
PHPUnit allows tests to be put into arbitrary groups. Groups of tests can be selected for execution or excluded from execution when the test suite is run (see the @group annotation, The Command-Line Test Runner and XML Configuration File documentation in the PHPUnit manual for additional details.)

To add a test (or class) to a group, use the  annotation in the docblock preceding the code. For example:

""

Eight groups are currently used in the MediaWiki unit tests:
 * Broken: Put broken tests into group Broken. Tests in this group will not be run (as is configured in tests/phpunit/suite.xml).
 * Database: Tests that require database connectivity should be put into group Database.
 * Destructive: Tests that alter or destroy data should be put into group Destructive.
 * Fundraising: Tests related to WMF fundraising (CentralNotice, DonationInterface, etc.)
 * Search: Tests that use MediaWiki's built-in search put into group Search.
 * SeleniumFramework: Tests that require SeleniumFramework to be installed should be put in group SeleniumFramework.
 * Stub: Put test stubs into group Stub. Tests in this group will not be run (as is configured in tests/phpunit/suite.xml</tt>).
 * sqlite: Tests that use SQLite should be put into group sqlite.
 * Upload: Tests that upload files should be put into group Upload.
 * Utility: Currently unused by any test. Tests in this group will be not be run (as is configured in tests/phpunit/suite.xml</tt>).

To test only a particular group, use the --group flag from the command line: php phpunit.php --group Search

Developing best practices
Developers should avoid inventing new conventions or borrowing conventions from other frameworks; using the already well-established PHPUnit conventions will serve to make MediaWiki's tests both useful and usable. Pay particularly close attention to the sections in the PHPUnit manual on writing and organizing tests.

Failing
Usually tests code shouldn't do, but throw an exception:

This would show up as an error in the testing summary rather than bringing the whole test suite down.

How to help

 * Make sure the tests run on your system -- if they don't, either something's broken or the tests are making bad assumptions, and it needs fixing either way!
 * Find and fix any testing problems on non-MySQL database backends:
 * (SQLite is used for test runs on http://integration.mediawiki.org/ci/ )
 * /PostgreSQL
 * /Oracle
 * /MS SQL Server
 * Find code that needs testing and write tests