Manual:PHP unit testing/Writing unit tests

The [ https://phpunit.readthedocs.io/en/9.5/index.html PHPUnit Manual] provides good instructions for understanding and developing unit tests. Pay particularly close attention to the sections on [ https://phpunit.readthedocs.io/en/9.5/writing-tests-for-phpunit.html writing] and [ https://phpunit.readthedocs.io/en/9.5/organizing-tests.html organizing] tests.

Developing best practices
Developers new to unit testing in MediaWiki should use as a starting point – it contains helpful comments that will ease the process of learning to write unit tests.

Another resource is the [ https://web.archive.org/web/20160303184343/http://assets.en.oreilly.com/1/event/45/PHPUnit%20Best%20Practices%20Presentation.pdf slides] from the [ https://web.archive.org/web/20150925211211/http://www.oscon.com/oscon2010/public/schedule/detail/12510 PHPUnit Best Practices] talk that Sebastian Bergmann gave at OSCON 2010.

Developers should avoid inventing new conventions or borrowing conventions from other frameworks; using the already well-established PHPUnit conventions will serve to make MediaWiki's tests both useful and usable.

Unit tests should follow [ http://www.artima.com/weblogs/viewpost.jsp?thread=126923 A Set of Unit Testing Rules] by Michael Feathers.

Write testable code
MediaWiki was not written with the objective of being testable. It uses global variables all over the place and static methods in many places. This is a legacy that we have to accept, but try not to introduce these things into new code, and try to change going forward.

A good resource might be Miško Hevery's [ https://web.archive.org/web/20200219224539/http://misko.hevery.com/2009/03/09/guide-to-testability-is-now-downloadable/ Guide to Testability]. ([ https://web.archive.org/web/20200829024409/http://misko.hevery.com/about/ Miško Hevery] is [one of?] Google's Agile Coaches.)

Test conventions
The filename must end with. Use as a starting point.

setUp and tearDown

 * Must be s.
 * tearDown should be in the opposite order of setUp.
 * setUp starts with calling its parent.
 * tearDown ends with calling its parent.

Assertion functions

 * Must be s.
 * The name of the function should be in lowerCamelCase and begin with the word ; e.g.,.
 * Whenever possible, refer to the most important method being tested; e.g., is tested in.

Data providers

 * Must be s.
 * The name of the data provider should be in lowerCamelCase and begin with the word ; e.g.,.
 * Should not instantiate MediaWiki Services, since data providers are called before .  For example, instead of , use  to avoid creating a.

Telling a story
Test output should tell a story. The output format is a good way to view this story: a test suite execution is displayed as a set of statements about the test classes, along with whether or not they have passed. The statements (unless customized) are the test method names with corrected capitalization and spacing.

The annotation can be used to customize the message that is displayed. It is not currently used in the MediaWiki codebase.

See [ https://phpunit.readthedocs.io/es/latest/other-uses-for-tests.html "Other Uses for Tests"] for more information.

Number of assertions
Only one assertion per test unless there is a good reason (expensive tests may need to be grouped).

Grouping tests
PHPUnit allows tests to be put into arbitrary groups. Groups of tests can be selected for execution or excluded from execution when the test suite is run (see the [ https://phpunit.readthedocs.io/en/8.0/appendixes.annotations.html#appendixes.annotations.group @group annotation], [ https://phpunit.readthedocs.io/en/8.0/textui.html The Command-Line Test Runner] and [ https://phpunit.readthedocs.io/en/8.0/appendixes.configuration.html XML Configuration File] documentation in the PHPUnit manual for additional details.)

To add a test (or class) to a group, use the annotation in the docblock preceding the code. For example:

Several functional groups are currently used in MediaWiki unit tests:


 * API: Tests that exercise the MediaWiki API.
 * Broken: Put broken tests into group Broken. Tests in this group will not be run (as is configured in ).
 * Database: Tests that require database connectivity should be put into group Database. NOTE: this causes temporary tables to be overlayed over the real wiki database, so test cases can perform database operations without changing the actual wiki.
 * Destructive: Tests that alter or destroy data should be put into group Destructive.
 * Search: Tests that use MediaWiki's built-in search put into group Search.
 * SeleniumFramework: Tests that require SeleniumFramework to be installed should be put in group SeleniumFramework.
 * Stub: Put test stubs into group Stub. Tests in this group will not be run (as is configured in ).
 * sqlite: Tests that use SQLite should be put into group sqlite.
 * Upload: Tests that upload files should be put into group Upload.
 * Utility: Currently unused by any test. Tests in this group will be not be run (as is configured in ).

In addition, tests may also be grouped based on development team:


 * Fundraising
 * EditorEngagement
 * Internationalization
 * etc.

To test only a particular group, use the flag from the command line:

php phpunit.php --group Search

or if you use the Makefile in core/tests/phpunit :

make FLAGS="--group Search" target

where target can be phpunit, safe, etc.

Coverage
The [ https://phpunit.readthedocs.io/en/8.0/code-coverage-analysis.html PHPUnit documentation has a chapter about coverage]. There is a [ https://doc.wikimedia.org/cover/mediawiki-core/ coverage report for mediawiki core] generated twice a day. As the [ http://phpunit.de/manual/current/en/appendixes.configuration.html forceCoversAnnotation option] should be in effect the test should be marked with [ http://phpunit.de/manual/current/en/appendixes.annotations.html#appendixes.annotations.covers @covers annotations] to express which parts of the code the test actually checks (as opposed to code that is just run but whose results are never tested for with assertions).

Note that requires fully-qualified class names (unlike Doxygen annotations such as  ).

Databases
When testing database-dependent code, you should put your test case in the  Database  group (see above). That tells MediaWikiTestCase to setup a database connection for you to use in. Normally, this uses a separate temporary database, with certain limited data prefilled by, including a ' UTSysop ' user and a ' UTPage ' title. A test case can add additional data to the database by overriding (which by default does nothing).

If you want to use another DB table (say, for your extension), it is not available by default for testing. To add the table to the temporary database constructed during testing, you need to add it to, for example:

However, please note that only the table schema is copied over from your actual database, not the existing data in that table.

You can directly test the current contents of the database with.

Here are some examples from extensions that you can look at for reference:


 * from Extension:BounceHandler
 * from Extension:PageAssessments

Tests that are not in the  Database  group still run against a temporary cloned database (even if they ignore and instead use e.g.  directly); however, this database is only set up once for the whole test run, and not reset between test runs. Tests should avoid relying on this safety feature if possible.

Maintenance scripts
Test cases for should inherit from , to handle the different output channels used by maintenance scripts.

The base test case method will instantiate your  object for you, if you specify the class to construct by providing the mandatory  in your subclass:

In the unlikely event that you want to do something special to instantiate the class under test, you can override the method, but hopefully this isn't needed.

By default, the maintenance script output will be suppressed and ignored. If you wish to test output (this is a good idea), use code like:

Failing
Usually tests code shouldn't do, but use the phpunit fail method:

This would show up as a failure in the testing summary rather than bringing the whole test suite down.