MediaWiki API integration tests

MediaWiki performs end-to-end integration testing for the Action API and REST API using the API-Testing library. The library is implemented in JavaScript for node.js, using the SuperTest HTTP testing library, the Chai assertion library, and the Mocha testing framework. Integration tests are run as part of continuous integration for MediaWiki Core and can be added to MediaWiki extensions and Wikimedia services.

Installation
After installing node.js and npm, use npm to install Mocha and the API-Testing package.

Configuration
To run the tests, you need access to a MediaWiki installation. We recommend using a Docker container to set up a wiki instance to test against; MediaWiki-Docker-Dev uses docker-compose to provide everything you need to run MediaWiki.

To configure API-Testing, copy the into the root folder of your application, rename the file .api-testing.config.json, and add the information for your test wiki. For a MediaWiki extension, include the config file in the root of MediaWiki Core. The example config is designed to work with a default setup of MediaWiki-Docker-Dev, just add the test wiki's secret key.

For automated testing, set the  environment variable to point to the correct configuration file. The configuration file is evaluated in the following order:


 * 1)   if set
 * 2)   if it exists

Configuration example

Configuration schema


 * Full base URI of the MediaWiki installation to target. Must end with a slash.
 * Full base URI of the MediaWiki installation to target. Must end with a slash.


 * Name of the wiki's main page
 * Name of the wiki's main page


 * Login credentials for a user that has bureaucrat privileges (most importantly, the right to add users to groups to grant them privileged access)
 * Login credentials for a user that has bureaucrat privileges (most importantly, the right to add users to groups to grant them privileged access)


 * Replace with, which can be found in the wiki's LocalSettings.php. If you are using MediaWiki-Docker-Dev, LocalSettings.php can be found under config/mediawiki/.
 * Replace with, which can be found in the wiki's LocalSettings.php. If you are using MediaWiki-Docker-Dev, LocalSettings.php can be found under config/mediawiki/.

Service setup
For non-MediaWiki applications, set the environment variable  to point to your service. For example:

Continuous integration
The Wikimedia continuous integration (CI) infrastructure supports automatic API testing for MediaWiki Core and extensions. When configured for an extension, the CI job sets up a test instance of MediaWiki, runs the MediaWiki Core tests, and then runs the extension tests. Accordingly, this integration should not be used for extensions that interfere with the behavior of Core and would cause Core API tests to fail.

To enable API tests in CI
1. Add the  script to your extension's package.json file.

MyExtension/package.json example:

2. Enable the  container in.

zuul/layout.yaml example:

After you have submitted the patch, make sure it passes experimental by committing and empty change to your extension repo and run the experimental checks.

Resetting the target wiki
Before running tests, it's advisable to ensure a known state of the wiki the tests run against. While tests should be written to be robust against pre-existing content, e.g. by randomizing all resource names, a known base state is useful. Also, test runs tend to pollute the wiki a lot, so a reset is bound to save space, even if not done for every test run.

The easiest way to achieve a known state of the wiki is to take a snapshot of a known state, preferably right after installation when the wiki contains just one page and one user, and then load that dump into the database before running tests. For convenience, two pairs of scripts are supplied to achieve this: one pair for use with a local MediaWiki installation and another pair for a MediaWiki-Docker-Dev environment.

Local snapshots
If you have MediaWiki installed locally, you can use:

This saves a snapshot of a wiki in the given tar file. The  parameter is the database name. If not given,  is used, which is the default name proposed by the MediaWiki installer. The  parameter allows the database host to be specified, in case it's not localhost.

This restores the snapshot in the given tar file. The tar file contains the name of the wiki database the snapshot was taken from. If the  parameter is not given, the dump will be loaded into that same database. The name of the database is also shown in the confirmation prompt.

Before you can use these scripts, you need to configure the location of your MediaWiki installation in bin/local.env:

Set this to something like  or wherever you have installed MediaWiki.

MediaWiki-Docker-Dev snapshots
If you have your wiki instances managed by MediaWiki-Docker-Dev, you can use:

This saves a snapshot of a wiki in the given tar file. The  parameter is the database name, which is the name you gave your wiki when running the  script. If not given, is used, which is the name of the wiki pre-installed by MediaWiki-Docker-Dev.

This restores the snapshot in the given tar file. The tar file contains the name of the wiki database the snapshot was taken from. If the  parameter is not given, the dump will be loaded into that same database. The name of the database is also shown in the confirmation prompt.

Before you can use these scripts, you need to configure the location of your MediaWiki-Docker-Dev installation in bin/local.env:

Set this to something like  or wherever you have installed MediaWiki-Docker-Dev.

Running specific tests
You can run individual test files or directories containing test files by invoking Mocha directly and pointing it to the desired path:

For more information on running Mocha tests and controlling the output, see the Mocha docs.

Writing tests
In MediaWiki Core, integration test files are stored in the. Action API tests are stored in the ; REST API tests are stored in the. For MediaWiki extensions, add tests to the tests/api-testing directory of the extension.

Each test file corresponds to a test suite covering an area of functionality. A test suite is defined by a top-level  function that takes two parameters:


 * a string describing the feature being tested
 * a function containing the test code

Within the  function, you can use the   hook to set up preconditions and the   function to create individual test cases. You can break up a long, complex test suite by nesting additional  functions under the top-level function.

Within a test suite, each test case is executed as an  function. These asynchronous functions contain  expressions to execute test steps in sequence and   expressions to evaluate responses. Assertions can use any methods supported by the Chai  interface, such as ,   (using a regular expression), and.

Here's a template that shows the main sections of a test suite:

Generating random strings
When writing tests, use random values whenever possible. This allows tests to function when run against a wiki that already contains content from previous test runs. To generate a random string for use in a test, use the  function.

Handling page titles
The API testing tool includes functions to help you manage wiki page titles.

Fixtures
To manage accounts and sessions, the API testing tool provides convenient fixtures that you can reuse across tests.

To open a wiki session and log in using a fixture, instantiate the account asynchronously using a fixtures function.

Custom accounts
If you're planning to make a permanent change to an account (for example: blocking an account) or if you need an account with a custom prefix, you can create a custom account instead of using a fixture. To open a wiki session and log in using a custom account, define a session using the  function, and log in using the   method. Applying a prefix to the username is optional.

Account properties
Once logged in, you can access information about the account using the,  , and   properties.

Anonymous users
To create an anonymous user, define the account using the  function, but omit the   function. This opens a new session without logging in, resulting in an anonymous user.

Working with wiki pages
The API testing tool provides helpful methods for interacting with wiki pages, including editing a page, exploring page history, and retrieving page HTML. To create and edit a page, use the  function to generate a random title and the   method to edit the page. To validate the edit, you can use the  method to get the HTML of the page and the   method to check for the edited text.

Calling the Action API
The Action API list, meta, and prop modules provide access to information about wiki pages and users. The testing tool provides methods that let you make GET requests to these modules within tests.

Other Action API calls
To call any module in the Action API, use the  method.

Some Action API calls require a token. See individual API action docs for token requirements. For example, to patrol a page, make a POST request to the  action. In the API:Patrol docs, we can see that this call requires a token, which we can get using the  method.

The  method fails if the response contains an error code. To test for expected errors, you can use the  method.

Action API example test
The Recent Changes test suite contains two tests cases that validate a user's ability to access recent changes for a page.

Calling the REST API
The testing tool provides methods that let you make requests to the MediaWiki REST API. To open a REST API session, use the  function within the top level   function. REST API sessions are anonymous.

To make a request, use the corresponding method for the HTTP request method:


 * : Make an HTTP GET request to the REST API
 * : Make an HTTP POST request to the REST API
 * : Make an HTTP PUT request to the REST API
 * : Make an HTTP DELETE request to the REST API

These methods can take up to three arguments:


 * The endpoint path as a string, starting after the version number (Example:  for the compare revisions endpoint)
 * If supported by the endpoint, a request body as an object
 * If supported by the endpoint, the content-type as a string. Defaulting to

See the REST API docs for endpoint paths and request body requirements.

REST API example test
Here's an example test suite with two tests for the get revision endpoint.

Contributing
The API tests are hosted on Gerrit and mirrored on GitHub. To open a patch request, see the guide to using Gerrit for Wikimedia projects.

To review open tasks or file a bug report, visit Phabricator.

Sharing feedback
To share your feedback about this page or to ask a question about API tests, leave a comment on the talk page.