MediaWiki API integration tests

The API testing tool is a standalone project that contains end-to-end integration tests for the MediaWiki Action API and REST API. You can run the tests locally by downloading the testing tool and configuring it to access a test wiki. The tests are implemented in JavaScript for node.js, using the SuperTest HTTP testing library, the Chai assertion library, and the Mocha testing framework.

Installation
To run the API test suite, you need to have node.js and npm installed.

Installation of the testing environment can then be done with npm:

Test wiki setup
To run tests, you need to have a MediaWiki installation to run them against. Ideally, tests would run against an empty, freshly initialized wiki each time, but tests should be written to also function when run against a wiki that already has content from previous test runs and manual testing.

The recommended way to set up a wiki instance to test against is to spin up a docker container that provides a wiki. The MediaWiki-Docker-Dev makes this simple, just follow the instructions provided in the README there. MediaWiki-Docker-Dev uses docker-compose to provide all necessary services for MediaWiki, such as a database and caching.

Configuration
The testing tool needs some configuration that tells it how to access the target wiki. A default configuration is contained in the  file:

The default values were chosen to "Just Work" with a default setup of MediaWiki-Docker-Dev (see above), you just have to supply the target wiki's in the   configuration option. can be found in the wiki's LocalSettings.php - if you are using MediaWiki-Docker-Dev, this file can be found under.

To use a custom configuration file, create a `config.local.json file in the root folder. For automated testing, an environment variable `API_TESTING_CONFIG_FILE` can be set to point to the correct configuration file.

The configuration file is evaluated in the following order:


 * 1)   if set
 * 2)   if it exists

If you have a custom setup, you need to provide the following configuration settings:

Must end with a slash. (most importantly, the right to add users to groups to grant them privileged access).
 * : Full base URI of the MediaWiki installation to target.
 * : The name of the wiki's main page.
 * : Login credentials for a user that has bureaucrat privileges

Resetting the target wiki
Before running tests, it's advisable to ensure a known state of the wiki the tests run against. While tests should be written to be robust against pre-existing content, e.g. by randomizing all resource names, a known base state is useful. Also, test runs tend to pollute the wiki a lot, so a reset is bound to save space, even if not done for every test run.

The easiest way to achieve a known state of the wiki is to take a snapshot of a known state, preferably right after installation when the wiki contains just one page and one user, and then load that dump into the database before running tests. For convenience, two pairs of scripts are supplied to achieve this: one pair for use with a local MediaWiki installation and another pair for a MediaWiki-Docker-Dev environment.

Local snapshots
If you have MediaWiki installed locally, you can use:

This saves a snapshot of a wiki in the given tar file. The  parameter is the database name. If not given,  is used, which is the default name proposed by the MediaWiki installer. The  parameter allows the database host to be specified, in case it's not localhost.

This restores the snapshot in the given tar file. The tar file contains the name of the wiki database the snapshot was taken from. If the  parameter is not given, the dump will be loaded into that same database. The name of the database is also shown in the confirmation prompt.

Before you can use these scripts, you need to configure the location of your MediaWiki installation in bin/local.env:

Set this to something like  or wherever you have installed MediaWiki.

MediaWiki-Docker-Dev snapshots
If you have your wiki instances managed by MediaWiki-Docker-Dev, you can use:

This saves a snapshot of a wiki in the given tar file. The  parameter is the database name, which is the name you gave your wiki when running the  script. If not given, is used, which is the name of the wiki pre-installed by MediaWiki-Docker-Dev.

This restores the snapshot in the given tar file. The tar file contains the name of the wiki database the snapshot was taken from. If the  parameter is not given, the dump will be loaded into that same database. The name of the database is also shown in the confirmation prompt.

Before you can use these scripts, you need to configure the location of your MediaWiki-Docker-Dev installation in bin/local.env:

Set this to something like  or wherever you have installed MediaWiki-Docker-Dev.

Running all tests
To run all tests containing in the test directory:

Running specific tests
You can run individual test files or directories containing test files by invoking Mocha directly and pointing it to the desired path:

For more information on running Mocha tests and controlling the output, see the Mocha docs.

Writing tests
Action API tests are stored in the /test directory; REST API tests are stored in the /test/REST directory. Each file corresponds to a test suite covering an area of MediaWiki functionality. A test suite is defined by a top-level  function that takes two parameters:


 * a string describing the feature being tested
 * a function containing the test code

Within the  function, you can use the   hook to set up preconditions and the   function to create individual test cases. You can break up a long, complex test suite by nesting additional  functions under the top-level function.

Within a test suite, each test case is executed as an  function. These asynchronous functions contain  expressions to execute test steps in sequence and   expressions to evaluate responses. Assertions can use any methods supported by the Chai  interface, such as ,   (using a regular expression), and.

Here's a template that shows the main sections of a test suite:

Generating random strings
When writing tests, use random values whenever possible. This allows tests to function when run against a wiki that already contains content from previous test runs. To generate a random string for use in a test, use the  function.

Handling page titles
The API testing tool includes functions to help you manage wiki page titles.

Fixtures
To manage accounts and sessions, the API testing tool provides convenient fixtures that you can reuse across tests.

To open a wiki session and log in using a fixture, instantiate the account asynchronously using a fixtures function.

Custom accounts
If you're planning to make a permanent change to an account (for example: blocking an account) or if you need an account with a custom prefix, you can create a custom account instead of using a fixture. To open a wiki session and log in using a custom account, define a session using the  function, and log in using the   method. Applying a prefix to the username is optional.

Account properties
Once logged in, you can access information about the account using the,  , and   properties.

Anonymous users
To create an anonymous user, define the account using the  function, but omit the   function. This opens a new session without logging in, resulting in an anonymous user.

Working with wiki pages
The API testing tool provides helpful methods for interacting with wiki pages, including editing a page, exploring page history, and retrieving page HTML. To create and edit a page, use the  function to generate a random title and the   method to edit the page. To validate the edit, you can use the  method to get the HTML of the page and the   method to check for the edited text.

Calling the Action API
The Action API list, meta, and prop modules provide access to information about wiki pages and users. The testing tool provides methods that let you make GET requests to these modules within tests.

Other Action API calls
To call any module in the Action API, use the  method.

Some Action API calls require a token. See individual API action docs for token requirements. For example, to patrol a page, make a POST request to the  action. In the API:Patrol docs, we can see that this call requires a token, which we can get using the  method.

The  method fails if the response contains an error code. To test for expected errors, you can use the  method.

Action API example test
The Recent Changes test suite contains two tests cases that validate a user's ability to access recent changes for a page.

Calling the REST API
The testing tool provides methods that let you make requests to the MediaWiki REST API. To open a REST API session, use the  function within the top level   function. REST API sessions are anonymous.

To make a request, use the corresponding method for the HTTP request method:


 * : Make an HTTP GET request to the REST API
 * : Make an HTTP POST request to the REST API
 * : Make an HTTP PUT request to the REST API
 * : Make an HTTP DELETE request to the REST API

These methods take two arguments:


 * The endpoint path as a string, starting after the version number (Example:  for the compare revisions endpoint)
 * If supported by the endpoint, a request body as an object

See the REST API docs for endpoint paths and request body requirements.

REST API example test
Here's an example test suite with two tests for the get revision endpoint.

Contributing
The API tests are hosted on Gerrit and mirrored on GitHub. To open a patch request, see the guide to using Gerrit for Wikimedia projects.

To review open tasks or file a bug report, visit Phabricator.

Sharing feedback
To share your feedback about this page or to ask a question about API tests, leave a comment on the talk page.