Selenium/Getting Started/Run tests targeting Quibble

This page describes how to run Selenium tests locally against your own Quibble MediaWiki install.

All commands have been tested on macOS and Ubuntu. Everything works on Ubuntu. There are some problems on macOS. No testing was done on Windows.

Advantages

• It will be fast, since the target machine is local. MediaWiki core Selenium test run takes about 3 minutes on my machine. (As of February 2026.)
• Works without an internet connection.
• Doesn't require installing ChromeDriver or FFmpeg (compared to bare installs).
• Doesn't require executing npm packages directly on your machine (which is scary).

Disadvantages

• You need to have Docker installed.

Prerequisites

• Git.
• Docker. See Docker CE for Linux, Docker for Mac, or Docker for Windows.

Install MediaWiki Core

You only have to do these steps once.

# get the latest image
docker pull docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest

# create a folder to hold all files, quibble is just an example name
mkdir quibble
cd quibble

# create required folders
mkdir -p ref/mediawiki/skins

# clone required repositories
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/core ref/mediawiki/core.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/vendor ref/mediawiki/vendor.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/skins/Vector ref/mediawiki/skins/Vector.git

# create required folders and make them writable by any user
mkdir cache
chmod 777 cache
mkdir -p log
chmod 777 log
mkdir -p src
chmod 777 src

# install MediaWiki
docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --command true

All log files are located in log folder.

For --entrypoint=quibble-with-supervisord see T397810.

Run Selenium Tests for MediaWiki Core

In the previous step, MediaWiki Core was installed. If you only want to run Selenium tests, you can speed things up by skipping MediaWiki installation (with --skip-zuul and --skip-deps).

Run All Tests

To run only Selenium tests (and skip other tests) use --run selenium.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --run selenium

Run All Tests Faster

If you set CI environment variable, tests will run in parallel and they should finish faster. I tested it on a couple of machines. Core tests took about 2-3 minutes, depending on the machine. With CI environment variable set, they took about 40 seconds on both machines.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e CI=true \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --run selenium

Run One File

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --command "npm run selenium-test -- --spec tests/selenium/wdio-mediawiki/specs/BlankPage.js"

tests/selenium/wdio-mediawiki/specs/BlankPage.js is the file name.

Run One Test

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --command "npm run selenium-test -- --spec tests/selenium/specs/user.js --mochaOpts.grep 'should be able to create account'"

'should be able to create account' is the test name.

Open Shell

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --command bash

In the shell you can, for example, run a test.

nobody@f928f5d19250:/workspace/src$ npm run selenium-test -- --spec tests/selenium/wdio-mediawiki/specs/BlankPage.js

There is another way to open shell (with --shell) but it is broken at the moment. See T418461.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --shell

Open MediaWiki in a Browser

Run the command for opening shell (so the container is running). Please notice --network host.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  --network host \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --command bash

Visit http://127.0.0.1:9413 in your browser.

The above command works only on Linux. On macOS, use the next command. Please notice -p 9413:9413.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -p 9413:9413 \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --command bash

In case MediaWiki is not available, you can check on which port it is running.

nobody@b1a019322d8b:/workspace/src$ env | grep MW_SERVER
MW_SERVER=http://127.0.0.1:9413

Install MediaWiki Core and an Extension

When installing an extension, you have to reinstall MediaWiki. Once the extension is installed, you can run it's tests without installing the extension again. Just use --skip-zuul and --skip-deps.

On macOS, if you get stderr: 'warning: failed to remove node_modules/: Directory not empty', remove node_modules/ with rm -rf src/node_modules.

Extension with No Dependencies

Install MediaWiki Core with an Extension. In this example, we will use Extension:Echo.

# if not already in quibble folder, go there
cd quibble

# create required folders
mkdir -p ref/mediawiki/extensions

# clone required repositories
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/extensions/Echo ref/mediawiki/extensions/Echo.git

# set up MediaWiki and run all tests
docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e ZUUL_PROJECT=mediawiki/extensions/Echo \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --command true

Run All Tests

If you have already installed the extension, you can skip installation and just run the tests.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e ZUUL_PROJECT=mediawiki/extensions/Echo \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --run selenium

Run One File

If you have already installed the extension, you can skip installation and run just one file.

docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e ZUUL_PROJECT=mediawiki/extensions/Echo \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --skip-zuul \
  --skip-deps \
  --command "cd extensions/Echo && npm run selenium-test -- --spec tests/selenium/specs/notifications.js"

Extension with Dependencies in extension.json

Every time you want to install a new extension, you have to reinstall MediaWiki Core. In this example, we will use Extension:CampaignEvents.

CampaignEvents has Extension:cldr as an dependency in extension.json.

{
	"requires": {
		"extensions": {
			"cldr": "*"
		}
}

Install MediaWiki Core with CampaignEvents and it's dependency, cldr. Please notice --resolve-requires.

# if not already in quibble folder, go there
cd quibble

# create required folders
mkdir -p ref/mediawiki/extensions

# clone required repositories
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/extensions/CampaignEvents ref/mediawiki/extensions/CampaignEvents.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/extensions/cldr ref/mediawiki/extensions/cldr.git

# set up MediaWiki and run all tests
docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e ZUUL_PROJECT=mediawiki/extensions/CampaignEvents \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --resolve-requires \
  --run selenium

Extension with Dependencies Not in extension.json

Every time you want to install a new extension, you have to reinstall MediaWiki Core. In this example, we will use Extension:ProofreadPage.

ProofreadPage has no dependencies in extension.json, but for Selenium tests to pass, Extension:ParserFunctions has to be installed.

Install MediaWiki Core with ProofreadPage and it's dependency, ParserFunctions. Please notice mediawiki/extensions/ParserFunctions after --run selenium.

# if not already in quibble folder, go there
cd quibble

# create required folders
mkdir -p ref/mediawiki/extensions

# clone required repositories
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/extensions/ProofreadPage ref/mediawiki/extensions/ProofreadPage.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/extensions/ParserFunctions ref/mediawiki/extensions/ParserFunctions.git

# set up MediaWiki and run all tests
docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e ZUUL_PROJECT=mediawiki/extensions/ProofreadPage \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --run selenium \
  mediawiki/extensions/ParserFunctions

Install MediaWiki Core and a Skin

Every time you want to install a new skin, you have to reinstall MediaWiki Core. In this example, we will use Skin:Minerva Neue, since it is the only skin with Selenium tests.

Minerva Neue has no dependencies in skin.json, but for Selenium tests to pass, Extension:MobileFrontend has to be installed.

Install MediaWiki Core with MinervaNeue and it's dependency, MobileFrontend. Please notice mediawiki/extensions/MobileFrontend after --run selenium.

# if not already in quibble folder, go there
cd quibble

# clone required repositories
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/skins/MinervaNeue ref/mediawiki/skins/MinervaNeue.git
git clone --bare https://gerrit.wikimedia.org/r/mediawiki/extensions/MobileFrontend ref/mediawiki/extensions/MobileFrontend.git

# set up MediaWiki and run all tests
docker run -it --rm \
  --entrypoint=quibble-with-supervisord \
  -v "$(pwd)"/cache:/cache \
  -v "$(pwd)"/log:/workspace/log \
  -v "$(pwd)"/ref:/srv/git:ro \
  -v "$(pwd)"/src:/workspace/src \
  -e ZUUL_PROJECT=mediawiki/skins/MinervaNeue \
  docker-registry.wikimedia.org/releng/quibble-bullseye-php83:latest \
  --run selenium \
  mediawiki/extensions/MobileFrontend