Design System Team/Test environments

There's a range of test environments for Codex, both outside and inside of MediaWiki to your delivery. This page documents these environments.

Codex repository

Test suite

The Codex monorepo contains various tests for the different packages. You can run npm run test locally to run all tests. Jest is used for unit testing and will output a coverage report in the console after running the tests. Read more about unit testing in Codex.

Netlify previews

When a patch is pushed to the Codex repository in Gerrit, an instance of the documentation site will be generated and launched via Netlify. Access the site by visiting the URL with the pattern https://PATCH_ID_NUMBER--wikimedia-codex.netlify.app/, swapping out PATCH_ID_NUMBER for the Gerrit patch ID (the number at the end of the URL, not the Change-Id). For example, the Netlify preview for this patch is available at https://953740--wikimedia-codex.netlify.app/.

You can also view the Sandbox page by adding /sandbox to the URL, e.g. https://953740--wikimedia-codex.netlify.app/sandbox.

Continuous integration (CI)

The Codex repository in Gerrit relies on Jenkins to run a set of tasks for patches. Currently there are 2 jobs:

Tests: Builds the library, runs all tests, and diffs the compiled CSS
Netlify preview: Deploys the current branch to Netlify

You can click on the console output of these jobs for an individual patch to view things like:

Test build:
- Test failures
- The Jest coverage report
- The CSS diff, which is an un-minified, prettified diff of the compiled CSS. This is useful to ensure that style or token changes have no unintended effects.
Netlify preview:
- Build failures
- The link to the Netlify preview (at the bottom of the output)

You can always view the status of in-progress jobs on the Zuul status page.

MediaWiki integrations

CodexExample

CodexExample is a MediaWiki extension meant to be an example of how to use Codex within MediaWiki. It contains uses of Vue components, CSS-only components, icons, and design tokens, along with comments explaining exactly how to use them. The README also identifies where these things are used in the code.

Pixel

Pixel is a visual regression testing tool developed by WMF engineers for testing with BackstopJS in MediaWiki. Pixel contains a dockerized MediaWiki environment, including a set of skins and extensions. Config can be added to run snapshot testing of a reference versus a test case. This can be done locally by pulling down the repository, installing it, taking reference snapshots, and running tests. There is also a live instance with a web UI that runs tests of the latest release branch of core and all extensions and skins versus the master branch, once per hour.

For Codex, we test a SpecialPage generated by the CodexExample extension. To easily run the tests, you can open and merge a patch to update the version of Codex in CodexExample (see CodexExample README). The next time the live Pixel tests run, you will see results based on the newly updated master branch of CodexExample.

Instructions for running Pixel tests on an unmerged Codex patch:

Clone the Pixel repo if you haven't already
If you haven't run Pixel in a while (or ever), run git pull, npm install, and ./clean.sh
If you are running MediaWiki through Docker, make sure it isn't running. Check that http://localhost:8080 doesn't show anything. If it is running, run docker compose down in the MediaWiki directory.
On some systems, you also have to delete the 'report' directory every time at start, because of weird permissions issues. sudo rm -rf report . You may or may not need this.
Take the 'reference' screenshots (think of this as the "before" in your before/after comparison) with ./pixel.js reference -g codex . Not passing any further arguments means it'll use the main branch of Codex. (If this is your first Pixel run since running clean.sh this will take a while because it has to do a bunch of setup; otherwise it should only take 2-3 minutes).
Take the "test" screenshots (the "after") with ./pixel.js test -g codex -c 1056227 . The -c flag makes it check out the Gerrit change with that number. Expect about 2-3 minutes) for this step.
Open report/codex/index.html in a browser to see the results. This should happen automatically when Pixel finishes running.

Patchdemo

Patchdemo is an invaluable tool for generating a sharable MediaWiki instance based on a patch or set of patches. One use case is testing the core patch to update Codex. Some tips for using patchdemo:

You can add more extensions by expanding the "Choose included repos" toggle. For example, you might include CodexExample if you want to see the component demos.
If you want to test TypeaheadSearch or any project that relies on content, check the "Proxy articles from wikipedia.org" box to import content into your test instance
You can log in to several existing accounts in your test instance with password "patchdemo1":
- Patch Demo (admin)
- Alice
- Bob
- Mallory (blocked user)

Test and beta wikis

There are various test wikis available to test features in a safe environment that is at least somewhat similar to production. The test wikis run the latest production code, while the beta versions run the master branch of core and other repositories. Beta wikis are useful for testing features after they have been merged but before they have ridden the deployment train. Some useful beta sites include: