Design Systems Team/Test environments

The Design Systems Team uses a range of test environments for its work with Codex, both outside and inside of MediaWiki. This page documents these environments.

Test suite
The Codex monorepo contains various tests for the different packages. You can run  locally to run all tests. Jest is used for unit testing and will output a coverage report in the console after running the tests. Read more about unit testing in Codex.

Netlify previews
When a patch is pushed to the Codex repository in gerrit, an instance of the documentation site will be generated and launched via Netlify. You can access the site by visiting the URL with the pattern, swapping out PATCH_NUMBER for the gerrit patch ID (the number at the end of the URL). For example, the Netlify preview for this patch is available at https://953740--wikimedia-codex.netlify.app/.

You can also view the Sandbox page by adding  to the URL, e.g. https://953740--wikimedia-codex.netlify.app/sandbox.

CI
The Codex repository in gerrit relies on Jenkins to run a set of tasks for patches. Currently there are 2 jobs:


 * 1) Tests: Builds the library, runs all tests, and diffs the compiled CSS
 * 2) Netlify preview: Deploys the current branch to Netlify

You can click on the console output of these jobs for an individual patch to view things like:

You can always view the status of in-progress jobs on the Zuul status page.
 * Test build:
 * Test failures
 * The Jest coverage report
 * The CSS diff, which is an un-minified, prettified diff of the compiled CSS. This is useful to ensure that style or token changes have no unintended effects.
 * Netlify preview:
 * Build failures
 * The link to the Netlify preview (at the bottom of the output)

VueTest
VueTest is a MediaWiki extension meant to be a testing ground for Vue and Codex work, internal to the Design Systems Team. VueTest uses an internal version of Codex, allowing us to update the version past the version that's included in MediaWiki. It contains a special page of component demos that are pulled from the  package. This page can be used to view Codex components in a MediaWiki context, and it is used for visual regression testing with Pixel (see below). See the VueTest README for more information.

CodexExample
CodexExample is a MediaWiki extension meant to be an example of how to use Codex within MediaWIki for Codex users. It contains uses of Vue components, CSS-only components, icons, and design tokens, along with comments explaining exactly how to use them. The README also identifies where these things are used in the code.

Pixel
Pixel is a visual regression testing tool developed by WMF engineers for testing with BackstopJS in MediaWiki. Pixel contains a dockerized MediaWiki environment, including a set of skins and extensions. Config can be added to run snapshot testing of a reference versus a test case. This can be done locally by pulling down the repository, installing it, taking reference snapshots, and running tests. There is also a live instance with a web UI that runs tests of the latest release branch of core and all extensions and skins versus the master branch, once per hour.

For Codex, we test a SpecialPage generated by the VueTest extension. To easily run the tests, you can open and merge a patch to update the version of Codex in VueTest (see VueTest README). The next time the live Pixel tests run, you will see results based on the newly updated master branch of VueTest.

Patchdemo
Patchdemo is an invaluable tool for generating a sharable MediaWiki instance based on a patch or set of patches. One use case is testing the core patch to update Codex. Some tips for using patchdemo:


 * You can add more extensions by expanding the "Choose included repos" toggle. For example, you might include VueTest if you want to see the component demos.
 * If you want to test TypeaheadSearch or any project that relies on content, check the "Proxy articles from wikipedia.org" box to import content into your test instance
 * You can log in to several existing accounts in your test instance with password "patchdemo":
 * Patch Demo (admin)
 * Alice
 * Bob
 * Mallory (blocked user)

Test and beta wikis
There are various test wikis available to test features in a safe environment that is at least somewhat similar to production. The test wikis run the latest production code, while the beta versions run the master branch of core and other repositories. Beta wikis are useful for testing features after they have been merged but before they have ridden the deployment train. Some useful beta sites include:


 * Beta English Wikipedia
 * Beta Commons
 * Beta Wikifunctions