Selenium/How-to/Run tests using Quibble Jenkins job

As of September 2019, several Quibble Jenkins job run in test and gate-and-submit pipelines for a several repositories. They run automatically every time you push a mediawiki/core patch to Gerrit.

Advantages

• You do not have to install anything on your machine.
• You are probably already familiar with Gerrit and Jenkins.
• While the tests are running, you can continue using your machine as usual, because everything is happening in Jenkins.
• Debugging failed tests will be reasonably easy, since you will see logs and screenshots of the test run.

Disadvantages

• It will not be fast. It takes from about 7 to about 25 minutes for Jenkins jobs to run for mediawiki/core.
• You will need internet connection while running the tests.
• You might not be familiar with Gerrit or Jenkins, so you will have to learn how to use yet another tool.

Check if the Jenkins job is configured to run for a repository

In layout.yaml look for extension-gate or extension-quibble template:

Example for mediawiki/core:

- name: mediawiki/core
  template:
    - name: extension-gate

Example for mediawiki/extensions/RelatedArticles:

- name: mediawiki/extensions/RelatedArticles
  template:
    - name: extension-quibble

Run the Jenkins job

If the job is in test pipeline for a repository, uploading patch to Gerrit will run the job automatically. When all Jenkins jobs configured for the repository are finished, jenkins-bot will leave a comment in Gerrit with links to jobs. Click any job with Quibble in name. Find Selenium tests in the left hand side sidebar. If there are no Selenium links in the sidebar, click Full Log link at the top of the page.

Example:

...
> wdio ./tests/selenium/wdio.conf.js
...
Spec Files:	 6 passed, 6 total (100% completed) in 00:01:31
...