Selenium/How-to/Run tests using selenium-daily-SITE-EXTENSION Jenkins job

For this example, we will target English Wikipedia at the Beta Cluster. The job runs once every day, but you can also start them manually.

Advantages

• You do not have to install anything on your machine.
• You are probably already familiar with Jenkins.
• While the tests are running, you can continue using your machine as usual, because everything is happening in Jenkins.
• Debugging failed tests will be reasonably easy, since you will see logs and screenshots of the test run.
• It will be reasonable fast. It takes 1-5 minutes for a Jenkins job to run. (As of October 2019.)

Disadvantages

• You will need internet connection while running the tests.
• You might not be familiar with Jenkins, so you will have to learn how to use yet another tool.

Check if the Jenkins job is configured to run for a repository

• You can find selenium-daily-SITE-EXTENSION at Jenkins Selenium view.

• In mediawiki.yaml look for mediawiki-wdio-selenium project. (As of October 2019.)

- project:
    name: mediawiki-wdio-selenium
    project: MediaWiki
    recipients: betacluster-alerts@lists.wikimedia.org cparle@wikimedia.org qa-alerts@lists.wikimedia.org zfilipin@wikimedia.org
    repository: mediawiki/core
    jobs:
        - 'selenium-daily-{sitename}-{project}':
            sitename:
                - beta:
                    mw_server: 'https://en.wikipedia.beta.wmflabs.org'
                - betacommons:
                    mw_server: 'https://commons.wikimedia.beta.wmflabs.org'

• In mediawiki-extensions.yaml look for wdio-selenium-daily project. (As of October 2019.)

- project:
    name: wdio-selenium-daily
    project:
        - CirrusSearch:
            recipients: discovery-alerts@lists.wikimedia.org qa-alerts@lists.wikimedia.org # @dcausse @EBernhardson
            repository: mediawiki/extensions/CirrusSearch
            site: en.wikipedia
...

Run the Selenium tests

• Log in to Jenkins.
• Go to a job. (Example: selenium-daily-beta-MediaWiki.)
• Click Build with Parameters.
• Click Build.
• In Build History a new job will be blinking at the top of the list, for example #106.
• Click job number, in this case #106.
• Click Console Output.
• If the job has been running for a while, there will be Full Log link at the top of the page. You can click it to load the entire console output.

...
+ npm run-script selenium-daily
...
Spec Files:	 6 passed, 6 total (100% completed) in 00:00:29 
...