Reading/Web/QA

Browser tests

 * What are browser tests? Quality_Assurance/Browser_testing
 * Setup browser tests - Quality_Assurance/Browser_testing/Setup_instructions;
 * Run browser tests - Quality_Assurance/Browser_testing/Running_tests;
 * Have Jenkins execute browser tests for every patchset submitted to Gerrit - Continuous_integration/Browser_tests;
 * Write your own browser tests - Quality_Assurance/Browser_testing/Writing_tests
 * Useful links - Reading/Web/Selenium_tests;
 * Some tips on writing automation tests - Reading/Web/Automation CheatSheet.

Running browser tests on the beta cluster
Set the following variables, using an existing account for username and password: Note that certain tests may require your username to have certain permissions set.

Simulating browser test run on an unmerged patchset.
The way we test such change is to create a copy of the existing Jenkins job, disable all notifications and then fetch and checkout the patchset.

Steps:
 * For example, you want to test 298959
 * Log in to Jenkins
 * Click New Item link in the left sidebar
 * A form would show
 * Item Name: selenium-(repository)-(gerrit-patch-number), so in this example selenium-Wikidata-298959
 * Copy existing Item: selenium-(repository), so in this example selenium-Wikidata
 * Save form
 * Go to the job configuration
 * Click Source Code Management > Git > Advanced
 * Change Refspec from +refs/heads/*:refs/remotes/origin/* to refs/changes/59/298959/1
 * You can get refs/changes... from Gerrit patch
 * Go to the patch 298959
 * click Download
 * URL in Pull will be git pull https://zfilipin@gerrit.wikimedia.org/r/wikidata/browsertests refs/changes/59/298959/1
 * You only need the last section, refs/changes/59/298959/1
 * Change Branch Specifier (blank for 'any') from master to FETCH_HEAD
 * Uncheck Build periodically
 * Delete the sections:
 * Execute shell (only one that starts with # Attempt to figure out MediaWiki branch being used and fetch it out)
 * Editable Email Notification
 * IRC Notification
 * Save and trigger a build of that item-name to run browser tests.

Troubleshooting "flakey tests"
Sometimes our beta cluster integration jobs fail, even though they pass when we merge code and a manual test shows that functionality is working as expected. This is often due to problems with the beta cluster's availability or performance and how the tests are written not the thing they are testing.

Use you can point Selenium jobs to unmerged patches, to test changes to browser tests by cloning an existing job by clicking "New Item" on the dashboard and using "copy from". This results in a duplicate job with a different name e.g. https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MinervaNeue-BugFix/

You can use this job to point run tests from another code path:
 * Click configure
 * Find on page "gerrit". In build "Execute shell" you'll see an explicit checkout command. You can update this line to check out any code for that repository.
 * Run the build (using "Build now")

Current browser test results

 * MobileFrontend
 * Other tests