Browser testing/Guidelines and good practices
|This page is obsolete. It is kept for historical interest only. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date.|
See Selenium instead.
|It has been suggested that this article or section be merged with Quality Assurance/Browser testing/Writing tests. (Discuss)|
- 1 Guidelines for writing browser tests
- 1.1 Do not put comments into Features/Scenarios
- 1.2 Say exactly what the test step does
- 1.3 Alphabetize Given/When/Then steps in the foo_steps.rb files
- 1.4 Avoid TODO and WIP
- 1.5 Be simple instead of clever
- 1.6 Don’t use element locators in steps files
- 1.7 Always have an assertion in a Then step
- 1.8 Don’t worry too much about DRY code
- 1.9 Use quotes for arguments in Feature steps
- 1.10 Prefer CSS selectors for page elements
- 1.11 Ruby stuff
Guidelines for writing browser tests
NOTE: If you are just starting with Cucumber and Page Objects, read the Quality Assurance/Browser testing/Very Basic Howto
Our recommendations boil down to this:
Be neat and be conscientious
Don’t be clever
Some examples of being neat and conscientious:
Do not put comments into Features/Scenarios
The intent of Features and Scenarios is to describe what it means for the feature in question to be functioning correctly. In a sense, all of the foo.feature files are 100% comments. If you have an urge to put a comment into a Feature or Scenario, consider improving the description of the test step instead.
For example, this
And I have created a Flow topic # which is not hidden (this is implicit from the above step)
should be something like
And I have created a Flow topic that is not hidden
Say exactly what the test step does
Don't do this:
Given(/^I am anonymous$/) do on(FlowPage).logout_element.click visit FlowPage end
Do this instead:
Given(/^I log out from a logged in session$/) do on(FlowPage).logout_element.click visit FlowPage end
(Except you should never log out of a test, see below)
Don’t do this:
Given(/^the page has re-rendered$/) do sleep 10 end
Do this instead:
Given(/^I pause for ten seconds$/) do sleep 10 end
(Except don’t use sleep! See below)
Alphabetize Given/When/Then steps in the foo_steps.rb files
Given aa Given ab When aa When ab Then aa
This makes it much easier to maintain the tests into the future.
Avoid TODO and WIP
If you must have TODO and WIP things in your test files, don’t check them into the master branch. These make maintenance and understanding difficult for others. Far better to have minimal clear test coverage than complex test coverage not implemented.
Be simple instead of clever
Some examples of being simple instead of clever:
Never use sleep()
Sleep is always a bad practice and it is (almost) never necessary. If you can’t find a way to make the test work without sleep(), ask Chris or Zeljko or Nik or Dan, or send a note to the QA mail list, we’ll sort it out.
We share users among all the test repos. If you need to be an anonymous user for a test, create a separate test. Logging out from your test logs out the user for every other test that is running.
If you need to be logged in to create some test data in a Given step, use the API instead, see Quality Assurance/Browser testing/Writing tests#API login.
Conditionals are a smell
When a test has a conditional, you won’t know which branch the test took until it fails. This can mask actual problems. The only exception is e.g. if you want to re-load a page some number of times and break out of the loop if a condition is met.
Loops are a smell
Cucumber allows “Scenario Outlines” (https://github.com/cucumber/cucumber/wiki/Scenario-Outlines) Use these instead of loops within the foo_steps.rb file.
- (Here is an exception to the loops and conditionals rule. Note that this is in support of the test steps and not implicit in the test:)
def poll_for_new_notifications(number_of_polls) number_of_polls.to_i.times do step 'I am on the "Selenium Echo flyout test page" page' break if on(ArticlePage).flyout_link_element.class_name =~ /mw-echo-unread-notifications/ end end
Don’t use element locators in steps files
Element locators should always be in the page object in the foo.page file. This abstraction is critical for maintenance.
Instead of this
Then(/^I see only one topic on the page$/) do on(FlowPage).div_element(css: '.flow-topic', index:1).when_not_present
Then(/^I see only one topic on the page$/) do expect(on(FlowPage).flow_second_topic_heading_element).not_to be_visible
Always have an assertion in a Then step
In the old style, this is the word “should”. In the new style, this is the word
expect. Sometimes when AJAX is involved this means that you have to do something that seems like duplication but it is not:
Then(/^the top post is an open discussion$/) do on(FlowPage) do |page| page.flow_first_topic_moderation_msg_element.when_not_present expect(page.flow_first_topic_moderation_msg_element).not_to be_visible end end
The following is not correct, because
when_not_present is not an assertion:
Then(/^the top post is an open discussion$/) do on(FlowPage).flow_first_topic_moderation_msg_element.when_not_present #THIS IS NOT AN ASSERTION end
Don’t worry too much about DRY code
A couple of things about DRY:
Simple is better than complicated. Repeating steps and code is OK in these sorts of tests. Better to be readable, understandable, and maintainable than DRY and hard to understand.
And a note about the Page Object design pattern: PO allows to address particular page elements in different contexts according to our understanding. Say for an example we have a button with id=my_button. We might address the same button in different contexts:
and this is perfectly OK, because it allows us to understand how the tests are using that element.
Use quotes for arguments in Feature steps
Don't do this:
And the Reason of the first topic is This is a bikeshed
Do this instead:
And the Reason of the first topic is "This is a bikeshed"
and it is implemented like so:
Then(/^the Reason of the first topic is "(.*?)"$/) do |text| on(FlowPage).flow_reason_element.text.should match text end
Prefer CSS selectors for page elements
Typically CSS selectors are more generally useful than e.g. class selectors
The page_object framework allows us an extraordinary level of abstraction. If you find yourself writing a whole lot of raw Ruby, you almost certainly should be using some aspect of the page_object framework instead. Again, feel free to ask Chris, Zeljko, Nik, Dan, and the QA mail list if you find yourself going down a Ruby rat hole.