Manual:Coding conventions/Selenium-Ruby

This page describes coding conventions we follow when writing browser tests. The links section of the mediawiki-selenium README file lists the repositories that contain browser tests. There are three types of Cucumber test files in the MediaWiki codebase: Cucumber features files, Cucumber step definition files and page object files.

Željko Filipin comments I pair with people from different teams and I work on the mediawiki-selenium gem, so I tend to see a lot of Ruby, Selenium, Cucumber and page-object code. I have noticed some code that I like and some code that I do not like.

To move the code towards what I like, I have been working on this page. I plan to implement the changes myself, if nobody does it before I have the time to do it. I do not have a timeline for that.

I would really appreciate if you would read the page (it should take you 5 minutes or so) and let me know if you agree or disagree with my thoughts, if I missed something, if something needs to be explained, if the page should have more text and less code or vice versa...

You can let me know in the page discussion section, at QA mailing list, at freenode IRC channel (I am zeljkof there). Also, feel free to update the page.

Whitespace
Selenium tests should use the same whitespace convention (tabs, spaces...) that the repository already uses. That makes it easy for developers to work on tests. This convention is currently used in most places.

tests/browser
If possible, Selenium tests should be located in  folder. This convention is currently used in most places.

Cucumber feature files
For example file see any file in  folder of qa-browsertests repository. Feature files usually contain features and scenarios.

Scenarios

 * Every scenario should be as simple as possible. It is better for a feature to have more smaller scenarios than just a few big scenarios that try to test everything. This convention is currently used in most places.
 * The scenarios are supposed to be as human-readable as possible. Do not turn them into a programming language - they are supposed to be a communication tool between users, product managers, testers and developers.
 * Avoid mentioning implementation details in the scenarios.
 * Whenever possible, specify actual test data strings in Scenarios and pass those to the tests using the regex capture aspect of Cucumber.

Example of article title defined in the page object (bad)

Fixed: article title is passed from the Scenario to the step to the page object. (good):

also

Example of text typed in to page only in a step and checked only in a step (bad):

Fixed, text to be typed into page is specified in the Scenario and result to be checked also specified in the Scenario (good):

Alphabetically sorted
If a feature or scenario has more than one tag, they should be sorted alphabetically. This convention is currently used in most places.

Example (good):

Required tags
Every feature in a feature file should have site- and browser- specific tags. Scenarios inherit tags from the features they belong to (see Cucumber tags documentation), so if for example the entire feature runs on a specific browser, you only need tag the feature at the top of the file, not individual scenarios.
 * A site-specific tag is for example  or  . The tag specifies where the feature or scenario should run. This convention is currently used in most places.
 * is a special case of a site-specific tag is . If the feature or scenario runs fine on a clean wiki, it should be tagged . This convention is currently not used.  Except for a single test in the /qa/browsertests repo, at the moment we tag features or scenarios that are known to fail on a clean wiki with.
 * A browser-specific tag is for example  or  . These tags specify which browsers can run the feature or scenario. This convention is used in most projects that have CI browser tests.  We also tag features or scenarios if they are known to fail with a specific browser, for example.

Optional tags
Some features or scenarios can have an optional tag.
 * If the feature or scenario requires the user to log in, it should have  tag. This convention is currently used in most places.
 * If the feature or scenario requires custom browser configuration, it should be tagged . This convention is currently used in most places.
 * We need to tag tests that are quick or slow to run. My suggestion is  and   tags.   tag could be used to create a Jenkins job that would run after every patch set submission to Gerrit, or every time a commit is merged into master branch.   tag could be used to create a Jenkins job that would run once a day. This convention is currently not used.

Extension tags
If your feature requires the presence of optional MediaWiki extensions, be sure to include an  tag for each. These tags will help keep your test suite more deterministic by allowing Cucumber to skip features that would otherwise fail falsely due to the wiki's configuration.

@extension-visualeditor Feature: VisualEditor Mobile

Scenario: VisualEditor Provides Bold Given I am logged into the mobile website When I look at the VisualEditor toolbar Then I see a bold button

Before attempting to execute your feature, Cucumber will check that the wiki has these extensions installed and enabled. If any of the dependencies aren't met, the runner will skip the feature and warn the user.

Cucumber step definition files
For example file see any file in   folder of qa-browsertests repository. Step definition files usually contain,   and   steps.

Page object pattern
Direct calls to Selenium function are not supposed to be used. Use PageObject. This convention is currently used in most places.

Example (good):

Example (bad):

Simplicity
In general, code in step definition files should be as simple as possible. Ideally, just one or two lines per step. All complicated code should be moved to page objects. This convention is currently used in most places.

Example (good):

Grouped by type
Steps should be grouped by type. and  steps should be grouped in one group,   steps should be grouped separately.

Example (good):

This convention is currently not used. At the moment we do not combine   and   steps in one group.

Alphabetically sorted
Inside a group, steps should be sorted alphabetically by step name. That moves steps with similar name close to each other. Steps with similar name usually have similar functionality, making them good candidates for merging. This convention is currently used in most places.

Example (good):

Assertions
We are using rspec-expectations assertions in step definition files. Assertions should be used only in  steps. Using assertion in  or   step is usually a sign that the scenario is too big and should be split into two or more smaller scenarios. This convention is currently used in most places.

or
We should use rspec-expectations  syntax.

Example (good,  syntax):

This convention is currently not used. At the moment we are using old   syntax.

Example (bad,  syntax):

Page object files
For example file see any file in   folder of qa-browsertests repository. Step definition files usually contain page URL, page elements and methods.

URL
Page URL is optional. It is used in step definitions when a  or   step needs to go directly to a page, or when a   step needs to check page URL. This convention is currently used in most places.

Example (good):

Simple
Simple page elements should be defined using page-object Ruby gem API. This convention is currently used in most places.

Example (good):

page-object gem shortcuts
Code is more readable if shortcuts are not used.

Example (good):

Example (badbad):

Complicated
Elements that are complicated to find should pass blocks finding the elements to the page-object API. This convention is currently used in most places.

Example (good):

Methods
If a page has complicated functionality, a method in its page class is usually the best place for it. This convention is currently used in most places.

Example (good):

If the method returns page element, it's name should end in. This convention is currently used in most places.

Example (good):

Example (bad):

Hooks
Complex, exceptional behavior needing to be used by multiple tests may be specified in the file under support/hooks.rb

Some examples from the VisualEditor repository:

A hook to keep the browser open if an env var is set:

A "Before" hook that will set up an edited page for manipulation by tests:

A "Before" hook that sets up a page and selects a particular string on the page to be manipulated by tests: