Quality Assurance/Writing feature descriptions

Executable Specifications
Acceptance Test Driven Development starts with a clear description of how a feature should behave. These plain-language specifications are the most important part of the whole project. The better the specifications of the behavior of particular features, the better the automated tests will be. For example, we might describe how Search should work in any given wiki:

On any random page in any wiki, type "main" into the search box. A list of results should appear, and "Main Page" should be the top result. On another random page in any wiki, type "ma" into the search box and click the Search icon. I should then be on a page with a set of search results, and "Main Page" should be one of the results."

Any descriptions of this nature may be added to the test backlog page at http://www.mediawiki.org/wiki/Qa/test_backlog

For those interested in going deeper, we have a formal syntax for making executable specifications in the "Cucumber" feature of our test framework. Cucumber demands specifications in a particular form, Given/When/Then:


 * A Given statement should only ever mention starting conditions.
 * A When statement should always be some sort of verb, or action.
 * A Then statement always has the words "should" or "should not", or equivalent.

Feature: My Feature Scenario: Testing some aspect of My Feature Given When  Then 

Any Given/When/Then statement may also have an arbitrary number of "And" clauses:

Scenario: Testing some complicated aspect of My Feature Given And When  And  And  Then  And 

So the example for Search above (which is in fact already automated) becomes

Feature: Search

Scenario: Search suggestions Given I am at random page When I search for: main Then a list of suggested pages should appear And Main Page should be the first result

Scenario: Fill in search term and click search Given I am at random page When I search for: ma     And I click the Search button Then I should land on Search Results page

Any thoughtful description of a feature to be tested is welcome at http://www.mediawiki.org/wiki/Qa/test_backlog. Specifications in Given/When/Then format are particularly welcome. For more examples of how to make Given/When/Then specifications, see our existing Feature files at https://github.com/wikimedia/qa-browsertests/tree/master/features

Technical Contributions
For information about contributing code and implementing Feature tests using Page Objects and Cucumber, see QA/running and writing browser tests