Selenium/Ruby/Workshops/Search features

WHAT: Writing Wikipedia search feature descriptions in plain English for our automated testing process. Let's feed the Test backlog with descriptions like this:

Feature: Search

Scenario: Search suggestions Given I am at random page When I search for: main Then a list of suggested pages should appear And Main Page should be the first result

WHEN: We will start with a discussion and slide demo to be streamed on a public channel on Wednesday, March 13, 2013 at 17h UTC and we will be helping volunteers during the rest of the week. This is an ongoing activity: you can arrive / leave at any time.

WHERE: IRC channel. After joining on IRC, say Hello browser testers! and we will welcome you with instructions and a simple task. chrismcmahon, zeljkof and qgil will do their best covering timezones. You can also use the Discussion page to ask any questions.

WHO: Anybody interested, including you! The only requirement is basic level of plain English in order to describe the features to be tested automatically. It is that simple.

See also How to contribute.

This is a weekly QA activity organized by the MediaWiki Group Browser testing. Join us!

Focus
Here is a list of solved and open bugs that we want to test automatically:


 * Search for simple text misses some results
 * Incorrect UTF-8 processing
 * Search ignores user settings

There are more examples in Bugzilla.

Automated Tests in Plain English

 * Q: Why automate tests at the browser level?
 * A: They find bugs!


 * Q: How do we know what to test?
 * A: Browser tests are most useful for:
 * Features under development and near release. We want them to work properly in all browsers when they are deployed.
 * Features heavy in javascript. Different browsers interpret javascript in different ways, which is a potential source of error.
 * Features that require navigation through the application. Unit tests don't navigate.
 * High risk features. If something breaks often, we should have a test for it.
 * UploadWizard is a good example of a feature that is fragile when it is being changed.
 * Features that need to work all the time. Wikipedia doesn't work if Edit or Search is broken.


 * Q: How can I help?
 * A: If you know how something should work, you can tell us how to test it. You don't have to be programmer, either.

Examples are Always Handy
Browser tests can identify problems at every level in ways that unit tests and integration tests cannot. Here are some examples of issues we have identified by way of failing browser tests that point to problems with a code mistake, how code manifested in a particular browser, and how the underlying environment itself was configured.

A recent collaboration. This is a great test, it's javascript-heavy and involves navigating through the application.
 * Paired with the E3 team on an acceptance test.
 * A little while later the test found pretty important bug
 * Note that E3 is actively testing this feature. Browser tests find bugs when humans aren't looking
 * We have video of all of our test runs. This one clearly shows the error.

An older test
 * This particular feature worked correctly every browser except for IE9

A test that found a configuration bug unrelated to software.
 * After the EQIAD data center migration this test discovered that the "texvc" library had not been migrated to the new data center.

A recent collaboration with some challenges:
 * Paired with Language team. This test is in process, the way the browser determines whether the Serbian language is presented in Latin characters or Cyrillic characters is complicated.
 * We're telling the browser that we want Serbian in Latin characters, but the browser shows us pages in Serbian using Cyrillc characters

We have Mobile browser tests too
 * They need some love

Demo time!
We will start with a live demo. Search demo: create Scenarios for Search tests
 * WMF is preparing to make some changes to how Search works in Wikipedia
 * Search is pretty important! We need some good tests for Search.
 * Both desktop and mobile browsers need search tests.]