Selenium/PHP/Selenium Framework

Purpose
SeleniumFramework provides a standardised way of writing Selenium tests for MediaWiki and its extensions. It is based on PHPUnit tests.

Set up the environment
In order to work properly, the framework builds on some external dependencies:
 * You need to set up a Selenium server, either Selenium RC or Selenium Grid
 * The PEAR PHPUnit 3.4 library must be accessible within your PHP path
 * Also, PEAR Testing_Selenium must be accessible within your PHP path

Configure the framework
Next, you have to tell the framework about your environment. This should be done in a file called LocalSettings.php. These options are currently available:

Also in LocalSettings.php, you should add a list of classes containing the actual test suites:

The classes must be registered with the autoloader. For the SimpleSeleniumTestSuite, this has also been done in includes/AutoLoader.php. For extensions, this can be done in the extension file using this command:

For quick testing, you can also register autoload classes in LocalSettings.php.

Writing tests
In order to add a Selenium test, you have to create a test suite and a test file (YourExtensionSeleniumTestCase.php). Here is a simple example which types some text in an article, saves the article and checks whether the text is still present. Put this in your extension/selenium directory with the name YourExtensionSeleniumTestCase.php:

Now, you need a test suite, which is best put into the same directory with the name YourExtensionSeleniumTestSuite.php:

As of now, you have to manually add the test suites to run in LocalSettings.php. This will be extended by a command line argument in the future where you can override these settings and run an arbitrary set of tests.

Running the tests
The test runner is located in maintenance/tests/RunSeleniumTests.php. In order to run the tests from a console, you just need to execute the test runner in a commandline:

php RunSeleniumTests.php

There is also a special page called Special:Selenium, which can be used to trigger the tests. Be aware that in order to use this, you have to have the selenium permission (see section on configuration).

Working example
A working example can be found at the PagedTiffHandler extension.

Architecture
These files are part of the framework:
 * RunSeleniumTests.php includes test suites specified in extension directories.
 * selenium/Selenium.php provides access to selenium and implements common tasks.
 * selenium/SeleniumLoader.php loads all the neccessary classes for the framework.
 * selenium/SeleniumTestSuite.php starts and stops selenium tests.
 * selenium/SeleniumTestCase.php provides some additional assertions.
 * selenium/SeleniumTestListerner.php is the interface to result logging.
 * selenium/SeleniumConsoleLogger.php and selenium/SeleniumHTMLLogger.php produce the actual output.
 * selenium/SimpleSeleniumTestSuite.php and selenium/SimpleSeleniumTestSuite.php provide an example test case.
 * Individual tests are located with the extensions in the directory selenium.

Test styleguides

 * A test should leave at best no traces in the wiki.
 * If this is not possible, the test should leave the wiki in a state which allows the test to be re-run.

Add common test tasks
Please add to this list...
 * Trigger user preferences
 * Edit a page (initial support already implemented)
 * Upload images (already in PagedTiffHandler tests)
 * Show edit and history mode
 * Revert pages

Add common assertions
Please add to this list
 * Text within an article
 * Text in Headlines
 * Text in Table of Contents
 * Error messages

Improvements

 * Make test result logging more talkative
 * Check prerequisites of MediaWiki configuration (initial support in PagedTiffHandler tests)
 * Find a way to execute selenes recorded via Selenium IDE
 * Refactor naming conventions
 * Ways to configure the wiki for specific test suites or extensions.
 * A way to bring the database to a clean initial state before each test
 * Use the mediawiki extension to write collaborative Selenium tests

Further ideas
A test (or a single test suite) should be able to reconfigure the wiki according to its needs. Here, some serious security issues arise. This might be a possible way: If the hook is not triggered in LocalSettings, no reconfiguration can take place.
 * Add a hook 'LocalSettingsEnd' at the end of LocalSettings.php
 * Add some URL parameter which indicates which test is being run
 * Within the extension, the hook code now changes the configuration

Error about Testing_Selenium (0.4.3 is installed from pear along with PHPUnit) not being found

 * Replace /PEAR/ by the actual path to the repository

General Tips

 * Ignore port A proxy time you out port 4444.
 * Uncomment any tests in RunSeleniumTests.php to run different test suites?

Experiences: Recent changes in handling of framework

 * PEAR PHPUnit 3.4 and PEAR Testing_Selenium: Check whether properly installed, esp. PHPUnit/Framework.php
 * Installation of selenium-rc is pretty easy:
 * Download selenium rc
 * start "java -jar selenium-server.jar"