Selenium/PHP/Test framework deployment/Requirements

Wikimedia Foundation projects Test framework deployment Develop a Selenium test framework with samples and demos that can be run on the grid. Group: Release Engineering Start: 2011-07-01 End: 2013-12 Team members: Markus Glaser, Ryan Lane, Rob Lanphier, Mark A. Hershberger, Priyanka Dhanda, Dan Nessett

These are some initial notes on Selenium integration with MediaWiki. They should be expanded as work progresses.

Framework requirements

• Tests should match the way the current tests work as closely as possible. Use the PHPUnit tests as a guide.
• Tests should be configurable to run against a localhost RC/grid, or a remote RC/grid.
• Tests should be able to reconfigure target wiki dynamically.
• Extensions should be able to define tests, and to define how target wikis should be re-configured.
• Tests need to run serially, or we must find a way to run in parallel.
• Tests should only be run for certain portions of codebase.
• Extensions should also be able to define which portions of core being changed should require a test to be run.
• Tests need to be able to report to CodeReview; should be optional on the part of the test runner.

Code Review integration

• Tests should be run per commit by a code-review/SVN monitor cron
  • Should be configurable for multiple wikis, so tests can run in parallel
  • Should be configurable to be usable in non-WMF environments
• For each new revision, the monitor should do the following:
  1. Picks appropriate wiki, and runs svn up
  2. Reconfigures wiki based on test suite requirements
  3. Runs appropriate test suites for relevant changed code
  4. Reports results to Code Reviews

WMF Selenium Grid

WMF has a Selenium Grid cluster in the Tesla architecture. You can see what environments are available by visiting the console. You can access the grid by using the Selenium framework, and connecting to http://grid.tesla.usability.wikimedia.org on port 80 using the selenium framework of your choice (we should all be using PHP).

Initially the grid will be used by the Usability Initiative for automated testing for releases, then will be used by Code Review. We hope to open the grid for developer testing later.

Continuous Integration

• Continuum
  • PHPUnit chapter on continuous integration
  • Presentation on Selenium end to end testing

Currently installed for testing on host: ci.tesla.usability.wikimedia.org. If you need access, post your wiki name, requested shell account username, and reason for access below:

• Wiki username
  • Shell account name
  • Reason for access

Selenium framework

A detailed description of the existing selenium framework can be found at SeleniumFramework

Project plan

• Selenium/Deployment