Language Testing Plan/TCMS Guide

A test case management system can be used to organise numerous repetitive tests over several versions of a product release. It also provides simple interfaces to create, manage and execute tests for users with different permission levels. For the projects maintained by the WMF Language Engineering team, we are evaluating the use of Testlink as a test case management system. As a start we have the included the tests for the Universal Language Selector that are manually run prior to MLEB releases. The following sections document the workflow used to create and execute the tests. Users with admin rights can create and manage tests. Users with tester roles can execute tests assigned to them.

More information about using Testlink can be found in the official User Guide

Log-in and Roles

 * login - create a new user at http://language-tcms.wmflabs.org/firstLogin.php by filling in the form. Registered users can then log into the TCMS system using http://language-tcms.wmflabs.org/login.php
 * Users can be assigned different roles depending on the extent of their work inside the TCMS system. For the sake of simplicity, in the Testlink instance on language-tcms, we use 2 roles: admin and tester.
 * The admin users prepare the tests, plan them and assign them to testers
 * The testers can execute the tests assigned to them
 * Test assignments are done after test plans for the extensions/features have been drawn up
 * Before tests can be assigned, they have to be created and planned

Test Creation
Tests within the TCMS system are organised as Test Projects with Test Specifications. Test Specifications include Test Suites which are used to group the invidual Test cases.

Test Plans

 * Test Plans can be created from the 'Test Plan Management' link on the Desktop View
 * Click on the 'Create' button to add a Test Plan. In the dialog displayed, the name, description of the plan can be added. It can also be marked as being 'Active' and 'Public'
 * After creating the Test Plan, the next step would be to add a Build and Test Set
 * The Build indicates the build release or version that is set to be tested. Unless specified the Test Plan cannot be marked for execution.
 * This can be created by clicking on the 'Build / Releases' link from the Desktop View. For our example, we used Build title: ULS MLEB 2014.3; Description: The build for the ULS tests for MLEB 2014.3 release Test Build Creation.png
 * Next step would be to create the 'Test Set' i.e. the Test Suites and/or individual Test Cases that will be used to test the build that has been included in the Test Plan
 * Click on 'Add/Remove Test Cases' from the Desktop view to see the dialog for adding test cases
 * Select the Test Plan from the left panel and click on a Test Suite (from the tree) that contains test cases. The Test Cases will be listed along with checkboxes that can be used to make selections Test Case Add Remove.png
 * Select the Build for tests and (optionally) the user you would like to assign the test to. The test assignment can also be done at a later stage.
 * Select the test cases (all or individual) using the checkboxes and then click on 'Add Selected' button
 * The view will change to reflect the selections made. Test Case Added.png
 * Similarly test cases can also be removed.
 * Repeat this process of selection for all the Test suites and/or Test cases as appropriate for the Test Plan

Replication

 * Test Projects, Test Suites, Test Cases, and Test Plans can all be replicated from the TCMS interface at the time of creation. The forms will provide options from existing sets that can be chosen as the base. These can also be exported/imported using XML files.
 * New sets are necessary not just for creating newer versions of tests, but also for creating duplicate sets of Test Plans if more testers join in later

Test Assignment

 * Test cases can also be assigned to users after they have been created.
 * The users will have to be granted 'Tester' roles by the Admin or Leader for tests to be assigned to them
 * Click on the 'Assign Test Case Execution' link on the Desktop view
 * In the displayed dialog, select the Test Suite that contains the Test Cases to be assigned Test Case Assignment.png
 * Make the selections using the checkbox and dropdown next each Test Case and click 'Save'
 * The testers will now be able to see the tests assigned to them when they log-in Tester Desktop View.png

Test Execution

 * Log in using the user account credentials
 * Select the Test Project, Test Plan and then Click on Execute Tests from the Desktop View to start testing
 * All the assigned test cases will be displayed in the left panel
 * Select a test case and view the instructions
 * At the end select if the test has passed successfully (i.e all expectations have been met) or failed Test Execution Result.png
 * Repeat for all the tests. Tests can also be skipped