MediaWiki Language Extension Bundle/Testing MLEB
The MediaWiki Language Extension Bundle (MLEB) is a collection of MediaWiki extensions that can be used to add internationalization features to a wiki and to make it ready for multi-lingual content. The target date for releasing a new version of MLEB is the last Wednesday of every month. Ahead of the release, the extensions in MLEB are tested against the last 2 stable versions of MediaWiki. This Bundle is maintained by the WMF Language Engineering team.
This document will help prepare the setup to test the MediaWiki Language Extension Bundle and outline the test procedure.
Setting up MLEB for Testing
The two latest stable versions of MediaWiki and the extensions that make up the Bundle are required to be set up before the tests can be run.
- Versions: The extensions are tested against the last 2 stable MediaWiki versions. These can be identified through the tags used in the gerrit repo. The development version is generally the latest 'master'.
- Downloading: The MediaWiki versions can be checked out from the gerrit repo. Having them in two separate locations is convenient for the test set up.
cd <dir-core> git clone core cd <dir1-location> cp <core-location>/core . git checkout -b branchname tagnumber1 cd <dir2-location> cp <core-location>/core . git checkout -b branchname tagnumber2
- Installation and Configuration - Install and setup a basic configuration of MediaWiki in all the locations using the standard Installation and Configuration procedure.
After configuring MediaWiki, the extensions need to be installed in all the testing instances. The extensions can be obtained from gerrit. Unless otherwise specified the 'master' branch is tested. Most of the extensions do not need any configuration beyond the standard settings in LocaleSettings.php. Follow the Installation and Configuration instructions to configure these Extensions on the MediaWiki instances that you set up in the previous step.
The list of extensions:
Testing and Reporting
The tests are run to check feature integration and to isolate any possible regressions. Before starting the tests, please make sure that you have completed the installations and configurations that have been mentioned in the sections above. This is particularly important so that any unexpected outcomes from the tests can be traced to the correct versions.
Currently, most of the tests are run against the two major extensions - Universal Language Selector and Translate. Some of these tests are being automated. The tests are written as Scenarios i.e.
- The test is to be executed at a certain state of use of the extension
- The test is a set of one or more actions within that particular scenario
- The test has an expected outcome
Scenario: Display language change is previewed immediately Given I clicked the cog icon to open the 'Language Settings' panel When I select a language different than "English" for display language Then I should see the text in the language panel in the selected language
The complete list of test scenarios can be found in the following links:
The test scenarios are not usually modified unless there is a change made to the functionality that is being tested.
How to test
The tests that have not been automated are tested manually. It can be a long process, but its often the easiest way to confirm that things work as they are expected to.
- Step 0: Open a mediawiki instance from your local system that you had earlier installed
- Step 1: Choose the extension you would like to test
- Step 2: Choose the web-browser to run the test on. At present we are short on hands and do not get enough time to test on all the Grade-A browsers.
- Step 3: Go to the page with the test scenarios for that particular extension. (List of Test Scenarios)
- Step 4: Follow the steps described in each of the scenarios.
- Step 5: Observe the outcome. If it matches the expected 'Then' condition, you can move to the next scenario. If not, go to Step 6.
- Step 6: Make a note of the exception. You will need to report this, preferably through Bugzilla.
Repeat all the steps on the other MediaWiki instance as well.
Reporting the exceptions
Go to bugzilla.wikimedia.org and file a bug against the extension (you will need an account). The developers maintaining the extensions will be notified when a new bug is reported.
Make sure to include the following information:
- the steps you had followed
- the outcome you expected
- the outcome you received
- the browser you were testing on
- the mediawiki version
- Optionally, you could also add the entire text for the scenario as additional information.
Make sure to report only one issue in one bug. If you see the problem recurring across browsers and MediaWiki versions, then list them as additional information in the same bug.
Generally, it is expected that following all the steps mentioned above will be enough to run the tests. However, if at any point of time you need help you can find someone on the IRC channel #mediawiki-i18n or send a mail to mediawiki-i18n mailing list.