- Send out user satisfaction survey to get baseline data on use and issues.
- Provide a pre-release JS framework to early adopter teams.
- Release the full featured version
- If successful, help other teams migrate old and/or create new tests with the new framework through workshops and education.
- If successful, deprecate MW-Selenium and migrate remaining projects to new JS framework.
- Send out follow up user satisfaction survey
|Major work is on hold/de-prioritized for Q1 (due to staffing restrictions).||Provide a pre-release JS framework to early adopter teams.||Release the full featured version||
||If successful, deprecate MW-Selenium and migrate remaining projects to new JS framework.|
- Higher levels of test adoption by both WMF Engineering and our community of volunteer developers will improve the overall quality of new features going out to all of our users (third-party reusers of MediaWiki as well).
- JS framework with fewer and less obscure dependencies could make end-to-end testing dramatically more accessible to new and experienced developers alike—many more of our developers understand JS toolchains than those for Ruby.
- ratio of # changes in test code by project owners vs. those made by members of Release Engineering (higher is better)
- Over the fiscal year we should see an improvement in the ratio of changes in test code by project owners vs by members of RelEng over the baseline (TBD) (higher is better). Each quarter should show improvement as we implement the new features and increase adoption.
- User satisfaction of the new tool based on results of a user survey.
- A non-trivial improvement in user satisfaction in the end-to-end test framework, as compared with the baseline (baseline survey to be sent in 16/17 Q4).