VisualEditor/Planning and testing

From mediawiki.org

This is a directory of pages related to the planning and testing process of the VisualEditor extension. Some pages document deprecated aspects of the software, and are kept for historical purposes.

Planning[edit]

  • "Forward-look" documents:

Testing[edit]


Weekly Test Plans[edit]

The Visual Editor QA team uses the page VisualEditor/Weekly Test Plans to track features being added in each weekly product release cycle, and capture essential notes on what sort of testing will be needed to verify each feature for production release.

QA tests[edit]

JavaScript tests[edit]

The VisualEditor JavaScript QUnit tests are in test subdirectories of some modules. They need to run in a browser. You can visit Special:JavaScriptTest/qunit to run all tests (which will take a long time to complete in your browser), or choose an individual module from the dropdown. Append ?filter=ve to the URL to run all modules matching "ve" which will run all VisualEditor tests plus some others ( try it on test2wiki). You can also run VisualEditor's QUnit tests from the commandline using the Grunt JavaScript task runner, which will start up a PhantomJS headless browser to run them.

Grunt can also run the JSHint code checker. See Gruntfile.js.

Browser tests[edit]

modules/ve-mw/test/browser is a set of browser tests that perform some VE actions.

You can run these tests yourself by installing the necessary software, see Quality Assurance/Browser testing/Running tests; the easiest setup is to run a MediaWiki-Vagrant virtual machine and enable the roles browsertests and visualeditor.

For the tests to work

  • the test username (MEDIAWIKI_USER) needs an account on the test wiki (MEDIAWIKI_URL)
  • the test username's Preferences > Editing "Enable VisualEditor" must be checked
  • The test wiki must be configured so that VE is enabled for the User namespace (in $wgUseVisualEditorNamespaces).

Selenium browser tests[edit]

https://github.com/inez/VE-tests/ has additional browser tests that test basics such as typing, breaking a paragraph into two paragraphs, bold, list, etc.

These test are written in Java and depend on Maven and Selenium; they load the VE demo web page in VisualEditor/demos/ve . To run them you must install

You can use FirefoxDriver instead, or load VE at a different URL, by editing test/java/BaseTest.java

Continuous integration tests[edit]

When developers submit changes to VisualEditor, and later when other developers approve ("+2") them in gerrit, Jenkins runs a subset of VE tests: jshint, PHPcs, qunit tests.

WMF QA regularly runs browser tests in multiple browser at https://wmf.ci.cloudbees.com, visiting pages on wikis, including beta labs (which is running master) and test2wiki (which is running the most current wmf branch). Click the [r-ve] tab to see all VisualEditor runs.

Dirty diff checking[edit]

There is a bot that runs VisualEditor, picks a random article, saves it, and checks for diffs arising from round-trip editing and Parsoid parsing. Source here, output here. Any dirty diff is a bug in either VisualEditor or Parsoid, even normalizations with no visible change (whitespace, quoting arguments)

TODO should improve dirtydiffbot to indicated whether it triggered VE's dirtywarning message, issue filed on github

Other[edit]