Parsoid/Visual Diffs Testing

From MediaWiki.org
Jump to: navigation, search

The code for generating visual diffs is mirrored from the integration/visualdiff repo on gerrit to github at:

  • diffserver/ has code for running a visual-diff server for generating diffs on demand.
  • testreduce/ has code for running mass visual diff testing via the testreduce setup, and for configuring the testreduce server.

Visual diff service[edit]

WMF runs a visual diffing testing setup on ruthenium to compare Parsoid's HTML rendering with the PHP parser's HTML rendering.

TODO: This diff server has not yet been puppetized and restarted on ruthenium after reimaging.

Visual-diff testreduce server[edit]

WMF runs a visual diffing testing setup on ruthenium to compare Parsoid's HTML rendering with the PHP parser's HTML rendering. This fetches the HTML from the production Parsoid cluster and the production Mediawiki cluster, takes screenshots via phantomjs, and uses the uprightdiff code to compare them.

The following information about all the code and configs and settings is just an FYI. These services are completely puppetized and you shouldn't be modifying these config files directly on ruthenium. Puppet will automatically overwrite your changes every 30 minutes.

The visualdiff code is in /srv/visualdiff. The service is managed by systemd and the configs are in /lib/systemd/system/parsoid-vd and /lib/systemd/system/parsoid-vd-client.

The testreduce server config is in /etc/testreduce/parsoid-vd.settings.js. See that file to access the mysql database used to store testing titles and test results.

The testreduce client config is in /etc/testreduce/parsoid-vd-client.settings.js

The upright code is in /srv/uprightdiff. The binary has been built on ruthenium itself. This build is the only part of the whole setup that is unpuppetized. This has to be done whenever uprightdiff code is updated.

Test results are accessible at http://parsoid-tests.wikimedia.org/vd_testreduce/

Updating the visual diff code[edit]

cd /srv/visualdiff
# Please verify that you are in the ruthenium branch
git pull
# Restart the testreduce clients to use the new diffing code
sudo service parsoid-vd-client restart

Updating the testreduce server code[edit]

Note that this testreduce repository is used by the Parsoid Roundtrip Testing services as well. It should be safe to do this update for both testing services, but just a reminder.

cd /srv/testreduce
# Please verify you are in the ruthenium branch before pulling
git pull
sudo service parsoid-vd restart

Managing the testreduce clients and looking at logs[edit]

To {stop,restart,start} all clients on a VM (not normally needed):

 sudo service parsoid-vd-client stop
 sudo service parsoid-vd-client restart
 sudo service parsoid-vd-client start

Client logs are in systemd journals and can be accessed as:

### Logs for the parsoid-rt-client service
# equivalent to tail -f <log-file>
sudo journalctl -f -u parsoid-vd-client
# equivalent to tail -n 1000
sudo journalctl -n 1000 -u parsoid-vd-client
 
### Logs of the parsoid-vd testreduce server
sudo journalctl -f -u parsoid-vd