Parsoid/Visual Diffs Testing

The code for generating visual diffs is on
 * has code for running a visual-diff server for generating diffs on demand.
 * has code for running testreduce clients, and for configuring the testreduce server.

Visual diff service
WMF runs a visual diff service based on this code. The code is deployed in /usr/lib/parsoid_vd/diffserver. The service is managed by upstart. The upstart config for the service is in /etc/init.d/parsoid_vd. The diffing/resemblejs options config is in /usr/lib/parsoid_vd/diffserver/diffserver.settings.js sudo service parsoid_vd stop sudo service parsoid_vd start sudo service parsoid_vd restart (something is broken with the config currently and this doesn't work) Logs are in /var/log/parsoid_vd/parsoid_vd.log. (Hmm .. maybe we should s/parsoid_vd/diffservice/g or something else?)

This visual diffing service is accessible at, e.g. http://parsoid-tests.wikimedia.org/visualdiff-item/diff/enwiki/Magnetosphere_of_Jupiter.

Visual-diff testreduce server
The testreduce server instance for doing mass visual-diff testing is in /usr/lib/testreduce_vd. (This is just a symbolic link to /usr/lib/parsoid-clone/tests/server and is shared with the testreduce instance used for article roundtrip testing.) The testreduce_vd server settings is in /etc/testreduce_vd/settings.js. See that file to access the mysql database used to store testing titles and test results.

Testreduce server logs are in /var/log/testreduce/testreduce_vd.log.

Updating the testreduce server code
cd /usr/lib/parsoid-clone git pull sudo service testreduce_vd restart Remember that this code is shared with article roundtrip testing. It should be safe to do this update for both testing services, but just a reminder.

Test results are accessible at http://parsoid-tests.wikimedia.org/visualdiff/

Testing clients
The visualdiff testing clients run on ruthenium. The clients look at the parsoid version deployed in production and re-runs tests whenever the deployed version changes.

The clients are managed/restarted by upstart. The upstart config is in /etc/init/parsoid-vd-client.conf</tt>. The diffing/resemblejs options/config is in /usr/lib/parsoid_vd/testreduce/config.js</tt> -- you might notice that these settings are slightly different from the settings for the diffserver in /usr/lib/parsoid_vd/diffserver/diffserver.settings.js</tt>. This is because the diffserver settings are used to generate diff images that humans look at and examine and the different errortype and no-grid settings make the diff easier to interpret. However, for the testreduce clients, where the images are never seen by anyone, we trade off the diff image quality for a slightly faster run.

To {stop,restart,start} all clients on a VM (not normally needed): sudo service parsoid-vd-client stop sudo service parsoid-vd-client restart sudo service parsoid-vd-client start

Client logs are in /var/log/upstart/parsoid-vd-client.log</tt>.