Parsoid/Visual Diffs Testing

The code for generating visual diffs is mirrored from the integration/visualdiff repo on gerrit to github at:
 * has code for running a visual-diff server for generating diffs on demand.
 * has code for running mass visual diff testing via the testreduce setup, and for configuring the testreduce server.

Visual diff service
WMF runs a visual diffing testing setup on ruthenium to compare Parsoid's HTML rendering with the PHP parser's HTML rendering.

TODO: This diff server has not yet been puppetized and restarted on ruthenium after reimaging.

Visual-diff testreduce server
WMF runs a visual diffing testing setup on ruthenium to compare Parsoid's HTML rendering with the PHP parser's HTML rendering. This fetches the HTML from the production Parsoid cluster and the production Mediawiki cluster, takes screenshots via phantomjs, and uses the uprightdiff code to compare them.

The following information about all the code and configs and settings is just an FYI. These services are completely puppetized and you shouldn't be modifying these config files directly on ruthenium. Puppet will automatically overwrite your changes every 30 minutes.

The visualdiff code is in /srv/visualdiff. The service is managed by systemd and the configs are in /lib/systemd/system/parsoid-vd and /lib/systemd/system/parsoid-vd-client.

The testreduce server config is in /etc/testreduce/parsoid-vd.settings.js. See that file to access the mysql database used to store testing titles and test results.

The testreduce client config is in /etc/testreduce/parsoid-vd-client.settings.js

The upright code is in /srv/uprightdiff. The binary has been built on ruthenium itself. This build is the only part of the whole setup that is unpuppetized. This has to be done whenever uprightdiff code is updated.

Test results are accessible at http://parsoid-tests.wikimedia.org/vd_testreduce/

Updating the testreduce server code
Note that this testreduce repository is used by the Parsoid Roundtrip Testing services as well. It should be safe to do this update for both testing services, but just a reminder.

Managing the testreduce clients and looking at logs
To {stop,restart,start} all clients on a VM (not normally needed): Client logs are in systemd journals and can be accessed as: