SVG benchmarks

For rasterizing SVG images we've used librsvg for some time. Another contender which we didn't choose originally is Batik. There are several respects in which librsvg is unsatisfactory and Batik might be preferable.

librsvg: In favour: fast; totally free/libre. Against: Lots of GNOME-ish dependencies, which makes tracking updates difficult; We have to maintain a security patch to disable the loading of external URL resources, since librsvg maintainers weren't interested in it; Lots of rendering bugs (but it's improved over time); Unclear fonts/text rendering situation.

Batik: In favour: more mature; higher quality rendering; built-in security mode. Against: slow!; doesn't (or at least didn't) run on free/libre Java (GCJ etc).

Benchmark
Brion did a quick test of SVG rendering (19 Dec 2006), with 50 images pulled off Commons. Test run on a MacBook, 2 GHz Intel Core 2 Duo, in Ubuntu running under Parallels (using one virtual processor). rsvg is 2.16.0 as apt-get'd via Ubuntu; batik is 1.6, running on Sun's J2SE 6 runtime.


 * rsvg: 18.07 seconds; 361.46ms per image
 * batik: 108.44 seconds; 2168.84ms per image

Batik is probably slowed down by having to start up the VM and JIT on every call. It likely can be sped up by using a daemon process and handing off render requests to that.

Batik also refused to render one file that rsvg did render.

ivar notes that to avoid JDK startup time you can use nailgun.

Additional notes

 * Batik command-line renderer appears to scale image height to be proportional with the given width. This could produce width/height inconsistencies between specified heights from MW and results.
 * I'm pretty sure Batik has a mode for increased safety (rejecting external links), which we have to maintain as a patch hack for librsvg. Not having to maintain a patch is attractive.
 * We frequently get requests about font support. It'd be nice to know how to answer such questions.
 * rsvg's resource usage can be limited with ulimit etc easily; what about a batik daemon? How to stop giiaaannttttt files from eating up RAM without taking down the server?
 * You'd have to look at jvm args... -- chris

Further testing
David Gerard ran a test of ImageMagick 6.2.4, Inkscape 0.45.1, Batik 1.7.0 (on Sun Java 1.6.0_03-b05) and rsvg 2.18.2 on 57 Commons images, including some in the megabytes. Those which reliably gave an error from the renderers were left out. Test box was a Dell Latitude D610, Pentium M 1.6 GHz, 512MB memory, running Kubuntu 7.10. All test applications except Batik were Ubuntu repository versions.

All four programs were run through the 57 images 115 times and timed with /usr/bin/time. X running in Failsafe (konsole as only app), nothing else running.

Worrying:
 * Batik's JVM crapped itself several times for no apparent reason - 39 times in the run, i.e. 39 in 4485 attempted renderings. Not on any particular SVG either.
 * Inkscape thrashed the disk. (Single physical disk in laptop.)

(Possible explanation: that rsvg is a lib whereas Inkscape is a full app - could the overhead of starting and stopping the app be significant?)

What this suggests: use rsvg if possible (anything Unix), use Inkscape if rsvg's dependencies are too troublesome (e.g. on Windows, where Inkscape is available as a standalone app) and you have a lot of memory.

See /test 20080112 for full data.

Update
Hk kng re-run the tests in 2009 using a http server solution for Batik: batikd. For my reasoning, see meta:Talk:SVG image support. Full test results are at /test 20090816. WikiMedia Commons presents a gallery of the test images.

A comparison of CPU times is not as easily done, since the real image conversion for Batik is done in the separate server process. The wallclock times look like this: