Extension:Cargo/Performance testing

A PHP script was created (you can see its full contents here) that attempts to run equivalent queries on matching data sets using Cargo and Semantic MediaWiki

The wiki chosen, Discourse DB, is a site run by Yaron Koren, the author of Cargo. It is a wiki of political commentary. The site stores most of its information using both SMW and Cargo. At the time of testing, Discourse DB was running SMW 2.2 and Cargo 0.8.

Query URLs were created for both Cargo and SMW that get, in CSV format, the earliest 100 opinion items on the site, sorted by date in ascending order, that were published in The Wall Street Journal; for each one, the query gets its name, author(s) and publication date. You can see the Semantic MediaWiki URL here, and the Cargo URL here. If you click on either of those URLs, you will get a CSV download. If you download both CSV sets, you can see that there are some small differences between the result sets; but overall the data is fairly similar, and both contain 100 rows, plus a header row.

The script fetches each URL 10 times at a time, and alternates between the two URLs, to get average retrieval times.

After several runs of the script, the average run time for the Semantic MediaWiki query was determined to be 1.02 seconds, while the average run time for the Cargo query was determined to be 0.69 seconds. For this particular data set, then, Cargo's querying is approximately 50% faster than Semantic MediaWiki's.

Anyone should feel free to run the script themselves, as it is or with any changes; please let Yaron know by email (yaron57@undefinedgmail.com), or write on the talk page, if you get different or interesting results. If you run similar test on another wiki, that would also be very helpful to know about.