Extension:GnuplotBasic

Purpose
This GnuplotBasic extension allows you to produce and display graph images (like mathematical functions, line charts, histograms etc.) within a MediaWiki instance. It uses the (free) plotting program Gnuplot for creating these graphs. It allows for 2- or 3-dimensional graphs, all within the potentials of the Gnuplot program itself. The graphs can be based on mathematical function(s) and they can use plotdata from the wiki page.

GnuplotBasic works, together with the Gnuplot progam, fully within your MediaWiki Server environment.

Note from the author (January 12th 2009) ''I will publish the missing images on this page ASAP (on Jan 16th) as soon as my account is upgraded and I am allowed to upload them. Sorry for any inconvenience.''

Use
Include the GnuplotBasic call between the tags.

A GnuplotBasic call in wiki markup consists of a plotscript-part plus an optional plotdata-part (see structure on the right). The plotscript has the Gnuplot commands, you can find these in the Gnuplot manual. The plotdata has your data (in rows and columns) to be plotted. The plotscript-part always comes before the plotdata-part.

If you use plotdata include it within  and within. There can be no more Gnuplot commands after.

The only required Gnuplot command in the plotscript-part is the  command, e.g.  . The   command should be the last plotscript command in order for any settings to take effect.

Sample plots simple
Here are some simple samples of what you can do with Gnuplot with the use of GnuplotBasic: Comment can be used both in the plotscript-part and the plotdata-part. This can be especially helpful for documenting columnheaders when using plotdata.
 * You can put comment in your GnuplotBasic call after the " " character. Best practice is to do this at the beginning of a line.

Sample plots more advanced
Here are some more advanced samples of what you can do with Gnuplot with the use of GnuplotBasic: Note that these two samples use font verdana 8. Advanced (True-Type) fonts are required when rotating text (you'll probably need rotating text making histograms). Make sure the Gnuplot program can find the fonts you use in the plotscript, see installation.

Installation and configuration
Got you interested with these samples? Here's how to install it. Installation consists of a few steps. Note that in the descriptions below  is the root install path of your MediaWiki. You'll need a version of Gnuplot. You can download it from the Gnuplot website. Follow its installation instructions suiting your choice of platform (Windows or Unix). Check the environment variable  if you need or like to use more advanced fonts rather than the build in ones. Save the PHP script below as file GnuplotBasic.php in the directory Define  as absolute path to the executable of the Gnuplot program e.g.:
 * Install the Gnuplot plotting program
 * Install this extension
 * Add the following lines to the file
 * Make sure that directory  is writable.

Configuration options

Optionally you can configure three other settings for GnuplotBasic in. If you're comfortable with the default settings you can omit these settings. You can change the default terminal used for all graphs e.g.:
 * , the default GnuplotBasic terminal setting is.
 * Note: This default setting can be overruled in a plotscript.


 * , the default GnuplotBasic size setting is . You can change the default size used for all graphs e.g.:
 * Note: This default setting can be overruled in a plotscript.


 * , the default GnuplotBasic refresh rate setting is . GnuplotBasic creates graph images in directory   with unique (automatically calculated) filenames based on the content of the call (including its optional plotdata). You can set the behavior of GnuplotBasic in case the graph image file already exists. You can define whether or not GunuplotBasic re-creates a new graph image when it already exists by defining the refresh rate. Some example settings:
 * Note: The default refresh rate = "-1" because this prevents unnecessary re-creation of graph images each time a page is viewed and the content of the call hasn't changed.

PHP Script
Cut and paste this whole PHP script and save it as  in. to use Gnuplot', 'url' => 'http://www.mediawiki.org/wiki/Extension:GnuplotBasic', );

$wgGnuplotCommand = '/usr/bin/gnuplot'; // Unix
 * 1) $wgGnuplotCommand = 'C:\\Wamp\\bin\\gnuplot\\bin\\pgnuplot.exe'; // Windows

$wgGnuplotDefaultTerminal = 'set terminal png transparent small'; $wgGnuplotDefaultSize    = 'set size 0.5, 0.5'; $wgGnuplotRefreshRate    = -1; // Refresh never if gnuplot call hasn't changed (default)

// Avoid unstubbing $wgParser too early on modern (1.12+) MW versions, as per r35980 if ( defined( 'MW_SUPPORTS_PARSERFIRSTCALLINIT' ) ) { $wgHooks['ParserFirstCallInit'][] = 'wfGnuplotBasicExtension'; } else { $wgExtensionFunctions[] = 'wfGnuplotBasicExtension'; }

function wfGnuplotBasicExtension { global $wgParser; $wgParser->setHook( 'gnuplot', 'renderPlot' ); return true; }

function renderPlot( $gnuplotsrc ) { global $wgGnuplotDefaultTerminal, $wgGnuplotDefaultSize; global $wgGnuplotCommand, $wgGnuplotRefreshRate; global $wgUploadDirectory, $wgUploadPath;

// create directory for storing the plot files $gnuplotDir = "/gnuplot/"; $dest = $wgUploadDirectory. $gnuplotDir; if( !is_dir( $dest ) ) { mkdir($dest, 0777); chmod($dest, 0777); }	// make sure the path is correct for both Windows and UNIX chdir($dest); $dest = getcwd;

// get the filename of the graph to be produced $name = getFileName( $gnuplotsrc ); $graphname = $dest. DIRECTORY_SEPARATOR .$name; $fname = $graphname. ".tmp"; //gnuplot plotscript $dname = $graphname. ".dat"; //gnuplot plotdata

$makeGraph = true; if( ( file_exists( $graphname ) ) and ( ( $wgGnuplotRefreshRate ) <> 0 ) ) { // the file already exists it won't be refreshed when // it is newer than the refreshrate OR setting set to never refresh if( (time - filemtime($graphname)) < ($wgGnuplotRefreshRate) ) $makeGraph = false; if( ($wgGnuplotRefreshRate) == -1 ) $makeGraph = false; }

if( $makeGraph ) { // Step 1: Check if there is plotdata defined $pos1 = strpos($gnuplotsrc, " "); if( $pos1 !== false ){ // split gnuplotsrc in a plotscript-part and a plotdata-part $pos2 = strpos($gnuplotsrc, " "); if( $pos2 === false ) $pos2 = strlen($gnuplotsrc); $plotdata = substr($gnuplotsrc, $pos1+10, $pos2-($pos1+10)); $gnuplotsrc = substr($gnuplotsrc, 0, $pos1); // replace gnuplot inline filename(s) '-' to the plotdata filename $gnuplotsrc = str_replace("'-'", "'$dname'", $gnuplotsrc); // save the plotdata-part $handle2 = fopen($dname, 'w'); fwrite($handle2, trim($plotdata)); fclose($handle2); }

// Step 2: Parse the plotscript-part and do some basic syntax checking $unparsedScriptArray = explode( "\n", $gnuplotsrc ); $parsedScriptArray = array_filter( $unparsedScriptArray, "parsePlotScript" ); $gnuplotsrc = implode( "\n", $parsedScriptArray );

// write the default settings and the input code from wiki into a		// temporary file to be executed by gnuplot, then execute the command $handle = fopen($fname, 'w');

// if terminal and size are not set in the gnuplot source we do it here if( strpos($gnuplotsrc, 'set terminal ') === false ) { fwrite($handle, $wgGnuplotDefaultTerminal . "\n"); }		if( strpos($gnuplotsrc, 'set size ') === false ) { fwrite($handle, $wgGnuplotDefaultSize . "\n"); }

fwrite($handle, "\nset output '" . $graphname . "'\n");

// save the plotscript-part fwrite($handle, $gnuplotsrc . "\n"); fwrite($handle, "exit" . "\n"); fclose($handle);

// execute the gnuplot command $cmdlinePlot = $wgGnuplotCommand. ' ' . $fname; shell_exec($cmdlinePlot);

// cleanup temporary files unlink($fname); if( $pos1 !== false ) unlink($dname); }

return '  '; }

/*** * Function: parsePlotScript * Purpose : Basic syntax checks and replaces unwanted and potentially harmful commands. *          Note that gnuplot allows far more advanced syntax than is allowed *          and recognised here. This should however suffice most graph needs from *          within a MediaWiki instance. * Input  : &$scriptline - a line from the scriptpart (by reference because we do  *           some modifications here). * Output : false if the line ends up empty. */ function parsePlotScript( &$scriptline ) {

$unwantedCommands = array(		"`" => "",		"set output " => "xxx xxxxxx ",		"shell" => "xxxxx", // potentially harmful		"system" => "xxxxxx", // potentially harmful		"load " => "xxxx ", 		"call " => "xxxx ", 		"save " => "xxxx ", 		"cd '" => "xx ", 		"cd \"" => "xx ", 		"update " => "xxxxxx "	);

$scriptline = trim($scriptline); if( strlen($scriptline) == 0 ) return false;

// Step 1: Delete everything after the first comment character $p = strpos($scriptline, "#"); if( $p !== false ) { $scriptline = substr($scriptline, 0, $p); $scriptline = trim($scriptline); if( strlen($scriptline) == 0 ) return false; }	// Step 2: Replace unwanted commands with substitutes $scriptline = strtr($scriptline, $unwantedCommands); $scriptline = trim($scriptline); if( strlen($scriptline) == 0 ) return false;

return true; }

/*** * Function: getFileName * Purpose : Determines the unique name of the output file. *          The filename is computed by a hash function and therefore always unique *          for the script plus its (optional) inline plotdata. *          The file extension is extracted from de format in the "set terminal" *          command. If this format is not found or there is a syntax error it is *           set to "xxx". * Input  : $gnuplotsrc - the input code from the wiki markup * Output : The file name with its extension */ function getFileName ( $gnuplotsrc ) { // determine the file format of the plotted graph image - default is png $format = "png"; $tpos = strpos($gnuplotsrc, "set terminal "); if( $tpos !== false ) { $format = ''; $tpos = $tpos + strlen("set terminal "); // extract the fileformat from this command, this is considered // to be the next word either ending with " ", "\n" or "\t" $done = false; do { $char = substr($gnuplotsrc, $tpos, 1); $tpos = $tpos + 1; if( $char !== false ) { if( ($char == " ") || ($char == "\n") || ($char == "\t") ) { $done = true; } else { $format .= $char; }			} else { $done = true; }		} while( !$done );

$format = trim($format); if( (strlen($format) == 0) || (strlen($format) > 6) ) $format = "xxx"; }	$filename = md5($gnuplotsrc). "." . $format; return $filename; }

How does GnuplotBasic work?
Underneath you'll find a summary of the global workings of GnuplotBasic: GnuplotBasic saves this plotdata in a temporary file (e.g. )
 * A GnuplotBasic call is initiated from a wiki page written in wiki markup. A call starts with the  tag. GnuplotBasic receives a call every time a wiki page with this   tag is viewed by a user
 * GnuplotBasic receives the full text from the call between  ...
 * From this full text the unique graph image filename is determined (e.g. )
 * If the graph image already exists GnuplotBasic (re-)creates it (or not) depending on the defined refresh rate. If it doesn't exist GnuplotBasic creates it.
 * If plotdata is defined between  ...   then
 * GnuplotBasic does some basic checking of the plotscript-part and replaces unwanted commands with harmless/useless ones
 * GnuplotBasic then saves the plotscript-part in another temporary file (e.g. )
 * The command to execute the plotscript is then given to the Gnuplot program. The Gnuplot program creates the graph image (if the plotscript's syntax is correct).
 * The temporary file(s) are deleted
 * The graph image (e.g. ) is then shown on the page.
 * If anything has gone wrong, either a mistake in the plotscript syntax or an error occurred on the server, the text GnuplotBasic is shown.

Maintenance of your site
As described in the installation, GnuplotBasic creates graph images in the directory. The filenames of these images are automatically determined from the full content of the   call (including its optional plotdata). This means that every time the content of a   call changes the filename of the created image also changes.

A filename is for example:

The consequence is that over time the directory  becomes populated with unused (orphan) graph images. Periodically the wiki site administrator should do some cleanup in this directory. It is recommended to delete all the files in directory  once in a while. Graph images will be (re)created whenever a page is viewed and GnuplotBasic discovers the graph image doesn't exist.

Its origin
This extension is inspired by and on two other gnuplot extensions: Gnuplot-1 and Gnuplot-2. These two have the same name but are different. Thanks to the author(s) of those extensions to get me going. I started to tweak and change one of them to suit my needs and eventually created this new extension with its own functions. I hope this one will be of use to others as well.

Its development
This extension is developed and tested with Gnuplot 4.2, MediaWiki 1.13 and PHP 5.2 on a Windows machine. It should work just as fine on other versions/platforms, as long as PHP 5.x is used.

Its design choices
The design choices I have made for GnuplotBasic.
 * No files, inline plotdata only. GnuplotBasic only accepts plotdata that is part of the wiki markup. Handling external datafiles from an extension is error prone and the uploaded datafiles are hard to manage.
 * No "set output" in plotscript-part to a user defined image filename. Unique (generated) image filenames based on the full content of the GnuplotBasic call only.
 * RefreshRate option. This gives control over the rate of re-creating graph images. Having a unique image filename based on the content of the GnuplotBasic call enables this option.
 * GnuplotBasic regards all text on each line in a GnuplotBasic call after the first "#" it finds as comment. This approach is a bit simpler than what the Gnuplot program actually offers but it is sufficient for what is needed using Gnuplot from within MediaWiki.
 * GnuplotBasic does some basic parsing/filtering of the plotscript-part. Unwanted Gnuplot commands (and potentially dangerous ones that I've identified) are replaced by "x"-es. This can either cause a script not to work or display "x"-es in a label if that text happens to be the same as an unwanted Gnuplot command.
 * GnuplotBasic is called from MediaWiki with the  tag because this is shorter typing, rather than for example.

Why Gnuplot in the first place? Well there is an alternative for producing simple graph images, and that is extension Google Chart for MediaWiki (gchart4mw). Its simplicity and good looking graphs make it an attractive option (don't miss the bug fix on the extensions talk-page if you're thinking of using it). It also supports pie-charts and Gnuplot doesn't. However the downside of this alternative is that it always requires an Internet connection from a user browser. Plus, more serious, all the plotdata is send over the Internet to Google to make a graph each time a page is viewed. This is not so nice if you work with confidential or privacy sensitive data. Therefore I chose Gnuplot. It works fully within the wiki server environment. But... using Gnuplot also comes with some challenges and limitations, please read security limitations. This is a trade-off, it's your decision to make.

Its security limitations
Using Gnuplot, with the help of GnuplotBasic, on a MediaWiki based wiki site (either a local or public site) has some risks because of Gnuplot's extended command vocabulary/syntax and its acceptance of abbreviated commands. Authors using this extension can willingly or unwillingly use Gnuplot commands that have the potential of harming your system (even though some of these commands are filtered out). I am using it on a intranet based wiki, my assumption is that my users have good intentions and are not seeking to harm the system. Therefore I am accepting this (very small) risk.

You can find the list of Gnuplot commands that GnuplotBasic filters out in the PHP script. They are found in  in the definition of.

If you know of Gnuplot commands (or abbreviations) that I've missed that should be filtered out, please make note of that on the talk page of this extension. This way you can help make GnuplotBasic better!

Its name
It is called GnuplotBasic because:
 * It does some basic syntax parsing/filtering of Gnuplot commands in the plotscript-part
 * This parsing/filtering reduces what you can do with Gnuplot a little bit to its basic functions. If anyone wants to use Gnuplot's full capabilities this has to be done with Gnuplot's standard GUI interface.
 * The name needed to be different from the two already existing gnuplot extensions to prevent confusion.

Comments and Feedback
I welcome comments and feedback on this extension. Please use the talk page for this purpose.