Extension:DataTransclusion

The DataTransclusion extension allows individual data records to be retrieved from an external source such as a database or a web API. The fields from the data record are then passed to a template as template parameters for rendering. The design of DataTransclusion aims to make the mechanism flexible, so it can easily be adopted to different application, while remaining safe and secure to use even on large scale public wikis.

Usage
To use a record from a given data source, you need to know the name the data source was given in the configuration, and a template that will accept the fields in the records from that data source as parameters. Consider for example the following wiki markup:

In this example, there are three parameters, two of which are positional parameters, which must always be present in exactly this order:
 * the first parameter is the template to use. In the example, "Book" refers to the page.
 * the second parameter is the name of the data source. "openlibrary" refers to an entry in the  configuration variable (see below).
 * the parameter  is the name of a lookup key that was configured for this data source. In this case, the data source contains information about books, and it was configured to allow look-ups using a book's ISBN.
 * any further parameters will be passed through to the template, unless a field of the same name exists in the data record retrieved from the data source. In that case, the value from the data record overrides the value from the  call.

If the data source openlibrary was configured to provide the fields author and title, then in the template  these fields can be accessed using the normal triple-brace syntax for template parameters, i.e.   and   respectively.

Data sources may specify additional information fields that will always be the same for records from that source. These values may indicate e.g. the name or license of the source itself. In this case,  might have been configured to return The Open Library Project. This is especially useful when using the same template with data from different data sources, e.g. different bibliographical databases.

Note that data transclusion influences MediaWiki's parser cache - if a data source specifies that records from that source are cacheable for one hour, then pages containing data transcluded from that source will not be cached for longer than one hour.

Installation

 * 1) Download and extract the DataTransclusion extension
 * 2) Place the   directory in the   directory of your wiki installation.
 * 3) Activate the extension in your   using the line.
 * 4) Configure data sources using the   configuration variable. See below for details.

Configuration
All configuration of the DataTransclusion extension is done using the  variable in. is an associative array, with one entry for each data source. The logical name of the data source is the key for each entry, the value is itself an associative array of configuration options for that data source. Which configuration options can be used depends on which module is used for the respective data source. Some options however apply to all modules (or are used by the DataTransclusion core itself):


 * class: the name of the PHP class that implements the module to be used to access this data source. REQUIRED. Built-in modules are:
 * DBDataTransclusionSource for accessing records in a local database. See below for details.
 * WebDataTransclusionSource for accessing records via a simple web API. See below for details.


 * keyFields: list of fields that can be used as the key for fetching a record. REQUIRED.
 * optionNames: names of option that can be specified in addition to a key, to refine the output. Optional.
 * fieldNames: names of all fields present in each record. Fields not listed here will not be available on the wiki, even if they are returned by the data source. REQUIRED.
 * sourceInfo: associative array of information about the data source that should be made available on the wiki. This information will be present in the record arrays as passed to the template. This is intended to allow information about source, license, etc to be shown on the wiki. Note that DataTransclusionSource implementations may provide extra information in the source info on their own. Per default, the entry source-name in the sourceInfo is set to the data source's logical name.
 * cacheDuration: the number of seconds a result from this source may be cached for. If not set, results are assumed to be cacheable indefinitely. This setting determines the expiry time of the parser cache entry for pages that show data from this source. If cache is set, cacheDuration also determines the expiry time of ObjectCache entries for records from this source.
 * cache: the object cache to use. Should be given as one of the CACHE_* constants normally used with $wgMainCacheType, usually CACHE_DB. Default is CACHE_NONE. Can also be an ObjectCache instance. The given object cache will be used to cache records from this data source for the number of seconds specified by the cacheDuration option.

DBDataTransclusionSource
The DBDataTransclusionSource implements access to records in a local database (on the same server as the wiki database).

Additional configuration options:
 * query: the SQL query for fetching records. May not contain a GROUP or LIMIT clause (use querySuffix for that). The WHERE clause is automatically generated from the requested key/value pair. If query already contains a WHERE clause, the condition for the desired key/value pair is appended using AND. Note that subqueries are not supported reliably. REQUIRED.
 * querySuffix: additional clauses to be added after the WHERE clause. Useful mostly to specify GROUP BY (ORDER BY and LIMIT are pointless, since the query should always return only a single record).
 * keyTypes: associative arrays specifying the data types for the key fields (as given in keyFields). Array keys are the field names, the associated values specify the type as 'int' for integers, 'float' or 'decimal' for decimals, or 'string' for string fields.
 * keyFields: like for DataTransclusionSource, this is list of fields that can be used as the key for fetching a record. However, it's not required for DBDataTransclusionSource: if not provided, all keys listed in keyTypes will be allowed as key fields.

WebDataTransclusionSource
The WebDataTransclusionSource implements access to records via a simple RESTful web API (generally, JSON or WDDX fetched by a HTTP GET request).


 * url: base URL for building urls for retrieving individual records. If the URL contains placeholders of the form {xxx}, these get replaced by the respective key or option values. Otherwise, the key/value pair and options get appended to the URL as a  regular URL parameter (preceeded by ? or &, as appropriate). REQUIRED.
 * dataFormat: Serialization format returned from the web service. Supported values are 'php' for PHP serialization format, 'json' for JavaScript syntax, and 'wddx' for XML-based list/dicts. Default is 'php'.
 * dataPath: "path" to the actual data in the structure returned from the HTTP request. REQUIRED. For the interpretation of data pathes, see below.
 * fieldPathes: an associative array giving a "path" for each fied which points to the actual field values inside the record, that is, the structure that dataPath resolved to. Useful when field values are returned as complex records. If this is given, fieldNames can be omitted. In that case, the field that appear as keys in fieldPathes will be made available to the template. For the interpretation of data pathes, see below.
 * errorPath: "path" to error messages in the structure returned from the HTTP request. The path is evaluated as described in the section below. If an entry is found at the given position in the response structure, the request is assumed to have failed. REQUIRED.
 * httpOptions: array of options to pass to Http::get. For details, see Http::request.
 * timeout: seconds before the HTTP request times out. If not given, the timeout value given in httpOption is used. If both are not set, 5 seconds are assumed.

The response data is assumed to consist of nested arrays, indexed or associative. A "path" can be used to specify a particular item in hat structure. Pathes are given as indexed arrays (lists), where each entry in the path navigates one step "down into" this structure. Each entry can be either a string (for a lookup in an associative array), and int (an index in a list), or a "meta-key" of the form @@N, where N is an integer. A meta-key refers to the Nth entry in an ordered associative array: @1 would be "bar" in array( 'x' => "foo", 'y' => "bar" ). ''Note: this is arcane and ideosyncratic. It should probably be changed to use XPath.''

XmlDataTransclusionSource
Extension of WebDataTransclusionSource that allows to parse and process arbitrary XML. XmlDataTransclusionSource changes the way some of the options defined by WebDataTransclusionSource are evaluated - especifically, XmlDataTransclusionSource makes use of the W3C XPath specification for ifentifying parts of an XML document.


 * dataFormat: must be "xml" or end with "+xml" (e.g. "rdf+xml") if given. Defaults to "xml".
 * dataPath: XPath to the actual record data in the structure returned from the HTTP request. REQUIRED.
 * fieldPathes: an associative array giving a XPath for each fied which points to the actual field values inside the record, that is, the structure that dataPath resolved to. Useful when field values are returned as complex records. If given, fieldNames defaults to array_keys( fieldPathes ).
 * errorPath: XPath to error messages in the structure returned from the HTTP request. If an entry is found at the given position in the response structure, the request is assumed to have failed. For more complex detection of errors, override extractError. REQUIRED.