Extension:RDFIO

Current status: Tested with SMW 1.9.x (MW 1.23.x)
UPDATE, August 11, 2014 Version 1.9.6 released. This is an intermediate release featuring improved feedback on import success, an enhanced admin page, and internal refactoring for improved performance. The next release (due soon) will incorporate template handling.

UPDATE, June 25, 2014: RDFIO is tested with MediaWiki 1.23 and SMW 1.9. We also just created a screencast of the full installation of MediaWiki, SMW and RDFIO on a vanilla Xubuntu 14.04 installation: ''The description on YouTube contains links directly to the sections covering MediaWiki, SMW and RDFIO. Although the full screencast is 27 min, the RDFIO part is only 10 minutes long, if you have a working MW and SMW installation''
 * Watch Screencast: Installing Semantic MediaWiki with RDFIO on Xubuntu 14.04
 * Download Virtual Machine With Semantic MediaWiki and RDFIO already installed (2 GB .ova file)

''The file is available in OVA format, which should be compatible with most virtual machine players such as VirtualBox and VMWare. ''

After downloading and importing to VirtualBox / VMWare, start the machine, and click the "Open wiki" shortcut on the desktop!

UPDATE, 20131021: RDFIO is nearing basic feature completion. Many things are now tested with Unit testing and some system testing.

Introduction
This extension extends the RDF import and export functionality in Semantic MediaWiki by providing import of arbitrary RDF triples (not only OWL ontologies, as was the case before), and a SPARQL endpoint that allows write operations.

Technically, RDFIO implements the PHP/MySQL based triple store and accompanying SPARQL Endpoint provided by the ARC2 library. For updating wiki pages with new triples on import/sparql update, the Wiki Object Model extension is used.

The RDF import stores the original URI of all imported RDF entities in the Equivalent URI property, which can later be used by the SPARQL endpoint, instead of SMW's internal URIs, which thus allows to expose the imported RDF data "in its original formats", with its original URIs. This allows to use SMW as a collaborative RDF editor, in workflows together with other semantic tools, from which it is then possible to "export, collaboratively edit, and import again", to/from SMW.

This extensions was developed as part of a Summer of Code 2010 project. The project description can be found here. See also the status page, with info on how you can follow the project.

MediaWiki

 * see https://www.mediawiki.org/wiki/Installation

Semantic MediaWiki

 * See http://semantic-mediawiki.org/wiki/Help:Installation
 * To show the "Semantic factbox" on all pages, make sure to include this in your LocalSettings.php file:

$smwgShowFactbox = SMW_FACTBOX_NONEMPTY;

Wiki Object Model Extension
Install the Wiki Object Model according to instructions on Wiki Object Model page, or by following the following steps:


 * Check out the WikiObjectModel extension:

cd wiki/extensions svn checkout http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/WikiObjectModel


 * Add the following to your LocalSettings.php file:

include_once("$IP/extensions/WikiObjectModel/WikiObjectModel.php");

Installing RDFIO
Assuming you have followed the steps above to install the dependencies for RDFIO:


 * Install RDFIO by executing the following commands in a terminal:

cd wiki/extensions git clone https://github.com/samuell/RDFIO.git


 * Add the following to LocalSettings.php, to activate RDFIO:

include_once("$IP/extensions/RDFIO/RDFIO.php");
 * Add the following option to the end of LocalSettings.php:

$smwgOWLFullExport = true; It allows user defined properties for property articles (which is needed by the RDF import)

cd wiki/extensions/SemanticMediaWiki mkdir libs cd libs git clone https://github.com/semsol/arc2.git arc (RDFIO adds include lines for ARC) * Semantic Tools ** Special:RDFIOAdmin|RDFIO Admin ** Special:RDFImport|RDF Import ** Special:SPARQLEndpoint|SPARQL Endpoint ** Special:SPARQLImport|SPARQL Import
 * Create folder 'libs' in 'wiki/extensions/SemanticMediaWiki/', and navigate to it. Download the ARC2 library from here, then extract and place it in a new folder `wiki/extensions/SemanticMediaWiki/libs/arc`, so that the file `ARC2.php` is placed in the `arc` folder. Command line syntax (using the git client (In Ubuntu, install with sudo apt-get install git, if you don't have it)):
 * Log in to your wiki as a super user
 * Browse to http://[your-domain]/wiki/Special:RDFIOAdmin (You can click the "RDFIO Admin" link in your menu, if you followed the point above!)
 * Click the "Setup" button is to set up the database tables.
 * Note: If you already have semantic annotations in your wiki, you need to go to the article "Special:SMWAdmin" in your wiki, and click "Start updating data", and let it complete, in order for the data to be available in the SPARQL endpoint.
 * Edit the MediaWiki:Sidebar page and add the following wiki snippet, as an extra menu (I use to place it before just the "* SEARCH" line), which will give you links to the main functionality with RDFIO from the main links in the left sidebar on the wiki:


 * Create the article "MediaWiki:Smw_uri_blacklist" and make sure it is empty (you might need to add some nonsense content like { – }).
 * Now, try adding some semantic data to wiki pages, and then check the database (using phpMyAdmin e.g.) to see if you get some triples in the table named `arc2store_triple`
 * Access the SPARQL endpoint at http://[url-to-your-wiki]/Special:SPARQLEndpoint
 * Access the RDF Import screen at http://[url-to-your-wiki]/Special:RDFImport
 * Access the SPARQL Import screen at http://[url-to-your-wiki]/Special:SPARQLImport

Additional configuration
These are some configuration options, that you might want to adjust to your specific use case, and that go into your LocalSettings.php file>


 * 1)  RDFIO Configuration
 * 2) An associative array with base uris as keys and corresponding
 * 3) prefixes as the items. Example:
 * 4) array(
 * 5)       "http://example.org/someOntology#" => "ont1",
 * 6)       "http://example.org/anotherOntology#" => "ont2"
 * 7) $rdfiogBaseURIs = array;
 * 8) Query by /Output Equivalent URIs SPARQL Endpoint
 * 9) (overrides settings in HTML Form)
 * 10) $rdfiogQueryByEquivURI = false;
 * 11) $rdfiogOutputEquivURIs = false;
 * 12) $rdfiogPropertiesToUseAsWikiTitle = array(
 * 13)  'http://semantic-mediawiki.org/swivt/1.0#page',
 * 14)  'http://www.w3.org/2000/01/rdf-schema#label',
 * 15)  'http://purl.org/dc/elements/1.1/title',
 * 16)  'http://www.w3.org/2004/02/skos/core#preferredLabel',
 * 17)  'http://xmlns.com/foaf/0.1/name',
 * 18)  'http://www.nmrshiftdb.org/onto#spectrumId'
 * 19) Allow edit operations via SPARQL from remote services
 * 20) $rdfiogAllowRemoteEdit = false;
 * 1)  'http://purl.org/dc/elements/1.1/title',
 * 2)  'http://www.w3.org/2004/02/skos/core#preferredLabel',
 * 3)  'http://xmlns.com/foaf/0.1/name',
 * 4)  'http://www.nmrshiftdb.org/onto#spectrumId'
 * 5) Allow edit operations via SPARQL from remote services
 * 6) $rdfiogAllowRemoteEdit = false;
 * 1) Allow edit operations via SPARQL from remote services
 * 2) $rdfiogAllowRemoteEdit = false;
 * 1) $rdfiogAllowRemoteEdit = false;

Dependencies

 * Semantic MediaWiki Extension
 * Wiki Object Model Extension
 * ARC2 RDF library for PHP.

Editing Semantic MediaWiki from Bioclipse
Chemists and biologists using Bioclipse can now take their working data and export it to a wiki where their peers can make corrections, before importing it again for further analysis, etc. This workflow is possible today, as hinted in this blog post / screencast, and is the focus of current research/development (progress documented on the blog) in the Bioclipse group of prof. Jarl Wikberg at [http://www.farmbio.uu.se/researchgroup.php?fg=1 Dept. of Pharm. biosciences], Uppsala University.

Bugs, new feature request and contact information
Please reports bugs and feature requests in Bugzilla].

Important: Please add my e-mail address, samuel.lampa[at]gmail.com as "assign-to" or "cc", so that I get notified, and add the string " " to the subject line, in order to make the issues easy to collect.

General feedback can be given here on the talk page.

Change Log

 * 1.9.6 - 2014-08-11 - Enhanced admin page, better feedback on import/setup success, creates pages for import data sources, refactoring and tidying of internal functionality
 * 1.9.? - 2013-10-21 - RDFIO is nearing basic feature completion. Many things are now tested with Unit testing and some system testing.
 * 1.9.? - 2013-05-17 - Works with the new SMWSQLStore3
 * 1.9.0 - 2012-11-22 - Updated to work with SMW 1.6 and later. Much refactored code base. Depends now only on Wiki Object Model, rather than SMWWriter and Page Object Model.
 * 0.5.0 - 2010-09-17 - Numerous fixes to make remote SPARQL querying work (See repository updates for a list of all commits).
 * Improved file hierarchy
 * Made querying and output of/querying by Original URIs and Equivalent URIs configurable from LocalSettings.php in SPARQL endpoint (So this can be turned on for remote queries too)
 * In total five new configurable settings for LocalSettings.php (see here for full list):
 * $rdfiogQueryByOrigURI = true;
 * $rdfiogOutputOrigURIs = true;
 * $rdfiogQueryByEquivURI = false;
 * $rdfiogOutputEquivURIs = false;
 * $rdfiogAllowRemoteEdit = false;
 * Lots of serious bug fixes encountered when making SPARQL querying from Bioclipse/Jena work
 * 0.4.0 - 2010-08-16
 * Support for configuring extra namespace prefixes in LocalSettings.php
 * More options in RDF Import screen
 * Output SPARQL resultset as default for remote queries, and HTML for form queries
 * Enable output as Original URI/Equivalent URI also for XML Resultset output format
 * Refactorings (Merged EquivalentURIHandler and SMWBatchWriter classes, Broke out RDFIOPageHandler in separate file)
 * Many bugfixes
 * 0.3.0 - 2010-07-30 - Added output filtering options and other improvements.
 * Option to query by Equivalent URI
 * Refined SPARQL Endpoint screen
 * Option to output all Equivalent URIs (For RDF/XML format only)
 * Option to filter properties by ontology (when outputting equivalent URIs) by specified an URL to an OWL ontology definition. (For RDF/XML format only).
 * Much improved processing of SPARQL queries
 * Various refactoring
 * Fixed various bugs
 * Initialize query variable (r150)
 * Don't delete Original URI properties etc when deleting other facts (r151)
 * Fixed error in isURL check (r153)
 * 0.2.0 - 2010-07-20 - Important security improvements
 * Checking for appropriate user rights on all special pages
 * Improved code structure and comments
 * Various small fixes
 * 0.1.0 - 2010-07-21 - First release

Alternative triple store connectors / SPARQL Endpoints
One of the features of RDFIO is to connect Semantic MediaWiki with a triple store, and to provide a SPARQL Endpoint. There are (already) a few extensions that offer this feature. See this page for an overview of triple store connector features. (The idea behind RDFIO is mainly focusing on the RDF import functionality, and merge of some or all of the extensions is being discussed).