Extension:LinkedWiki

From MediaWiki.org
Jump to navigation Jump to search
MediaWiki extensions manual
OOjs UI icon advanced.svg
LinkedWiki
Release status: stable
LogoLinkedWiki.png
Implementation Parser extension , Special page , Data extraction, API , Media
Description The LinkedWiki extension allows you to read and save your data and reuse the Linked Open Data in pages or modules of your wiki.
Author(s) Karima Rafes (Karima Rafestalk)
Latest version 3.3.7
MediaWiki 1.33.1+
PHP 7.1+
Database changes No
License CC BY-SA 4.0
Download
README
Example http://daap.eu
Translate the LinkedWiki extension if it is available at translatewiki.net
Check usage and version matrix.
Issues Open tasks · Report a bug

The LinkedWiki extension lets you reuse Linked Data in your wiki. You can get data from Wikidata or another source directly with a SPARQL query. This extension also provides Lua functions for building your modules so that you can write your data in your RDF database.

Quick start[edit]

After installing this extension:

  1. Open the special page: SPARQL editor
  2. Select a SPARQL service (in your settings) or write the endpoint of your SPARQL service
  3. Insert a SPARQL query (examples of SPARQL queries)
  4. Select a visualization: HTML table or a Sgvizler2 visualization
    1. For a Sgvizler2 visualization, you can click on the button "See the doc" to find its available options.
  5. Check the result
  6. To finish, open the tab "Howto use this query in this wiki?" and copy the generated wiki code in a page of your wiki

See details : #sparql reuses your data in your wiki

Examples of visualizations[edit]

Maps[edit]

A map with the parameter chart=leaflet.visualization.Map
A datatable with the parameter chart=bordercloud.visualization.DataTable

Only 3 parameters are necessary to print a map in your wiki:

  1. a SPARQL query
  2. a SPARQL service (by default Wikidata)
  3. a visualization (charts or table, pivot, etc.)
{{#sparql:
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX pq: <http://www.wikidata.org/prop/qualifier/>
PREFIX psv: <http://www.wikidata.org/prop/statement/value/>
PREFIX p: <http://www.wikidata.org/prop/>
PREFIX bd: <http://www.bigdata.com/rdf#>
PREFIX wikibase: <http://wikiba.se/ontology#>
PREFIX wd: <http://www.wikidata.org/entity/>
PREFIX wdt: <http://www.wikidata.org/prop/direct/>

select distinct ?lat ?long  ?presidentLabel
where {
        ?president wdt:P39 wd:Q191954 ;
                    wdt:P19  ?villeNaissance ;
                     p:P39 ?s .

        ?villeNaissance  p:P625 ?coord .
        ?coord   psv:P625 ?coordValue .
        ?coordValue a wikibase:GlobecoordinateValue ;
                        wikibase:geoLatitude ?lat ;
                        wikibase:geoLongitude ?long .

        SERVICE wikibase:label {
             bd:serviceParam wikibase:language "fr" .
        }
}
|config=http://www.wikidata.org
|chart=leaflet.visualization.Map
}}

For the leaflet.visualization.Map visualization with OpenStreetMap, you can add several options.

{{#sparql:
...
|config=http://www.wikidata.org
|chart=leaflet.visualization.Map
|options=width=100%!height=500px
}}

You can also use the google.visualization.Map visualization and see the log to debug your query or the visualization.

{{#sparql:
...
|config=http://www.wikidata.org
|chart=google.visualization.Map
|options=width=100%!height=500px
|log=2
}}

You can replace the parameter config by the parameter endpoint with a SPARQL endpoint but if it does not work, if that does not work, you will have to create a specific configuration for this SPARQL service.

{{#sparql:
...
| endpoint = http://example.org/sparql
...
}}

HTML table[edit]

Example a HTML table with the SPARQL parser with templates

By default, #sparql builds a HTML table that can be customized with wiki templates.

This visualization supports the service SPARQL with credentials that have to describe in the LocalSettings.php .

Example:

{{#sparql:
select * where { ?x ?y ?z . } LIMIT 5 
| config = https://myPrivateSPARQLService.example.org/sparql
| headers = &nbsp;,name2,name3
| classHeaders= class="unsortable",, 
}}

Datatable[edit]

Another available table "DataTable", this Javascript visualization can be customized with HTML tags.

{{#sparql:
select * where 
 { ?x ?y ?z . } 
LIMIT 15
|config=http://www.wikidata.org
|chart=bordercloud.visualization.DataTable
|options=width=100%!height=500px
}}

Usage[edit]

Here Open Street Map but you can use other chart in your wiki.

Build SPARQL queries[edit]

The LinkedWiki extension gives two SPARQL editors. Flint Editor works with SPARQL endpoint 1.1 or 1.0 but sometimes, it doesn't work for example with Wikidata.

We develop a new SPARQL editor where you can select in one click an endpoint already defined in your configuration and read (and write, if you want) via SPARQL directly in this editor.

See details: Special pages to test your queries and to build your visualizations for your wiki

Visualize SPARQL results[edit]

The extension gives a parser #SPARQL to reuse your data and the Linked Open Data in your wiki.

You can use a new SPARQL endpoint or reuse a SPARQL service already defined in the configuration of your wiki.

See details :

Write data in the pages[edit]

The tag rdf lets to write directly in RDF/Turtle (1.0 or 1.1) on any page of wiki. All pages with this tag are inserted in the category "RDF page".

If the option "check RDF Page" is enabled, the wiki checks the RDF before saving the page. If there is an error, the wiki shows the line where there is a problem in the RDF code.

Example of page with the tag rdf to describe a RDF documentation:

Example of a RDF documentation with SHACL rules
<rdf>
prefix daapp: <http://daap.eu/wiki/Data:Project#>

prefix owl: <http://www.w3.org/2002/07/owl#> .
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix sh: <http://www.w3.org/ns/shacl#> 
prefix xsd: <http://www.w3.org/2001/XMLSchema#> 
</rdf>

=== GeneralMethod ===

Descriptions...

==== Definition  ====
<rdf>
daapp:GeneralMethod
  rdf:type owl:Class ;
  rdfs:label "General method"@en ;
  rdfs:label "Méthode générale"@fr ;
  rdfs:subClassOf owl:Thing .

</rdf>

==== Constraints  ====
<rdf constraint='shacl'>
daapp:GeneralMethod
  rdf:type sh:Shape ;
  sh:targetClass daapp:GeneralMethod ;
  sh:property [
      rdfs:label "Label" ;
      sh:minCount 3 ;
      sh:predicate rdfs:label ;
    ];
  sh:property [
      rdfs:label "hasCampaign"^^xsd:string ;
      sh:minCount 1 ;
      sh:nodeKind sh:IRI ;
      sh:predicate daapp:hasCampaign ;
      sh:class daapp:Campaign ;
    ] ;
.
</rdf>

Share data[edit]

IRIs (or URIs) of pages with the tag rdf are Cool IRIs. So via a HTTP request, a machine see only the RDF content and a human can see the RDF content and its description in natural language in the same page.

If your wiki is private, it is possible to open your private wiki only for your RDF database (see the installation).

Write data of main pages in a data namespace[edit]

A tab "Data" appears at the top of each page.

The LinkedWiki extension creates namespaces: Data and UserData. Users navigate in these namespaces via the tab "Data" on all main/users pages.

Only users in the group "Data" can change these namespaces. A user or a bot can use these namespaces to write a RDF/Turtle content in relation with the main pages.

Push private data in an open knowledge base[edit]

You can insert the tab "Push" on all pages, in order to push easily in a target wiki a wiki page of another wiki with its sub-pages, data, files, modules, etc.

Configuration of SPARQL services[edit]

Often, the configuration of SPARQL services is not trivial and very different in function of RDF databases. In the configuration of this extension, you can configure in detail the HTTP requests supported by your public SPARQL services as well as your private SPARQL services.

This extension supports the SPARQL services with credentials and the users of your Wiki can reuse your private data without seeing your credentials.

See details : Configuration of the LinkedWiki extension

With a module, you are able to read, save and check your data with an external knowledge base such as Wikidata.

Module : Lua class to read/write your data[edit]

Generally for users, a wiki page is like an object where they want to be able to add a new property. Unfortunately, RDF schemas can be complex and the contributors are rarely experts in RDF or in SPARQL.

The extension simplifies the work of contributors without imposing definitive RDF schemas on your data. With the Lua class of this extension, you can build your own module (for example an infobox) where you are able to add, read and check a property of your RDF database via a SPARQL service.

If you want to change your RDF schemas, you need to change simply your modules and refresh your database and all pages of your wiki via the special page "Refresh database".

See details : Use LinkedWiki in your modules

Write constraints and generate a SHACL report[edit]

The tag rdf supports the attribute constraint to precise how checking your data. All pages with this attribute are inserted in the category "RDF schema".

For the moment, LinkedWiki support only SHACL. If RDFUnit is installed, the special page "RDF test cases" generate the SHACL report of your database with the rules wrote in the wiki. This special page shows the last report calculated and can recalculate it (many minutes).

To enable constraints, you need to insert this attribut constraint='shacl' in the tag rdf. Example :

<rdf constraint='shacl'>
daapp:GeneralMethod
  rdf:type sh:Shape ;
  sh:targetClass daapp:GeneralMethod ;
  sh:property [
      rdfs:label "Label" ;
      sh:minCount 3 ;
      sh:predicate rdfs:label ;
    ].
</rdf>

Download instructions[edit]

You can download the last version with this link.

Installation of LinkedWiki[edit]

To install this extension :

  1. copy the extension in the folder extension/LinkedWiki of your wiki
  2. in the folder, execute composer install --no-dev and yarn install --production=true (or npm install --production). If you don't have install composer or yarn, see in this page : "How install composer and yarn ?".
  3. add the following lines to LocalSettings.php :
wfLoadExtension( 'LinkedWiki' );

// Insert your Google API key, if you use Google charts. 
// https://developers.google.com/maps/documentation/javascript/get-api-key
$wgLinkedWikiGoogleApiKey = "GOOGLE_MAP_API_KEY";

// Insert your OpenStreetMap Access Token, if you use OpenStreetMap via the Leaflet charts.
// https://www.mapbox.com/api-documentation/#access-tokens
$wgLinkedWikiOSMAccessToken = "OPENSTREETMAP_ACCESS_TOKEN";

You can now use the special page "SPARQL Editor" of your wiki to build a query with its visualization and copy an example of code with the parser #sparql in any pages of your wiki. On the service LinkedWiki.com, you can find examples of queries with their wiki text.


Configuration of local SPARQL services[edit]

By default, a query without endpoint or configuration is resolved by Wikidata (read only).

If you add a new SPARQL service and change the default SPARQL service of your wiki, you need to add parameters in your LocalSettings.php .

For example for a Virtuoso SPARQL service, you can add the configuration "http://database-test/data" :

$wgLinkedWikiConfigSPARQLServices["http://database-test/data"] = array(
	"debug" => false,
	"isReadOnly" => false,
	"typeRDFDatabase" => "virtuoso",
	"endpointRead" => "http://database-test:8890/sparql/",
	"endpointWrite" => "http://database-test:8890/sparql-auth/",
	"login" => "dba",
	"password" => "dba",
	"HTTPMethodForRead" => "POST",
	"HTTPMethodForWrite" => "POST",
	"lang" => "en",
	"storageMethodClass" => "DatabaseTestDataMethod",
	"nameParameterRead" => "query",
	"nameParameterWrite" => "update"
);

If you want to replace Wikidata by this SPARQL service, you need to add also this line:

$wgLinkedWikiSPARQLServiceByDefault= "http://database-test/data";

If you want to use this SPARQL service to save all RDF data of wiki, you need to add this line:

$wgLinkedWikiSPARQLServiceSaveDataOfWiki= "http://database-test/data";

Installation of advanced features[edit]

Make an infobox with lua functions[edit]

If you want to make an infobox with Lua functions of LinkedWiki, you need to install the Extension:Scribunto and the Extension:Capiunto.

Next, you can start to read the quick start with Lua.

Add a tab Data for main pages and user pages[edit]

A data tab appears at the top of each page.

LinkedWiki allows the tag rdf to write with RDF/Turtle directly in a page (their ontology or their SHACL rules, for example) but people prefer to separate natural language from RDF/Turtle in their wiki.

In your LocalSettings.php , the following line inserts a new tab Data on all main pages and user pages. After, you can give the rights to users to see or not this tab and to change or not the pages in the Data namespace.

wfLoadExtension( 'NamespaceRelations', "$IP/extensions/LinkedWiki/bundle/NamespaceRelations/extension.json" );

Add a tab Push[edit]

When you have finished working in private (ie, in a private wiki), you may want to push your pages (with their modules, templates, files and data pages) in another (public) wiki. This installation inserts a discrete tab "push" on your pages.

You can add push targets (wikis you can push content to) by adding elements to the $egPushTargets array. The array keys should be the names of your wikis and the values should point to the wiki root, without tailing slash. You can find your wiki root by clicking the history tab on a page, finding the '/index.php' in it, and taking everything that's left of that.

You need to create the logins and passwords via the Special: BotPasswords of targets.

Example:

wfLoadExtension( 'Push', "$IP/extensions/LinkedWiki/bundle/Push/extension.json" );

$egPushTargets["wikiTarget"] = "http://en.wikipedia.org/w";
$egPushLoginUsers["wikiTarget"] = "LOGIN_OF_BOT"; 
$egPushLoginPasswords["wikiTarget"] = "PASSWORD_OF_BOT";
$egPushTargetsNamespace['wikiTarget'][] = "Data";

Details: Install of push extension for LinkedWiki

Compatibility: Mediawiki 1.33.1+

Check the RDF/Turtle syntax before saving[edit]

In your LocalSettings.php , you can enable the feature "check RDF Page" with this line :

$wgLinkedWikiCheckRDFPage = true;

This feature uses RAPPER to parse the syntax Turtle (1.0 and 1.1) in the wiki.

This tool is installed in same time that Raptor2 of Redland. To install it in CentOS, the commands are :

yum install raptor2 
# or yum install redland 

# check
rapper -?

Generate a SHACL report[edit]

You need to install RDFUnit. This tool is experimental but the code is stable.

The extension waits RDFUnit in the folder /RDFUnit of your server (or link this folder). The special page "RDF Unit" shows the command line to test the installation and shows the SHACL report or the errors of RDFunit.

Here an example to install RDFUnit v0.8.21 (last release) in a CentOS server:

yum install redland  maven -y
cd /
git clone --depth 1 --branch v0.8.21 https://github.com/AKSW/RDFUnit.git
cd RDFUnit
mvn -pl rdfunit-validate -am clean install -DskipTests=true

rm -rf /usr/share/httpd/.m2/repository
mkdir -p /usr/share/httpd/.m2/repository 
cp ~/.m2/repository /usr/share/httpd/.m2/repository -R
chown apache:apache /usr/share/httpd/.m2 -R

semanage permissive -a httpd_t
chown apache:apache /RDFUnit -R

Refresh your RDF database with RDF pages of private wiki[edit]

If your wiki is private, the special page "Refresh RDF database" does not work without Extension:NetworkAuth. The RDF database with a SPARQL service need to read the RDF page of private wiki without credentials.

If your database is installed with the wiki on the same server, the configuration for Extension:NetworkAuth will be:

# Log-in unlogged users from these networks
$wgNetworkAuthUsers[] = [
        'iprange' => [ '127.0.0.1','::1','OR_ANOTHER_IP'],
        'user'    => 'NetworkAuthUser',
];

$wgNetworkAuthSpecialUsers[] = 'NetworkAuthUser';

You can find the good IP used by your SPARQL service in the HTTP logs after using the special page "Refresh RDF database".

Known issue[edit]

Errors about CURL[edit]

If, after the installation, you have errors about CURL, probably you need to install the lib php7X-curl in your server. Example with PHP 7.3 with ubuntu, debian, CentOS or fedora:

apt-get install php73w-common

or

yum / dnf install php73w-common

Questions ?[edit]

How install composer and yarn ?[edit]

For debian or fedora:

apt-get install yarn composer

or

curl -sL https://dl.yarnpkg.com/rpm/yarn.repo -o /etc/yum.repos.d/yarn.repo
yum / dnf install yarn composer

How to propose a new feature ?[edit]

How to declare a problem ?[edit]

See also[edit]