Extension:Semantic Drilldown

Semantic Drilldown is an extension to MediaWiki that provides a page for drilling down through a site's data, using categories and filters on semantic properties. It is heavily tied in with the Semantic MediaWiki extension, and is meant to be used for structured data that has semantic markup. You must have version 1.5 or later of Semantic MediaWiki installed for the Semantic Drilldown extension to work: the code will not work without it, or with an earlier version.

The "Browse data" page is the heart of the extension. It lists all the 'top-level' categories in the wiki; i.e., the categories that are not subcategories of another category, and the number of pages within that category. Each category name is a link to a 'drilldown' for the pages in that category. It lets the user select additional constraints to limit the number of results. These constraints come in two types:


 * Subcategories - if the category has any subcategories, those will show up in a "Subcategory" row. Each one will be a link that will let the user view only the pages that belong to that subcategory. The resulting drilldown page will include links for all the filters for the top-level category, and any subcategories of the selected subcategory. You can thus use the "Browse data" page to navigate through the entire category tree.


 * Filters - filters based on semantic properties can be manually created and added to any top-level category. Each such filter gets its own row within the constraints area, to let the user limit the results to only those pages that have certain values for a semantic property. There are four ways to set the possible values for a filter (described in detail below):
 * From property values - the default method; the filter simply shows all current values for this property.
 * Pages in a category - the filter's values can be all the pages that belong, either directly or through a subcategory, to a category.
 * By date range - results are grouped into date ranges, based on a specified time period.
 * Set manually - the filter's values can be set manually. For properties that are numbers, a numerical range can also be set for a filter value, instead of a single number.



In the display of the filter in the drilldown, values that do not have any results for them will not be displayed. The filter will also show two additional values: "Other" and "None". Pages that show up for "Other" are those that have a value for that filter's property other than one of the pre-specified values. Pages that show up for "None" are those that have no value for that property. "Other" and "None", like other filter values, will not show up if there are no results for them.

After any amount of clicking on different subcategories and filters, the user will be able to see, in the page header near the top of the page, the set of subcategories and filters he/she has clicked on, that currently set constraints on the result. The user can get rid of any constraint by clicking on the "x" next to its name in the header.

The drilldown 'results', i.e. the set of pages displayed at any given time during the drilling-down, are by default displayed in the same manner as in MediaWiki's category page; this display, however, can be customized, to show additional values for each page, or to show results in other formats such as tables, maps and timelines.

Code and download
You can download the Semantic Drilldown code in either one of these two compressed files:


 * semantic_drilldown_1.1.tar.gz
 * semantic_drilldown_1.1.zip

You can also download the code directly via SVN from the MediaWiki source code repository, at http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/SemanticDrilldown/. From a command line, you can call the following:

To view the code online, including version history for each file, you can go here.

Installation
After you've obtained a 'SemanticDrilldown' directory (either by extracting a compressed file or downloading via SVN), place this directory within the main MediaWiki 'extensions' directory. Then, in the file 'LocalSettings.php' in the main MediaWiki directory, add the following line somewhere below the calls for the Semantic MediaWiki extension (both the main 'include_once' line and the 'enableSemantics' line):

You may also wish to change the number value for the new "Filter" namespace, defined in SD_Settings.php; by default it is set to 170.

Also, if you have any custom namespaces declared, you should add the following declaration before the 'include_once' call in the file 'LocalSettings.php':

(Or, instead of 170, whatever number you want the "Filter" namespace set to.)

NOTE: The definition of $sdgNamespaceIndex and the call to SD_Settings.php must be placed after the initialization of any custom namespace definitions in LocalSettings.php. Otherwise, the Filter namespace will not initialize.

NOTE: This extension requires the database account used by MediaWiki to be able to create and drop temporary tables, as well as to create table indexes.

Languages supported
Semantic Drilldown has support for over 100 languages.

Authors
Semantic Drilldown was mostly written, and is maintained, by Yaron Koren, reachable at yaron57 -at- gmail.com. The code to display drilldown results was written by David Loomer. The 'combo box' jQuery input was created by Sanyam Goyal as part of the 2010 Google Summer of Code. The code for interfacing with the Page Schemas extension was created by Ankit Garg as part of the 2011 Google Summer of Code. Other developers have also made helpful contributions.

Version history
Semantic Drilldown is currently at version 1.1. See the entire version history.

Special pages
The extension defines three "special" MediaWiki pages:


 * Special:BrowseData - displays a drilldown interface for browsing all the data on the site. (See example of page)
 * Special:CreateFilter - lets a user create a new filter. (See example of page)
 * Special:Filters - lists all filter pages on the site. (See example of page)

Getting started
Before you set up Semantic Drilldown, you should have all the data structures on your site set up - properties, categories, templates, and, if you're using them, forms. See the Semantic Forms "Getting started" section for more on how to set these up.

After all this is done, and you've added some actual data, you should take the following steps:


 * Drill down through the data. As soon as you've installed Semantic Drilldown, you can go to the "BrowseData" special page and see what the category structure looks like on your site. There you can see what filters are needed or would be helpful for each category.


 * Create filters. Every filter you want to be able to drill down with has to be defined separately, on a page in the "Filter:" namespace. The easiest way to create filters is using the 'CreateFilter' special page (see above).


 * Add filters to categories. To add a filter to a category, simply add the tag Has filter::Filter :filter-name to the category's page. Filters should only be added to top-level categories.

Filter settings
Within the page for each filter, semantic tags need to be placed that define the filter. The allowed tags are:


 * - The only mandatory tag. Specifies which property this filter applies to.
 * - States that the possible values for this filter are all the pages that are members of a certain category.
 * - Adds a specific value allowed for this category. You can add as many such tags as you want to a filter.
 * - Used for date filters; indicates the period of time that values are divided into (options are "Month" and "Year").
 * - Sets an input type for the user to enter a filter value, instead of clicking on pre-specified values (options are "combo box" and "date range").
 * - Specifies the name of the filter which will be displayed on the screen.
 * - States that a filter should only be displayed for users if they have already selected a value from the specified filter.

If neither "Gets values from category", "Has value" or "Uses time period" are defined for a filter, the extension will simply list all current values of the property for the given set of pages.

The easiest way to add such properties to a filter is using the 'CreateFilter' special page; see above.

Example
Here is the relevant part of the source code for the 'Sources' category page at Discourse DB:

This category uses the filters Has filter::Filter:Type, Has filter::Filter:Circulation and Has filter::Filter:Country.

And here are the source codes for the three relevant filters - Type:

This filter covers the property Covers property::Publication type.

Circulation:

This filter covers the property Covers property::Has circulation. It has the values Has value::< 100,000, Has value::100,001 - 250,000, Has value::250,001 - 500,000, Has value::500,001 - 1,000,000 and Has value::> 1,000,000.

and Country:

This filter covers the property Covers property::Is published in country. It has the values Has value::Australia, Has value::Great Britain and Has value::United States.

Setting drilldown page title
You can manually set the title of the drilldown page for any specific category, by adding the special property "Has drilldown title" to the category's page. An example would be, in a page called "Category:Cities", the property:

Tag-cloud-style display of filter values
You can set the drilldown page to show the values for each filter and each subcategory in "tag-cloud" style, where the size of each value's name is dependent on the number of results it has. To do this, you need to add two values to your LocalSettings.php file, "$sdgFiltersSmallestFontSize" and "$sdgFiltersLargestFontSize"; these represent the font size of the names of the least-popular and most-popular filter values, respectively, in pixels. Here is an example:

Make sure to add these below the inclusion of Semantic Drilldown itself.

Setting the display of drilldown results
By default, the list of results, or pages that match the current set of filters, is displayed in the style of a category page, with a maximum of 100 results per page. You can change the number of results per page in your LocalSettings.php file, by setting the variable "$sdgNumResultsPerPage". To set the page to show 250 results per page, add the following:

More importantly, you can set the display of results as you would the display of Semantic MediaWiki's inline queries: that includes both additional properties that you would want to see displayed, the format (such as a map or calendar) in which results will be displayed. To do this, use the "Has display parameters" special property on the category page. It takes in a set of parameters like that of an inline query, although separated by semicolons instead of pipes. So, for instance, to display the drilldown results of a category on a Google Maps map, you could add the following to the category page:

(Note: the final '|' simply hides the ugly-looking value from appearing on the the category page; see "To hide the property" here.) You can see an example of a simple use of this functionality here; every event gets its city shown as well. This is done through a "Has display parameters" call found here.

There are several issues relating to the use of "Has display parameters":
 * Unfortunately, currently one format is allowed for any one category.
 * The "sort=" and "order=" parameters for inline queries will not work - pages will always be sorted by their name.

Changing the order of filters
By changing the order in which the "Has filter" tags appear on the category page, the order of the filters on the drilldown page can be changed.

Display of categories
By default, a list of all top-level categories on the wiki shows up on one side of the "Browse data" page, to let users navigate to the drilldown interface for each such category. However, this display can be modified in a number of ways.

Removing the list of categories
You may want to have the drilldown page show the data only for one category, and not display the list of other categories. To do that, just add the string "_single" to anywhere in the URL query string; this will remove the list of categories. An example of such a URL is "Special:BrowseData/Cars?_single" - see here for one example.

Note that you cannot use an internal link (i.e., with double square brackets) to link to such a page, since question marks don't work with internal links. Instead, to link to it, you should use an external link: you can hardcode the URL, or use a formulation like " [ /Special:BrowseData/category-name?_single link-description]", or use the fullurl magic word like this: " [ link-description]".

Showing category names as tabs
If you add the following line to LocalSettings.php:

...it will display categories as tabs at the top of the page, instead of in a vertical list on the side of the page. This option makes more sense for sites with a smaller number of categories (less than seven or so).

Excluding and including categories from the list
You may want certain categories to not show up in the top-level list of categories. You can easily remove a category by adding the following anywhere within the category's page: __HIDEFROMDRILLDOWN__

Conversely, there are some categories that you may want to show up in the main BrowseData list, even though they are not top-level categories. You can accomplish this as well, by adding the following to the category's page: __SHOWINDRILLDOWN__

Conversely, you can also set for categories to only show up in the list if you specifically include them, i.e. have an "opt-in" instead of an "opt-out" approach. You can do that by adding the following to LocalSettings.php:

If you do that, only categories marked with " __SHOWINDRILLDOWN__ " will appear. (The " __HIDEFROMDRILLDOWN__ " marking, on the other hand, will become irrelevant.)

Sites that use Semantic Drilldown
Here is a small sampling sites that use Semantic Drilldown in conjunction with Semantic MediaWiki:


 * CSI Wiki - software wiki produced by Computers and Structures, Inc.
 * Discourse DB
 * Domotiki - A French wiki about home automation
 * EPSA - European Public Sector Award winners and nominees
 * Food Finds
 * Structural Wiki - wiki for structural engineers
 * Technical Presentations
 * UNDP Inter-Municipal Cooperation
 * Verwaltungskooperation - Cooperation in Public Administration
 * WeCoWi

For a much more comprehensive listing, see the Semantic MediaWiki Community Wiki. (Which itself uses Semantic Drilldown to show the list.)

Mailing list
You should use the Semantic MediaWiki mailing list, semediawiki-user, for any questions, suggestions or bug reports about Semantic Drilldown. If possible, please add "[SD]" at the beginning of the subject line, to clarify the subject matter.

Hosting
Currently two wiki hosting sites offer support for Semantic Drilldown:
 * Referata - a site created and run by Yaron Koren. Wikis on Referata can use Semantic MediaWiki, Semantic Drilldown and a variety of related extensions; basic usage is free.
 * Wikia - wikis on Wikia can request Semantic MediaWiki, Semantic Drilldown and Semantic Forms to be turned on (they are not enabled by default). Be aware that these extensions can be several versions behind on Wikia; see here, for example, for the current version numbers. Additional SMW-based extensions are currently not available on Wikia.

Bugs and feature requests
You can submit bug reports and requests for new features at MediaWiki's Bugzilla, here.

The current list of known bugs and requested features for Semantic Drilldown can be found here.

Contributing patches to the project
If you found some bug and fixed it, or if you wrote code for a new feature, please create a patch by going to the main "SemanticDrilldown" directory, and typing:

Then go to the relevant bug report in Bugzilla, or create one if one doesn't exist (note, again, that Bugzilla is used for both bugs and feature requests), and attach this patch file to it.

If, for any reason, you don't wish to use Bugzilla, feel free to simply send this patch, with a description, to Yaron Koren.

Translating
Translation of Semantic Drilldown is done through translatewiki.net. The translation for this extension can be found here. To add language values or change existing ones, you should create an account on translatewiki.net, then request permission from the administrators to translate a certain language or languages on this page (this is a very simple process). Once you have permission for a given language, you can log in and add or edit whatever messages you want to in that language.