Extension talk:Semantic Drilldown

Filters show up in BrowseData, but empty
Odd problem: my filters do show up in Special:BrowseData, but the properties they are supposed to cover don't. Some of them even have a "Has label" tag, but that doesn't apply. It's weird because they were working until a few hours ago. I made a lot of tests, I even renamed the Category, moved all tags from the old category to the new one, changed the category tag in every relevant page (from the old name to the new one), applied nulledits to all property and filter pages, but no way: in property pages all values are fine, in filter pages the names of the covered properties are fine, in the category page the names of the filters are fine, but still: in Special:Browse Data I can see all filters, but no values. Note that I can see below the whole list of pages in the category. Again, everything was fine until a few hours ago. The only change I have made that may have interfered with Drill Down is deleting the category "General Templates" from my Templates using Replace Text extension. I read above that Stormraven used the same extension... Help?

Davide 05:05, 11 May 2009 (UTC)

Installation ... Upgrade?
Is there any issue when upgrading this extension? --Dmb 15:08, 22 January 2009 (UTC)


 * Support for old versions of Semantic MediaWiki gets removed from time to time; you can check the version history. Yaron Koren 15:47, 22 January 2009 (UTC)

Undefined variable: wgAmericanDates in SD_BrowseData.php on line 630
I get the following PHP Notice when I try to BrowseData on a category that "Has filter" with an input type of "date range": Notice: Undefined variable: wgAmericanDates in /path/to/wiki/extensions/SemanticDrilldown/specials/SD_BrowseData.php on line 630

I have tried explicitly setting "$wgAmericanDates = true" in LocalSettings.php, and the error remains.

Version Details:
 * MediaWiki 1.14.0
 * PHP 5.2.6
 * MySQL 5.0.45
 * Apache 2.2.3
 * OS: CentOS 5.2 (Linux Kernel 2.6.18-92.1.6.el5)

Semantic Extensions: --74.164.199.186 23:23, 23 March 2009 (UTC)
 * Semantic MediaWiki 1.4.2
 * Semantic Forms 1.5.4
 * Semantic Drilldown 0.5.4


 * Looks like you've found a bug. It'll be fixed in the next version of Semantic Drilldown, which will hopefully be released soon. Yaron Koren 23:39, 23 March 2009 (UTC)

Filters not displaying - Table 'wikidb.smw_attributes' does not exist
Hi Yaron and all:

Having an issue with getting my filters to show up on the BrowseData page. I have articles that have the properties I am trying to filter on, but when I click on the category in my BrowseData page, I get the following error:

Database error

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

 CREATE TEMPORARY TABLE semantic_drilldown_filter_values ( id INT NOT NULL, value VARCHAR(200) NOT NULL, INDEX sdfv_id_index(id) ) AS SELECT subject_id AS id, value_xsd AS value FROM `smw_attributes` WHERE attribute_title = 'Affects_Geographic_Region'

from within function "". MySQL returned error "1146: Table 'wikidb.smw_attributes' doesn't exist (localhost)".

Any idea as to where I veered off course? My install is MW is:

MediaWiki 	1.13.4 PHP 	5.2.8 (apache2handler) MySQL 	5.1.30-community-log

Thanks,

Dave Manzolillo DManzolillo-at-LMI.org

'''Update: Once I disabled Halo it worked. Thanks!'''

Has anyone been able to get SMW forms and drilldown to work with the Halo extension? The more I get into trouble, I find that disabling Halo/SMW+ extensions fixes my issue....


 * An ideal solution to this problem is right now being discussed on the SMW mailing list. Until there is one, there's an easy workaround in the SD code, that I plan to add in the next version. Yaron Koren 03:51, 1 April 2009 (UTC)

Filter Stopped Working
Hi. One of my filters has just stopped displaying in my browsedata page.

The property is still there and named the same. There is data in it.

Is there a limit on the number of objects in a filter?

The Filter: This filter covers the property Covers property::Manufacturer.

The Property: This is a property of type Has type::Page. It links to pages that use the form Has default form::Straight Razor Manufacturer.

Thanks.


 * No, there's no limit. Did anything change between when it worked and when it didn't? An extension upgrade, perhaps? Yaron Koren 03:52, 1 April 2009 (UTC)


 * I created a subcategory called "Manufacturers". Cannot think of any other changes.

Articles that belong to a Namespace are not listed but still counted in the Category
Hi all,

I am using Semantic Drilldown Version 0.5.6.

I noticed that when an article belongs to a namespace (e.g. KB in my case), the article will be counted in the category. However the article will not be returned in the list. I get the following message when all the articles of a category belong to a custom namespace:

There are no results for this report.

If I modify an article that does not belong to any Namespace and add it to the category, the article is properly listed.

Is this a normal behaviour? I am not good at PHP so it will take me few hours or more to understand how the extension is selecting which article is displayed, but if needed, I will do it :) . Is there an option I could configure to allow articles from specific namespaces to be returned?

Anyone faced the same problem? I searched but could not find anything about it, I apologize if it has already been answered.

Thank you. Yann.


 * Hi, I don't understand - if you create an article in the "KB" namespace, it gets added to a category automatically? What do you mean by a category here? Yaron Koren 12:46, 27 April 2009 (UTC)

No it is not added automatically to the namespace, sorry for the confusion. I use a form + template to make this happen. I mean by category that the following text is present in a page/article: [ [ Category : KB ] ] I added spaces on purpose here to avoid it to be parsed.

So basically if I have two articles with the category set with the following names:

KB:Article1 and Article2

only the second article (Article2) will show up in the list whereas both are counted on the top right box named "Choose a category" where you can switch between categories.

EDIT: I ended up by not putting the articles of the KB category in a custom Namespace


 * Okay, it looks like you've found a bug in Semantic Drilldown - namespaces that don't have SMW enabled for them don't show up in 'BrowseData'. Yaron Koren 15:27, 29 April 2009 (UTC)

Special:Browsedata showing incorrect category counts
Using this extension (0.5.6) on MW 1.15.0 with SMW 1.4.2, SRF 1.4.4. I have not started to try using any filters yet, but in "Special:BrowseData", all of the article counts on categories seem to range from 60% - 100% above the actual number of articles in the category. It seems that articles in multiple categories are being counted as one an instance of each the category to which it belongs (with ~600 articles, I'm not going to check each one to confirm, but eyeballing it, it seems right).

For example (not an actual example, but it's simpler to explain here), it will read:

"Fruit (300)" when there are only 200 articles in that category, "Edible yellow things (2)" when there is only 1 article in that category (Banana), but that article also belongs to the category "Fruit".

I'm not sure what the issue is (I know very little about PHP and less about MySQL), but perhaps this has something to do with the way the categorylinks table is handled in lines 150 or 203 of SD_BrowseData.php?

Drilldown using multiple values for in a single filter
Is it possible to drill down by selecting a multiple values for a single filter? For example, the following page has a filter based on an extensive list of keywords: http://www.structuralwiki.org/en/Special:BrowseData/Software?_single. I would like to be able to drill down by selecting two keywords in the first filter. However, as soon as I select one keyword, all the the other keywords in the list disappear and "(Add another value)" message appears. How can I add this another value? Thanks. Andrewok 03:44, 10 September 2009 (UTC)


 * You need to click the little arrow next to it, and that will display all the values again. It's a rather cryptic interface, I admit, and if you can think of something better (maybe just better text, maybe more than that), I'd like to hear it. Yaron Koren 03:59, 10 September 2009 (UTC)


 * I was thinking that perhaps the text "(Add another value)" could be a hyperlink with the same functionality as the little down arrow. I also noticed that selections from different filters are treated as AND condition, while selections of multiple values from a single filter is treated using OR condition. Sometimes desirable to select multiple values from the same filter using AND condition. I was able to achieve this functionality by using several filters for the same property (such as "keyword 1", "keyword 2", "keyword 3"), with each subsequent filter requiring selection to be made on the previous filter. Perhaps it would be worthwhile (if possible) to directly implement choice of OR or AND condition for multiple values selected from the same filter. It could be presented to the user as two hyperlinks: ("Add another OR value", "Add another AND value"). Andrewok 00:17, 13 September 2009 (UTC)

Excluding categories from the drilldown
Perhaps it's possible to exlude categories from the drilldown by adding "__HIDDENCAT__" to the category page. If I do so, it is excluded from drilldown - --Ulli 757 08:08, 4 October 2009 (UTC)


 * That's a very good idea. I had thought of excluding via a semantic property, or a special category, but never via a magic word; and that seems like the best solution. I'll try to add something like that in, but maybe call it __HIDEFROMDRILLDOWN__ instead. Yaron Koren 18:27, 5 October 2009 (UTC)


 * Okay, I finally implemented __HIDEFROMDRILLDOWN__ in version 0.7, along with its new best friend, __SHOWINDRILLDOWN__ . I think both of these will prove quite helpful. Thanks for the idea! Yaron Koren 20:00, 15 December 2009 (UTC)

Fix 0.6.2
Hey Yaron, cool - the last fix solved my problems with drilldown - in a german wiki I got sometimes an error on line 174 - but I was not able to analyse this; you are great! Many thanks! - kr --Ulli 757 21:31, 2 November 2009 (UTC)


 * I'm glad to hear it! As you probably noticed, I haven't implemented "HIDEFROMDRILLDOWN" yet - I'm not sure any more that that's the way to go; but it's good to see that it's not holding you up. Yaron Koren 22:04, 2 November 2009 (UTC)
 * You are right, I noticed myself that "HIDDENCAT" is not the solution in every case, but sometimes it helps :-) --Ulli 757 20:47, 3 November 2009 (UTC)

Removing list of category
If the example "Special:BrowseData/Cars ? _single" doesn't work, try it with "Special:BrowseData/Cars & _single" - --Ulli 757 16:14, 20 December 2009 (UTC)


 * That's true; it depends on the URL structure, i.e. whether the part before "Special:BrowseData" already contains a question mark. Yaron Koren 21:57, 20 December 2009 (UTC)

Filter on multi-valued property
This is not to be confused with.

I have tried to set up a filter on a multi-valued property (page;page) with the default settings (get values from property). I have also tried hard coding a possible value Has value::Page1;Page2. Neither seemed to work. Has anyone else got this working? 155.198.150.201 16:46, 23 February 2010 (UTC)


 * Indeed, there's no way to do this. If you want to filter on a single value from a multi-value property, I'd recommend making an additional property for just that value - if you're using templates to store data, it shouldn't be that hard to do. Yaron Koren 00:25, 24 February 2010 (UTC)


 * Thanks for the update. Yes I did trivially change my template from an n-ary property to two unary ones, but your suggestion is better, I should keep the n-ary and add two unary ones too (can hide them from display if I like).  Having looked at the source code, I didn't see a simple fix for this.  155.198.150.201 09:44, 24 February 2010 (UTC)

Drilldown counts contain duplicates (with patch)
One of my category trees has some dodgy multiple inheritance going on. For example (fictional), a page 'Highland White Terrier' will belong to categories 'Dog' and 'Terrier'. The category 'Terrier' belongs to category 'Dog' too. In the category and subcategory counts on the drilldown page, the 'Highland White Terrier' page gets counted twice everywhere (category headings, subcategory counts, filter counts). I have naively added some COUNT(DISTINCT ...) in the code to fix this and my patch (which should be ignored or at least treated with caution) is below:

Index: specials/SD_BrowseData.php

=
====================================================== --- specials/SD_BrowseData.php (revision 62828) +++ specials/SD_BrowseData.php (working copy) @@ -250,7 +250,7 @@        */        function getNumResults($subcategory, $subcategories, $new_filter = null) { $dbr = wfGetDB( DB_SLAVE ); -              $sql = "SELECT COUNT(*) "; +              $sql = "SELECT COUNT(DISTINCT sdv.id) "; if ($new_filter) $sql .= $this->getSQLFromClauseForField($new_filter); else @@ -280,7 +280,7 @@ END; foreach ($categories as $i => $category) { $category_children = SDUtils::getCategoryChildren($category, false, 5); -                      $category_str = $category. " (" . count($category_children) . ")"; +                      $category_str = $category. " (" . count(array_unique($category_children)) . ")"; if (str_replace('_', ' ', $this->category) == $category) { $text .= '                                             '; $text .= $category_str; Index: includes/SD_Filter.php

=
====================================================== --- includes/SD_Filter.php     (revision 62828) +++ includes/SD_Filter.php     (working copy) @@ -139,7 +139,7 @@               $smw_ids = $dbr->tableName( 'smw_ids' ); $prop_ns = SMW_NS_PROPERTY; $sql =<<<END -      SELECT $value_field, count(*) +      SELECT $value_field, count(DISTINCT sdv.id) FROM semantic_drilldown_values sdv JOIN $property_table_name $property_table_nickname ON sdv.id = $property_table_nickname.s_id

It seems a shame to add computational burden to everyone just because I have non-standard category hierarchy perhaps. 155.198.150.201 10:05, 24 February 2010 (UTC) (Bob again, I really should register...)

Broken Combobox/Search
I played with Drilldown about a year ago on a proof of concept installation and I like its visual method of getting to the data that matters. However, on coming back to Drilldown to try and implement it in my SM Wiki installation now I am coming up against a problem - namely when a combo box is used the drilldown simply doesn't show any results. If it was just me then I would presume that it is some specific of my setup, but this is also seems to be affecting discoursedb, the reference implimentation - e.g. this drill which should show one result but instead shows nothing. I guess it is the search code more than the use of the combo box itself that is causing the problem- some change in MW or SMW?

If I go back to my old proof of concept wiki (MW 1.14.0, PHP 5.2.3, MySQL 5.0.41, SMW 1.4.2, SD 0.5.8) then it all works correctly. Using SD 0.5.8 on my new installation (MW 1.15.1, SMW 1.4.3) makes no difference, with the lack of results still being there. MLCT 11:17, 15 June 2010 (UTC)


 * Ah, indeed - actually it looks like a MySQL issue: either the version is different or there's a different configuration, somehow. The same thing seems to have happened on Discourse DB, because I believe this was working before (I fixed it now on Discourse DB, but it's not an ideal fix). The issue seems to be that the "LIKE" operator used to be case-insensitive, but is now case-sensitive. Yaron Koren 18:46, 15 June 2010 (UTC)


 * ok, thanks, glad I wasn't going crazy and it is an identifiable issue! Is the workaround easy to implement i.e. a code change I can make to the drilldown code myself to get things working for now (as I see that LIKE is used in SD_AppliedFilter.php), or is it a bit more complex and in need of a properly released update to the extension? MLCT 11:35, 16 June 2010 (UTC)

Removing: Other (*) and None (*)
How would I remove "Other (*number)" and "None (*number)" from the results from drilldown filters, viewed when I "Browse data" using drilldown? Chendy 11:08, 5 August 2010 (UTC)


 * If you define the filter as getting its values from a property, instead of a category, they shouldn't appear... Yaron Koren 17:00, 5 August 2010 (UTC)

Sort order in combobox
When using the combo box input type in a filter covering a property for which all the allowed values have been defined, those values are not sorted as in their definition order, but alphabetically. Is there any possibility to change this behavior? Thanks for your great extension! --Thai 20:51, 4 January 2011 (UTC)


 * You can do it by setting the values manually on the filter page, using "Has value", but that's kind of a hack - I wish there were a better way, but at the moment there isn't. Yaron Koren 23:54, 7 January 2011 (UTC)

Displaying File Names
Would there be a way to display file names when using drilldown? For example, compare http://www.dexid.org/wiki/Special:BrowseData/Manuals to http://www.dexid.org/wiki/Category:Laptop_Manuals I'd love to use semantic drilldown to sort files, but as you can see it is very difficult to tell what file is what. Any input would be greatly appreciated! Thanks for all your work! --BrettFerguson 08:58, 25 May 2011 (UTC)


 * That's a good point. The fix was actually in SMW itself - I just checked in a fix to SMW that adds in the file name, here. Unfortunately, it might be a month or so before the latest version of SMW, 1.6, can be used fully with SD - but until then, I guess you could directly add in this change to your code. Yaron Koren 23:28, 25 May 2011 (UTC)

Using the "Modification Date" Property / "AND" concatenation of categories
I'm trying to use the "Modification Date" Property that comes with SMW to narrow down my search results by months (or a given time period). It appears that this property is not supported by SMW and thus creates problems when being used as filter. Is there a way to include the creation or modifcation date into my filters without the need of a new (duplicate) property?

Furthermore i'd like to concatenate several categories via "AND" selection in my drilldown browser. By using forms I'm i let users select between multiple several predefined categories as "Tags" to further classify my content. Can this be somehow used to narrow down my Search results?


 * I don't believe either of those two is easily doable. For the second one - being able to do "ands" is a definite weakness in SD. Yaron Koren 20:06, 8 June 2011 (UTC)