Extension talk:Semantic Drilldown

Filters show up in BrowseData, but empty
Odd problem: my filters do show up in Special:BrowseData, but the properties they are supposed to cover don't. Some of them even have a "Has label" tag, but that doesn't apply. It's weird because they were working until a few hours ago. I made a lot of tests, I even renamed the Category, moved all tags from the old category to the new one, changed the category tag in every relevant page (from the old name to the new one), applied nulledits to all property and filter pages, but no way: in property pages all values are fine, in filter pages the names of the covered properties are fine, in the category page the names of the filters are fine, but still: in Special:Browse Data I can see all filters, but no values. Note that I can see below the whole list of pages in the category. Again, everything was fine until a few hours ago. The only change I have made that may have interfered with Drill Down is deleting the category "General Templates" from my Templates using Replace Text extension. I read above that Stormraven used the same extension... Help?

Davide 05:05, 11 May 2009 (UTC)

Installation ... Upgrade?
Is there any issue when upgrading this extension? --Dmb 15:08, 22 January 2009 (UTC)


 * Support for old versions of Semantic MediaWiki gets removed from time to time; you can check the version history. Yaron Koren 15:47, 22 January 2009 (UTC)

Undefined variable: wgAmericanDates in SD_BrowseData.php on line 630
I get the following PHP Notice when I try to BrowseData on a category that "Has filter" with an input type of "date range": Notice: Undefined variable: wgAmericanDates in /path/to/wiki/extensions/SemanticDrilldown/specials/SD_BrowseData.php on line 630

I have tried explicitly setting "$wgAmericanDates = true" in LocalSettings.php, and the error remains.

Version Details:
 * MediaWiki 1.14.0
 * PHP 5.2.6
 * MySQL 5.0.45
 * Apache 2.2.3
 * OS: CentOS 5.2 (Linux Kernel 2.6.18-92.1.6.el5)

Semantic Extensions: --74.164.199.186 23:23, 23 March 2009 (UTC)
 * Semantic MediaWiki 1.4.2
 * Semantic Forms 1.5.4
 * Semantic Drilldown 0.5.4


 * Looks like you've found a bug. It'll be fixed in the next version of Semantic Drilldown, which will hopefully be released soon. Yaron Koren 23:39, 23 March 2009 (UTC)

Filters not displaying - Table 'wikidb.smw_attributes' does not exist
Hi Yaron and all:

Having an issue with getting my filters to show up on the BrowseData page. I have articles that have the properties I am trying to filter on, but when I click on the category in my BrowseData page, I get the following error:

Database error

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

 CREATE TEMPORARY TABLE semantic_drilldown_filter_values ( id INT NOT NULL, value VARCHAR(200) NOT NULL, INDEX sdfv_id_index(id) ) AS SELECT subject_id AS id, value_xsd AS value FROM `smw_attributes` WHERE attribute_title = 'Affects_Geographic_Region'

from within function "". MySQL returned error "1146: Table 'wikidb.smw_attributes' doesn't exist (localhost)".

Any idea as to where I veered off course? My install is MW is:

MediaWiki 	1.13.4 PHP 	5.2.8 (apache2handler) MySQL 	5.1.30-community-log

Thanks,

Dave Manzolillo DManzolillo-at-LMI.org

'''Update: Once I disabled Halo it worked. Thanks!'''

Has anyone been able to get SMW forms and drilldown to work with the Halo extension? The more I get into trouble, I find that disabling Halo/SMW+ extensions fixes my issue....


 * An ideal solution to this problem is right now being discussed on the SMW mailing list. Until there is one, there's an easy workaround in the SD code, that I plan to add in the next version. Yaron Koren 03:51, 1 April 2009 (UTC)

Filter Stopped Working
Hi. One of my filters has just stopped displaying in my browsedata page.

The property is still there and named the same. There is data in it.

Is there a limit on the number of objects in a filter?

The Filter: This filter covers the property Covers property::Manufacturer.

The Property: This is a property of type Has type::Page. It links to pages that use the form Has default form::Straight Razor Manufacturer.

Thanks.


 * No, there's no limit. Did anything change between when it worked and when it didn't? An extension upgrade, perhaps? Yaron Koren 03:52, 1 April 2009 (UTC)


 * I created a subcategory called "Manufacturers". Cannot think of any other changes.

Articles that belong to a Namespace are not listed but still counted in the Category
Hi all,

I am using Semantic Drilldown Version 0.5.6.

I noticed that when an article belongs to a namespace (e.g. KB in my case), the article will be counted in the category. However the article will not be returned in the list. I get the following message when all the articles of a category belong to a custom namespace:

There are no results for this report.

If I modify an article that does not belong to any Namespace and add it to the category, the article is properly listed.

Is this a normal behaviour? I am not good at PHP so it will take me few hours or more to understand how the extension is selecting which article is displayed, but if needed, I will do it :) . Is there an option I could configure to allow articles from specific namespaces to be returned?

Anyone faced the same problem? I searched but could not find anything about it, I apologize if it has already been answered.

Thank you. Yann.


 * Hi, I don't understand - if you create an article in the "KB" namespace, it gets added to a category automatically? What do you mean by a category here? Yaron Koren 12:46, 27 April 2009 (UTC)

No it is not added automatically to the namespace, sorry for the confusion. I use a form + template to make this happen. I mean by category that the following text is present in a page/article: [ [ Category : KB ] ] I added spaces on purpose here to avoid it to be parsed.

So basically if I have two articles with the category set with the following names:

KB:Article1 and Article2

only the second article (Article2) will show up in the list whereas both are counted on the top right box named "Choose a category" where you can switch between categories.

EDIT: I ended up by not putting the articles of the KB category in a custom Namespace


 * Okay, it looks like you've found a bug in Semantic Drilldown - namespaces that don't have SMW enabled for them don't show up in 'BrowseData'. Yaron Koren 15:27, 29 April 2009 (UTC)

Special:Browsedata showing incorrect category counts
Using this extension (0.5.6) on MW 1.15.0 with SMW 1.4.2, SRF 1.4.4. I have not started to try using any filters yet, but in "Special:BrowseData", all of the article counts on categories seem to range from 60% - 100% above the actual number of articles in the category. It seems that articles in multiple categories are being counted as one an instance of each the category to which it belongs (with ~600 articles, I'm not going to check each one to confirm, but eyeballing it, it seems right).

For example (not an actual example, but it's simpler to explain here), it will read:

"Fruit (300)" when there are only 200 articles in that category, "Edible yellow things (2)" when there is only 1 article in that category (Banana), but that article also belongs to the category "Fruit".

I'm not sure what the issue is (I know very little about PHP and less about MySQL), but perhaps this has something to do with the way the categorylinks table is handled in lines 150 or 203 of SD_BrowseData.php?

Drilldown using multiple values for in a single filter
Is it possible to drill down by selecting a multiple values for a single filter? For example, the following page has a filter based on an extensive list of keywords: http://www.structuralwiki.org/en/Special:BrowseData/Software?_single. I would like to be able to drill down by selecting two keywords in the first filter. However, as soon as I select one keyword, all the the other keywords in the list disappear and "(Add another value)" message appears. How can I add this another value? Thanks. Andrewok 03:44, 10 September 2009 (UTC)


 * You need to click the little arrow next to it, and that will display all the values again. It's a rather cryptic interface, I admit, and if you can think of something better (maybe just better text, maybe more than that), I'd like to hear it. Yaron Koren 03:59, 10 September 2009 (UTC)


 * I was thinking that perhaps the text "(Add another value)" could be a hyperlink with the same functionality as the little down arrow. I also noticed that selections from different filters are treated as AND condition, while selections of multiple values from a single filter is treated using OR condition. Sometimes desirable to select multiple values from the same filter using AND condition. I was able to achieve this functionality by using several filters for the same property (such as "keyword 1", "keyword 2", "keyword 3"), with each subsequent filter requiring selection to be made on the previous filter. Perhaps it would be worthwhile (if possible) to directly implement choice of OR or AND condition for multiple values selected from the same filter. It could be presented to the user as two hyperlinks: ("Add another OR value", "Add another AND value"). Andrewok 00:17, 13 September 2009 (UTC)

Excluding categories from the drilldown
Perhaps it's possible to exlude categories from the drilldown by adding "__HIDDENCAT__" to the category page. If I do so, it is excluded from drilldown - --Ulli 757 08:08, 4 October 2009 (UTC)


 * That's a very good idea. I had thought of excluding via a semantic property, or a special category, but never via a magic word; and that seems like the best solution. I'll try to add something like that in, but maybe call it __HIDEFROMDRILLDOWN__ instead. Yaron Koren 18:27, 5 October 2009 (UTC)


 * Okay, I finally implemented __HIDEFROMDRILLDOWN__ in version 0.7, along with its new best friend, __SHOWINDRILLDOWN__ . I think both of these will prove quite helpful. Thanks for the idea! Yaron Koren 20:00, 15 December 2009 (UTC)

Fix 0.6.2
Hey Yaron, cool - the last fix solved my problems with drilldown - in a german wiki I got sometimes an error on line 174 - but I was not able to analyse this; you are great! Many thanks! - kr --Ulli 757 21:31, 2 November 2009 (UTC)


 * I'm glad to hear it! As you probably noticed, I haven't implemented "HIDEFROMDRILLDOWN" yet - I'm not sure any more that that's the way to go; but it's good to see that it's not holding you up. Yaron Koren 22:04, 2 November 2009 (UTC)
 * You are right, I noticed myself that "HIDDENCAT" is not the solution in every case, but sometimes it helps :-) --Ulli 757 20:47, 3 November 2009 (UTC)

Removing list of category
If the example "Special:BrowseData/Cars ? _single" doesn't work, try it with "Special:BrowseData/Cars & _single" - --Ulli 757 16:14, 20 December 2009 (UTC)


 * That's true; it depends on the URL structure, i.e. whether the part before "Special:BrowseData" already contains a question mark. Yaron Koren 21:57, 20 December 2009 (UTC)

Filter on multi-valued property
This is not to be confused with.

I have tried to set up a filter on a multi-valued property (page;page) with the default settings (get values from property). I have also tried hard coding a possible value Has value::Page1;Page2. Neither seemed to work. Has anyone else got this working? 155.198.150.201 16:46, 23 February 2010 (UTC)


 * Indeed, there's no way to do this. If you want to filter on a single value from a multi-value property, I'd recommend making an additional property for just that value - if you're using templates to store data, it shouldn't be that hard to do. Yaron Koren 00:25, 24 February 2010 (UTC)


 * Thanks for the update. Yes I did trivially change my template from an n-ary property to two unary ones, but your suggestion is better, I should keep the n-ary and add two unary ones too (can hide them from display if I like).  Having looked at the source code, I didn't see a simple fix for this.  155.198.150.201 09:44, 24 February 2010 (UTC)

Drilldown counts contain duplicates (with patch)
One of my category trees has some dodgy multiple inheritance going on. For example (fictional), a page 'Highland White Terrier' will belong to categories 'Dog' and 'Terrier'. The category 'Terrier' belongs to category 'Dog' too. In the category and subcategory counts on the drilldown page, the 'Highland White Terrier' page gets counted twice everywhere (category headings, subcategory counts, filter counts). I have naively added some COUNT(DISTINCT ...) in the code to fix this and my patch (which should be ignored or at least treated with caution) is below:

Index: specials/SD_BrowseData.php

=
====================================================== --- specials/SD_BrowseData.php (revision 62828) +++ specials/SD_BrowseData.php (working copy) @@ -250,7 +250,7 @@        */        function getNumResults($subcategory, $subcategories, $new_filter = null) { $dbr = wfGetDB( DB_SLAVE ); -              $sql = "SELECT COUNT(*) "; +              $sql = "SELECT COUNT(DISTINCT sdv.id) "; if ($new_filter) $sql .= $this->getSQLFromClauseForField($new_filter); else @@ -280,7 +280,7 @@ END; foreach ($categories as $i => $category) { $category_children = SDUtils::getCategoryChildren($category, false, 5); -                      $category_str = $category. " (" . count($category_children) . ")"; +                      $category_str = $category. " (" . count(array_unique($category_children)) . ")"; if (str_replace('_', ' ', $this->category) == $category) { $text .= '                                             '; $text .= $category_str; Index: includes/SD_Filter.php

=
====================================================== --- includes/SD_Filter.php     (revision 62828) +++ includes/SD_Filter.php     (working copy) @@ -139,7 +139,7 @@               $smw_ids = $dbr->tableName( 'smw_ids' ); $prop_ns = SMW_NS_PROPERTY; $sql =<<<END -      SELECT $value_field, count(*) +      SELECT $value_field, count(DISTINCT sdv.id) FROM semantic_drilldown_values sdv JOIN $property_table_name $property_table_nickname ON sdv.id = $property_table_nickname.s_id

It seems a shame to add computational burden to everyone just because I have non-standard category hierarchy perhaps. 155.198.150.201 10:05, 24 February 2010 (UTC) (Bob again, I really should register...)

Broken Combobox/Search
I played with Drilldown about a year ago on a proof of concept installation and I like its visual method of getting to the data that matters. However, on coming back to Drilldown to try and implement it in my SM Wiki installation now I am coming up against a problem - namely when a combo box is used the drilldown simply doesn't show any results. If it was just me then I would presume that it is some specific of my setup, but this is also seems to be affecting discoursedb, the reference implimentation - e.g. this drill which should show one result but instead shows nothing. I guess it is the search code more than the use of the combo box itself that is causing the problem- some change in MW or SMW?

If I go back to my old proof of concept wiki (MW 1.14.0, PHP 5.2.3, MySQL 5.0.41, SMW 1.4.2, SD 0.5.8) then it all works correctly. Using SD 0.5.8 on my new installation (MW 1.15.1, SMW 1.4.3) makes no difference, with the lack of results still being there. MLCT 11:17, 15 June 2010 (UTC)


 * Ah, indeed - actually it looks like a MySQL issue: either the version is different or there's a different configuration, somehow. The same thing seems to have happened on Discourse DB, because I believe this was working before (I fixed it now on Discourse DB, but it's not an ideal fix). The issue seems to be that the "LIKE" operator used to be case-insensitive, but is now case-sensitive. Yaron Koren 18:46, 15 June 2010 (UTC)


 * ok, thanks, glad I wasn't going crazy and it is an identifiable issue! Is the workaround easy to implement i.e. a code change I can make to the drilldown code myself to get things working for now (as I see that LIKE is used in SD_AppliedFilter.php), or is it a bit more complex and in need of a properly released update to the extension? MLCT 11:35, 16 June 2010 (UTC)

Removing: Other (*) and None (*)
How would I remove "Other (*number)" and "None (*number)" from the results from drilldown filters, viewed when I "Browse data" using drilldown? Chendy 11:08, 5 August 2010 (UTC)


 * If you define the filter as getting its values from a property, instead of a category, they shouldn't appear... Yaron Koren 17:00, 5 August 2010 (UTC)

Sort order in combobox
When using the combo box input type in a filter covering a property for which all the allowed values have been defined, those values are not sorted as in their definition order, but alphabetically. Is there any possibility to change this behavior? Thanks for your great extension! --Thai 20:51, 4 January 2011 (UTC)


 * You can do it by setting the values manually on the filter page, using "Has value", but that's kind of a hack - I wish there were a better way, but at the moment there isn't. Yaron Koren 23:54, 7 January 2011 (UTC)

Displaying File Names
Would there be a way to display file names when using drilldown? For example, compare http://www.dexid.org/wiki/Special:BrowseData/Manuals to http://www.dexid.org/wiki/Category:Laptop_Manuals I'd love to use semantic drilldown to sort files, but as you can see it is very difficult to tell what file is what. Any input would be greatly appreciated! Thanks for all your work! --BrettFerguson 08:58, 25 May 2011 (UTC)


 * That's a good point. The fix was actually in SMW itself - I just checked in a fix to SMW that adds in the file name, here. Unfortunately, it might be a month or so before the latest version of SMW, 1.6, can be used fully with SD - but until then, I guess you could directly add in this change to your code. Yaron Koren 23:28, 25 May 2011 (UTC)

Using the "Modification Date" Property / "AND" concatenation of categories
I'm trying to use the "Modification Date" Property that comes with SMW to narrow down my search results by months (or a given time period). It appears that this property is not supported by SMW and thus creates problems when being used as filter. Is there a way to include the creation or modifcation date into my filters without the need of a new (duplicate) property?

Furthermore i'd like to concatenate several categories via "AND" selection in my drilldown browser. By using forms I'm i let users select between multiple several predefined categories as "Tags" to further classify my content. Can this be somehow used to narrow down my Search results?


 * I don't believe either of those two is easily doable. For the second one - being able to do "ands" is a definite weakness in SD. Yaron Koren 20:06, 8 June 2011 (UTC)

Using a hybrid of SMW and ordinary MW-- preventing non-SMW categories from showing up in Drilldown?
Hello -

I have three different categories that I would like to display as categories in semantic drill down: 1) Top level 2) Open Tasks (a category that includes all pages that need work) 3) Articles (a category with a list of articles that all fit a semantic form.

Each of these have either been included via __SHOWINDRILLDOWN__ or by virtue of being the "top level" category.

However, whenever a new category is created by someone who doesn't immediately place a category tag into it to indicate that it is not a top-level, the category ends up showing up on the category tree.

Do you have any suggestions to keep just those three categories showing up in the drill? Is there perhaps a way to prepopulate all new categories with the "no show in drilldown" option.


 * I can't think of any way to do it, but - is that a big issue? If you have a truly SMW-based setup, I would think users wouldn't be creating new categories that often. Yaron Koren 22:38, 12 September 2011 (UTC)


 * Thanks so much for the reply Yaron. Well, the problem is that this is my first wiki and I'm just discovering how everything in SMW works. I suppose that by "truly SMW-based", you mean that users have to input into forms to create new categories? Is that correct?


 * I am sort of hoping to have people doing a mix of both. The wiki is also very math-based and so I'd like to enable people to also be comfortable "Editing Source". My concern is that the wiki is just growing and so we don't have an exact taxonomy of every kind of webpage which would appear and there will probably be a lot of ad-hoc category creation. Thanks again for the reply, and please let me know if you think of anything else or if perhaps I am missing something in the SMW-setup that would make my life easier. Dc321 23:58, 12 September 2011 (UTC)


 * Hi - it actually has nothing to do with forms (though I definitely recommend using forms). But if you're using SMW, you probably don't need more than one category per page - any other piece of information can be stored via semantic properties. Yaron Koren 02:17, 13 September 2011 (UTC)


 * I am sort of hoping to have the wiki be a hyrbrid of both. For one part of the site there is a well-defined taxonomy and I'd like to take advantage of SMW, but I'd like to keep the other part free-form.


 * One thing I tried was to use external links to remove categories. However, I get the error that "There are no results for this report.". Am I missing a configuration step? I think that if I could just make the other category options go away and provide hard links to just the one category I want to be in SMW, that may be a solution.


 * Do you think that's a sensible way to use SMW for just a portion of the site? Dc321 02:25, 13 September 2011 (UTC)


 * Sure, that sounds sensible. I don't know what the specific issue you're having is. Yaron Koren 02:31, 13 September 2011 (UTC)


 * Alas -- this is covered by substituting "?" with "&", i'm going to add a new category... Thanks again Yaron, this is really revolutionary software you've created. Dc321 02:45, 13 September 2011 (UTC)

Why isn't the ?_single hack working to show only one category?
Asked and answered: As pointed out above in Removing list of category, you may need to use a "&" sign instead.

This question is not a new question, but I think it is more findable this wayDc321 02:50, 13 September 2011 (UTC)

Counts take a long time to update... old values that no longer exist stay around
I decided to change my template so that it defines a property with value "untagged" whenever the property wasn't entered in. I was expecting to see that instantly each article within the category of my filter that uses this property would then show up "untagged". However, only a few appeared. Strangely, it seems that more values appear over time.

I then decided to change the name of untagged to "000-untagged" so that it would show up first on the drill-down. Now, I have article counts for the old value -- "untagged" and many counts for the new one "000-untagged" still haven't appeared.

It seems strangely that each time I refresh the drilldown page, the count for missing articles increases. I definitely don't want to hit refresh a hundred times to update all of my articles.

Any help or advice would be greatly appreciated Dc321 01:00, 16 September 2011 (UTC)


 * When you resave a template, all pages that include that template get refreshed via MediaWiki jobs. You can read more here on how jobs are handled in MediaWiki. Yaron Koren 05:40, 16 September 2011 (UTC)


 * Sigh, I can't stump you can I? Thanks, yet again for all of the help! Dc321 21:11, 18 September 2011 (UTC)