Extension talk:Semantic Drilldown

Bugzilla
Is bugzilla component created for it yet? Should people submit bugs to SF component while it's being created? --Sergey Chernyshev 22:59, 10 December 2007 (UTC)


 * Oops, I had bad instructions for submitting bugs before; it's fixed now. No, there's no Bugzilla component yet for SD, but most likely there will be soon. Yaron Koren 00:35, 11 December 2007 (UTC)

Bug report and feature request
Just installed it. For some reason the filter shading is not showing up.

Also, all my categories are showing up on the top bar. I know that's how it's supposed to work, but my top-level categories include things like Help and Templates. Maybe there should be a way to set what should be showing up as categories? I guess the best way to do that would be to have an attribute set on the category's page. That does add a level of complexity. Maybe the attribute can be placed on categories that are excluded. --Tosfos 05:40, 12 December 2007 (UTC)


 * For filter shading - that should be specified in a CSS file, called "SD_main.css". My guess is that the URL that's specified for that file isn't readable, for one reason or another. Can you look in that page's source code, then try going to that CSS file in your browser, and seeing if that's the case?


 * Having an 'exclude from drilldown' attribute/property isn't a bad idea. Out of curiosity, though, why do you have a "Templates" category? If you have SF installed, you can just use 'Special:Templates' to see the whole list. Yaron Koren 15:18, 12 December 2007 (UTC)


 * You're right. That file was blank somehow. A reinstall fixed the "bug." (I had tried a reinstall but I foolishly used the same archive as the first install. Sorry.)


 * As for why I have a templates category, the truth is it's a holdover from my pre-SF days. But I still like it. I have many templates that aren't defined by forms, such as utility templates, and they're all neatly divided into sub-categories. --Tosfos 19:29, 12 December 2007 (UTC)

Classifying as "other"
I'm wondering how the extension decides what to list explicitly and what to list as "other." I've figured out that for a relation, if the related page doesn't exist, that relation is classified "other." Why? Also, I had one relation where everything was classified "other" until I put a "Gets values from category" in the filter. --Tosfos 01:10, 13 December 2007 (UTC)


 * Hi, the explanation for how the filter gets its values was lacking. I added the following to the documentation: 'If neither "Gets values from category" nor "Has value" are defined for a filter, the extension will assume that the property for this filter is an enumeration, and get its values from that property's "allowed values".' Does that make sense? For the first case, presumably you had "Gets values from category" set - pages that don't exist yet don't belong to a category. Yaron Koren 17:28, 13 December 2007 (UTC)


 * Sounds right, though I haven't tested it yet. The explanation doesn't list what happens if it doesn't fit the enumeration's allowed values. Thanks. --Tosfos 19:40, 13 December 2007 (UTC)


 * The documentation is admittedly skimpy - I added more of an explanation to the "Description" section. Please let me know if it makes sense now. Yaron Koren 21:34, 13 December 2007 (UTC)


 * I think it's good for a 0.1 version of documentation :). --Tosfos 02:17, 17 December 2007 (UTC)

Filters on filters
I know it's a little early for feature requests, but I'll put it here for future consideration.

I think it would be nice to be able have sub-filters of filters. A sub-filter would be a filter that doesn't show up as an option until its parent filter is selected. The extension already does this with sub-categories: the sub-sub-categories don't show up until the parent is selected.

Here's an example: Say in the Sources category, you wouldn't only want to filter it based on "country" but also based on "state" (or "city" or "province"). You wouldn't want "state" to show up as a filter until "country" was selected because it would show states of every country in the world. Since each state can only be in one country, it's much simpler to have the user first select the country and then the state.

I tried to think of a simple implementation. Perhaps each filter's definition can have an optional attribute called "requires filter" or something like that. In the above example, the Filter:State page would say requires filter:=Filter:Country. Under the current implementation, once the country is selected, only states in that country would return values, so states in other countries wouldn't show up.

I have more to say on the subject, but I would like to hear your opinion first. Thanks. --Tosfos 03:07, 17 December 2007 (UTC)


 * That's a great idea, and I believe someone else requested it early on. Thinking about it now, it probably wouldn't be hard to implement at all - it might even go into the next version. That also reminds me that I should get a Bugzilla "component" added for Semantic Drilldown... what else did you have to say about this proposed feature? Yaron Koren 17:21, 17 December 2007 (UTC)


 * I may be the one you're remembering. I was suggesting something similar: There are instances where it would make more sense to not show all the potential criteria for the higher levels of a drilldown... Otherwise wikis with a lot of criteria could get very cluttered at the higher levels.


 * Note that a side-benefit (if you consider it a benefit) of the proposed implementation is that it also implements the above suggestion. It would be possible to have one filter require a completely unrelated filter to be selected before being available. This raises another possibility. If I had, for example, three total filters, I might want to have the third filter show up after either of the other two are selected, etc. Implementing this might complicate things and I'm not sure it's so necessary. What do you think?


 * Another issue: If this feature is implemented, it wouldn't make sense to only allow filters on filters but not on sub-categories, IMHO. The same way I might want a filter not to show up after another filter was selected, I may not want it to show up until a certain sub-category is selected. Under the current implementation, filters can only be placed on a top-level category. But I may have five sub-categories, each one having its own unique filter. It would make more sense to have each of those filters on its related sub-category, rather than putting all five on the top-level category (which would also make the top-level filtering rather complicated, as above). I realize that under the current implementation it makes sense to place all of them in the top-level category. Since all five do show up as a top-level filter, it would be confusing if they all came from different places. I think that's all (for now). Thanks again. --Tosfos 02:57, 18 December 2007 (UTC)


 * Yeah, it might have been you, now that I think about it. Although maybe it was more than one person. As to your other suggestions: having a "either/or" dependency for filters seems like it would introduce a lot of complexity, both for administrators and for users trying to figure out the rules of the system. I also don't know when it would ever be useful. As for filters for subcategories - that sounds like a more useful feature, and it starts to resemble my original design for the extension, which allowed more flexibility. I'll think about which (or both) of these to add in to the next version. Yaron Koren 04:39, 18 December 2007 (UTC)

Problem after install
I've just installed version 0.3.4 onto my system. I'm running mediawiki 1.8 with Sematic 1.0 and MySql 5.045a.

I'm getting the following error when when I click on Browse data:

Database Error A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

(SQL query hidden)

from within function "". MySQL returned error "1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's' ORDER BY cl.cl_sortkey' at line 3 (localhost)". I do have properties, templates, categories and forms setup and working. Any ideas? Thanks!


 * Oh, that's not good. Try adding the line "$wgShowSQLErrors = true;" anywhere to your LocalSettings.php file. That will un-hide the SQL query, which should give a better idea of what's going wrong. If you still don't know, please put the SQL query here. Yaron Koren 17:46, 1 February 2008 (UTC)

Hi. Thanks for the fast response. The error is:

SELECT p.page_title, p.page_namespace FROM `categorylinks` cl 		JOIN `page` p on cl.cl_from = p.page_id 		WHERE cl.cl_to = 'ADP's' ORDER BY cl.cl_sortkey

from within function "". MySQL returned error "1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's' ORDER BY cl.cl_sortkey' at line 3 (localhost)". The error is obviously with the ' character. I have a category called "ADP", but nothing called "ADP's". Hmmm. Needs a \ in the query perhaps?


 * Hi, okay, you've found a bug in Semantic Drilldown - it can't handle category names that have an apostrophe in them. This will be fixed in the next version; for now, I'm pretty sure you do have a category called "ADP's"; if so, you can just delete it, and take care of the problem on your wiki. Yaron Koren 18:04, 5 February 2008 (UTC)

Aha! I had an old unused category called "ADP's". Of course, as it wasn't used it did not show up in the main Category list. I've deleted it and browse data now works. Great plugin :)

My Patch
Because I had some Problems with the CREATE INDEX SQL I made a small patch, also fixing the bad translation of Namespaces in German. Here it is. Index: includes/SD_Filter.php

=
====================================================== --- includes/SD_Filter.php	(revision 33871) +++ includes/SD_Filter.php	(working copy) @@ -102,7 +102,7 @@ 			FROM $table_name WHERE $property_field = '$query_property'"; 		$dbr->query($sql); -		$sql = "CREATE INDEX sdfv_subject_id_index ON semantic_drilldown_filter_values (subject_id)"; +		$sql = "ALTER TABLE semantic_drilldown_filter_values ADD INDEX sdfv_subject_id_index (subject_id)"; 		$dbr->query($sql); 	} Index: languages/SD_LanguageDe.php

=
====================================================== --- languages/SD_LanguageDe.php	(revision 33871) +++ languages/SD_LanguageDe.php	(working copy) @@ -20,7 +20,7 @@ var $m_Namespaces = array( 	SD_NS_FILTER		=> 'Filter', -	SD_NS_FILTER_TALK	=> 'Filter_talk' +	SD_NS_FILTER_TALK	=> 'Filter Diskussion' ); } Index: specials/SD_BrowseData.php

=
====================================================== --- specials/SD_BrowseData.php	(revision 33871) +++ specials/SD_BrowseData.php	(working copy) @@ -93,7 +93,7 @@ 		$dbr->query($sql); // create an index to speed up subsequent queries // (does this help?) -		$sql2 = "CREATE INDEX page_id_index ON semantic_drilldown_values (page_id)"; +		$sql2 = "ALTER TABLE semantic_drilldown_values ADD INDEX page_id_index (page_id)"; $dbr->query($sql2); }

Maybe you can test this Version, I think it does the same but maybe it denies problems with the CREATE INDEX Statement. DaSch 22:37, 24 April 2008 (UTC)


 * Hi, thanks for the patch, I'll add it in to the next version. Out of curiosity, what was the problem with the "CREATE INDEX" call? Yaron Koren 14:39, 25 April 2008 (UTC)


 * I think that many hosting provider does not allow this action so that I got an access denied Error on the BrowseData Special Page. DaSch 18:39, 25 April 2008 (UTC)

Group of Categories
In my Wiki there are many Categories. I think a possibility to group Categories could be useful. Maybe the first Category is used as group. So with only a few Categories you could put them all in one "Main-Category", when there are more then they could be grouped.

Another thing that I thin could be useful is that Filters in subcategories that are not in the maincategory are added. So depending on the subcategory you could add more filters. --DaSch 07:54, 13 June 2008 (UTC)


 * Maybe also useful could be options for grouping the property values. Or for only displaying values that have more then X entries. --DaSch 10:31, 13 June 2008 (UTC)

Using AND instead of OR for multiple values in a filter
Hi there Yaron and anyone else on this page. Many of my properties have multiple values. When I select more than one of these in the Drilldown, it shows me "Filter (x): Value1 (x) or Value2 (x)" and gives me all the items that have either Value1 or Value2 for that property. This is how it is supposed to work, I think. However, is there any way to by default show all items with both of selected values for that property? Thanks! Staeiou 13:36, 30 July 2008 (UTC)


 * No, there's no way to do that; I can't think of a way to allow both OR and AND on values while keeping the interface usable. I'm definitely interested in improving the Semantic Drilldown functionality, though, so if you can think of a way please email me about it. Yaron Koren 15:37, 30 July 2008 (UTC)

SQL Error in Browse Data after initializing
Hi

I just installed Semantic Drilldown 0.4.5 on SMW 1.2 using the new data store.

After initializing it for the first time, Browse Data returns this error :

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

SELECT COUNT(*) FROM semantic_drilldown_values sdv JOIN `kb_smw_inst2` inst ON sdv.id = inst.s_id WHERE inst.o_id IN (SELECT smw_id FROM smw_ids WHERE smw_namespace = 14 AND (smw_title = 'Document' OR smw_title = 'Definition' OR smw_title = 'Link' OR smw_title = 'Note' OR smw_title = 'Task' OR smw_title = 'Log_entry' ))

from within function "". MySQL returned error "1146: Table 'wikidb.smw_ids' doesn't exist

It looks like SD is trying to query the 'smw_ids' table instead of 'kb_smw_ids' table.

I am using the 'kb_' prefix for Media Wiki tables in my database.

Is this a bug or a sign that I did not initialize it correctly ?


 * Yes, that's a bug; support for the new SQL store is still a little bit in progress. I took this bug as an opportunity to release a new version, so if you get the new version 0.4.6, it should be fixed. Please let me know if you encounter more problems. Yaron Koren 14:15, 1 August 2008 (UTC)


 * 0.4.6 did the trick ! Thanks. Browse Data is now working as advertise... I will start working on filters and see what I can do with it.

Issue with undefined values
I have a property 'Is archived' with Type:Boolean. Some of my pages have that property set to True or False.... many pages do not have anything set for that property.

When I created an 'Archived' filter, the Drilldown page gave me 3 options for the filter : Yes, No and None.

Clicking on Yes or No works fine. Clicking on None results in a SQL error such as :

A database query syntax error has occurred. This may indicate a bug in the software. The last attempted database query was:

SELECT DISTINCT ids.smw_title AS title, ids.smw_title AS value, ids.smw_namespace AS namespace, ids.smw_sortkey AS sortkey FROM `kb_smw_ids` ids JOIN `kb_smw_inst2` insts ON ids.smw_id = insts.s_id AND ids.smw_namespace != 14 LEFT OUTER JOIN (SELECT subject_id, value_xsd FROM `kb_smw_atts2` WHERE p_id = 'Is_archived') na0 ON id.smw_id = na0.s_id JOIN `kb_smw_atts2` a0 ON ids.smw_id = a0.s_id WHERE insts.o_id IN (SELECT smw_id FROM `kb_smw_ids` cat_ids WHERE smw_namespace = 14 AND (smw_title = 'Location' OR smw_title = 'Building' OR smw_title = 'City' OR smw_title = 'Country' OR smw_title = 'Region' OR smw_title = 'Room' OR smw_title = 'Site' OR smw_title = 'State')) AND a0.p_id = (SELECT smw_id FROM `kb_smw_ids` WHERE smw_title = 'Is_archived' AND smw_namespace = 102) AND ((a0.value_xsd = '' OR a0.value_xsd IS NULL) ) ORDER BY sortkey LIMIT 250

from within function "". MySQL returned error "1054: Unknown column 'subject_id' in 'field list' (rndusddblade67.ompus.na.jnj.com)".

Could this be a bug ?

I am using SD 0.4.6, MW 1.13 and SMW 1.2.1 (with the new data store).

Thanks

- Laurent Alquier


 * Oh... that's really a bug. Once again I'm amazed that it hadn't been discovered yet. Anyway, I believe I fixed it, and snuck the fix in to SD 0.4.7, so if you upgrade to that one it should work. Thanks for the bug report. Yaron Koren 23:39, 28 August 2008 (UTC)


 * And I am amazed once again by the quick response to the bug report. 0.4.7 removes the error but does not return any result. The filter displays 'None (307)' but the page displays no results (instead of 307 pages) when I click on the link. I would surprised to see anything in that page actually, since it is not possible to query SMW for the absence of property value. Maybe there should be a warning on that empty page instead ? - Laurent Alquier


 * Hi, that's too bad. This wouldn't be a public wiki by any chance, would it? Yaron Koren 16:27, 29 August 2008 (UTC)


 * Unfortunately no. It's an intranet knowledge base I am trying to set up. SWM and your extensions are providing solutions to most of the issues I encountered with our old, more static knowledge base.


 * Hi Yaron. I'm having the same issue with my public wiki at http://vworld.fas.org.  I had just commented out the code in SD_BrowseData.php that displays "None," but it got overwritten when I upgraded to the latest version.  Perhaps you could take a look at it there and discover something that would help Laurent as well?  Thanks for all your hard work! Staeiou 18:03, 29 August 2008 (UTC)


 * Staeiou, you appear to be having the original bug - I think you got your version 0.4.7 before it was fixed. If you re-get the current version, that problem should go away. Yaron Koren 03:56, 31 August 2008 (UTC)


 * I just updated, and I still get the error:   SELECT DISTINCT ids.smw_title AS title, ids.smw_title AS value, ids.smw_namespace AS namespace, ids.smw_sortkey AS sortkey FROM `vw_smw_ids` ids JOIN `vw_smw_inst2` insts ON ids.smw_id = insts.s_id AND ids.smw_namespace != 14 LEFT OUTER JOIN (SELECT s_id, o_ids0.smw_title FROM `vw_smw_rels2` WHERE p_id = 'Use') nr0 ON ids.smw_id = nr0.s_id JOIN `vw_smw_rels2` r0 ON ids.smw_id = r0.s_id JOIN `vw_smw_ids` o_ids0 ON r0.o_id = o_ids0.smw_id WHERE insts.o_id IN (SELECT smw_id FROM `vw_smw_ids` cat_ids WHERE smw_namespace = 14 AND (smw_title = 'Virtual_Worlds' OR smw_title = 'MMORPGs')) AND r0.p_id = (SELECT smw_id FROM `vw_smw_ids` WHERE smw_title = 'Use' AND smw_namespace = 102) AND ((o_ids0.smw_title = '' OR o_ids0.smw_title IS NULL) ) ORDER BY sortkey LIMIT 250 from within function "".
 * MySQL returned error "1054: Unknown column 'o_ids0.smw_title' in 'field list' (localhost)"  Hmm. Staeiou 19:12, 1 September 2008 (UTC)

Hi, this is a strange one. Where's the URL for which you see this error? Yaron Koren 15:51, 2 September 2008 (UTC)
 * and  generate the error, while  successfully executes but returns no results when it should return many. Staeiou 19:27, 2 September 2008 (UTC)


 * Wow, untangling that bug proved rather tricky - I guess I just never tested the 'none' value for relations with SMWSQLStore2. Anyway, I believe it's fixed now, as is the bug below, in the new version, 0.4.8. Let me know if it works for you. Yaron Koren 20:53, 3 September 2008 (UTC)

Ambiguous columns
I figure I'd better bug report this too, while I'm here. When I have a filter that covers a property with allowed values (like this one), I find that I have to duplicate those in the Has values for the filter. If I don't, I get this error message:

SELECT o_id FROM `vw_smw_rels2` r JOIN `vw_smw_ids` p_ids ON r.p_id = p_ids.smw_id JOIN `vw_smw_inst2` insts ON r.s_id = insts.s_id JOIN `vw_smw_ids` cat_ids ON insts.o_id = cat_ids.smw_id WHERE p_ids.smw_title = 'Language' AND cat_ids.smw_namespace = 14 AND cat_ids.smw_title = 'Virtual_Worlds' GROUP BY o_id ORDER BY o_id

from within function "". MySQL returned error "1052: Column 'o_id' in field list is ambiguous (localhost)".

My Use filter enumerates them, but my language filter does not. This means that when you go to my browse data page and click on one of the languages, you get the above error. It is not that much work to add them (I already did but took them down to demo the bug), but would this be a quick fix on your end? Thanks!

Staeiou 18:13, 29 August 2008 (UTC)

Filters going unused
Hi, all! I've recently set up a gaming wiki (MediaWiki 1.12.0) with Semantic MediaWiki (1.2) and Semantic Drilldown (0.4.8), and I've barely begun tagging articles with semantic properties. I've set up a few filters associated with the properties I've used on the articles, and I've added them to a new top-level category. But when I go to Special:Browse data for that category, the filters don't show up. The two tagged articles appear just because they're in the category, but I've no way to filter them. Any clue what I'm doing wrong?

Here's the browse data page:.

Thanks,

Stormraven 17:29, 4 September 2008 (UTC)


 * Hm, that's odd. Which SQL store are you using, the original or 2? Yaron Koren 21:59, 4 September 2008 (UTC)


 * I've no idea, off hand. The version page says I'm using MySQL version 4.1.22-standard-log, but I'll confess that I know little about how it works.  I'll see whether I can find that information once I get home tonight and can get into some of the nuts and bolts underlying my wiki.  What's the difference between the original and 2?  Stormraven 22:08, 4 September 2008 (UTC)


 * Please look for a line that starts with '$smwgDefaultStore =' in your LocalSettings.php file. SD uses two different sets of code, depending on which one it is. Yaron Koren 22:22, 4 September 2008 (UTC)


 * That text does not appear in my LocalSettings.php file. In fact, no variable in my LocalSettings.php file starts with $smwg; they're all $wg variables.  Is this the right file to look in?  Stormraven 01:09, 5 September 2008 (UTC)


 * Okay, that's fine; it means you're using the original SMWSQLStore. Let me think about this. Yaron Koren 02:00, 5 September 2008 (UTC)


 * All right, thanks. I appreciate you helping out with this.  I presume it's just something I've done wrong, not a problem with the software.  Stormraven 02:02, 5 September 2008 (UTC)


 * I keep going over and over the problem, and I just can't find an error in what I've done. It's a very short path.  Drilldown appears to have been installed correctly.  The special pages are all working.  The category has filters, the filters cover properties, and the articles have both the category and the properties.  It should work.  It works for other users just fine.


 * The easiest explanation for why it shows no filters is that the software thinks there are no results under those filters. But I can't figure out why it would come to that erroneous conclusion.


 * &mdash; Stormraven 12:28, 8 September 2008 (UTC)

Yeah, it's probably a bug in Semantic Drilldown. Yaron Koren 13:22, 8 September 2008 (UTC)


 * I wish I could help you debug. I'm a Visual Basic programmer, but I don't know php at all. :(  &mdash;Stormraven 11:15, 9 September 2008 (UTC)


 * Okay, this is strange. I've been adding more and more semantic tags, and I've created some descriptive pages in the Property namespace for the tags I've used.  And suddenly some (but not all) of my filters are showing up in the Browse Data page.  I don't understand it.  But I'm glad to see at least a few of my filters. :) Stormraven 11:47, 11 September 2008 (UTC)


 * I found a problem on my end, and now all my filters are being used. I still don't really understand why they weren't at the beginning, but the problems seem to have gone away now.  &mdash;Stormraven 17:48, 12 September 2008 (UTC)


 * Well, that's great news; I guess it wasn't an SD bug after all. Please let me know if you figure out what was causing it. Yaron Koren 19:13, 12 September 2008 (UTC)


 * There was definitely some sort of bug early on, possibly in how the extension counts articles. I'll let you know if I have any further problems, with as much detail as I can.  Thanks!  &mdash;Stormraven 01:42, 14 September 2008 (UTC)

New development
Okay, this is strange. I've lost one of the filters that had been working for me. See this Browse Data page, where the browser for category Settings should be using (and until moments ago had been using) the filter World. I can't conceive of a reason why it should have stopped working. All I've been doing is adding semantic tags to articles, including the one that this filter works on, is geography of. &mdash;Stormraven 11:48, 15 September 2008 (UTC)


 * Oh, I did do one other thing that might have a bearing here. I used the ReplaceText extension to fix the capitalization of one of the targets of the semantic tag "is geography of," from "Real earth" to "Real Earth."  That might have some significant bearing, especially since the property page for "is geography of" still shows the old capitalization, while the articles themselves show the new version.  &mdash;Stormraven 12:12, 15 September 2008 (UTC)

New experiment on unused filters
Okay, I've created a new property and applied it to one article. I created a filter for it, then added it to a top-level category. Immediately I checked for the filter on the Browse Data page. It wasn't there. Not surprising, this is just the problem I had before.

I then created a new page in the Property namespace for the new property. I checked the Browse Data page again, but the filter still wasn't there. The new property's page says that no articles are using the property as yet, but I know there's exactly one.

I added the property to a second article, and again checked Browse Data. This time the second article showed up, but not the first one. The property's page now shows one article using the new property, the second one. The first still isn't shown.

So this is looking like a question of how Semantic MediaWiki counts properties in use, and possibly not a problem with Semantic Drilldown at all. When and how does SMW find pages using properties, and can I force it manually to rebuild its data? That might solve my problems.

Thanks!

Stormraven 17:42, 15 September 2008 (UTC)


 * That's very odd - I've never heard of that happening. Yeah, you can do a manual rebuild using the SMW_refreshData.php script, although you shouldn't need to in this case. Yaron Koren 16:14, 16 September 2008 (UTC)

Category sort order
I've got a character on my gaming wiki whose article is named "Shining Talon" Mandus, with the quotation marks (here). The article name is sorted as Shining Talon Mandus, without quotation marks, by using the pipe trick to create an alternate sort order (here). But on the Drilldown page Special:Browse data, the alternate sort order is not used (here). I'd like to see this addressed. Thanks! Stormraven 00:53, 6 September 2008 (UTC)