Extension talk:Semantic Drilldown

SDD don't show results in MW 1.32
I run SMW 3.0.2 and SDD 2.1 (56e40a2) 10:12, April 22, 2019 Just made un upgrade from 1.27 to 1.32 My drilldown configuration in my Category Tune is:
 * MediaWiki	1.32.0
 * PHP	7.2.17-0ubuntu0.18.04.1 (apache2handler)
 * MariaDB	10.1.38-MariaDB-0ubuntu0.18.04.1

The Special:BrowseData/Tune show all the filtered Properties with the correct number of pages per filter but when I follow one of the filter links the Special:BrowseData/Tune special page says: and for the filters: Am I miss something in config?
 * There are no results for this report
 * (There are no values for this filter)

NOTE: I run all (I think) the requested script, i.e. maintenance/update.php and SemanticMediaWiki/maintenance/rebuildData.php Thanks. Silkwood (talk) 19:40, 4 May 2019 (UTC)


 * Unfortunately, I have no easy way to test SD with the latest versions of either SMW or MW... unless you or someone else can give me access to their system. Otherwise, you may have to debug this on your own. Yaron Koren (talk) 15:19, 5 May 2019 (UTC)


 * I've sent you a PM by e-mail with access details. Thanks. Silkwood (talk) 10:44, 6 May 2019 (UTC)

Database PostgreSQL: Error 42601 Syntax-Error
I run a SMW (2.4.1) MW (1.27.1) with PHP (7.0.7) and PostgresSQL (9.4.9)

As long as I set only one filter in my drilldowninfo-tag everything on Special:BrowseData is fine. But as soon as a second (or third) filter the 42601 ERROR occur: My guess. PostgreSQL hast no CONVERT-Funktion. I also posted this for a while at Phabricator

Backtrace:

Any suggestion or solutions? Thanks a lot!


 * What's the full SQL call, do you know? Yaron Koren (talk) 14:18, 19 January 2017 (UTC)


 * --M art in (talk) 14:27, 19 January 2017 (UTC)
 * On Referata there is a similar bug - but more than one filters are possible there --M art in (talk) 14:53, 19 January 2017 (UTC)


 * Thanks for pointing out that bug on Referata, though it's a different bug. Yes, the issue does seem to be CONVERT with PostgreSQL - that function exists, but it has different syntax and (I think) behavior. I just checked in what I think are fixes for both bugs - if you get the latest code, hopefully it should work now. Yaron Koren (talk) 04:35, 20 January 2017 (UTC)


 * Thanks for the fast fix. The Problem with CONVERT was solved but now there is stll a database error 42883

Seems there are still some problems with PostgreSQL. The full SQL-query is this:

Could you please have a look again? --M art in (talk) 08:23, 23 January 2017 (UTC)


 * Ah, I didn't realize that IF was a problem too... I just checked in another fix, so hopefully now it works. Yaron Koren (talk) 18:04, 24 January 2017 (UTC)
 * Thank you very much for the fix! Works great now with postgresSQL. I really appreciate your patience with stupied users like me and your time you have to spend therefore. --M art in (talk) 08:44, 30 January 2017 (UTC)
 * Great. Sorry about the problems! Yaron Koren (talk) 22:12, 30 January 2017 (UTC)

Free text search
Hi. Is there a way to add a free-text search box pointed only to the filtered pages? I am developing a smw about Supreme Court orders fron my country, by now I have about 80k documents. Filters are great to narrow down the search, but a search box would be very helpfull. Greetings.
 * No, unfortunately. If you want this feature, you may have to install the Cargo extension (which I also wrote) - it is an alternative to Semantic MediaWiki that contains a drilldown feature, closely modeled after Semantic Drilldown, but which also supports free-text search. Yaron Koren (talk) 15:28, 25 January 2017 (UTC)
 * Thanks a lot. When you say alternative to smw, does it mean that both extensions cannot work together?
 * They can run on the same wiki, although each one stores data in its own location, and they can't query each other's data. Yaron Koren (talk) 17:03, 25 January 2017 (UTC)

Filtering Data type property
I have a property wich is Date type. It works fine but, when I set $sdgMinValuesForComboBox=1; the filter doesnt give any result. I would thank any idea..


 * That's strange. What if you set it to 2? Yaron Koren (talk)


 * No matter the value, the moment ComboBox is activated for date filter, it doesnt work anymore.
 * Without ComboBox, when I click on a date, the follow line is generated: ...&Fecha_de_publicacion=febrero_2010 (and works)
 * With ComboBox: ...&_search_Fecha_de_publicacion%5B0%5D=febrero+2010 (and it doesnt work)


 * Well, that's a bug in the extension. I guess I never tried setting that variable to a value low enought that it would kick in for dates, so I never saw the issue. Out of curiosity, why do you want a combo box to appear, and not individual links, even if there are only a few values? Yaron Koren (talk) 19:43, 25 January 2017 (UTC)


 * Date ComboBox creates a group for each month, I have documents from 84 different months. So I would need to set up a very high value for $sdgMinValuesForComboBox. Also there is another property (key word) that has more that a hundred values. So, a ComboBox is more functional for me.


 * There may be 84 months, but only up to 12 months are shown at a time in the drilldown interface. Yaron Koren (talk) 20:37, 25 January 2017 (UTC)


 * You are right. I just set up $sdgMinValuesForComboBox=13. I'll let you know how it worked. Thanks.
 * It took a while but the date filter is already showing only 12 groups. Thanks.

Performance
Hi. I have a Category with 42k pages already loaded and it will grow fast. By now, it takes one minute to load the drilldown page. I wolud appreciate any suggestion to keep a good performance. Thanks.


 * I'm surprised that it's taking that long to load - it may be one specific filter that is causing the problem. You could try removing the filters one at a time, to try to isolate the problem. In any case, another option is to install and use the Cargo extension, which provides its own very similar drilldown interface - either instead of or in addition to SMW+SD. Cargo has a more straightforward database structure, so I'm guessing that it's drilldown would be faster for your data. Yaron Koren (talk) 14:57, 7 February 2017 (UTC)

New release?
I have seen that several fixes especially concerning the querying for dates were authored since the last 2.0.2 release. Thus it will be cool to have a new version. --&#91;&#91;kgh&#93;&#93; (talk) 16:42, 1 May 2017 (UTC)
 * It would be immensely useful to have a Composer version so that it can be integrated with the rest of the Semantic MediaWiki suite of tools. This is the last major extension from Extension:Semantic Bundle that still has to be installed manually without Composer.

"Special:Filters" and "Special:Create filters" missing after upgrade
Hi, I've upgraded my MW installation:

MediaWiki          1.19.23 PHP                5.3.3 (apache2handler) MySQL              5.1.73 Semantic MediaWiki 1.7.0.2 Semantic Drilldown 1.1
 * FROM

MediaWiki          1.23.15 PHP                5.3.3 (apache2handler) MySQL              5.1.73 Semantic MediaWiki 2.4.1 Semantic Drilldown 2.0.2
 * TO

The wiki behaves ok, all existing filters are working as expected. However, both filter-related special pages seem to be gone. I've already run the SMW "Data repair and upgrade" multiple times but to no avail. Can this be a configuration setting issue in LocalSettings.php or some missing dependency? Any pointers welcome.

TIA --Nakohdo (talk) 15:07, 16 June 2017 (UTC)


 * Yes, this is not a bug - in version 2.0, those two special pages were removed, because filters are now defined in category pages, as opposed to in their own separate pages. (The "Filter:" pages are still supported, but they will stop being supported at some point, so I'd suggest converting everything over to use #drilldown_info.) Yaron Koren (talk) 15:31, 16 June 2017 (UTC)


 * Many thanks for this information, Yaron. This helps a lot. Nakohdo (talk) 13:04, 19 June 2017 (UTC)

#drilldownlink: Syntax for "filters" setting
Hi,

I don't quite understand the correct syntax for the "filters" parameter of #drilldownlink. The documentation says:


 * #drilldownlink

filters - the set of filters to apply, in the format "a=b&c=d&..."

Whereas the syntax for #drilldowninfo looks different:


 * #drilldowninfo

Could you perhaps provide a working example for #drilldownlink with filters set? Many thanks in advance.

--Nakohdo (talk) 09:34, 30 June 2017 (UTC)


 * Here's one example. Yaron Koren (talk) 14:03, 30 June 2017 (UTC)


 * Many thanks for the quick reply. However, that's an example for #drilldowninfo where the documentation is quite clear. I'm struggling with #drilldownlink. Nakohdo (talk) 14:19, 30 June 2017 (UTC)


 * Oh, oops. I can't think of any public examples of #drilldownlink, but for that earlier  example you could have a parameter like "filters=Language=English&Data type=Education". Yaron Koren (talk) 18:46, 30 June 2017 (UTC)

Is the drilldown info cached somewhere?
I am running into an issue I can't remember experiencing with Drilldown. No matter what I change in the #drilldowninfo tag in my category, the Special:BrowseData page still shows an earlier version of the drilldown filters. It is behaving as if the drilldown filter info was cached somewhere. --Lalquier (talk) 12:30, 12 March 2018 (UTC)


 * I don't think so... I can't think of why that would be happening for you. Yaron Koren (talk) 13:20, 12 March 2018 (UTC)


 * That's very odd. I was hoping a full rebuildData would clear things up but it didn't. For some reason, the filters displayed on the Special:BrowseData page are not the same filters that are defined in the category page.I will let you know if I figure this one out. --Lalquier (talk) 10:15, 14 March 2018 (UTC)


 * Update - I even tried to delete the call to #drilldowninfo in my category page and I can still see the Special:BrowseData page for that category, so it is really cached somewhere. I tried restarting the web server, forcing an exclusion to the mediawiki cache for that page, clearing my browser cache with no result. Very odd. --Lalquier (talk) 11:23, 15 March 2018 (UTC)


 * Update 2 - so it took a few days but the cache was eventually cleared on its own and the changes were applied. I still don't know which cache is causing such a delay between mediawiki, apache and the browser but at least I know it sorts itself out eventually. --Lalquier (talk) 01:22, 16 March 2018 (UTC)

Properties set by a parser hook cannot be filtered
Property filtering works When setting or Has type::Text, but it doesn't when we set the property types through an customized parser hook. IE:. Any idea why?


 * No idea. When you use the parser hook/function, does the data get set correctly? Does it show up in query results, for instance? Yaron Koren (talk) 00:42, 26 March 2018 (UTC)


 * A quick update after some further digging. Data gets set correctly, and the culprit is not the parser function but a query. If on a property page we put an ask-query like, filtering on that property ceases to works. Remove the query from the property page, and it still doesn't work until on the Special:SMWAdmin page we run the 'Schedule disposal' task. The filtering works again, until we put back the ask-query on the property page. I had wanted to show this on the SMW sandbox, but SemanticDrilldown is not installed there. --Remco de Boer 12:53, 27 March 2018 (UTC)

DB Unexpected Error
I run a SMW (2.5.8) MW (1.31.0) with PHP (7.0.31)

When going to https://dicoado.org/dico/Sp%C3%A9cial:BrowseData I get this error : I tried a rebuild data. Even completely deleted all SMW data and also ran the MW update.php maintenance script. I really don't know what the cause of that problem is. Any hint ? DSwissK (talk) 20:47, 11 September 2018 (UTC)


 * I concur, tried MariaDB and Mysql 5.7.32, same issue


 * It's the same thing here with MW1.30.0 SMW 3.0.0 SD 2.0.2 Carchaias (talk) 09:44, 19 October 2018 (UTC)


 * Same with me: MW 1.31.1, MariaDB 10.2.15, SMW 3.0.0, SD 2.0.2


 * Same with me with Mediawiki 1.31 and Semantic Mediawiki 2.5.8 or 3.0. This error:

--Manu.wikidebats (talk) 14:00, 13 February 2019 (UTC)

Compatibility with SMW 3?
Hi Yaron, is current version of Semantic Drilldown compatible with SMW3? I tried to upgrade an older wiki from SMW2/Drilldown 1.4 to MW1.31 and SMW3/Drilldown latest master via git and it does not appear to work.
 * since SD 2.1. you can use SMW 3.0.1 BUT not together with elasticStore --M art in (talk) 08:27, 13 February 2019 (UTC)

branch compatability
Branch REL_130 yes

Branch REL_131 no

Branch REL_132(stable) no

Tag 2.1 only if PHP < 7.1, issues with the major breaking change in PHP from version 7.0 to 7.1, function count Parameter must be an array or an object that implements Countable


 * Compatibility with what? Yaron Koren (talk) 16:55, 14 February 2019 (UTC)


 * MW REL1_30 works with SDD REL1_30, MW REL1_31 does not work with SDD REL1_31, MW REL1_32 does not work with SDD REL1_32, SDD version 2.1 works with MW REL1_32 but if PHP > 7.1 ( I said 7.0) above you get a lot of warnings about the breaking change in count function "count: Parameter must be an array or an object that implements Countable in /var/www/html/extensions/SemanticDrilldown/specials/SD_BrowseData.php  on line 973" and 999 and 1008. Using mediawiki  docker release REL1_32 uses PHP 7.2


 * What's the error message for MW 1.31 and higher? Yaron Koren (talk) 02:27, 15 February 2019 (UTC)


 * Never mind, now I get it. You really should never use those "REL_..." branches for Semantic Drilldown - or for any other of my extensions - because they just represent a random snapshot in time; there's no coordination with MediaWiki's own release schedule. As for the "Countable" warnings - sorry about that; I checked in a fix, so if you use the latest code, those warnings should go away. Yaron Koren (talk) 05:02, 20 February 2019 (UTC)

Version 2.1 is running with MW 1.32.0, SMW 3.0.2, PHP 7.3.5, MariaDB 10.1.37
I would like to share my experience with the download links - this may be helpful for others to get the extension running with the specified configuration: MW Kappa (talk) 16:04, 3 June 2019 (UTC)
 * Download link https://www.mediawiki.org/wiki/Special:ExtensionDistributor/SemanticDrilldown leads to version 2.0.2 - which is NOT running with this configuration
 * Github link https://github.com/wikimedia/mediawiki-extensions-SemanticDrilldown/archive/master.tar.gz leads to version 2.1 - which is running with this configuration


 * In general, ExtensionDistributor shouldn't be used for any of my extensions. Yaron Koren (talk) 13:07, 4 June 2019 (UTC)

Call to undefined method SMWQueryProcessor::processFunctionParams since SMW 3.1.0

 * MW 1.31.x
 * SD master (2019-10-26)
 * SMW 3.1.0

SMW 3.1.0 removed long deprecated functions from SMWQueryProcessor.

This throws specials/SD_BrowseData.php: Call to undefined method SMWQueryProcessor::processFunctionParams from SD_BrowseData.php line 1305

Potential fix: list( $querystring, $params, $printouts ) = SMWQueryProcessor::getComponentsFromFunctionParams( $display_params, false ); --Planetenxin (talk) 13:11, 26 October 2019 (UTC)
 * 1) SMWQueryProcessor::processFunctionParams( $display_params, $querystring, $params, $printouts );


 * The place to report this would be on Phabricator. 「 ディノ 奴 千？！」☎ Dinoguy1000 09:37, 27 October 2019 (UTC)


 * Planetenxin - does that code change fix the problem for you? Yaron Koren (talk) 23:19, 27 October 2019 (UTC)


 * Good that you asked Yaron, indeed there was a copy&paste issue. Now, the above fix works for me.--Planetenxin (talk) 09:05, 28 October 2019 (UTC)


 * Okay, I merged in this change - thanks. Dinoguy1000 - it's also fine to put bug reports here. Yaron Koren (talk) 21:36, 28 October 2019 (UTC)


 * Aah, apologies. I've seen in the past that generally when an extension has a Phabricator project, that is the preferred location for bug reports, which is why I said as much here. 「 ディノ 奴 千？！」☎ Dinoguy1000 23:26, 28 October 2019 (UTC)


 * Phabricator is indeed usually better, but for simple bugs like this one, anything is fine - email too. Yaron Koren (talk) 01:38, 29 October 2019 (UTC)

Format datatables does not load with SematicDrilldown
Hi,

after an update of MediaWiki and all extensions the pages using SemanticDrilldown do no longer show results with format=datatables. Using format=datatables with common ask queries works without problems. Other formats also work without problem together with SemanticDrilldown.


 * MediaWiki: 1.33.1
 * PHP: 7.1.33
 * Semantic MediaWiki: 3.1.1
 * Semantic Drilldown: 2.1 (d75d8b5) 22:36, 15 March 2019
 * Semantic Result Formats: 3.1.0

The firefox javascript console shows an error: "jQuery.Deferred exception: value is null".

Is SemanticDrilldown known to be working with MediaWiki 1.33.1? Is there any other MediaWiki version that is known to be working?

Thanks! Uli


 * I don't know. Are SMW queries with "format=datatables" working correctly? Yaron Koren (talk) 03:53, 26 November 2019 (UTC)


 * Yes, SMW queries are working correclty, SemanticDrildown not.
 * Today I have tested a fresh MediaWiki 1.33.1 installation and configured the installation using the web interface. Semantic Drilldown is taken from git.
 * I added two pages with different (text) properties, added the properties with type text and added a simple SemanticDrilldown statement with format=datatables to the category page. Executed all jobs using runJobs.php script. When going to the Special:BrowseData page the filters are displayed but the datatables do not load (only the three progress dots are shown).

Allowing for more than one value of a property
In a ReplicationWiki test version this example shows one can allow more than one value for a property (in this case years 2012, 2013, and 2014 for article year): https://replication.uni-goettingen.de/replication/wiki/index.php/Special:BrowseData/Studies?Journal=%5B%5B%3ACategory%3AJAE%7CJAE%5D%5D&Article_year[0]=2012&Article_year[1]=2013&Article_year[2]=2014 (The markup language makes links break at certain characters like in this case, this is another problem for which I'm searching a fix.) This can however only be made work by adding the additional years manually to the URL. When trying to click for it as indicated "Article year: (Click arrow to add another value)" the options are only "other" or "none". Yaron wrote in an email this seems to be a bug but he cannot currently fix it. Anyone else? I created a bug report at phabricator. ReplicationWiki-fan (talk) 15:22, 7 December 2019 (UTC)