Extension talk:Semantic Internal Objects

Duplication of Objects

 * MediaWiki 	1.15.1
 * SIO: 0.6.3

I created a simple template:

This template is called on a page named "History" just as example:

Fine.

As the template does not make any output I ask for these entries:

With the first load of the page anything is fine. If I reload .. all the entries double. If I reload again .. i get one more entry. Anyway: If I switch to edit-modus I still see only one object-definition .. if I save then I have one entry and the game restarts.

This error does not occur always. I make some edits .. and it is gone. I make some new edits und is is back again.

Do you have any idea?

Update: I want to cry. I cleaned the whole History-Page and startet with minimalistic entries to see, at which point the duplication occurs. It does not. I restored my old content .. and there is no error anymore -.-

Update 2: One Entry more and I have the duplication again

Update 3: Just to clarify: The amount of SIO does not change if I simply ask for them. It only changes, if I load the page, where these sio are defined.

--109.164.246.152 15:54, 17 January 2011 (UTC)


 * That's not good - it means the old duplication problem is still there. I was never convinced that it went away entirely - but the number of ways in which it can occur has definitely decreased. Anyway, don't cry! I bet a solution can be found. Could you try to replicate this problem on a public wiki, like scratchpad.referata.com? Yaron Koren 19:16, 17 January 2011 (UTC)


 * Thanks for the sandbox. http://scratchpad.referata.com/wiki/History. Anyway: At the moment I cannot reproduce the error, neither there nor on my own wiki. One little thing at referata: Sometimes the content of the first entry is missing.


 * Update: Missing content: Does occur, if I have a link link in the content and no definition for the attribute as has type::Text

--109.164.246.152 13:09, 18 January 2011 (UTC)


 * Okay, that makes sense - properties of type "Page" (which is what properties are by default, if you don't specify anything else) can't include links. It's not a bug. Yaron Koren 13:58, 19 January 2011 (UTC)


 * Duplicate-Error: I am still not able to reproduce the error at will, anyway, I see some conditions:
 * - the duplication occurs only, if definition and askquery are on the same page
 * - after duplication come into effect, it is displayed with every ask-query anywhere.
 * - IF the error occurs, it seems to occurs everywhere: I got it again .. now i see duplicates also on pages/sio which did not change during the last days.
 * - Update: Maybe there is a dependency with the Job_queue. I have many queued jobs at the moment due to changes in central templates. From time to time the duplication is gone, although no edit on the page has been made. With every reload from now on, it reappears. I'll wait until my job_queue is empty and retry afterwards.
 * -Update 2: Alright .. there IS a clear dependency with job_queue: I could reproduce the error on different pages with differend SIO as long as the job_queue had entries. As soon as the queue is cleared, the error does not occur anymore.
 * --193.247.250.55 09:04, 24 January 2011 (UTC)

Update 4: I have been using the semantic internal object in my mediawiki for holding news items. Each news item consists of news article,news date and project field(project filed holds the project which this news is related to). I would like to give my observation about duplication of field in internal object.
 * - Duplication Error: This error happens time to time which is shown in my #ask result. I have been trying to find how we get to this state so I can reproduce this bug. The only field that get duplicated is news article field.
 * - As mention above, this bug seems to be a state that the system enters for a period of time and then the system automatically comes out of it and behave normally.
 * - I had the "#ask" query in the same page as my news items but even after removing the ask query from that page I still got into duplication state.
 * - As mention on update 3, when the system enters this state you will see this anywhere that has any ask query.
 * - The only way that I can reproduce this is when there is jobs queued and I make changes to any of my news items(Removing, Adding or modifying news item on any page).
 * - Even if I have jobs queued and I do any changes which does not effect my internal object, This problem will not occurs.

I am also getting duplicated content of SIO field data when displaying multiple objects in a table. I uploaded an example to the scratchpad here. I also can't reproduce the problem on the scratchpad, but I believe it has to do with MediaWiki page cache settings as noted on the scratchpad page. I've verified that I don't see the problem on my wiki with caching turned on, but I must have it turned off. --Dmulter 23:57, 23 February 2011 (UTC)

Type to use
Hello,

What type are we supposed to put in the property page describing a semantic object ?


 * The main property should be of type "Page", if that's what you're asking about. Yaron Koren 15:31, 7 February 2011 (UTC)

Combining page properties and semantic properties
Hello, is possible to combine page properties and semantic object properties in a query ?

For example, let's say we have recipes separated by continents. The recipe page would have a from_continent::europe property, and then it would have the SIO.

If at some opint you want to make a list of all recipe from europe using flour, it would seem normal to do

but it doesn't seem to work.

I also tried a category approach

but no success either.

Is there a special syntax in the query declaration to refer to the SIO's parent's properties ?

--61.114.185.196 08:47, 23 February 2011 (UTC)


 * Yes - your query should look like " Is part of recipe.From continent::EuropeHas ingredient::flour ". Yaron Koren 13:27, 23 February 2011 (UTC)

Combining regular semantic queries with SIO?
Dear Yaron Koren, my question is probably best asked using an example:

Template/form
Say you have a Form:Speed skaters for listing contestants on articles about ice skating races. Some properties are specific to each race, such as "Year of race" and "Sponsor". Other properties are specific to each contestant and so are handled by multiple instances of a SIO call to #set internal, for instance:

In Template:Speed skaters the SIO call for the champion would look something like this:

Likewise, for the contestant who came second:

Etcetera

The query
Now say one wants to build a query, sorted by year, which gives an overview of all races and their contestants, with a column for first-place champions, another for second place, a third for bronze, etc. The first set of columns (i.e. which comes before the names of the contestants) consists of a column for the names of the races, another for the year in which the race had been organised and another for the sponsor. To help you visualise what I mean here, consider the following (again completely fictitious) example:

Etcetera

Or if you want to include the "average speed" of each contestant:

Etcetera

The circumstances are obviously entirely imaginary, but the kind of situation portrayed here looks fairly common to me. In fact, I've run into a very similar one. So my question would be: what query should be used, if possible at all, to produce a table like the one sketched above? Regards, Cavila 12:31, 25 February 2011 (UTC)


 * I see the issue - you want to display the year and sponsor in the same query as the other stuff. Unfortunately, SMW queries don't allow the display of "linked properties". But I believe there's a workaround - you can have each internal object store the necessary values. So you could add to #set_internal something like:


 * Race has sponsor=
 * Race has year=
 * Yaron Koren 15:42, 25 February 2011 (UTC)
 * A belated thanks (I don't know how I missed this, sorry)! Although it's not entirely what I meant, it's a useful workaround that I will need to bear in mind for related situations. A complementary approach I guess would be to define separate properties for first place, second place, etc. That way at least each position (1, 2, etc.) would get a separate column. Cavila 10:07, 18 April 2011 (UTC)

Selecting SIOs of a specific page
Hi there. I got al little problem with the design of a query.

I have a wiki that is used to document our company's interfaces. We have a ticket tracker and so on, but we want to store some important tickets/change requests in the wiki as well. So on these article pages I have one or more set_internal like I use the category property in querys to only show a assignment of a specific type.

But what I'm missing is an easy way to select SIOs only from a specific page or category.

I think I could store the Category and the Page in the SIO as well, but maybe there is an easier way.


 * Hi - if that #set_internal call is meant to go on a page named "27526", then it is indeed unnecessary - you can just do a query like " {{#ask:is assigned to::27526|... ". A property named "category" is also confusing, since that doesn't correspond to a MediaWiki category, but that's another story. Yaron Koren


 * In our system, an article could have assignments other articles of different, specific categories. E.g. a ticket could be assigned to an Interface and/or an Change Request. Maybe here again I could use different properties for that, but I wanted to keep it as simple as possible.

Need to resave the pages to get SIO appearing
Hello, I saw that there was this problem before with this extension and saw in the version log that it was fixed, but I am still encountering this issue.

My setup:


 * Mediawiki 1.16.1
 * Semantic Internal Objects (Version 0.6.3)
 * Semantic MediaWiki (Version 1.5.6 alpha)

I declare SIO on pages from the Category A all formatted with the same template, and have a Query on the same page fetching the data and putting it in a nicer display, but I noticed that I need to save a page twice to get the SIO actually be taken in account. If it was only about the page you just saved it would not be a huge issue, just a bother, but my main worry is when it comes to templates. A small modification in the template will force a refreshlink job on all the pages using it (around 900, processed with runJobs.php), but after the refreshlink job the SIO disappear, and I have then to go on each page again and resave it. Purge doesn't have any impact and there isn't any job left in the queue. Obviously, it's not really convenient to manually resave every page if someone changes a letter in the main template.

Do you have any idea ? --61.114.185.196 06:43, 19 April 2011 (UTC)


 * Hi - this may be two different, unrelated problems - though I'm surprised that either one is happening. If possible, could you try to replicate at least one of the issues on a public wiki, like http://scratchpad.referata.com? That would make it easier to debug. Yaron Koren 13:48, 19 April 2011 (UTC)


 * I can confirm this particular problem on MW 1.16.2, SIO 0.6.3, SMW 1.5.6 Remco de Boer 12:18, 3 May 2011 (UTC)
 * Using 'repair and update data' on Special:SMWAdmin seems to 'fix' the problem. I'm wondering whether this could be some sort of race condition. I could not create a simple test case; I suspect this behaviour only occurs when a large number of pages is placed on the job queue after e.g. a template edit. I inspected the database, and noticed duplicate rows in smw_rels2 for a page that did not show its internal objects. Perhaps there is some weird interference between a 'simultaneous' forced refresh (purge) of a page and a refreshLinks action that is waiting on the job queue? Remco de Boer 13:07, 3 May 2011 (UTC)
 * Just a wild idea: if duplicate database rows are (part of) the problem, couldn't SIO just blindly remove rows that are identical to the new data before inserting? Especially in smw_rels2 this should be easy to do (don't know about any other involved database tables). I don't think you would even have to check whether such duplicate rows exist in the db, because a "DELETE FROM" statement that deletes nothing is just fine. Remco de Boer 13:19, 3 May 2011 (UTC)

Lists of values separated by punctuation marks other than commas
Hi Yaron. Is there a way we can use other punctuation marks, such as semi-colons, rather than commas, as delimiters in lists of values? Many of my article titles already include a number of commas, so the prescribed method would not work for me. Regards, Cavila 17:17, 25 April 2011 (UTC)


 * There's no way to do that at the moment (other than by making a change to the PHP code), but it makes sense to add that as an option. Would it be enough for you to have it as a wiki-wide setting (i.e., set it to ";" for every #set_internal call in the wiki), or would you need to have different delimiters for different calls? Yaron Koren 23:17, 3 May 2011 (UTC)
 * Well, I've been using commas where they do not present any problem, but I presume that manually replacing them with semi-colons is still doable at this stage. So a site-wide solution for #set_internal calls only would be welcome. Are you thinking of a minor PHP patch/change? Cavila 18:40, 4 May 2011 (UTC)

Chained Searches on SIOs
Suppose that we use SIOs to describe a dependency graph. How can we explore the 2nd and 3rd level dependencies?

Example
Using the plain-old properties to describe a dependencies-schema we would write, for instace, the following 3 pages: Page C: Client of::Page B, Client of::Page A Page B: Client of::Page A Page A: No dependency.

In order to search for all 2nd level clients of A, and 2nd level dependencies of C, we would ask for chained properties like that: * 2nd level Clients     : * 2nd level Dependencies :

With SIOs we could be more expressive, adding, for instance, the dependenc-type, and the 3 pages would then become like that: Page C:  Page B:   Page A: No dependency.

Question
Is this possible to re-phrase the former 2 #ask queries to get similar results?

Thank you in advance, Ankostis 07:59, 18 May 2011 (UTC).


 * Hi - first, there's an error in your #set_internal calls - they should look like " ". But I'm not sure there's a way to do chained queries using SIOs, at least not in an easy way - if you wanted to do it, you might need to make use of extensions like VariablesExtension and ArrayExtension. Yaron Koren 17:10, 18 May 2011 (UTC)
 * Thank you Koren, i'm already using those extensions, rather heavily, and i think i can see what you mean. For instance, for a single page it would take something along the lines:

...
 * But i cannot use the above workaround with the powerfull '|format=graph' result-extension. And in general, it is quite awkward idiom compared to the "natural" feeling of plain-old properties.  Maybe you should take this example into account when (planning to) integrate SIOs into the core of SMW, and provide a better syntax.
 * (regarding the #set_internal error in my example, and my new suggestion above, i typed them out of my head, did not tried them first - error corrections are more than welcomed) Ankostis 09:31, 19 May 2011 (UTC)


 * This is actually strictly an SMW querying issue, not an SIO issue - the same problem would happen if dependencies had their own pages. I should note that there's nothing to prevent you from calling the plain SMW "Client of" property, in addition to the #set_internal call. Yaron Koren 13:03, 19 May 2011 (UTC)

Not displaying anything?
So I have a company that has the following SIO statement in it:

I have another page that has the following ask statement:

I have properties of date type set for "Has buy date" and "Has sell date". I have no idea what property type the "Has transaction" is supposed to be, I've tried "Page". In any case every single property shows up as "Unused"

Could someone help me get this running?

Thanks,

Mats


 * That's weird. Yes, "Has transaction" should be of type "Page". If you have a regular #set call on that company page, does it work? Yaron Koren 03:22, 16 June 2011 (UTC)


 * I reinstalled mediawiki with SIO and now everything is working. Thanks for clarifying the type for me.
 * Cheers :)

Set Object on Another Page?
I'm using SIO to record votes on a series of objects defined in a page. I want to have the votes set by a subpage so that they can be ignored when viewing the page itself. I can transclude the subpages to get the SIO objects on the page but that has a lot of other issues. The page I'm working on is here.

Is there a way to set an SIO object on a parent page from a subpage?

Jamie Thingelstad 04:14, 27 June 2011 (UTC)


 * No. For voting, you could consider using a dedicated voting/polling extension, instead of SMW/SIO... Yaron Koren 16:20, 27 June 2011 (UTC)


 * Thanks for the reply. I looked at the dedicated voting extensions and there are some good ones. However, they become a different data universe. I found that I wanted to do Semantic Queries of voting data along with other data. I think I can do what I'm looking to do using transclusion (it works that way now), it's just a little more complicated for the end user. Jamie Thingelstad 12:08, 28 June 2011 (UTC)

Search for SIO sorted by other Attributes
A person's page attributes are set like this, since one person can be leader of more countries but the person's rank is always global:

If I now ask for this data, the rank is not shown:

Is this at all possible?--VolkoV 08:18, 1 August 2011 (UTC)


 * You should change "Leader of" to "Leadership of", since that's the correct description here. Also, I think you should take "Leader of - " out of the other property names. Finally, within the #set_internal call, add "|Leader rank=1"; and change "?Rank" in the query to "?Leader rank". Yaron Koren 19:21, 1 August 2011 (UTC)


 * Thanks for the answer - the problem is (above is a very abstract example), that I have a lot (~20) of normal attributes like the rank for all the persons (~1500) and one person can only have one rank (or one picture etc.) but can be a leader of multiple countries (in average ~3). So I do not want to include all these attributes to every leadership leading to an unneeded multiplication of attributes. Therefore the question on how to include "normal" attributes in SIO-Queries.--VolkoV 05:06, 2 August 2011 (UTC)


 * Well, the real problem is in SMW itself - that chained properties, like, in this case, "?Leadership of.Rank", can't be included as printout properties. You can query on chained properties, but you can't display them. Until such a thing is allowed (or until RDF/SPARQL is fully supported, including by SIO), I think my solution is the only real one. Yaron Koren 07:02, 2 August 2011 (UTC)


 * Well, thank you anyways! If I do not want to sort, I just put another query in the output template - works fine. Have to go without sorting for now then...--VolkoV 08:23, 2 August 2011 (UTC)

Troubles when upgrading to SMW1.6/SIO0.6.5
Hello, I got the following errors after updating Semanic Mediawiki to 1.6 and SIO to 0.6.5, and after running the table upgrade+data repair. It happened on pages with SIO during the repair process.

(Requête SQL cachée)

depuis la fonction « SIO::updateAtts2Data ». La base de données a renvoyé l’erreur « 1054 : Unknown column 'value_unit' in 'field list' (localhost) ».

The root cause is this statment in SemanticInternalObjects_body.php line 153:

// getUnit was removed in SMW 1.6 if ( method_exists( $value, 'getUnit' ) ) { $upAttr['value_unit'] = $value->getUnit; }

It was assuming that getUnit would indeed be completely removed in SMW 1.6, but it is actually still present in SMW_DV_Number.php, SMW_DV_Quantity.php and SMW_DV_Temperature.php. So it tried to insert empty values in the now non-existent "smw_atts2.value_unit" column.

--Meng Ly 07:42, 2 August 2011 (UTC)


 * Hi - yes, indeed, this was a bug. It was already fixed in SVN last week, but I still haven't released a new version with the fix - I'll try to do that soon. You can see the fix here to apply it to your own code, until then. Thanks for the effort in hunting it down. Yaron Koren 19:58, 2 August 2011 (UTC)

Limiting queries to search only within a single object set on the same page
Hello. Maybe I'm just being dumb here, but I can't figure this out. Let's say I have a single page with a set of data that gets added to the page via a Semantic Forms multiple instance template. It ends up looking like this in the source:

Let's say I want to query this data on another page, like so:

You'd expect this to return no results. However, the query returns the schedule for Billy Bob, because parts of his schedule match Sundays and other parts match 7:00. What can I do to limit the query? Thanks. --Blicarea 01:18, 11 August 2011 (UTC)


 * What does the "Event" template look like? My guess is that it contains normal SMW tags in addition to #set_internal - it should only contain #set_internal. Yaron Koren 04:28, 11 August 2011 (UTC)


 * Thanks, Yaron. Actually though, the Event template only contains #set_internal.

--Blicarea 11:23, 11 August 2011 (UTC)


 * That's strange, then - maybe you've found a bug in SMW's querying. Maybe that colon in the time is causing the error? Yaron Koren 22:20, 11 August 2011 (UTC)

Repetition issues again
Dear Yaron, sorry for bringing such bad news. I found that I’m still having trouble with property values being stored twice or even 'thrice'. In an earlier situation in MW 1.16 / SMW 1.5.6, composing calls to set_internal actually made things worse: property values increased tenfold, at which point I had to remove the queries to prevent potential database overload.

These latter issues may or may not have been fixed (I really don’t know yet), but the former remains with regard to one particular data type: Text. The replication issue seems to disappear once this property is converted to one of type String – on the downside, text strings longer than 255 characters are not supported.

Regards, Cavila MW 1.17, MySQL 5.1.57, Php 5.2.17 23:05, 14 August 2011 (UTC)
 * Mmmm, now one of my pages returns an "Internal error: Illegal property key ........", with a backtrace pointing to errors in all sorts of php-files (SMW_DI_Property.php, SemanticInternalObjects_body.php, SIOHandler, /Preprocessor_DOM.php, Parser.php, etc.). Any clue what's going on here? Cavila MW 1.17, MySQL 5.1.57, Php 5.2.17 14:16, 15 August 2011 (UTC)


 * Yikes. Can you put the whole backtrace on pastebin or some such? Yaron Koren 17:32, 15 August 2011 (UTC)


 * OK, I'll send you an e-mail with further details (merely because I don't want to disclose my identity in public, that's all). Cavila MW 1.17, MySQL 5.1.57, Php 5.2.17 19:36, 15 August 2011 (UTC)

SIO with forms
Could someone show me an example on how to use semantic forms to populate an SIO per template instance? I want to create a separate SIO for every template instance (on a multiple template form), in order to later query the objects in a n-ary matter. Or if it's possible to use a form to directly populate multiple SIOs, and be able to display SIO properties and its parent page properties (has other templates) at once.

Duplication in a Text Property

 * MediaWiki: 1.16.5
 * Semantic Internal Objects: 0.6.6
 * Semantic MediaWiki: 1.5.6
 * Semantic Result Formats: 1.5.3
 * Semantic Maps: 0.7.7
 * Semantic Forms: 2.2.1
 * Semantic Compound Queries: 0.2.9
 * Semantic Forms Inputs: 0.4.1

I am getting duplication for text-type properties inside #set_internal when querying the data with #ask. Sometimes re-saving the data page resolves the issue, but recently, it hasn't help. Any thoughts?


 * You could try upgrading your versions of Semantic MediaWiki and MediaWiki - I doubt that will help, but you never know, and it's a good idea anyway. Is there a set series of steps you can take, to reproduce the problem? If you could find such a thing, that would be extremely helpful in debugging it. Yaron Koren 20:53, 2 September 2011 (UTC)


 * We are experiencing the same problems when using SIO together with Semantic Forms. I thought it might be a matter of interaction between Semantic Forms and SIO, but it is not. -- achimbode
 * SMW-Versions: MediaWiki 1.16.5, Semantic MediaWiki (Version 1.5.6), Semantic Forms (Version 2.1.2), Semantic Forms Inputs (Version 0.4.1)
 * Tried to reproduce it, but couldn't either: http://smw.referata.com/wiki/Sandbox/SIO_bug_type_text. I'll keep trying as soon as I find the time. Feel free to help.
 * I noticed that you hard coded the templates into the page. I utilize a semantic form which is triggered with button to enter the data into my tables. The data is stored it's own unique page, and I then am able to perform an #ask query from any page. I'll need to see if I can reproduce the problem by hard coding the templates into a page. If not, the issue may be the creation of the semantic internal object via a semantic form.
 * We suspect the job queue to be responsible for some of the double entires. We tested the latest SIO trunk on 1.18alpha, and saw sporadically double entries on pages that have been updated through the job queue. Only after the page was saved manually the original dataset was restored (we use SIO to store datasets to be displayed by jqPlot therefore it is obvious when double entries are present). --MWJames 02:23, 8 September 2011 (UTC)
 * I've placed some text on the sandbox page that 100% of the time on my system (specs on that page) will cause text-type properties to save (or at least display) two values after the initial save. --41.222.91.165 11:48, 15 September 2011 (UTC)

Yes - I should have specified before that my strong guess is that, at this point, all SIO duplication-related issues come from resaving the template on which a #set_internal call is made - that causes jobs to be added to the job queue to refresh every page that uses that template, and those jobs can (I guess) interfere with resaves of those same pages. But I haven't been able to actually reproduce this issue, which means that debugging it is hard. Yaron Koren 02:30, 8 September 2011 (UTC)

Browse whit Drilldown
Is it possible to somehow use page Special:BrowseData of Semantic Drilldown with Semantic Internal Objects? --Pastakhov 10:26, 19 September 2011 (UTC)


 * Unfortunately, no. It's something I would like to add to Semantic Drilldown, though. Yaron Koren 13:37, 19 September 2011 (UTC)