Extension talk:Cargo

Trouble Adding Table to a particular Template
For the life of me I can't see to get a particular template to create tables. I'm starting with Page Schemas and have something like this...

    Author  text   Text    Update</Label>  text</InputType> </semanticforms_FormInput>  Text</Type> </cargo_Field> </Field> </Template>

This is my actual template that gets generated:

This is the "Status" template. It should be called in the following format: &lt;pre>

&lt;/pre> Edit the page to see the template text.

Originally it was a multiple instance, with a datefield, author (user page), and textbox/wikitext. I've tried changing the name, order, datatypes, etc. My other templates are coming through but not this one or anything like it. I execute runJobs on cmd line. In the API url call I get something like this...

/api.php?action=cargorecreatetables&template=Status

{   "error": { "code": "internal_api_error_DBQueryError", "info": "[7817f089] Database query error" } }

Usually I get an error when I then try to edit the page with form stating the table doesn't exist, but what's weird is that eventually the form works and I can edit data, but no data is stored to the database, and still no table for this one actually exists in the mysql database.

If I run cargoRecreateData it seems to skip this template also.

I appreciate any direction to help debug this.--Kc5vcx (talk) 03:01, 2 November 2015 (UTC)


 * My guess is that the field called "Update" is causing the problem, because that is already an SQL keyword. If so, that's a bug in Cargo - it should either handle that name or reject it. Please let me know, though. Yaron Koren (talk) 14:13, 2 November 2015 (UTC)


 * Ahh, yes, that's it! Thanks, works now after renaming the field. --Kc5vcx (talk) 15:22, 2 November 2015 (UTC)

Timeline output format displays blank box
Here's the question I was going to ask: The timeline format (as here) would be ideal for my data, but unfortunately all I get is a blank space. It is exactly the same as the example on discoursedb.org, and similar to the example on mwstake.org. I'm using MW1.25.3 and Cargo 0.6.

Then I realised I should use Cargo 0.10 so did "get checkout master". It all works fine now. So don't use the REL1_25 branch with MW 1.25.3 :-)

Also the calendar format showed an empty calendar on the iPhone but that also now works fine. Jonathan3 (talk) 19:19, 5 November 2015 (UTC)

Timeline and calendar formats don't work properly with Foreground skin tabs
I would like to use three tabs for an events page - for Table, Timeline and Calendar formats.

Each format works fine, one after the other, without using tabs. But...

In tab 1, only Table and Timeline work. Calendar does not.

In tabs 2 or 3, only Table works. Calendar and Timeline do not.

Is there something I can do to fix this?

The tabs are done as follows. Each "section" is a tab.



<p class="title" data-section-title>Timeline ...

It looks similar to the problem here though probably is unrelated. Jonathan3 (talk) 20:48, 5 November 2015 (UTC)

I have discovered that the timeline format DOES work, in any tab, if there is a cargo query outside the tabs structure on the same page. Jonathan3 (talk) 20:57, 5 November 2015 (UTC)


 * I would check the JavaScript console for errors, if you know how to do that. Also, does this only happen with the Foreground skin? Yaron Koren (talk) 01:16, 6 November 2015 (UTC)


 * Thanks Yaron. The problem isn't there for the Vector skin (the tabs don't show as tabs with Vector, but everything appears one after the other - as it does with Foreground if you don't use tabs).
 * I got the following error initially:

Test_page:1 XMLHttpRequest cannot load http://domain.co.uk/index.php?title=Special:CargoExport&tables=Event&&fields=_pageName%2CDate&&order+by=_pageName&limit=100&format=timeline. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://www.domain.co.uk' is therefore not allowed access.
 * After adding a str_replace to add the www that error stopped appearing (I don't really understand this to be honest so it may have coincided with something else).
 * Also I was getting an error "Caught exception: SyntaxError: Unexpected end of input" which may have been related, and which also no loger appears.
 * The only other things are just lots of "Use of "wgCargoMonthNamesShort" is deprecated. Use mw.config instead."
 * Jonathan3 (talk) 02:22, 6 November 2015 (UTC)


 * So, you're no longer getting errors? (You can disregard those warnings.) And is the actual problem still happening? Yaron Koren (talk) 02:43, 6 November 2015 (UTC)


 * Sorry for not being clear. It all seems so random to me so I'm just trying different things. I'm not getting those errors any more but the original problem persists (plus at the moment timeline format doesn't work in tab 1 either). I'll email you a URL if that would help (I don't want to publish it). Jonathan3 (talk) 02:47, 6 November 2015 (UTC)


 * Ah - yes, please do. Yaron Koren (talk) 02:51, 6 November 2015 (UTC)

Are "Semantic_Extra_Special_Properties" meta data not available for Cargo either ?
If yes I suggest to add "Semantic Extra Special Properties + SMW" in the line "Storage of page metadata"/column "SMW-based system" of the "Features checklist" in the page Extension:Cargo/Cargo_and_Semantic_MediaWiki.

Cheers,

Nicolas NALLET (talk) 16:03, 6 November 2015 (UTC)


 * Good point - I just added it in. Yaron Koren (talk) 16:49, 6 November 2015 (UTC)

Advanced SQL Joins in cargo_query for multiple instance templates
I'm probably over complicating things in my data store, but I'm building a project tracker on SMW using Cargo. Our projects can have multiple milestones with planned/expected/actual dates I want to track. Rather than hard-coding a potential list of these milestones and respective dates (this list could get long) in my form, I made a multiple instance dates template where somebody can select from usual milestones and add others. The problem is I then need to link it back to the project and this obviously complicates the query. It's been awhile since I've done SQL but I can accomplish what I want in phpmyadmin with something like this...

select Projects._pageName,Dates.Planned_Design_Start from cargo__projects Projects JOIN (select _pageName, Planned_date as Planned_Design_Start from cargo__project_dates where Milestone="Design Start") Dates ON Projects._pageName = Dates._pageName

Unfortunately a simple join doesn't really organize the way I need and produces something like this...

Not sure why the aref's aren't coming through here, but that's not part of the issue on my real page.

I'm assuming I need to translate my data more directly like the first example for the various results formats to process it, but I'm just getting there. I was trying to embedded it in the cargo syntax, maybe put the join in the tables= field, but it obviously won't let me use SELECT for the subquery, and $wgCargoAllowedSQLFunctions in localSettings.php also won't let me allow it. Obviously I could simplify my data structure and that's why I'm working through this, but wanted to explain my example to help justify. Any thoughts on enabling something like this or how I might hack around it? Might be nice if I could just give it a raw sql query (this is on a secure internal network, so less concerned about security).

Also, as I'm debugging, is there a good way to display the raw sql query text it generates like at the top of the page with a variable? I can introduce some types of errors and get it, which I've found helpful debugging what I'm trying to do with my data.

Thanks again.--Kc5vcx (talk) 18:35, 6 November 2015 (UTC)


 * Hi - I think I see the issue: you want to show more than one "row" of data from that "Dates" table across the same row of results. Unfortunately, I don't think you can do that with a call to #cargo_query, given its constraints. There is one way, if you use a "template" format and then include a query within that template; but that's a hack, and anyway it won't let you display anything more interesting than a (hacked-together) table. What I would suggest instead is to hardcode the fields for "planned date" and "expected date", since every page should have those. And if there are other dates that you'd expect every page to have, I'd hardcode those as well. Actually, maybe it's better to hardcode everything, unless there are really a lot of other date types, in which case you can use that setup you came up with for all the other types.
 * A "debug" format would be great; I keep meaning to add that in. Yes, I too introduce errors in order to see the SQL, which is not great. Yaron Koren (talk) 19:39, 6 November 2015 (UTC)


 * I found I can sort of do what I want my concatenating the field names in a compound query, in this case to send to the calendar result format, like so...

However, there's something wrong with the AS statement on a date field because the calendar format equivalent is empty. If I remove the =date field, then I get the first query on my calendar, but not the second because it's in a second column. It looks like cargo is including the __precision table in the query behind the scenes and I presume formatting it correctly to send to the calendar. Using the = in fields which correlates to the SQL AS for alias, looses this formatting or whatever. I seem to need to be able to combine my dates all into one column for them to show up on the calendar. So is there a way to force the date formatting calendar needs? I've been trying the DATE and DATE_FORMAT functions on both sides.--Kc5vcx (talk) 22:38, 6 November 2015 (UTC)


 * This was indeed a bug, and the problem lay with the 'precision" fields like you said. Thanks for diagnosing it. I believe I just fixed it in the Cargo code. Yaron Koren (talk) 00:48, 11 November 2015 (UTC)


 * Thanks, I'm still having a bit of an issue though after playing with my sample above. For some reason only one (maybe the last) of the second query in the compound query gets put on the calendar.  What I'm trying to do is have color coded events, blue is the default planned date.  A second red calendar shows where the expected date is not equal to the planned.  If I use a format table I see a couple applicable entries, but in the calendar format only one shows up in red and the others not at all.  Is there anything different in the data output between them?  I confirmed they're all the same simple looking date format in mysql with precision 1.  The rest of the blue are there and shouldn't conflict with names or dates.  I haven't been able to figure out another way to view the raw calendar data to further debug.  Any thoughts? --Kc5vcx (talk) 16:41, 11 November 2015 (UTC)


 * There's a way to view the raw calendar data, though it's a little bit of a hassle; I use it all the time, though: (1) View the page source, and find the URL that includes "Special:CargoExport". (2) Replace all the "&amp;amp;"s in the URL with just "&"s. (3) Add a "start=" and "end=" value to that URL, representing the range of dates you want to find, like "&start=2014-01-01&end=2015-01-01". (4) Go to that URL. Yaron Koren (talk) 00:10, 12 November 2015 (UTC)


 * As I suspected the raw calendar data is a subset that I get in the table or even the csv format, so something else in the processing is filtering it out. The functions for the different types look significantly different in the export code with additional date stuff for the calendar, if I get anywhere looking at it I'll update.--Kc5vcx (talk) 16:48, 12 November 2015 (UTC)


 * UPDATE: If I comment out the below section where it appends the start/end dates to the where clause of the query I get my expected result. There must be something conflicting with the multiple dates, but not obvious to me yet without seeing the produced sql.

$where = $sqlQuery->mWhereStr; if ( $where != '' ) { $where .= " AND "; }			$where .= "(";			foreach ( $dateFieldRealNames as $j => $dateField ) {				if ( $j > 0 ) {					$where .= " OR ";				}				$where .= "($dateField >= '$startDate' AND $dateField <= '$endDate')";			}			$where .= ")"; $sqlQuery->mWhereStr = $where;


 * Is there any particular reason for this rather than the user limiting the results on their own? --Kc5vcx (talk) 20:55, 12 November 2015 (UTC)


 * It's because the calendar only shows one month/week/day at a time; so the page only queries for results from that time period. Yaron Koren (talk) 21:04, 12 November 2015 (UTC)

Job run rate "too high"?
The Known bugs and planned features page states: "If a table is recreated via the web interface and the "job run rate" (set via $wgJobRunRate) is too high, it can lead to some pages not getting their data stored." What is too high? Alternatively, what range is all right (e.g. 0.1 to 1)? Thanks Jonathan3 (talk) 20:51, 6 November 2015 (UTC)


 * I don't know; I should do more testing. I think it's worked for me fine with a rate of 1, but caused problems when I used a rate of 10. Yaron Koren (talk) 21:54, 6 November 2015 (UTC)


 * Thanks for explaining. Jonathan3 (talk) 22:24, 6 November 2015 (UTC)

Boolean field always 0 in database
I have a boolean field ("Active" in a "Jobs" adverts table). When I create the job page using Semantic Forms, the tick box makes it "Yes" or "No" as appropriate. But when it appears in the Cargo database table it's always 0. I'm sure I'm missing something simple..... Thanks Jonathan3 (talk) 22:28, 6 November 2015 (UTC)

P.S. When I edit the database directly (changing one of the 0s to 1) it is reflected in the Special:CargoTables/Jobs page, but when I "recreate data" on Template:Jobs it returns everything to 0. Jonathan3 (talk) 22:40, 6 November 2015 (UTC)

P.S. And when I change the Job page so that "Active" is "1" (instead of "Yes") this leads to Special:CargoTables/Jobs page showing 1. So I suppose I need to know either: (a) how to get SF to make the values 1/0 rather than Yes/No, or (preferably) (b) how to get Cargo to recognise "Yes" and save "1" to the database.Jonathan3 (talk) 22:46, 6 November 2015 (UTC)


 * I was going to try to help, but I just realized the same thing is happening to me. --Kc5vcx (talk) 22:50, 6 November 2015 (UTC)


 * Yikes! It looks like the setting of Boolean values never worked correctly. I guess it's not a very popular type. Sorry about that; I think I just fixed the problem in the Cargo code. Yaron Koren (talk) 20:20, 8 November 2015 (UTC)


 * Looks fixed to me. --Kc5vcx (talk) 16:41, 11 November 2015 (UTC)

Error: the SQL function "AND" is not allowed.
When I use the following query I get the error in the topic title: |where=Active=1 AND (Closes >= NOW or Closes IS NULL)

But when I use this it works fine: |where=(Closes >= NOW or Closes IS NULL) AND Active=1

For future reference, where am I going wrong? Jonathan3 (talk) 23:13, 6 November 2015 (UTC)


 * That was a bug - sorry about that! I just fixed it in the code. Yaron Koren (talk) 19:16, 10 November 2015 (UTC)


 * Thank you very much (again). Jonathan3 (talk) 18:58, 11 November 2015 (UTC)

Date precision
Here is written about the possibility of storing of incomplete dates (year or year+month). But I was not able to set incomplete dates by #cargo_store. What is the syntax for this? --StasR (talk) 15:41, 17 November 2015 (UTC)


 * There's no special syntax - the code tries to figure out the precision of each date. What was the date you tried to set, and what happened? Yaron Koren (talk) 17:00, 17 November 2015 (UTC)


 * Template:Cargo1


 * Page


 * View table

This table has 8 rows altogether. Page   descr       value Cargo2 31.03.2014  2014-03-31 Cargo2 2014-03-31  2014-03-31 Cargo2 .03.2014    1970-01-01 Cargo2 2014-03-    1970-01-01 Cargo2 03.2014     1970-01-01 Cargo2 2014-03     2014-03-01 Cargo2 2014        2014 Cargo2 2011        2010
 * Database content

_ID descr       value     value__precision 1 31.03.2014  2014-03-31  0               2  2014-03-31  2014-03-31  0               3  .03.2014    1970-01-01  0               4  2014-03-    1970-01-01  0               5  03.2014     1970-01-01  0               6  2014-03     2014-03-01  2               7  2014        2014-01-01  3               8  2011        2011-01-01  3
 * I realized that the syntax is defined, but there are problems in Table View ("2014-03" → "2014-03-01" and "2011" → "2010"). In addition, value__precision for complete date contradicts its description (0 vs. 1). --StasR (talk) 18:07, 17 November 2015 (UTC)


 * There were a few issues there. The date parsing obviously is not perfect - Cargo uses PHP's own date parsing, for what it's worth. Although those failing dates are in rather awkward formats; are there any that would specifically be useful to handle correctly? The year-off-by-one thing was strange; I just checked in a fix for that. Thanks for letting me know. As for the "precision" field showing the wrong value, I'm guessing that's just because you're using an older version of Cargo. Yaron Koren (talk) 20:08, 17 November 2015 (UTC)


 * I use Cargo REL1_25. Cargo master and MW 1.25 — is it good? --StasR (talk) 21:33, 17 November 2015 (UTC)


 * Cargo master View Table:

This table has 8 rows altogether. Page   descr       value Cargo2 31.03.2014  2014-03-31 Cargo2 2014-03-31  2014-03-31 Cargo2 .03.2014    1970-01-01 Cargo2 2014-03-    1970-01-01 Cargo2 03.2014     1970-01-01 Cargo2 2014-03     март 2014       - formally OK, but long form :-( Cargo2  2014        2014 Cargo2  2011        2011            - OK!!! --StasR (talk) 21:44, 17 November 2015 (UTC)


 * Great! I think spelling out the month name, when it's only the month and year, is good - don't you agree? That's the usual way it's written out. As for the three failing syntaxes, are there any that you think are a problem? Yaron Koren (talk) 00:47, 18 November 2015 (UTC)
 * The problem with the wrong year left (the master version is only shifted):


 * And about long forms. It is very good for 'View table', but very inconvinient for templates etc. --StasR (talk) 07:36, 18 November 2015 (UTC)


 * Oops! I forgot to add in that same fix for year/month-only dates. I just checked in what I think is a fix. As for the month formatting: if you want a specific format for the date, I would recommend using the DATE_FORMAT function within the query, as opposed to doing formatting within a template. Wouldn't that work? Yaron Koren (talk) 14:39, 18 November 2015 (UTC)


 * Yes, I will use the DATE_FORMAT function. Thanks for the help! --StasR (talk) 14:57, 18 November 2015 (UTC)

Update to 0.10 version - make my sql request on error ( function SUBSTRING_INDEX)
Hello Yaron, I'm back, still using your Cargo ! I tried to update my cargo to the last 0.10 version, but I have this error message : Error: the SQL function "SUBSTRING_INDEX" is not allowed. Allmost all my queries are concerning by this pb ... here one exemple of such a query : Guillaume Prêcheur (talk)


 * Hi Guillaume! Yes - the variable $wgCargoAllowedSQLFunctions was added, so you just need to add that function to it; see here. Yaron Koren (talk) 14:47, 18 November 2015 (UTC)
 * I try to add this setting wgCargoAllowedSQLFunctions line after the Cargo inclusion, but still have an SQL error (Error: the SQL function "COMPETENCE:',COMPETENCE,'|',SUBSTRING_INDEX" is not allowed.) for the same request - with this settings :

require_once( "$IP/extensions/Cargo.0.10/Cargo.php" ); $wgCargoAllowedSQLFunctions[] = 'SUBSTRING_INDEX'; Am I wrong again somewhere ?--Guillaume Prêcheur (talk) 16:26, 18 November 2015 (UTC)


 * I think you're running into a bug that I may have fixed last week; I would recommend getting the latest version of the code, if you're not using that now. Yaron Koren (talk) 16:35, 18 November 2015 (UTC)
 * I'm using the downloaded zip one. I'll try with the git version.--Guillaume Prêcheur (talk)
 * I tried with the master branch of the git sources. Same problem :-(. --Guillaume Prêcheur (talk)


 * Alright. What is the exact query you're calling, that leads to this error? I assume it's not the one from above. Yaron Koren (talk) 15:27, 19 November 2015 (UTC)
 * The query is the same that the one above (I add some  tags to display it completly !) --Guillaume Prêcheur (talk)


 * Ah, now it makes sense. And that's very interesting - it looks like your query had a rather specific combination of punctuation that Cargo's parsing failed on. I just checked in what I think is a fix - if you update your code, hopefully it will work correctly. Yaron Koren (talk) 14:19, 20 November 2015 (UTC)


 * Ok the first query is now working. But this next one need the declaration of CHAR

If I simply 2 lines It doesn'work (CHAR is not allowed) I tried something like : It's still doesn't work - what'is the right syntaxe to add 2 functions ? --Guillaume Prêcheur (talk)
 * 1) $wgCargoAllowedSQLFunctions[] =  'CHAR'
 * 2) $wgCargoAllowedSQLFunctions[] =  'INDEX_SUBSTRING'
 * 1) $wgCargoAllowedSQLFunctions[] =  array('CHAR','INDEX_SUBSTRING');


 * Ah - you've found another bug. Sorry about that. This one only happens with recent versions of MediaWiki (1.25.3 or higher). If you get the latest version of Cargo, the problem should go away - and then you should use the first of those approaches. Yaron Koren (talk) 17:50, 20 November 2015 (UTC)


 * Yep - It works ! You fixed it ! thanks - --Guillaume Prêcheur (talk)

Changing lots of content at once
If I were to have a table with "Content" (wikitext) and "Accurate as at" (date) for the columns, is there an easy way to change the dates of multiple pages/rows? For instance, for periodic checks that the content is still accurate, so that readers can be assured that the content is up to date. I imagine that Extension:Replace Text might work, but is there another way? Thanks Jonathan3 (talk) 15:45, 18 November 2015 (UTC)


 * I would think Replace Text would be the easiest approach. (I don't know how useful that "Accurate as at" field actually would be, but that's another story...) Yaron Koren (talk) 15:58, 18 November 2015 (UTC)


 * Thanks. I'm interested in your views on the date field... Jonathan3 (talk) 21:42, 18 November 2015 (UTC)


 * Well, how informative is that field if it's just going to be updated en masse? It doesn't inspire a lot of confidence that anyone is actually checking the accuracy of each page. Yaron Koren (talk) 22:15, 18 November 2015 (UTC)


 * I see what you mean, but: (a) it boils down to how much people trust the site already; (b) in all likelihood I'll be checking multiple pages each time rather than all pages en masse, just becuase of time constraints, which would lessen your concern; (c) I think Cargo will be useful because I will be able to (for my own purposes) list pages by last-checked-date to prioritise future checks. Jonathan3 (talk) 22:24, 18 November 2015 (UTC)

Changing structure of database once created
I guess this may have the same answer as my other question (about changing lots of data at once) - when you have created your data structure, template, form, etc, is there an easy way, for example, to add a column or change a column name, without having manually to edit all the pages? Thanks Jonathan3 (talk) 15:48, 18 November 2015 (UTC)


 * Sure - just change the template and form, and recreate the Cargo table. The pages that call that template don't need to be changed, although if a field name changes, that's probably a good idea. Yaron Koren (talk) 17:10, 18 November 2015 (UTC)


 * I guess that if a field name changes, the pages calling the relevant template would all need to be changed, as otherwise the data wouldn't end up in the Cargo table - is that right? Looking at Replace Text, it looks like you would just use the "Replace only in category" filter to change only the relevant pages. Jonathan3 (talk) 22:29, 18 November 2015 (UTC)


 * That's right. Yaron Koren (talk) 23:09, 18 November 2015 (UTC)

DATE_FORMAT problem
has type Text,  is the same text, but has type Date.

Result is: --StasR (talk) 09:02, 21 November 2015 (UTC)


 * Yes, you found another bug - sorry about that. I'm guessing you were the first person to try using DATE_FORMAT. I just fixed this in the code, I think. Yaron Koren (talk) 15:08, 22 November 2015 (UTC)

Request for new function : Template label for bar chart format output
Hello Yaron, I would like to be able to display bar chart labels throw templates to create links to pages for instance.

The image above corresponding of what I dreamed about.

--Guillaume Prêcheur (talk)


 * You may recall that you asked for something similar a few months ago. Unfortunately, the answer is still the same: links would be difficult to implement within the bar chart display. Yaron Koren (talk) 18:42, 25 November 2015 (UTC)

Request for new function : SQL function processing and template evaluation for exhibit format output
Hell Yaron (again) I'm trying the exhibit format queries output to help my users navigating into my skill database.

Sql functions
I'm using sql string formating functions into the field definition of my queries to format the labels. Example : ... fields=CONCAT(,SUBSTRING_INDEX(MagDBCompetenceCollaborateur.Competence,'/',-1),)=Compétences ) ...



The exhibit output doesn't take in account the functions into the selection views. In the image above the string before the "\" should not be desplayed (ok for the and  )

template in the result tab


The exhibit output doesn't evaluate wiki syntaxe into the result view : In the image above :
 * column Label : the string before the "\" should not be desplayed and the label should be displayed as a link
 * column Collaborateur: the template is not evaluated.

--Guillaume Prêcheur (talk)