Extension talk:Cargo

Referential integrity?
Does Cargo enable me to restrict the contents of a column to the values contained in another table?

The desired end result is a form where the values in that column are not free-entry but rather selected from a drop down list that is populated from another table.


 * This is really a Page Forms question, but yes - in the form definition, you can have something like " ". Yaron Koren (talk) 02:05, 14 November 2016 (UTC)

Cargo Tables don't work after restoring MW database
We had a scenario where we needed to restore a database backup. Between the time the dump was made and restored, Cargo tables had been created so there were a few cargo__XXX tables not properly accounted for. After restoring the old database, attempts from within MediaWiki to recreate the Cargo table (ie: from the Template page) or to remove the Cargo table (from the special page) would both fail with errors. We had to drop the extra cargo_XXX tables from our database manually before proceeding.

This isn't really a "bug", since it was our fault the data was out of sync. However, it would be nice if the Delete Cargo Table special page or Recreate Data action were smart enough to assist with this (probably uncommon) scenario. Otherwise posting here in case others run into this error.


 * Alright, this is good to know. And sorry about the problem. I believe if you had recreated the data using the command-line cargoRecreateData.php script, it would have worked correctly, for what it's worth. Yaron Koren (talk) 00:51, 20 October 2016 (UTC)

Database error when saving/deleting templates/categories
I have a fresh install of MediaWiki 1.27 on MySQL 10.1.18-MariaDB-cll-lve server using PHP 5.6. I installed Cargo and Semantic Forms extensions and tried to set up a database structure. When I try to create/save/update templates or categories, I get the following error: "Database error: A database query error has occurred. This may indicate a bug in the software.". I have identified that it is caused by Cargo extension - when I disable Cargo extension, the error is not generated. Domain is http://everyhotsauce.com

Any ideas?


 * Please see the first item here. Yaron Koren (talk) 16:17, 19 October 2016 (UTC)


 * Yoren, thanks for the quick reply. I realized I missed the step in the installation to run the PHP update script. Looks to be all good now. --Hotsaucerer (talk) 17:32, 19 October 2016 (UTC)

Problems with list fields
Hi

I'm using cargo for some time now and I accumulated some problems with list fields:
 * 1) Recreating cargo tables produces php notices Undefined index in Cargo/parserfunctions/CargoStore.php on line 293 and 315 (not really a problem, since error reporting is only activated on my dev system where I noticed this)
 * 2) Having a field definition in Semantic Forms with cargo table and cargo field set to point to a list field produces a "Table does not exist" when accessing the form for the first time (aka: the table is emtpy). One has to remove cargo table and cargo field, create at least one entry and then they can be added again. Methinks that has to do something with type ahead suggestions (propagation of the autocomplete field).
 * 3) Most anoying: I have some list fields of type page whose *__full field exceeds the 300 characters leaving me with one of two options: I can cargo query normally for the field in the table, knowing very well that the result will be cut [0] or I can create a join and then loose the linking to the pages having to reintroduce it manully in the query [1]. Note: size in the #cargo_store definition does neither apply to page fields in generell nor to the *__full field of the list. Furthermore I have a bot gathering some data using a generic function which has no way of knowing, whether it queries a list field or a normal field making it impossible to use the join workaround.

I would appreciate any input on this...

[0] Normal query

[1] Join query

On system (amongst others): MW 1.23.14 Semantic Forms 3.7 Cargo 1.0.1

Tobias (talk) 12:18, 1 July 2016 (UTC)
 * Just noticed, calling index.php?title=Group:ToManyMembers&action=pagevalues yields the same unwanted trimmed result, indicating this also accesses the *__full field. :(
 * Tobias (talk) 10:00, 5 July 2016 (UTC)


 * Hi, sorry for the delay. For #2 - what version of SF are you using? (It's not 3.7, because that hasn't come out yet.) For #3, I just checked in a small change that I think fixes this issue - I didn't fully test it. Yaron Koren (talk) 04:38, 7 July 2016 (UTC)


 * I'm sorry, you are right. I'm on SF 3.6 (0dae7d5) 16:28, 4. Mai 2016. Thanks for responding. I'll try your fix asap. Tobias (talk) 05:56, 7 July 2016 (UTC)

Not getting anything but page name
I've got queries that were working before I updated to mediawiki 1.27 and the latest version of Cargo via git, but now they only return the page name. None of the other fields in the query return anything, but when I go the page and look at the page values the values are there, and stay there after a purge.

--Cody3647 (talk) 17:00, 14 July 2016 (UTC)
 * Not working
 * Working


 * Very strange! I haven't seen this before. It looks like the issue is with the display of any field that has an alias - whether it's "Page" instead of "_pageName" or "npc permission" instead of "npc_permission", etc. Beyond that, I don't know. Could you try to narrow down whether it's due to the difference in the MediaWiki vs. the Cargo versions? Yaron Koren (talk) 18:08, 14 July 2016 (UTC)


 * I've updated Cargo on the mediawiki 1.26 installation via git and everything seems to be working, though the version listed in history hasn't updated. --Cody3647 (talk) 18:54, 14 July 2016 (UTC)


 * Okay - it must be the MediaWiki version, then. My guess is that the issue is with extension.json, which is only called for MW 1.27 and higher. Could you try commenting out the extension.json stuff, at the top of Cargo.php, in the non-working wiki, and see if that fixes the problem? Yaron Koren (talk) 20:28, 14 July 2016 (UTC)


 * I changed the version compare to 1.28, and the 1.27 install is still not working. --Cody3647 (talk) 22:11, 14 July 2016 (UTC)


 * Oh well. I guess there are three options: you can try to figure out the problem yourself, you can give me login access on your server and I can try to figure it out, or I can upgrade my own wiki to 1.27 and see what happens. (I do plan to upgrade my wiki anyway, but I didn't have any specific timeframe for that.) Yaron Koren (talk) 00:14, 15 July 2016 (UTC)


 * Looking at the sql query that is being passed in the $res object in CargoSQLQuery::run, its getting extra ` around aliases.  .  I commented out   in CargoGalleryFormat::display fixes it, but instead of adding it to the stylesheet like a normal wiki gallery does, it adds it in a style block in the head.  Only a problem because it then overwrites some changes I'd made to the gallery css through the skin. --Cody3647 (talk) 00:09, 16 July 2016 (UTC)


 * Did you really report this problem before? I don't remember hearing about it before. But yes, you're right - the 'gallery' format was only displaying correctly when there was no page caching. I just checked in a change that disables the page cache when the gallery format is called, which is my go-to fix for cases like these. Hopefully that solves this problem. Yaron Koren (talk) 01:30, 18 July 2016 (UTC)


 * I took a look again, and I'm not sure the cache needs to be disabled. The following two lines, before I was using only one, seem to work and adds it into the external stylesheet loaded on page load rather than the dynamic style block added after page load, so no jumping from list to gallery anymore whether just refreshed or purged.  Also, just FYI: Missing Gallery Module for Packed-hover gallery
 * --Cody3647 (talk) 02:27, 18 July 2016 (UTC)


 * Ah! That sounds familiar; although I didn't think it was the same issue. Anyway - isn't my solution superior, in that it doesn't overwrite any CSS? Or does it not really matter? Or does my solution not work for you? Yaron Koren (talk) 02:33, 18 July 2016 (UTC)


 * I don't think it overwrites anything, just adds the module into the stylesheet like it would if you just used the gallery tag. I'm not opposed to turning off the cache, I just know that its generally useful?  I know I have a number of pages with multiple queries, several galleries and others, and just figure that its better to have those cached rather than not.  --Cody3647 (talk) 02:37, 18 July 2016 (UTC)


 * Alright - if there's no overwriting, then yours is indeed the better solution. I just checked it in. Thanks! Yaron Koren (talk) 14:52, 18 July 2016 (UTC)

List of URL data type leads to SQL error
I have a template with a Cargo declaration of the form:

When trying to view the resulting table, I get an SQL syntax error (1064) reported in CargoSQLQuery::run. The reported query is as follows: SELECT `_pageName` AS `Page`,CONCAT('[', Field1, ' URL]') AS `Field1`,CONCAT('[', Field2, ' URL]')__full AS `Field2`,`Field3` AS `Field3`  FROM `cargo__Tests`    ORDER BY `_pageName` LIMIT 100 The problem seems to be with the concatenation of the parts of Field2 call. This is with Cargo 1.0.1 (5ed9912) on MediaWiki 1.27.0 and MariaDB 10.0.25. -- Paul T (talk) 20:28, 1 August 2016 (UTC)


 * Sorry for the delay. Yes, this was a bug - I just checked in a fix for it. Yaron Koren (talk) 17:00, 4 August 2016 (UTC)


 * Thank you. That works! Paul T (talk) 08:52, 5 August 2016 (UTC)

Undefined constants when specifying additional fields for _pageData table
Following the instructions in on the "Storing page data" part of the Storing data page, I have added $wgCargoPageDataColumns[] = CARGO_STORE_CREATION_DATE; to LocalSettings.php, below Cargo's require_once call. When trying to create the table, I get the following: $ php setCargoPageData.php PHP Notice: Use of undefined constant CARGO_STORE_CREATION_DATE - assumed 'CARGO_STORE_CREATION_DATE' in /var/www/html/w/LocalSettings.php on line NNN The same happens for the other constants. The _pageData table is successfully created, but it doesn't include any of the optional fields.

Have I missed some subtlety, or is this a bug? Many thanks! Paul T (talk) 10:52, 10 August 2016 (UTC)


 * Ah, it looks like the handling for the page data constants with extension.json - which is used with MediaWiki 1.27 and higher - doesn't work. I just changed it to a setup that's probably better in all cases. The constant names have changed - you should replace "CARGO_STORE_" with "CargoPageData::" in each of the names, so it would look like:

$wgCargoPageDataColumns[] = CargoPageData::CREATION_DATE;
 * If you get the latest code, please try this out and let me know if it works for you. Yaron Koren (talk) 13:52, 10 August 2016 (UTC)
 * Thanks. With the latest code and that change, I'm getting this: PHP Fatal error: Class 'CargoPageData' not found in /var/www/html/w/LocalSettings.php on line NNN
 * Paul T (talk) 15:16, 10 August 2016 (UTC)
 * That's not good. Are you calling Cargo in LocalSettings.php with wfLoadExtension, by any chance? Also, what version of MediaWiki are you running? Yaron Koren (talk) 02:01, 11 August 2016 (UTC)
 * No, it's: require_once( "$IP/extensions/Cargo/Cargo.php" );
 * ... which has been working fine in all other respects. The attempt to set $wgCargoPageDataColumns[] is immediately below that. This is Cargo 1.0.1 (e0dbafe) with MediaWiki 1.27.0 on PHP 5.6.24 on Debian. Paul T (talk) 08:29, 11 August 2016 (UTC)


 * Okay, I finally tested this out directly. It turns out that constants unfortunately can't be used with extension.json at all; so I had to change the settings to be strings instead. So now the relevant call is instead: $wgCargoPageDataColumns[] = 'creationDate'; ...and the other possible values are 'modificationDate', 'creator', 'fullText', 'categories' and 'numRevisions'. Sorry about all the problems; hopefully this works for you now. Yaron Koren (talk) 18:00, 12 August 2016 (UTC)


 * That works -- thank you. Paul T (talk) 18:43, 15 August 2016 (UTC)

Hiding fields from drilldown but not from table view
I have a minor feature suggestion. The "hidden" parameter in a field definition allows a field to be hidden from drilldown. It would be good to be able to do that without also hiding it from the Special:CargoTables view. The particular use case I have in mind is for fields containing filenames: it's good to be able to see that they're there in the table view, but they're of no use in a drilldown (and just add clutter) because they're unique. This could be done by adding another keyword, or perhaps it might just make sense to exclude file fields from drilldowns by default? Paul T (talk) 19:02, 15 August 2016 (UTC)


 * Good point - the latter makes the most sense. I just added "File" to the list of types not handled by Special:Drilldown. Yaron Koren (talk) 15:52, 16 August 2016 (UTC)


 * Thank you - that's much neater! Paul T (talk) 21:57, 16 August 2016 (UTC)

Duplicated Records
I'm having a problem with mysteriously duplicated records in a couple of tables. I've read in the archives that a few people probably had the same problem. It might be due to me having used the recreate table functionality, but I thought I did it for only one of the two tables. What's the current wisdom to eliminate or minimize the issue? -- manu3d (talk) 22:02, 15 August 2016 (UTC)


 * The most foolproof way seems to be to call the "cargoRecreateData.php" script - once you're sure that there are no more "recreate data" jobs still waiting to be run. (You can call MediaWiki's own runJobs.php script to be sure.) Yaron Koren (talk) 15:54, 16 August 2016 (UTC)


 * We are having a similar problem: ending up with duplicated records after editing pages using forms. This particularly seems to happen when there are list fields involved. I think I may have tracked down the cause. When making edits that cause these duplicates, these errors appear in Apache's error.log:

PHP Notice: unserialize: Error at offset 193 of 200 bytes in /var/www/html/w/extensions/Cargo/Cargo.hooks.php on line 128, referer: [site url]/w/index.php?title=[page title]]&action=formedit PHP Warning: Invalid argument supplied for foreach in /var/www/html/w/extensions/Cargo/Cargo.hooks.php on line 129, referer: [site url]/w/index.php?title=[page title]&action=formedit
 * These are in the deletePageFromSystem function that is called as the first step in the onPageContentSaveComplete hook, so I think they are causing (some of) the original records for the page not to be deleted, such that the new version then becomes a duplicate. The unserialize error seems to be caused by the contents of the field_tables field in the cargo_tables table being truncated to 200 characters by the database schema, so I suppose this problem will only show up on tables with several list fields. Paul T (talk) 07:03, 17 August 2016 (UTC)


 * Changing the type of field_tables from varchar(200) to text (and recreating everything) seems to fix the problem. The table on our site that was accumulating duplicates now has a field_tables value that is 337 characters long. This issue seems to be specific to MySQL/MariaDB, as /sql/Cargo.pg.sql already used a text type with unlimited length. Paul T (talk) 10:48, 17 August 2016 (UTC)

Page moves do not seem to work
I noticed that moving a page (i.e. changing its title) did not seem to lead to the Cargo tables being updated: the table still showed the old title. I can see that it is meant to be handled, using the onTitleMoveComplete hook. I think the problem may be related to this, which appeared in Apache's error.log: PHP Warning: Parameter 3 to CargoHooks::onTitleMoveComplete expected to be a reference, value given in /var/www/html/w/includes/Hooks.php on line 195, referer: [site url]/wiki/Special:MovePage/[original page name] This is with Cargo 1.1 (3e1b9f3) on MediaWiki 1.27.0. Paul T (talk) 06:23, 17 August 2016 (UTC)


 * Yikes! It looks like you're seeing this issue, which is a bug in either MediaWiki or HHVM. Are you using HHVM/HipHop by any chance? If not, what version of PHP are you using? Yaron Koren (talk) 12:29, 17 August 2016 (UTC)


 * No, it's nothing exotic: PHP 5.6.24-0+deb8u1 (apache2handler), which is the default for Debian. From the command line, it looks like this:

$ php -v PHP 5.6.24-0+deb8u1 (cli) (built: Jul 26 2016 08:17:07) Copyright (c) 1997-2016 The PHP Group Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2016, by Zend Technologies
 * Paul T (talk) 13:28, 17 August 2016 (UTC)


 * Alright. This seems to be a bug in MW 1.27; hopefully it'll get fixed soon. Yaron Koren (talk) 18:24, 23 August 2016 (UTC)


 * It looks like this bug might be with us for a while. I just checked in a workaround to it in the Cargo code, that seems like it will work with all MediaWiki versions. Thankfully it was a pretty easy fix. Yaron Koren (talk) 18:47, 13 September 2016 (UTC)

Unnecessary error when a field value in a where clause contains the name of a virtual field
When one of the values passed in a where clause of cargo_query call happens to contain the name of a list field, an error is thrown incorrectly.

Here's a minimal test case that shows the problem. If you define a table as follows:

and run this query:

It results in this error: Error: operator for the virtual field 'Test.Field2' must be 'HOLDS', 'HOLDS NOT', 'HOLDS LIKE' or 'HOLDS NOT LIKE'.

Obviously, this is a contrived example. It causes a problem in practice, however, when a page title that's used as part of a query happens to contain the name of a list field.

This seems to be caused by the preg_match call on line 676 of CargoSQLQuery.php matching on the entire clause, rather than just the field names. Disabling that error by commenting out the test seems to make the query work. However, clearly there must be a better fix. This is with Cargo 1.0.1 (3f84330) or 1.1 (3e1b9f3). Paul T (talk) 13:58, 23 August 2016 (UTC)


 * That does seem like a bug... until there's a real solution, you could probably get around it with a hack like replacing that string with something like "CONCAT( 'A string that contains the text Fie', 'ld2 somewhere within it ')". Yaron Koren (talk) 18:10, 23 August 2016 (UTC)


 * Thanks. Alas, that doesn't work in our case, as the where clause in question is actually of the form " Field HOLDS ". The other workaround would be to change the field names to be less likely to crop up in the data, but for now I've just commented out that line in CargoSQLQuery.php, and we'll live without that error message. Paul T (talk) 18:38, 23 August 2016 (UTC)

Updating a field
Hi, I'd like to update a single field in a database table but when I do this it creates a complete table row with just the updated fields in it. How can I just update a single entry?


 * Sorry, I don't understand what exactly you want to do, or what happened when you tried it. Yaron Koren (talk) 22:09, 5 September 2016 (UTC)

Apologies, what I'd like to do is similar to this,

If I try to do it with;

Then it adds a second row into the table rather than updating the existing field on the existing row.


 * I'm still not quite sure I understand, but you should generally have only one #cargo_declare, and one #cargo_store, call within a template, except for special cases. Do you have more than one? If so, you should combine them into one, then recreate the table. Yaron Koren (talk) 15:30, 6 September 2016 (UTC)

Missing filter breadcrumb in drilldown when selecting from a combobox
We have noticed what looks like a bug in the Special:Drilldown. Normally, there is a trail of breadcrumbs at the top of the page, giving the fields and values on which you have filtered, each with a cross icon (filter-x.png) that you can click on to remove that filter.

However, this doesn't seem to work when you filter on a field that has a large number of values, triggering the use of a combobox, rather than an in-line list of values. The relevant fieldname still appears in the header, but no value and no cross icon. Looking at the URL, I can see that the problem arises when the filter filter appears in the form _search_FieldName[0]=Value, rather than simply FieldName=Value or FieldName[0]=Value.

This is problematic because (a) it's not clear to the user what they've already filtered on, and (b) there's no easy way for them to remove the filter. Increasing $wgCargoDrilldownMinValuesForComboBox to avoid the use of comboboxes is a temporary workaround, but will hurt usability, as some of the lists are very long.

This is on Cargo 1.1 (3e1b9f3). Paul T (talk) 16:41, 7 September 2016 (UTC)


 * Oops! I believe that's due to some typos entered into the code three months ago. Sorry about that. I just checked in what I think is a fix. Yaron Koren (talk) 18:06, 7 September 2016 (UTC)


 * Thank you -- yes, that fixes it. Paul T (talk) 18:48, 7 September 2016 (UTC)

Special:ViewData returning blank tables
The Special:ViewData page does not seem to be working for us. It seems to run the query correctly, in that it throws appropriate SQL errors if you get the field names wrong, and the numbers in the header are plausible, e.g. "Showing below up to 68 results in range #9 to #76." However, it doesn't actually display any data. If you ask for table format, you get a table with the appropriate rows and columns, but with all value cells blank. For other formats, you get a string of commas, or empty bullet points. This occurs both for a call resulting from the "More..." link below a properly working cargo_query call, and for simple, manually constructed queries on a test table. This is on Cargo versions 1.1 (f3599cf) or (3e1b9f3), on MediaWiki 1.27.0 on PHP 5.6.24 (on Apache) and MariaDB 10.0.26 or 5.5.47. Paul T (talk) 19:23, 7 September 2016 (UTC)


 * That's very strange. Does this always happen for one table, or can you reproduce the problem with any table? Yaron Koren (talk) 22:02, 7 September 2016 (UTC)


 * It seems to be any table, including a trivial test table. Paul T (talk) 09:19, 8 September 2016 (UTC)


 * I have no idea. This seems unlikely, but maybe check the HTML to see if the values are showing up but are somehow being rendered invisible? Yaron Koren (talk) 13:46, 8 September 2016 (UTC)


 * To continue publicly an offline discussion: this seems to be a compatibility problem with MediaWiki 1.27: From that version onwards, MediaWiki automatically quotes fieldnames; Cargo's quoting (which was necessary with earlier versions) leads to them being double-quoted and failing to match the expected names when formatting the result. Cargo code for most other queries already checks the MediaWiki version and only adds quotes on older versions, but the Special:ViewData page uses separate code that didn't have this test. I've sent Yaron a short patch that fixes it for me. Paul T (talk) 10:35, 12 September 2016 (UTC)


 * To confirm, the current git version of the Cargo code contains a patch from Yaron that fixes this. Thank you! Paul T (talk) 14:32, 15 September 2016 (UTC)


 * Thank you also! Yaron Koren (talk) 18:16, 15 September 2016 (UTC)

Change formatting of cells in dynamic table
I'd like the vertical align of these to be "top" rather than "middle". What should I change to make this work? Thanks in advance. Jonathan3 (talk) 19:46, 23 September 2016 (UTC)


 * Just answered my own question... add .cargoDynamicTable td {vertical-align:top !important} to MediaWiki:Common.css :-) Jonathan3 (talk) 19:54, 23 September 2016 (UTC)

Error: the SQL function "SUBSTRING_INDEX" is not allowed.
This would be useful for me. I've tried fields=SUBSTRING_INDEX(Description, '==', 1) to take the first part of the Description, up to the first heading, but get the error above. Is there a way round this?

I have to confess I'm still using version 0.10 (c930560). Thanks in advance. Jonathan3 (talk) 20:08, 23 September 2016 (UTC)


 * You just need to add it to the array $wgCargoAllowedSQLFunctions - see here. Yaron Koren (talk) 20:58, 23 September 2016 (UTC)


 * I've added $wgCargoAllowedSQLFunctions[] = 'SUBSTRING_INDEX'; in the line after require_once( "$IP/extensions/Cargo/Cargo.php" ); but still get the error message. I get a similar message for LEFT even if I add it similarly. What am I doing wrong? Thanks Jonathan3 (talk) 13:37, 26 September 2016 (UTC)


 * I don't know, that's very strange... maybe try upgrading Cargo? Also, what version of MediaWiki are you using? Yaron Koren (talk) 14:14, 26 September 2016 (UTC)


 * I'm afraid I'm still using MW 1.25.3. I'll upgrade it as soon as I can, and will update this page to let you know if the problem disappears. Thanks. Jonathan3 (talk) 14:33, 26 September 2016 (UTC)

Changing column names
I have a table with a "Description" column, but would like to (a) change "Description" into "Summary", and (b) add a "Detail" column next to it. How do I change this? Incidentally, I have other tables with "Description" columns too. Thanks in advance. Jonathan3 (talk) 13:42, 26 September 2016 (UTC)


 * Change the template (including the #cargo_declare and #cargo_store calls it), then go to the "Recreate data" tab at that template page and hit the button. Yaron Koren (talk) 14:09, 26 September 2016 (UTC)


 * Cheers. What will this do with those existing pages with text under the "Description" heading? Will that text be kept and assigned to one of the new columns? Jonathan3 (talk) 14:42, 26 September 2016 (UTC)


 * No, those will all have to be modified as well. If there are a lot of such pages, the Replace Text extension may be useful. Yaron Koren (talk) 19:31, 26 September 2016 (UTC)

Form for user to search data
Would it be possible to add a form for the user to search data? I guess I mean something like a customised version of Special:ViewData. Thanks. Jonathan3 (talk) 22:50, 30 September 2016 (UTC)


 * You can use the Semantic Forms Special:RunQuery feature for that. Yaron Koren (talk) 17:54, 2 October 2016 (UTC)

Cargo & SMW, 1.27.1
Mediawiki 1.27.1, SMW 2.4.1, Cargo master.

When Cargo is on, on Special:AllPages lost all links related to SMW :-( --StasR (talk) 14:37, 8 October 2016 (UTC)


 * Yes, that's a new SMW "feature" - see here. Yaron Koren (talk) 13:07, 9 October 2016 (UTC)


 * Thanks! --StasR (talk) 18:53, 9 October 2016 (UTC)

setCargoPageData.php terminating with error 1214
Hi Yaron

I was adding some of the optional fields to the _pageData table following the documented instructions when it terminated with the following error:

Error: 1214 The used table type doesn't support FULLTEXT indexes (localhost)

This occurs because I am using MySQL v 5.5 which needs "ENGINE=MyISAM" on the create table instruction to allow FULLTEXT indices.

I had a look at CargoUtils.php which has code to trap for this situation but the code wasn't executing because the global $wgCargoDBtype is null. If I explicitly set $wgCargoDBtype in the public static function getDB to be $dbType after that variable is set then the create table function works as expected.

I wondered if I have misunderstood something which has resulted in $wgCargoDBtype being null when running setCargoPageData.php?

Many thanks, Duncan 14 Oct 2016


 * Sorry about that! That was a bug in Cargo; I think I just fixed it. Yaron Koren (talk) 14:45, 14 October 2016 (UTC)

Installation fails with git version and MW 1.25.5
PHP Fatal error: Class 'Database' not found in /var/www/html/mediawiki-1.25.5/extensions/Cargo/CargoUtils.php on line 56


 * Oops! Someone else made a change to the code about a month ago that removed compatibility for MW versions before 1.27. I think I just fixed it... if you get the latest code, please let me know if you still see a problem. Yaron Koren (talk) 21:06, 30 October 2016 (UTC)

Formatting of floating-point numbers
Hi Yaron,

We've come across a bit of an oddity when displaying the results of queries that contain float values that happen to be integers: the number of decimal places shown in the formatted number depends on the length of the number. For example, 6 is formatted correctly as "6", whereas 24 becomes "24.0", 100 becomes "100.00", and 1000 becomes "1,000.000".

I think the culprit is line 168 of CargoQueryDisplay:

$numDecimalPlaces = strlen( $value ) - strrpos( $value, '.' ) - 1

... which needs a special case for when '.' isn't found.

On a related note, it would be nice to be able to right-align numerical columns by default. The CSS classes based on the field names make this possible, but you have to handle each numerical field separately. If you could use CSS classes to indicate the data type, this would be easier. Also, I can't see a way to do this with dynamic tables, as there don't seem to be any classes defined there. Many thanks. Paul T (talk) 20:00, 1 November 2016 (UTC)


 * That's true... what do you think a good default number of decimal places would be - 0? 1? The CSS is also a pretty good idea. I don't think it's possible within dynamic tables, i.e. the DataTables JS library, though, unfortunately. Although numbers in dynamic tables are right-aligned automatically, no? Yaron Koren (talk) 03:19, 2 November 2016 (UTC)


 * Apologies for the slow response. The patch you've done to avoid adding decimals works nicely. I suspect use cases for floats vary so much that there's no sensible default for the number of decimal places. A nice feature (but not one that we really need) would be to be able to specify a default display format in the Cargo table definition (in a similar manner to how the "link text" and "hidden" attributes are dealt with).


 * On the CSS issue, I see that the DataTables javascript library does have a "columns.className" option. It also has an order option that it might be handy to expose: at present, it seems to sort the data by the first field, undoing any ordering specified in the cargo_query call. Paul T (talk) 13:15, 16 November 2016 (UTC)

Sparse matrix
Is it possible to implement a list field with named elements to store a sparse matrix? May be, is there any other way? --StasR (talk) 13:39, 2 November 2016 (UTC)


 * Yes - what's the extra challenge with a sparse matrix? Yaron Koren (talk) 13:52, 2 November 2016 (UTC)


 * For each row of the matrix need to store a plurality of pairs of "index (key) - value" and I need to be able to get the value for the key. Of course, I can myself organize a separate table, but it will require a lot of calls cargo_store. --StasR (talk) 14:13, 2 November 2016 (UTC)


 * Oh, I see - you want to store a sparse matrix within each page. If so, what about a multiple-instance template, where the template contains "key" and "value" fields? Yaron Koren (talk) 14:27, 2 November 2016 (UTC)


 * The tempate call looks like this: . It is logical to convert it into a function  . And you save _rowID, _key, _value in the separate table. --StasR (talk) 15:29, 2 November 2016 (UTC)


 * Alright. Is your problem solved now? Yaron Koren (talk) 16:09, 2 November 2016 (UTC)


 * Now I use an separate no-sparse table. It is useful for experiments, but it is bad for real work (especially because the number of columns will grow with time). --StasR (talk) 16:25, 2 November 2016 (UTC)


 * If so, what about a multiple-instance template, where the template contains "key" and "value" fields? — Perhaps I did not understand exactly what you advise :-( --StasR (talk) 16:31, 2 November 2016 (UTC)


 * Do you know what multiple-instance templates are? Yaron Koren (talk) 17:09, 2 November 2016 (UTC)


 * I found it in PageForms description. If I understand correctly, this is multiple cargo_store calls. This is true? --StasR (talk) 17:54, 2 November 2016 (UTC)


 * Yes. Yaron Koren (talk) 18:37, 2 November 2016 (UTC)


 * It is too prodigally for my task. I fear timeout. --StasR (talk) 18:43, 2 November 2016 (UTC)

How many key/value pairs could a single page have? I thought you said it was "sparse". :) Yaron Koren (talk) 03:12, 3 November 2016 (UTC)


 * One page is shaping up to 200-300 main table records. Each main table record usually includes 10-20 pairs. --StasR (talk) 07:42, 3 November 2016 (UTC)


 * Okay, now I don't understand again... a page holds main table records, which themselves hold key/value pairs? That sounds like you basically want to store a three-dimensional array of data on every page. What's a main table record? Yaron Koren (talk) 13:10, 3 November 2016 (UTC)

Fatal_exception_of_type_MWException_on_RunQuery_Template
Hi, was there any resolution on this issue?

Fatal_exception_of_type_MWException_on_RunQuery_Template

We are finding the same problem - CONCAT is breaking when used in a query form, but works otherwise. We're using Semantic Forms 3.7 and Cargo 1.0.1. --Bgrenon (talk) 19:12, 4 November 2016 (UTC)


 * Try upgrading Cargo. You might as well upgrade Semantic Forms while you're at it, i.e. switch to Page Forms. Yaron Koren (talk) 20:23, 4 November 2016 (UTC)


 * Upgrading Cargo seemed to work. (Haven't tried upgrading to Page forms yet).  Thanks! --Bgrenon (talk) 10:44, 10 November 2016 (UTC)

Problem with the Cargo Query Calendar format
Hi

I've been trying to get this to work with Cargo 1.1.1 and MW1.26.4 and encountered a few problems.

The page call I'm using is

Firstly CargoExport was failing with:

A database query error has occurred. This may indicate a bug in the software. Query: SELECT `_pageName`,`Event_Start_Date_Time` AS `Event Start Date  Time`,`Event_Start_Date_Time__precision` AS `Event Start Date Time__precision` FROM `cargo__Articles` WHERE DATEDIFF(Event_Start_Date_Time,NOW) >= -30 AND ((Event_Start_Date_Time >= '2016-10-30' AND Event_Start_Date_Time <= '2016-12-11)) ORDER BY `_pageName` LIMIT 100 Function: CargoSQLQuery::run Error: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''2016-12-11)) ORDER BY `_pageName` LIMIT 100' at line 1 (localhost)

I managed to fix this in function displayCalendarData by adding an apostrophe before the final ')' on the following line:

$where .= "($dateField >= '$datesLowerLimit' AND $dateField <= '$datesUpperLimit)";

The calendar was still not displaying events and gave the following warning:

Notice: Undefined index: Event_Start_Date_Time__precision in /var/www/vhosts/cmswiki.farm/httpdocs/x/mw-1.26/extensions/Cargo/specials/CargoExport.php on line 131

I managed to fix this by removing the str_replace clause from the following line in the same function:

$startDatePrecisionField = str_replace( ' ', '_', $startDateField ). '__precision';

I am also having problems getting the Calendar Format to recognise the color parameter although I've yet to work out this one.

Finally I have tried to use an alternative Cargo field for the calendar event title by using the following field parameter.

fields=_pageName=Title,Description,Event_Start_Date_Time

This works if I use the default table format but not within the Calendar format. I'd like to do this because I'm using Semantic Title where the Cargo field Title is the semantic title for the page.

I wondered if I might have been doing something wrong to cause the above issues or whether the fixes I've made are OK?

Many thanks

Duncan, 11 Nov 2016

Hi

I've now had a look at the problem displaying colours in the calendar format and found a possible fix for that too. In CargoCalendarFormat.php function queryAndDisplay there is a block of code:

if ( $querySpecificParams != null ) { if ( array_key_exists( 'color', $querySpecificParams[$i] ) ) { $queryParams['color'][] = $querySpecificParams[$i]['color']; } else { // Stick an empty value in there, to					// preserve the order for the queries // that do contain a color. $queryParams['color'][] = 'red'; }

If this is changed to the following then colours display OK:

if ( $displayParams != null ) { if ( array_key_exists( 'color', $displayParams ) ) { $queryParams['color'][] = $displayParams['color']; } else { // Stick an empty value in there, to					// preserve the order for the queries // that do contain a color. $queryParams['color'][] = ''; }

As before I'm not really sure I understand this code fully so this change may mess up something else. Would you advise?

Kind regards

Duncan 12 Nov 2016


 * Sorry about all the problems you ran into!
 * The missing-parenthesis problem: I actually discovered this bug about a week ago, and checked in a fix for it then (it was the same fix you found).
 * Problem with underscores in date fields: thanks for that fix; I'll probably check it in soon.
 * Alternative title field: there's a way to do this, and it's actually mentioned in the documentation, though it's easy to miss. Instead of an "=Title" alias, I think you just need to set the alias to be "=name".
 * The multiple-colors problem: I have no idea where that 'red' value came from; it's not now, and doesn't look like it ever was, in the Cargo code. Could it be that you changed that line yourself during testing? Anyway, this, at least, looks like a non-bug. Yaron Koren (talk) 02:22, 14 November 2016 (UTC)


 * Hi Yaron


 * Thank you for your responses and fixes. You do a fantastic job producing all this great code, so I'm glad if I can help identify the occasional minor problems. On the colours question, you are right the 'red' was part of my testing trying to track down how colors were being set, although I am not sure this effects the problem I was encountering.  What my testing seemed to show is that the colour parameter is being passed to the function as part of the $displayParams variable rather than in the $querySpecificParams array variable.  As a result, the following line never found the colour parameter being passed from the cargo query:

if ( array_key_exists( 'color', $querySpecificParams[$i] ) ) { $queryParams['color'][] = $querySpecificParams[$i]['color']; } else ...


 * When I changed that line to the following it seems to pick up the colour OK:

if ( array_key_exists( 'color', $displayParams ) ) { $queryParams['color'][] = $displayParams['color']; } else ...


 * Is it possible I've inadvertently affected how the color parameter is passed to the queryAndDisplay function?.


 * Many thanks


 * Duncan 15 Nov 2016


 * Oh, I didn't notice the rest of the code changes. Could it be that you're putting the 'color' parameter in the wrong place in the #cargo_compound_query call? It should be within each "sub-query". Yaron Koren (talk) 14:39, 16 November 2016 (UTC)


 * Ah now I see where I may be going wrong. Perhaps if I explain what I'm trying to do maybe you will tell me if it is possible?


 * What I'd like is to display pages from an Articles table on the calendar. Each Article has an Event Type field that looks up on a separate Event Types table.  Each Event Type has a Colour field associated with it.


 * By joining the Articles and Event Types table, I hope to find the colour associated with each and then have the Calendar format use this to select the background colour for each calendar entry. Originally I assumed I wouldn't need to use Cargo's compound query which would leave me with a simple query such as:


 * From what you say above, the Calendar format requires you use Cargo's compound query to use the colour parameter. I have tried this using a literal value for the colour parameter and this does indeed work .  However, if I try to use a variable, such as Event_types.Colour which changes for each calendar entry, this doesn't seem to work. For example:


 * I wonder if what I am trying to do is possible and, if so, where I'm going wrong? I was able to do this when I used SMW as the data source but I know quite a lot changed in the query function for Cargo.  It looks like it might be possible if I do a compound query on the same table to select each event type and set its colour using a literal value, but this seems a bit inefficient and probably makes maintenance more complicated.


 * If it is not possible to use a field variable to set the background colour, is this a feature you might consider for the future? It would also be useful to send other parameters to the calendar for each event such as the end date and time so that it displays events that span several hours or days.


 * Many thanks


 * Duncan 17 Nov 2016


 * Ah, now it makes sense. No, you can't get the color value from the database. I know Semantic Result Formats has such a feature, with the "Has color" special property, but I decided against it for Cargo, because it seemed like the query was the more natural place to set that information. If you want the same page to be displayed with a different color in two different calendars, and the color information is set in the page itself, it's awkward (though possible). But I'm curious - in what way do you think setting the color info in the query is less efficient? Yaron Koren (talk) 14:53, 17 November 2016 (UTC)


 * Thanks Yaron, my thinking is this. Say I have two types of event - Show and Race.  I want the calendar events with type Show to have a green background and those of type Race to be red. I imagine (although this is just speculation as I haven't tried it yet) that the compound query would look like this?


 * The reason I supposed this to be inefficient was because I thought it may have to perform the join and then select the right type of articles twice, once for each parameter of the compound query. If it had been possible to pick up the colour for each calendar event from the join, then I had imagined I would only have to perform the join once and there would be no need to select a subset of the records for each event type. You've probably realised I'm a bit of a novice so apologies if I've go the wrong end of the stick but I'd be interested to know how this works.


 * Kind regards


 * Duncan 17 Nov 2016


 * Hi again. I now tested a version of the compound query that tests for the colour of the event type and sets the background colour as follows.  This does achieve the effect I want, although I don't really have enough data to tell about its efficiency:


 * I also thought about the use case that you gave whereby one might want to display the same events with different colours on different calendars. This might still be possible in the same way as it is now whilst still allowing the query to pick up a variable (database value) if one was set.  This might be done if the code could distinguish between whether a literal or variable database value was passed with the Color parameter.  If the Color parameter was a literal it could function as now and apply that value to each calendar event. On the other hand, if the Color parameter was a variable database value that might be used instead for each calendar event. That way one might achieve both use cases depending on the type of value passed in the Color parameter.


 * As mentioned I am out of my depth here but if you pointed me to the place where the programme sets the values for each of the calendar event entries I could see if I can get this working, if you think this might be a useful feature?


 * Many thanks, Duncan 17 Nov 2016


 * That's true, this approach of getting the color from the database would indeed be more efficient. And I think your first instinct was right, that this sort of thing should be handled via #cargo_query and not #cargo_compound_query - since it's not a compound query at all. The only thing I would change is that there should be a new parameter for this, probably named "color field=", to avoid confusion with "color=", which would keep only taking hardcoded colors. It does sound like a possibly useful feature, and if you want to have a go at creating a patch for it, feel free - I'll add it in to the code. Yaron Koren (talk) 20:02, 17 November 2016 (UTC)


 * Thanks Yaron. It's taken me a while to work this out but it's been an interesting journey.  I'm not sure I know how to create a patch so I've just included the code changes below.  I've tested them and they seem to work OK.  Also the Color parameter still seems to function as before.  However as mentioned I am a novice so you might want to check I haven't done anything daft!


 * I've also enabled cargo query with calendar format to display events that span more than one day correctly if an end date is supplied (see query example below). And to round things off I've enabled the text colour to be specified (also see example below).  This was necessary in case one used light background colours to ensure sufficient contrast for the text.


 * I'm hoping you will be able to adopt these changes so I can feel I've given somthing back to the Cargo community!


 * This is the updated version of the queryAndDisplay function in CargoCalendarFormat.php

function queryAndDisplay( $sqlQueries, $displayParams, $querySpecificParams = null ) { $this->mOutput->addModules( 'ext.cargo.calendar' ); $ce = SpecialPage::getTitleFor( 'CargoExport' ); $queryParams = $this->sqlQueriesToQueryParams( $sqlQueries ); $queryParams['format'] = 'fullcalendar'; $queryParams['color'] = array; foreach ( $sqlQueries as $i => $sqlQuery ) { if ( $querySpecificParams != null ) { if ( array_key_exists( 'color', $querySpecificParams[$i] ) ) { $queryParams['color'][] = $querySpecificParams[$i]['color']; } else { // Stick an empty value in there, to					// preserve the order for the queries // that do contain a color. $queryParams['color'][] = ''; }			}			if ( $displayParams != null ) { if ( array_key_exists( 'color field', $displayParams ) ) { $queryParams['color field'][] = $displayParams['color field']; } else { // Stick an empty value in there, to					// preserve the order for the queries // that do contain a color. $queryParams['color field'][] = ''; }			}			if ( $displayParams != null ) { if ( array_key_exists( 'text color', $displayParams ) ) { $queryParams['text color'][] = $displayParams['text color']; } else { // Stick an empty value in there, to					// preserve the order for the queries // that do contain a color. $queryParams['text color'][] = ''; }			}		}		if ( array_key_exists( 'width', $displayParams ) ) { $width = $displayParams['width']; // Add on "px", if no unit is defined. if ( is_numeric( $width ) ) { $width .= "px"; }		} else { $width = "100%"; }

$attrs = array(			'class' => 'cargoCalendar',			'dataurl' => $ce->getFullURL( $queryParams ),			'style' => "width: $width"		);

if ( array_key_exists( 'view', $displayParams ) ) { $view = $displayParams['view']; // Enable simpler view names. if ( $view == 'day' ) { $view = 'basicDay'; } elseif ( $view == 'week' ) { $view = 'basicWeek'; }			$attrs['startview'] = $view; } else { $attrs['startview'] = 'month'; }		if ( array_key_exists( 'start date', $displayParams ) ) { $attrs['startdate'] = $displayParams['start date']; }		$text = Html::rawElement( 'div', $attrs, '' );

return $text; }

}


 * This is the updated version of the displayCalendarDatafunction in CargoExport.php

/**	 * Used for calendar format */	function displayCalendarData( $sqlQueries ) { $req = $this->getRequest;

$colorArray = $req->getArray( 'color' ); $textColorArray = $req->getArray( 'text_color' ); $colorFieldArray = $req->getArray( 'color_field' );

$datesLowerLimit = $req->getVal( 'start' ); $datesUpperLimit = $req->getVal( 'end' );

$displayedArray = array; foreach ( $sqlQueries as $i => $sqlQuery ) { foreach( $sqlQuery->mFieldStringAliases as $fieldName => $fieldAlias ) { if ( $fieldName == $colorFieldArray[0] ) { $colorFieldName = $fieldAlias ; }			}

$dateFieldRealNames = array; $dateFieldAliases = array; foreach( $sqlQuery->mFieldDescriptions as $alias => $description ) { if ( $description->mType == 'Date' || $description->mType == 'Datetime' ) { $dateFieldAliases[] = $alias; $realFieldName = $sqlQuery->mAliasedFieldNames[$alias]; $dateFieldRealNames[] = $realFieldName; }			}			$where = $sqlQuery->mWhereStr; if ( $where != '' ) { $where .= " AND "; }			$where .= "(";			foreach ( $dateFieldRealNames as $j => $dateField ) {				if ( $j > 0 ) {					$where .= " OR ";				}				$where .= "($dateField >= '$datesLowerLimit' AND $dateField <= '$datesUpperLimit')";			}			$where .= ")"; $sqlQuery->mWhereStr = $where;

$queryResults = $sqlQuery->run; foreach ( $queryResults as $queryResult ) { if ( array_key_exists( 'name', $queryResult ) ) { $eventTitle = $queryResult['name']; } else { $eventTitle = reset( $queryResult ); }				$title = Title::newFromText( $queryResult['_pageName'] ); $startDateField = $dateFieldAliases[0]; $startDate = $queryResult[$startDateField]; $startDatePrecisionField = $startDateField. '__precision'; $startDatePrecision = $queryResult[$startDatePrecisionField]; if ( array_key_exists( $colorFieldName, $queryResult ) ) { $eventColor = $queryResult[$colorFieldName]; } else { $eventColor = $colorArray[$i]; }

$curEvent = array(					// Get first field for the title - not					// necessarily the page name.					'title' => $eventTitle,					'url' => $title->getLocalURL,					'start' => $queryResult[$dateFieldAliases[0]],					'end' => $queryResult[$dateFieldAliases[1]],					'color' => $eventColor,					'textColor' => $textColorArray[$i]				); if ( $startDatePrecision != CargoStore::DATE_AND_TIME ) { $curEvent['allDay'] = true; }

$displayedArray[] = $curEvent; }		}		print json_encode( $displayedArray ); }


 * This is an example of cargo query using the new capabilities


 * Many thanks


 * Duncan 18 Nov 2016

Oh wow, great! I'll add this in. One question: the "text color" parameter certainly seems like it could have its uses, but, if "color field" is going to be used, shouldn't there really be a corresponding "text color field" parameter? Yaron Koren (talk) 16:17, 18 November 2016 (UTC)