Extension talk:Cargo

Looking for a way around duplicate rows
I have the infamous duplicate rows bug. Fortunately, this isn't actually much of a problem most of the time, since I can just add "|group by=_pageID" to most Cargo DB queries and then it will just return distinct rows from the database, even if they are actually doubled in the database. However, I have not yet found a way around the duplicate row problem when I query an aggregate function like SUM or COUNT. In particular, I am trying to do something like this:

Unfortunately, I can't just tack "|group by=_pageID" onto the cargo_query, since that causes the aggregate function SUM to work on a per-group basis (i.e., instead of returning a single row containing the total sum of the Bronze column, it returns many rows, and each row tells me the sum of Bronzes with that particular _pageID is in the database). This wouldn't be a problem if subqueries were possible or something like that, but I haven't been able to find a Cargo-friendly way of grouping by _pageID and then counting/summing the resulting rows. Any thoughts? Thanks in advance! NathanielJ (talk) 17:22, 12 October 2017 (UTC)


 * I was running into this problem the other day. No solution, but one thing I'm thinking about is putting a unique index on all user-facing columns, to prevent the dupes from ever being saved. I'm not sure if that'd just make Cargo fail to insert in a ugly way though. It mightn't be too hard to add  to the insert queries; seems like something Cargo might already do anyway, I'm not familiar with the code. Sam Wilson 22:44, 12 October 2017 (UTC)


 * I was actually going to write a totally different reply - explaining that these are not good options, the best solution is to just use the command-line option for now, etc., when Sam Wilson's suggested solution made me realize that there's a very reasonable solution for this problem that I had been neglecting until now. The code can simply check, before adding a new row to a table, whether a row already exists with that exact same set of data, and then stop adding that row if there is one. I had thought about that solution various times before, but had always rejected it because there can in fact be valid duplicates - a page can have multiple, identical calls to a template, for various reasons. However, I've never heard of this actually happening, and conversely, unwanted data duplication has been a major problem. So I just added this check to the code - in the current Cargo code, a row that looks like a duplicate will simply not get saved. Hopefully there's a nicer long-term fix, but for now, the issue of data duplication may be over! If duplication is a problem for anyone, please try out the latest code. If it seems to work, I'll probably release a new Cargo version soon. Yaron Koren (talk) 16:00, 13 October 2017 (UTC)


 * If anyone's reading this - there was actually a bug in that fix, which I think I just fixed, so if you try to use this new feature, please make sure to get the very latest version. Yaron Koren (talk) 21:41, 20 October 2017 (UTC)


 * Thanks for these changes, Yaron Koren. For those able to run the very latest code, have these fixes (there are several) resolved the issue of row duplication? —Pangaearocks (talk) 17:10, 24 October 2017 (UTC)


 * I believe so, though I'm not making any guarantees yet. But it works for me. Yaron Koren (talk) 17:42, 24 October 2017 (UTC)

Links/images sometimes displaying as plaintext
I've read through the archives and didn't see a similar issue, so hopefully haven't missed something obvious. I'm using the following query to generate rows:

The template being used to format it includes icon images. Half the time this will display on the page as intended, but on occasion it just breaks and just shows the HTML instead:

 Ac 10% Up

It usually rights itself if I modify the query in some way and resave the page. I'm not really sure what's causing it. Any help would be appreciated. --Rurikawa (talk) 13:40, 13 October 2017 (UTC)


 * Is it the same images that fail, or is it a different set every time? And if it's the same images, does there seem to be any pattern to which images fail? Yaron Koren (talk) 16:01, 13 October 2017 (UTC)


 * It's all of the images at once. There doesn't seem to be a pattern. It happens with another similar query elsewhere as well. Rurikawa (talk) 06:38, 14 October 2017 (UTC)


 * For what it's worth, this happens on my wiki as well, and it's not limited to images --- it happens with any HTML thatis created by format=template. I believe that it may be some kind of conflict with Extension:ConfirmEdit (do you have it installed?), and purging the page's cache seems to fix the issue. NathanielJ (talk) 02:36, 20 October 2017 (UTC)


 * Ah yes, we do have ConfirmEdit installed. Cheers for that! Rurikawa (talk) 20:45, 25 October 2017 (UTC)

Cargo query error
After making new templates and changing truckloads of pages, I'm finally ready to start connecting some of the data with Cargo. However, I ran into a snag, and wonder if this is due to running Cargo 1.3.1, a silly error on my part (I'm learning as I go), or something else. Could somebody please elaborate?

This is the test query I'm trying out. Quest_poe1.location is a List of Page

This yields a rather long error message: https://i.imgur.com/ABlUVBV.png

(These are the Cargo tables) —Pangaearocks (talk) 16:54, 15 October 2017 (UTC)


 * You just need to put a table name before "_pageName" in "fields=" - both tables have a "_pageName" field, so the query doesn't know which one to use. Yaron Koren (talk) 03:28, 16 October 2017 (UTC)


 * Thanks. That got me another error, so I must be doing something wrong somewhere. Have attempted so many different things, but this looks right to me? Yet I'm told . Quest_poe1.location is a list field.


 * In a different attempt I tried join on with HOLDS, which yielded another error message. The info here is correct I assume? HOLDS example —Pangaearocks (talk) 05:35, 16 October 2017 (UTC)


 * Yes - given that "location" is a list, it needs to be "join on=Quest_poe1.location HOLDS Location._pageName". Yaron Koren (talk) 13:24, 16 October 2017 (UTC)


 * Thank you - that helped and got rid of the errors :) We have run into another one, though, which is a strange one. When running a query, we get many more rows than should be there. There are hits on pages that don't contain the data we are searching for in the query. I've been in touch with another admin who knows and works with SQL, and we've run different sorts of queries on different tables. He suggests it might be a caching issue. Are there any settings that might cause this, and which we can change so that queries yield the right results immediately? We've resorted to force refreshing pages, which has got rid of some of the many duplicates an erroneous hits.


 * Running the above code (with your fix), here is an example of the hits we get, on this test page. Many of these have nothing to do with "Caed Nua", and others don't have any location set yet. Another example was getting all 5 rows when searching for 'Iron' on Nation.products, which also is a list field, using HOLDS. —Pangaearocks (talk) 19:43, 16 October 2017 (UTC)


 * What's an example of a page that shouldn't be in the results? Yaron Koren (talk) 01:58, 17 October 2017 (UTC)


 * Quite a few actually. For instance His Better Half, Into the White Void, Safe Haven, Songs of the Wild, The Termal Pearl. —Pangaearocks (talk) 15:49, 17 October 2017 (UTC)


 * This looks like a bug with the handling of HOLDS in the "join on" clause. There was a fix for "HOLDS" handling that was done in August, here - I think this was after the version you have. I would recommend upgrading to Cargo 1.4 - and ideally to the very latest code, which contains a possible fix for the duplicates problem - to see if that solves your problem. Yaron Koren (talk) 01:46, 18 October 2017 (UTC)


 * Thanks for confirming this was due to a bug (we're on the 1.3.1 version). As the wiki is located on a wiki farm (Gamepedia), I can't update extensions myself, but the good news is they'll update to Cargo 1.4.0 in the near future, so that should get rid of the HOLDS query issue. That is great news, because it means I can start connecting data on pages from multiple tables. The most recent commit won't be included, but another version of Cargo is released, I hope they can roll out that update too in the near future. Sounds like a good workaround for the duplicate rows issue, which could help a great deal of people :) At least right now, we have a bigger problem with tables not filling out properly, so if you manage to find a fix for that too, it would be great. —Pangaearocks (talk) 20:02, 19 October 2017 (UTC)


 * The good news is that Cargo has been updated to 1.4. The bad news is that the query with the double HOLDS is still throwing doubles and triples of data, and false positives. There aren't doubles or triples in the actual database, so shouldn't both these issues have been solved since upgrading to 1.4? Do you know what else might be wrong? I've tried many Purges/Refresh and editing the query page, but it hasn't solved it. Query page. —Pangaearocks (talk) 19:01, 26 October 2017 (UTC)

I still think this is a bug with the handling of "HOLDS", although clearly it hasn't been fixed yet. Right now that page isn't showing anything relevant - could you please get this problem to appear again? Yaron Koren (talk) 03:52, 30 October 2017 (UTC)


 * Sorry about that, I changed the tablename. But I intended to come back to you, so I'm glad you already stated there is probably an issue with HOLDS. Have made some other queries with a single HOLDS, and I still get false positives, and duplicate entries. Some of those don't have a location set, yet get hits in queries. Others have locations set, but not what I'm querying for, yet shows up in the results. Check out the same page again, where I made some other queries. One of them is:


 * If you manage to find out what's wrong and fix it, please put out another version of Cargo in the not-too-distant future :) I'm also really intrigued about the table rebuild commit the other day. —Pangaearocks (talk) 18:35, 30 October 2017 (UTC)


 * Long URLs so I hope it works: by using a different query, you can see some of the places where it goes pear-shaped. On "The Trials of Durance" quest, only two locations are set: Picture proof. But using the query in the link above (with the View Data option), it gets a whole string of hits, including what the query in the previous post queried for (Copperlane Catacombs). Hope this may help in locating the underlying issue. —Pangaearocks (talk) 17:08, 31 October 2017 (UTC)

How to fill the table for calendar display?
Hi, I have just started with Mediawiki and one of the basic functionality I'd like to have is calendar displayed on my main page. So I installed Cargo with external mysql database, I created simple template:

To jest szablon „Urlopy”. Powinien zostać wywołany w następującym formacie:

 // Here it was with one / but I wasn't able to paste it here correctly with it. Edytuj stronę, aby zobaczyć tekst szablonu.

I see the table is created, but I can't figure out how to add data into it. Could you please help? Daniel


 * Now you just need to create pages that call the "Urlopy" template - you can do it hand, or set up forms to do it using Page Forms, or do a mass creation from a file using Data Transfer. Yaron Koren (talk) 13:11, 18 October 2017 (UTC)


 * Working perfectly. Thank you! Daniel

fullcalendar.io locale change
Hi, is it possible to change fullcalendar.io locale? I know it can be done (according to the fullcalendar documentation:  ), but I have no idea how to apply it in Cargo.


 * I'm amazed to see that all of the "calendar" format display is hardcoded in English! I thought at least some of it was getting translated, but I guess not. I'll have to look into this. Yaron Koren (talk) 17:34, 20 October 2017 (UTC)

Performance testing
Regarding this test, which was apparently run over two years ago: Performance testing Has other more recent tests been carried out, comparing SMW and Cargo, maybe on a larger scale as well? It would be interesting to know more about the speed and resource differences between the two setups. —Pangaearocks (talk) 19:53, 19 October 2017 (UTC)


 * Yes - that test was run a while ago, and now it can't even be re-run, because that wiki no longer has SMW installed on it. I'm not aware of any other performance tests. Yaron Koren (talk) 17:35, 20 October 2017 (UTC)

When do tables need to be re-created?
Figured it was best to put this in a separate 'thread'. As I'm busy converting templates from using SMW (or often nothing) to Cargo, fields may need changes along the way, as I discover the template isn't quite ideal. For instance changing a field from Page to List of Page. Or more simple changes like auto-categorisation, or formatting different things in the infobox.

How serious must the changes be to require a re-creation of the Cargo table itself? Is Cargo able to spot slight changes to the template and update the fields and data accordingly, or should we recreate the table if there have been any changes to the cargo_declare or cargo_store sections? —Pangaearocks (talk) 20:10, 19 October 2017 (UTC)


 * I think only changes to #cargo_declare require recreating the data - if only #cargo_store is changed, I think just the template re-save will trigger all the pages' data to get refreshed. I could be wrong about that, though. But yes, any change to #cargo_declare that would cause the DB table schema to be modified requires a recreate. Yaron Koren (talk) 17:38, 20 October 2017 (UTC)

No blank alias anymore?
It used to be possible to have  to keep the field name or an alias from displaying in the final format. Not sure when it stopped working and I am still on 1.3.1. But removing the equal sign or adding text for an alias fixes the issue. Error: no field named "" found for any of the specified database tables. --192.231.41.175 21:57, 21 October 2017 (UTC)

Special characters in page names
I'm having some issues with a few pages that have special characters in them. What is the best way to deal with this, given that we want pages to reflect the real names of characters in the game? Some examples are:
 * Will-O'-Wisp (database error, but can use " instead of ' when querying here. Used ' since documentation stated it was preferred over ")
 * All Hands on Deck, with people called Waldr "Three Fingers" and "Lefty" Lisc. I've used semicolon to separate List of... which means that here the names get incorrectly separated given that &quot; is used. See the page values.
 * In the CargoTable it shows up with wrong separation, but oddly it is correct when querying.

Sorry for the overload of questions and images, but since I want to use Cargo and drop SMW, I'd like to clear out all big and minor issues :) —Pangaearocks (talk) 17:07, 24 October 2017 (UTC)


 * The first issue probably requires a change to the wikitext, unfortunately - you can have something like "" instead of "" in the template. The second one seems like a bug in Cargo, though thankfully just with display, not with storage - that's why the data shows up correctly some of the time. Yaron Koren (talk) 17:53, 24 October 2017 (UTC)

Join Conditions
Took a look at the code, specifically in CargoSQLQuery.php, and started wondering about join conditions. The code uses "LEFT OUTER JOIN" and "RIGHT OUTER JOIN". But not "INNER JOIN". W3schools.com has a nice comparison image, so won't this cause some issues?

Thinking about this query for instance, and remembering that some of the problems I had before was with pages where no locations (list field) were set:

When there is no locations set in the page, the intention (in my mind at least) is that the query shouldn't get results, but with LEFT OUTER JOIN it will, because the first table will always yield hits. Maybe this isn't an issue most of the time, I don't know, but when the field you query for is empty, it looks like there will be false positives. Maybe this is what is happening in the HOLDS query I got help with before?

Naturally I don't understand all the code in that Cargo SQL page, and maybe there are perfectly valid reasons for using outer joins. There are certainly uses for it. But my understanding has been that INNER JOIN is the "default" use, because you typically want to see results that are unique to both (or more) tables. —Pangaearocks (talk) 02:38, 6 November 2017 (UTC)

Display event end time in fullcalendar
Hello, is there any way to display not only a start time, but an event end time also, in case that the event is not full day? Regards, Daniel


 * Yes - you can use the "start" and "end" field aliases; see here. Let me know if it doesn't make sense, or it doesn't work. Yaron Koren (talk) 14:27, 7 November 2017 (UTC)


 * Hi Yaron, "start" and "end" field aliases are working for me when I use them to extend an event for a few days with 'Date' type fields. But not working, when I use to two 'Datetime' type fields. Still I have displayed only start hour. Any ideas?


 * That's an interesting quirk I didn't know about, of the JS calendar library that Cargo uses, FullCalendar - by default, it only shows the start time, not the end time, of events with times. I don't know why it does that, but I agree that it's better to show both, so I just checked in a change to the code so that now it also shows end times. Yaron Koren (talk) 16:55, 9 November 2017 (UTC)

Can't get template to create tables
In the past I successfully had Cargo installed on a testmachine and was able to use templates to fill tables. Now developers have installed cargo for me on our main site and I'm running into trouble: I create a template with #cargo_declare and #cargo_store but after saving, the template page does not show the "Create Table" link. Are there any other ideas of what I could check? MidiaWiki 1.28 / Cargo 1.4
 * I checked in the mysql database that the cargo_pages and cargo_tables tables were created after running mediawikis update.php
 * I manually ran, no error was returned but also no other message displayed
 * I ran  and it returned a simple:
 * in between I also tried saving out the template again with slight modifications.
 * on the Special Pages I can see the cargo relevant pages but Cargo Tables page only shows "The following tables are defined:"


 * I'm guessing that there's a problem with the #cargo_declare call. Could you pastebin the template contents? Yaron Koren (talk) 01:05, 8 November 2017 (UTC)


 * Thanks Yaron, here it is: https://pastebin.com/dh57iEL7


 * That's very odd - the template definition looks totally correct, and I actually tried it out myself (including running that API action), and the table was created correctly. Is that exactly the template you used? Or did you simplify it, or change the language or something, from what's on your wiki? Yaron Koren (talk) 04:08, 8 November 2017 (UTC)


 * No that's exactly the template as I have it. (http://wiki.derivative.ca/index.php?title=Template:Glossary) I'll have to inquire with the website people what else they did when installing mediawiki. Also removed Cargo and installed it again via an git submodule - same thing. Perhaps I should start disabling more and more extensions - could there be a conflict? Although I suppose I would see an error somewhere.


 * Oh, it's a public wiki. This is very odd indeed... do you have access to the database for this wiki? If so, it would be very helpful to run the following query on it: " SELECT * FROM page_props WHERE pp_propname = 'CargoTableName' ". It looks like something is getting messed up in saving to that table. Yaron Koren (talk) 12:11, 8 November 2017 (UTC)


 * Hi Yaron, it returns an empty set. The propname column just contains displaytitle, notoc and forcetoc entries when i do a select * --66.207.201.146 16:37, 8 November 2017 (UTC)

Alright. Well, that explains why you're seeing this problem... although I have no idea why that's happening. The only thing I can think of is an option you suggested before, that some other extension there is conflicting with saving data to the page_props table. I would suggest temporarily disabling all extensions except for Cargo, then re-saving that template, and seeing if that makes a difference. If it does, then it's a matter of isolating the extension(s) causing the problem. Yaron Koren (talk) 22:05, 8 November 2017 (UTC)