Extension talk:Cargo

Looking for a way around duplicate rows
I have the infamous duplicate rows bug. Fortunately, this isn't actually much of a problem most of the time, since I can just add "|group by=_pageID" to most Cargo DB queries and then it will just return distinct rows from the database, even if they are actually doubled in the database. However, I have not yet found a way around the duplicate row problem when I query an aggregate function like SUM or COUNT. In particular, I am trying to do something like this:

Unfortunately, I can't just tack "|group by=_pageID" onto the cargo_query, since that causes the aggregate function SUM to work on a per-group basis (i.e., instead of returning a single row containing the total sum of the Bronze column, it returns many rows, and each row tells me the sum of Bronzes with that particular _pageID is in the database). This wouldn't be a problem if subqueries were possible or something like that, but I haven't been able to find a Cargo-friendly way of grouping by _pageID and then counting/summing the resulting rows. Any thoughts? Thanks in advance! NathanielJ (talk) 17:22, 12 October 2017 (UTC)


 * I was running into this problem the other day. No solution, but one thing I'm thinking about is putting a unique index on all user-facing columns, to prevent the dupes from ever being saved. I'm not sure if that'd just make Cargo fail to insert in a ugly way though. It mightn't be too hard to add  to the insert queries; seems like something Cargo might already do anyway, I'm not familiar with the code. Sam Wilson 22:44, 12 October 2017 (UTC)


 * I was actually going to write a totally different reply - explaining that these are not good options, the best solution is to just use the command-line option for now, etc., when Sam Wilson's suggested solution made me realize that there's a very reasonable solution for this problem that I had been neglecting until now. The code can simply check, before adding a new row to a table, whether a row already exists with that exact same set of data, and then stop adding that row if there is one. I had thought about that solution various times before, but had always rejected it because there can in fact be valid duplicates - a page can have multiple, identical calls to a template, for various reasons. However, I've never heard of this actually happening, and conversely, unwanted data duplication has been a major problem. So I just added this check to the code - in the current Cargo code, a row that looks like a duplicate will simply not get saved. Hopefully there's a nicer long-term fix, but for now, the issue of data duplication may be over! If duplication is a problem for anyone, please try out the latest code. If it seems to work, I'll probably release a new Cargo version soon. Yaron Koren (talk) 16:00, 13 October 2017 (UTC)


 * If anyone's reading this - there was actually a bug in that fix, which I think I just fixed, so if you try to use this new feature, please make sure to get the very latest version. Yaron Koren (talk) 21:41, 20 October 2017 (UTC)


 * Thanks for these changes, Yaron Koren. For those able to run the very latest code, have these fixes (there are several) resolved the issue of row duplication? —Pangaearocks (talk) 17:10, 24 October 2017 (UTC)


 * I believe so, though I'm not making any guarantees yet. But it works for me. Yaron Koren (talk) 17:42, 24 October 2017 (UTC)


 * Like I was saying, I manually change the settings of the mySQL tables created by cargo so that either the _pageID is unique (when applicable) or that a combination of fields is. I haven't updated cargo in a while :-S and it seems to do the trick. Maybe specifying by default that the ID for each table is in fact the pageID (and not the ID column), unless stated otherwise by the user, would do the trick as far as checking for duplicates. It does not solve the bug in itself, but unless we are in the (rare) situation where duplicate recordings are acceptable, it seems to prevent the bug from appearing in my wiki (and so far I haven't witnessed a recording not being updated because of my toying with the database structure). --Loizbec (talk) 18:05, 15 December 2017 (UTC)

Links/images sometimes displaying as plaintext
I've read through the archives and didn't see a similar issue, so hopefully haven't missed something obvious. I'm using the following query to generate rows:

The template being used to format it includes icon images. Half the time this will display on the page as intended, but on occasion it just breaks and just shows the HTML instead:

 Ac 10% Up

It usually rights itself if I modify the query in some way and resave the page. I'm not really sure what's causing it. Any help would be appreciated. --Rurikawa (talk) 13:40, 13 October 2017 (UTC)


 * Is it the same images that fail, or is it a different set every time? And if it's the same images, does there seem to be any pattern to which images fail? Yaron Koren (talk) 16:01, 13 October 2017 (UTC)


 * It's all of the images at once. There doesn't seem to be a pattern. It happens with another similar query elsewhere as well. Rurikawa (talk) 06:38, 14 October 2017 (UTC)


 * For what it's worth, this happens on my wiki as well, and it's not limited to images --- it happens with any HTML thatis created by format=template. I believe that it may be some kind of conflict with Extension:ConfirmEdit (do you have it installed?), and purging the page's cache seems to fix the issue. NathanielJ (talk) 02:36, 20 October 2017 (UTC)


 * Ah yes, we do have ConfirmEdit installed. Cheers for that! Rurikawa (talk) 20:45, 25 October 2017 (UTC)

Cargo query error
After making new templates and changing truckloads of pages, I'm finally ready to start connecting some of the data with Cargo. However, I ran into a snag, and wonder if this is due to running Cargo 1.3.1, a silly error on my part (I'm learning as I go), or something else. Could somebody please elaborate?

This is the test query I'm trying out. Quest_poe1.location is a List of Page

This yields a rather long error message: https://i.imgur.com/ABlUVBV.png

(These are the Cargo tables) —Pangaearocks (talk) 16:54, 15 October 2017 (UTC)


 * You just need to put a table name before "_pageName" in "fields=" - both tables have a "_pageName" field, so the query doesn't know which one to use. Yaron Koren (talk) 03:28, 16 October 2017 (UTC)


 * Thanks. That got me another error, so I must be doing something wrong somewhere. Have attempted so many different things, but this looks right to me? Yet I'm told . Quest_poe1.location is a list field.


 * In a different attempt I tried join on with HOLDS, which yielded another error message. The info here is correct I assume? HOLDS example —Pangaearocks (talk) 05:35, 16 October 2017 (UTC)


 * Yes - given that "location" is a list, it needs to be "join on=Quest_poe1.location HOLDS Location._pageName". Yaron Koren (talk) 13:24, 16 October 2017 (UTC)


 * Thank you - that helped and got rid of the errors :) We have run into another one, though, which is a strange one. When running a query, we get many more rows than should be there. There are hits on pages that don't contain the data we are searching for in the query. I've been in touch with another admin who knows and works with SQL, and we've run different sorts of queries on different tables. He suggests it might be a caching issue. Are there any settings that might cause this, and which we can change so that queries yield the right results immediately? We've resorted to force refreshing pages, which has got rid of some of the many duplicates an erroneous hits.


 * Running the above code (with your fix), here is an example of the hits we get, on this test page. Many of these have nothing to do with "Caed Nua", and others don't have any location set yet. Another example was getting all 5 rows when searching for 'Iron' on Nation.products, which also is a list field, using HOLDS. —Pangaearocks (talk) 19:43, 16 October 2017 (UTC)


 * What's an example of a page that shouldn't be in the results? Yaron Koren (talk) 01:58, 17 October 2017 (UTC)


 * Quite a few actually. For instance His Better Half, Into the White Void, Safe Haven, Songs of the Wild, The Termal Pearl. —Pangaearocks (talk) 15:49, 17 October 2017 (UTC)


 * This looks like a bug with the handling of HOLDS in the "join on" clause. There was a fix for "HOLDS" handling that was done in August, here - I think this was after the version you have. I would recommend upgrading to Cargo 1.4 - and ideally to the very latest code, which contains a possible fix for the duplicates problem - to see if that solves your problem. Yaron Koren (talk) 01:46, 18 October 2017 (UTC)


 * Thanks for confirming this was due to a bug (we're on the 1.3.1 version). As the wiki is located on a wiki farm (Gamepedia), I can't update extensions myself, but the good news is they'll update to Cargo 1.4.0 in the near future, so that should get rid of the HOLDS query issue. That is great news, because it means I can start connecting data on pages from multiple tables. The most recent commit won't be included, but another version of Cargo is released, I hope they can roll out that update too in the near future. Sounds like a good workaround for the duplicate rows issue, which could help a great deal of people :) At least right now, we have a bigger problem with tables not filling out properly, so if you manage to find a fix for that too, it would be great. —Pangaearocks (talk) 20:02, 19 October 2017 (UTC)


 * The good news is that Cargo has been updated to 1.4. The bad news is that the query with the double HOLDS is still throwing doubles and triples of data, and false positives. There aren't doubles or triples in the actual database, so shouldn't both these issues have been solved since upgrading to 1.4? Do you know what else might be wrong? I've tried many Purges/Refresh and editing the query page, but it hasn't solved it. Query page. —Pangaearocks (talk) 19:01, 26 October 2017 (UTC)

I still think this is a bug with the handling of "HOLDS", although clearly it hasn't been fixed yet. Right now that page isn't showing anything relevant - could you please get this problem to appear again? Yaron Koren (talk) 03:52, 30 October 2017 (UTC)


 * Sorry about that, I changed the tablename. But I intended to come back to you, so I'm glad you already stated there is probably an issue with HOLDS. Have made some other queries with a single HOLDS, and I still get false positives, and duplicate entries. Some of those don't have a location set, yet get hits in queries. Others have locations set, but not what I'm querying for, yet shows up in the results. Check out the same page again, where I made some other queries. One of them is:


 * If you manage to find out what's wrong and fix it, please put out another version of Cargo in the not-too-distant future :) I'm also really intrigued about the table rebuild commit the other day. —Pangaearocks (talk) 18:35, 30 October 2017 (UTC)


 * Long URLs so I hope it works: by using a different query, you can see some of the places where it goes pear-shaped. On "The Trials of Durance" quest, only two locations are set: Picture proof. But using the query in the link above (with the View Data option), it gets a whole string of hits, including what the query in the previous post queried for (Copperlane Catacombs). Hope this may help in locating the underlying issue. —Pangaearocks (talk) 17:08, 31 October 2017 (UTC)

How to fill the table for calendar display?
Hi, I have just started with Mediawiki and one of the basic functionality I'd like to have is calendar displayed on my main page. So I installed Cargo with external mysql database, I created simple template:

To jest szablon „Urlopy”. Powinien zostać wywołany w następującym formacie:

 // Here it was with one / but I wasn't able to paste it here correctly with it. Edytuj stronę, aby zobaczyć tekst szablonu.

I see the table is created, but I can't figure out how to add data into it. Could you please help? Daniel


 * Now you just need to create pages that call the "Urlopy" template - you can do it hand, or set up forms to do it using Page Forms, or do a mass creation from a file using Data Transfer. Yaron Koren (talk) 13:11, 18 October 2017 (UTC)


 * Working perfectly. Thank you! Daniel

fullcalendar.io locale change
Hi, is it possible to change fullcalendar.io locale? I know it can be done (according to the fullcalendar documentation:  ), but I have no idea how to apply it in Cargo.


 * I'm amazed to see that all of the "calendar" format display is hardcoded in English! I thought at least some of it was getting translated, but I guess not. I'll have to look into this. Yaron Koren (talk) 17:34, 20 October 2017 (UTC)

Performance testing
Regarding this test, which was apparently run over two years ago: Performance testing Has other more recent tests been carried out, comparing SMW and Cargo, maybe on a larger scale as well? It would be interesting to know more about the speed and resource differences between the two setups. —Pangaearocks (talk) 19:53, 19 October 2017 (UTC)


 * Yes - that test was run a while ago, and now it can't even be re-run, because that wiki no longer has SMW installed on it. I'm not aware of any other performance tests. Yaron Koren (talk) 17:35, 20 October 2017 (UTC)

When do tables need to be re-created?
Figured it was best to put this in a separate 'thread'. As I'm busy converting templates from using SMW (or often nothing) to Cargo, fields may need changes along the way, as I discover the template isn't quite ideal. For instance changing a field from Page to List of Page. Or more simple changes like auto-categorisation, or formatting different things in the infobox.

How serious must the changes be to require a re-creation of the Cargo table itself? Is Cargo able to spot slight changes to the template and update the fields and data accordingly, or should we recreate the table if there have been any changes to the cargo_declare or cargo_store sections? —Pangaearocks (talk) 20:10, 19 October 2017 (UTC)


 * I think only changes to #cargo_declare require recreating the data - if only #cargo_store is changed, I think just the template re-save will trigger all the pages' data to get refreshed. I could be wrong about that, though. But yes, any change to #cargo_declare that would cause the DB table schema to be modified requires a recreate. Yaron Koren (talk) 17:38, 20 October 2017 (UTC)

No blank alias anymore?
It used to be possible to have  to keep the field name or an alias from displaying in the final format. Not sure when it stopped working and I am still on 1.3.1. But removing the equal sign or adding text for an alias fixes the issue. Error: no field named "" found for any of the specified database tables. --192.231.41.175 21:57, 21 October 2017 (UTC)


 * Sorry for the very long delay on this, and for the problem. I finally looked into it, and yes, apparently this stopped working due to a code change in January 2017. (!) I just checked in a fix. Yaron Koren (talk) 16:21, 2 January 2018 (UTC)

Special characters in page names
I'm having some issues with a few pages that have special characters in them. What is the best way to deal with this, given that we want pages to reflect the real names of characters in the game? Some examples are:
 * Will-O'-Wisp (database error, but can use " instead of ' when querying here. Used ' since documentation stated it was preferred over ")
 * All Hands on Deck, with people called Waldr "Three Fingers" and "Lefty" Lisc. I've used semicolon to separate List of... which means that here the names get incorrectly separated given that &quot; is used. See the page values.
 * In the CargoTable it shows up with wrong separation, but oddly it is correct when querying.

Sorry for the overload of questions and images, but since I want to use Cargo and drop SMW, I'd like to clear out all big and minor issues :) —Pangaearocks (talk) 17:07, 24 October 2017 (UTC)


 * The first issue probably requires a change to the wikitext, unfortunately - you can have something like "" instead of "" in the template. The second one seems like a bug in Cargo, though thankfully just with display, not with storage - that's why the data shows up correctly some of the time. Yaron Koren (talk) 17:53, 24 October 2017 (UTC)

Join Conditions
Took a look at the code, specifically in CargoSQLQuery.php, and started wondering about join conditions. The code uses "LEFT OUTER JOIN" and "RIGHT OUTER JOIN". But not "INNER JOIN". W3schools.com has a nice comparison image, so won't this cause some issues?

Thinking about this query for instance, and remembering that some of the problems I had before was with pages where no locations (list field) were set:

When there is no locations set in the page, the intention (in my mind at least) is that the query shouldn't get results, but with LEFT OUTER JOIN it will, because the first table will always yield hits. Maybe this isn't an issue most of the time, I don't know, but when the field you query for is empty, it looks like there will be false positives. Maybe this is what is happening in the HOLDS query I got help with before?

Naturally I don't understand all the code in that Cargo SQL page, and maybe there are perfectly valid reasons for using outer joins. There are certainly uses for it. But my understanding has been that INNER JOIN is the "default" use, because you typically want to see results that are unique to both (or more) tables. —Pangaearocks (talk) 02:38, 6 November 2017 (UTC)

Display event end time in fullcalendar
Hello, is there any way to display not only a start time, but an event end time also, in case that the event is not full day? Regards, Daniel


 * Yes - you can use the "start" and "end" field aliases; see here. Let me know if it doesn't make sense, or it doesn't work. Yaron Koren (talk) 14:27, 7 November 2017 (UTC)


 * Hi Yaron, "start" and "end" field aliases are working for me when I use them to extend an event for a few days with 'Date' type fields. But not working, when I use to two 'Datetime' type fields. Still I have displayed only start hour. Any ideas?


 * That's an interesting quirk I didn't know about, of the JS calendar library that Cargo uses, FullCalendar - by default, it only shows the start time, not the end time, of events with times. I don't know why it does that, but I agree that it's better to show both, so I just checked in a change to the code so that now it also shows end times. Yaron Koren (talk) 16:55, 9 November 2017 (UTC)


 * Cool! Thank you for such a quick support. BTW. I see in the same commit 'agendaWeek' option. Is it possible to set it as default view? 'week' makes 'basicWeek'


 * Ah, good point, I forgot about that. I just checked in a fix for this. Yaron Koren (talk) 23:06, 10 November 2017 (UTC)

Can't get template to create tables
In the past I successfully had Cargo installed on a testmachine and was able to use templates to fill tables. Now developers have installed cargo for me on our main site and I'm running into trouble: I create a template with #cargo_declare and #cargo_store but after saving, the template page does not show the "Create Table" link. Are there any other ideas of what I could check? MidiaWiki 1.28 / Cargo 1.4
 * I checked in the mysql database that the cargo_pages and cargo_tables tables were created after running mediawikis update.php
 * I manually ran, no error was returned but also no other message displayed
 * I ran  and it returned a simple:
 * in between I also tried saving out the template again with slight modifications.
 * on the Special Pages I can see the cargo relevant pages but Cargo Tables page only shows "The following tables are defined:"


 * I'm guessing that there's a problem with the #cargo_declare call. Could you pastebin the template contents? Yaron Koren (talk) 01:05, 8 November 2017 (UTC)


 * Thanks Yaron, here it is: https://pastebin.com/dh57iEL7


 * That's very odd - the template definition looks totally correct, and I actually tried it out myself (including running that API action), and the table was created correctly. Is that exactly the template you used? Or did you simplify it, or change the language or something, from what's on your wiki? Yaron Koren (talk) 04:08, 8 November 2017 (UTC)


 * No that's exactly the template as I have it. (http://wiki.derivative.ca/index.php?title=Template:Glossary) I'll have to inquire with the website people what else they did when installing mediawiki. Also removed Cargo and installed it again via an git submodule - same thing. Perhaps I should start disabling more and more extensions - could there be a conflict? Although I suppose I would see an error somewhere.


 * Oh, it's a public wiki. This is very odd indeed... do you have access to the database for this wiki? If so, it would be very helpful to run the following query on it: " SELECT * FROM page_props WHERE pp_propname = 'CargoTableName' ". It looks like something is getting messed up in saving to that table. Yaron Koren (talk) 12:11, 8 November 2017 (UTC)


 * Hi Yaron, it returns an empty set. The propname column just contains displaytitle, notoc and forcetoc entries when i do a select * --66.207.201.146 16:37, 8 November 2017 (UTC)

Alright. Well, that explains why you're seeing this problem... although I have no idea why that's happening. The only thing I can think of is an option you suggested before, that some other extension there is conflicting with saving data to the page_props table. I would suggest temporarily disabling all extensions except for Cargo, then re-saving that template, and seeing if that makes a difference. If it does, then it's a matter of isolating the extension(s) causing the problem. Yaron Koren (talk) 22:05, 8 November 2017 (UTC)


 * Hi Yaron, I did get a bit further with this after our webdevs installed MediaWiki 1.29.1 I am now able to create tables and they are displayed in the database. My next problem was that the table would not be populated and eventually I found out that I should not use the term Long as a column in the cargo table. I think I'm getting a bit further with this - thanks for all the help--66.207.201.146 20:15, 8 December 2017 (UTC)

How to delete table
Hi Yaron,

I have a problem with one table that makes PHP 500 Error when I go to Special:CargoTables. So I decided to create it again, but how to remove the old version? I removed table template, it doesn't exists anymore in my DB, and I am able not to open Special:CargoTables. But next to my table name it says "This table is registered, but does not exist!". Registered where?

Cheers, Daniel


 * That means it's "registered" (has a row) within the cargo_tables DB table. I can't remember now if there's a way to get out of that state (row in cargo_tables but no actual table) from within the web interface - you may have to manually delete that row from the table, if you have SQL access. Yaron Koren (talk) 01:20, 13 November 2017 (UTC)


 * This rings a bell with me. I think I had a problem recreating a table which had been deleted. I think the solution was to remove it from limbo by deleting a row in the database before recreating the table. Jonathan3 (talk) 01:29, 14 November 2017 (UTC)

Calendar not displayed with version based on FullCalendar v3.6.2
Hi Yaron,

I was using Cargo version based on FullCalendar v2.3.2. After all your updates that I have asked for in the recent days I decided to switch to the latest version which is using FullCalendar v3.6.2. Unfortunately I can't display calendar now. Even simple query that I had earlier doesn't show it: Only header buttons are presented (but even header title is missing).

My MediaWiki is 1.28.2 and jQuery 1.11.3

If you could please help also with it. Cheers, Daniel


 * I was worried that would happen... MediaWiki switched from jQuery 1 to jQuery 3 in version 1.30, and I was hoping that there could be one version of the FullCalendar library that would work with both versions, but that doesn't seem to be the case. I just checked in a change so that Page Forms now includes two different versions of FullCalendar (2.9.1 and 3.6.2), and uses one or the other depending on the MediaWiki version. If you can, please upgrade to the very latest code and let me know if that fixes the problem for you. Yaron Koren (talk) 22:27, 13 November 2017 (UTC)

List of authors with Christian and surname separate
I would like to have a list of authors for each item. Some authors will be organisations, some will be individuals. For the individuals I'd like to be able to output the names as "Joe Bloggs" or "Bloggs, J" or "Bloggs, Joe" or any other combination. What would be the best way to do this?

I was thinking of saving the names (for individuals) with a delimiter like a colon between the two parts, e.g. "Bloggs:Joe". Are there parser functions that, where the ":" is present, would be able to extract/separate the parts for display purposes? I guess for lists of multiple authors I could then use arraymap to do this for each author. Or would I need to use Scribunto/Lua, or something else, instead? Thanks. Jonathan3 (talk) 23:23, 11 November 2017 (UTC)

P.S. I suppose firstname1, surname1, firstname2, surname2, firstname3, surname3, etc for perhaps 10 pairs might do the job, but it's not as neat and doesn't use Cargo's List functionality. Jonathan3 (talk) 23:26, 11 November 2017 (UTC)


 * Some combination of #arraymap and #explode might do the trick. Conversely, you could have a separate template just to hold a full author name, and then make that a multiple-instance template within the form - if that makes sense. Yaron Koren (talk) 01:18, 13 November 2017 (UTC)


 * I will have to read the documentation to understand the multiple-instance template idea. What difference would it make to the way the main table is treated by the drillldown page, and queries etc? Jonathan3 (talk) 09:05, 13 November 2017 (UTC)


 * If you use a separate template for names, then they would go into a separate table - querying would still be possible, using a "join", but drilling down on all the data together would not. (One of the to-dos for Cargo is to allow drilling down on multiple related tables.) If drilling down on names is important, you may be better off keeping everything in one template and table, although there would be some level of hackery involved. Yaron Koren (talk) 15:52, 13 November 2017 (UTC)


 * Yes, I'd like to be able to drilldown on names, so that to-do sounds good. In the meantime I'll likely use the format "Bloggs, Joe" with the comma as a delimiter so that it looks OK in drilldown etc, but use when I want to show it in another format. Jonathan3 (talk) 22:07, 13 November 2017 (UTC)


 * "Some combination of #arraymap and #explode might do the trick." I can confirm that this does work, in conjunction with the template output type (to allow the parser functions to be used). I used List of String and the semi-colons look fine in between "Bloggs, Joe"-format names. Jonathan3 (talk) 01:33, 14 November 2017 (UTC)

size parameter for Wikitext fields?
Do Wikitext type fields also have a default length of 300, as String and Text do? The docs currently say: —Sam Wilson 02:20, 23 November 2017 (UTC)


 * Yes, the default for "Wikitext" is also 300. The documentation for "size" was way off! I guess I just didn't update it in a long time. I updated it now. Yaron Koren (talk) 04:32, 23 November 2017 (UTC)


 * Thanks! Sam Wilson 05:00, 23 November 2017 (UTC)

Add Row With Some Static Data and Some Data from Another Row
I'm trying to add a new row to a cargo table which has some new data and also uses data from an existing row for "everything else". For example, if I have:

And I want to add a new row, called "Tom", with a new age, "61", but use Bob's other data (size, weight), my template has the following information:

Name = Tom Age = 61 Use-Existing-Row = Bob

How do I go about this? I'm trying to use a query for Bob's data in the Size and Weight fields (in the "#cargo_store:" section):


 * Size =

But I don't end up creating a new row, much less inserting new and old data into it. How do I do this? (Please let me know if this example doesn't give enough info). Thanks! --Nuada99 (talk) 03:47, 1 December 2017 (UTC)


 * Is this wikitext snippet from a page or a template? If it's from a page, it shouldn't be calling #cargo_store directly; and if it's from a template, the value (like 'Bob') shouldn't be hardcoded. Yaron Koren (talk) 00:07, 1 December 2017 (UTC)

Templates, which are feeding into templates, actually. :) So, there's one table, but two templates which feed into it: Template:Silly



So far, very basic. Then there's several basic pages, which make use of this template, to provide data:

Here's where it gets more complicated. There's a second template, Silly-Mod, which should build additional rows into the same "Silly" table, but re-use data, based on other rows, as specified:



I then create a new page, called "Tom", which calls this new template:

In a perfect world. this would return:
 * Tom
 * 61
 * Medium
 * 175

And the "Silly" table would have 5 rows (one for Bob, Alice, Ted, George, and Tom).

Instead, I get an HTML 500 error when I try to save the page for "Tom", I have a permanent message on the "Special:Cargo Tables" page, which says "Note: One or more of these tables are currently being populated, via the job queue." and the table "Silly" never goes above 4 rows.

I'm sure I'm just doing something stupid here, but I really don't know how to proceed. Thanks for being so responsive! --Nuada99 (talk) 03:46, 1 December 2017 (UTC)


 * Alright. That seems like it should work... if you take out the #cargo_query calls from the template, does it work? Yaron Koren (talk) 03:35, 4 December 2017 (UTC)


 * I think I've resolved this by storing Silly-Mod into a new table, and just dynamically pulling data into the display portion of the template (via queries). After some thought, I didn't really need to store the re-used row data a second time. Thanks so much for your help, and great job with Cargo! I'm excited by all the possibilities of it. -Nuada99 (talk) 18:47, 4 December 2017 (UTC)

Max Columns?
Is there a known maximum number of columns that a single table can hold in Cargo? I see that the max in MySQL is 4096, and I'm nowhere near that, but I am trying to build a table with around 450 columns, and I'm getting an HTTP 500 error when I try to build the table. I can send the template file with the fields, but I don't want to post it here, because it's HUGE. --Nuada99 (talk) 23:29, 6 December 2017 (UTC)


 * That's very large! I don't think I've heard of anyone else creating a table with anywhere near that many columns, so you may be in uncharted territory here. I would try creating the table via the command-line script cargoRecreateData.php, if you can - you at least won't get a server timeout. Yaron Koren (talk) 02:44, 7 December 2017 (UTC)


 * So, I tried the command line script (php cargoRecreateData.php --table Monster), and received a 'Template does not exist' error. Now, when I try to view the Special:CargoTables page, I see an HTML 500 error.


 * To try to correct this, I tried commenting out all of the Cargo extension bits in LocalSettings.php, and then restarting mysqld, and running Maintenance.php, then un-commenting the Cargo back in, but the problem still exists. I also rebooted the server, which still didn't fix the issue.


 * So, I deleted the template which was trying to create that table, but I'm not sure how to delete the table itself, or manually clear the issue with Special:CargoTables.


 * In the meantime, I've tried creating small test tables, which still works.


 * Sorry to be such a hassle. Still trying to wrap my head around this. Any suggestions for getting a clean start with Cargo? At this point, there is no table data I feel a need to preserve. --Nuada99 (talk) 14:49, 7 December 2017 (UTC)


 * Restarting MySQL, Apache etc. won't make any difference - the problem is just that some operations are taking too long to do. Just to be clear: is the page that's timing out just "Special:CargoTables", or "Special:CargoTables/Monster" (or whatever the table name is)? I'm guessing it's the latter. As for getting rid of the bad data - there are probably better ways to do it, but you can always go the "cargo_tables" DB table and delete either just the "Monster" row, or all rows. Yaron Koren (talk) 15:24, 7 December 2017 (UTC)


 * Both "Special:CargoTables", or "Special:CargoTables/Monster" are displaying an html 500 error when I load them. (See ). I'll try deleting the rows from "cargo_tables" in the DB when I get some time. Hopefully that will let me ease into this without any lingering weirdness getting in the way. --Nuada99 (talk) 16:26, 7 December 2017 (UTC)


 * Ah, I didn't know that this was a public wiki. And I thought that maybe "Monster" was just a code name, given that this is a monster-sized template, but no, it's a template for a monster. :) This is really a massive template, and the fact that it declares a 450-field table is maybe not even the most relevant aspect of it - it contains a whole lot of logic, including lots of transclusions, which may be slowing things way down. I still have no idea why Special:CargoTables isn't working, but it doesn't look like a timeout (my first guess) - I would recommend this addition to see the actual error message. Yaron Koren (talk) 19:01, 7 December 2017 (UTC)

Cant get Cargo to use custom DB name
I've defined
 * $wgCargoDBtype
 * $wgCargoDBserver
 * $wgCargoDBname
 * $wgCargoDBuser
 * $wgCargoDBpassword

in LocalSettings.php

But Cargo still creates the tables in the mediawiki DB Can someone tell me what am I doing wrong?


 * Maybe you included those lines before, instead of after, the inclusion of Cargo. Maybe existing tables now need to be recreated. Maybe you've mistyped the custom database details and Cargo defaults to the MW database. These are all just guesses! Jonathan3 (talk) 22:54, 9 December 2017 (UTC)

Cargo and $wgRunJobsAsync
Manual:Job_queue and Manual:$wgRunJobsAsync state that $wgRunJobsAsync is now set to false by default. Is there an ideal setting when running Cargo? Thanks Jonathan3 (talk) 22:50, 9 December 2017 (UTC)


 * Oh, that's interesting. It shouldn't matter in theory, although there have indeed been problems for some people using "async", which I assume is why they changed the default back, so turning it off seems safer. Yaron Koren (talk) 00:56, 11 December 2017 (UTC)

List (...) of output in queries
In queries where the only output is a field which is defined as "List (...) of Wikitext" (for instance), is there a way to split up the fields without using a separate results template? The little dot doesn't show up very well on proper pages, so I'm thinking about something slightly more prominent, or perhaps a &lt;br/&gt; or something like that. I tried with arraymap in the fields part of the query, but that got me an error message. Looked up CONCAT but I suppose that won't work?

Here is the current query, to give an idea, and the template: The idea of the template is that it fetches the effects of a given spell or enchantment and suchlike, so that the page of it can effectively be an ID for use in infoboxes. Makes it easier to maintain without listing the actual effects in every infobox datafields. –Pangaearocks (talk) 15:19, 12 December 2017 (UTC)


 * Furthermore, how can I split up the list of values? Am trying out with the below code for testing, but it just lists the whole string with the tiny dot in between, as a single bulletpoint list.
 * What I need is to get at each enchantment (for instance "Disorienting"), and then fetch the effects of that enchantment and show it in pages. And repeat it for all enchantments connected to an item and suchlike. --Pangaearocks (talk) 18:52, 12 December 2017 (UTC)
 * What I need is to get at each enchantment (for instance "Disorienting"), and then fetch the effects of that enchantment and show it in pages. And repeat it for all enchantments connected to an item and suchlike. --Pangaearocks (talk) 18:52, 12 December 2017 (UTC)


 * It's true that the little dot ("middot", or &middot;) is not that conspicuous. I don't think a "br" tag would be a good idea - it might not be clear where the separation was. A bulleted list could work. What do you think about replacing it with a bullet ( &bull; )? It would be a little more prominent. Tied in with that, a "span" tag, with a class, could be put around those bullets, to allow for setting the color, etc. What do you think? (I don't know why the #arraymap call isn't working.) Yaron Koren (talk) 20:18, 12 December 2017 (UTC)


 * &bull; would make the separation a little more obvious visually, and it looks like it's a little different from the usual bullet-list item (so shouldn't be too confusing when used together). After much more trouble than envisioned, I finally managed to get these queries kind-of to behave like I wanted, on https://pillarsofeternity.gamepedia.com/Disorienting. Struggled to remove the grey "effects" label, but maybe it's better kept. At least it works now, without too many nested templates. Would be really grand if it was easier to use variables and normal wikitext without nesting so many templates. I was working with 4 nested templates here before giving up and using "ul".


 * Quite possible I did something wrong, but I couldn't get the "List of" items to show up separated. So I had to resort to using the __full version. This is what is (currently) in use:


 * Split up a little here for easier readability. That  gives me a headache :blush: --Pangaearocks (talk) 23:58, 12 December 2017 (UTC)


 * Do you want help with this? If this is just a temporary workaround until the bullets get bigger, it might not be worth spending much more time on. Although it looks like you're now displaying more than just a simple list anyway. Yaron Koren (talk) 03:56, 13 December 2017 (UTC)


 * Thank you for the offer. I think it works fine enough for now, though, so we should be good (and in the future, a bigger separator will make it better). However, if you know of a reasonably easy way to remove the "effects: " label, that would be preferable, as it takes up some space and isn't really needed here.


 * More generally, and I'm not sure if this isn't possible due to security/other concerns, but it would be marvellous if it was possible to customise the output from queries a bit easier, without the need for (often several) nested templates just for display purposes. For instance a way to mix Wikitext, Cargo table fields, and variables. That way we could for instance output . We could then for instance have a a template that dealt with each row, and in this template there is a query, with output such as the above. That should allow us to have fewer nested templates, and make it easier to adapt output to a page type's needs. I have no idea if this is impossible, or a huge task to code, but as an end user it would be very useful :) –Pangaearocks (talk) 18:01, 13 December 2017 (UTC)


 * I think you can get rid of the label by having "effects=", instead of just "effects", in the "fields=" parameter of the relevant query. Though I still don't really understand this query, so maybe that won't work. As to the other request - can't that just be done with the "template" format? Yaron Koren (talk) 18:17, 13 December 2017 (UTC)


 * Previously I tried with an empty alias, but unfortunately it didn't work. I get an error message:  With any input like ...effects="" or ".", the output becomes   so there doesn't appear to be an easy way to remove the label(s). Which leads to inflation of nested template use, and leads me to the above suggestion. It's certainly correct that the suggested output can be done in a template, but the benefit of the ability to mix this in the output field is that we can cut down on template use, particularly for simple output. Or conversely, for a nested layout structure, where the last 'loop' is simple, like in the example of "Disorienting".


 * To try to explain the process on that page. First query for pages with a given enchantment, and set up the table headers. List those pages (so a row template). If you then want to use more special output than e.g. "ul", you need yet another template, and quite possibly another one. This structure quickly gets confusing, especially for people who haven't actually written it. If it was possible to use custom output format, we could cut down on additional template use, and put it in the Cargo query instead. Naturally, if the output is complex, a template should be used anyway, but for simple output this custom way would be preferable. In general, one of my wishes for the future is that Cargo becomes easier to customise, particularly in the area of output, as this is what the user sees. It's fantastic to have such well-working default format as table, ul, etc, but I often need to make adjustments, such as adding icons behind an Integer value, putting several fields into the same cell, or similar. Or like above, to mix fields and/or Wikitext/variables. I tried to play around with CONCAT, but just got error messages (and what I would need is the inverse of CONCAT anyway).


 * Lastly, it would be excellent with an easier way to list or work with the individual "list of.." items than having to use the 'complex' _rowID and such queries from the "list of.." (sub-)table. If this is actually possible, then please let me know how :) I tried, but it just listed the whole field (with middot between values), and I couldn't get #arraymap to split it up either. Hence why I had to resort to the long-form listed here. –Pangaearocks (talk) 17:09, 14 December 2017 (UTC)


 * If you send the results of your Cargo query to a template output type then it's possible to split up the List fields into individual elements using #arraymap. I think this is what you're referring to in your last question, but sorry if I've missed the point here. Let's say you want to add an icon after each element in a List. You could call the template with something like this: . Then in TemplateA:  . Then in TemplateB:   I guess the icon HTML could display a different icon depending on the relevant value. Or maybe it could be a further Cargo query. Jonathan3 (talk) 20:48, 14 December 2017 (UTC)

I just checked in a fix for the blank aliases problem, for what it's worth. I hope to release a new version of Cargo soon, with the fix (and many other changes). Yaron Koren (talk) 16:24, 2 January 2018 (UTC)


 * Thanks for the fix and info, Yaron. This will make these pages look better, and it's also nice to have blank aliases in some table headers, like for icons. As for the tips above from Jonathan3, I haven't look more into that part, but thanks for the info. I wasn't aware it was possible to use |values=XXX. —Pangaearocks (talk) 08:30, 10 January 2018 (UTC)

Ranking output
Are there any plans to be able to add row numbers to output (not the row IDs) based on the order? I.E., T-SQL's ROW_NUMBER but without requiring MSSQL or having to add that function to the allowed SQL functions. This would be useful for partial aggregate functions (e.g., SUM returns the sum of the entire table regardless of the LIMIT setting). --pcj (talk) 14:58, 14 December 2017 (UTC)


 * Sorry, I don't understand how this would be useful for functions like SUM - can you give a little more detail? Yaron Koren (talk) 15:09, 14 December 2017 (UTC)
 * Referring to this table specifically: https://dota2.gamepedia.com/Special:CargoTables/professional_players
 * With the query (ViewData), the limit parameter has no effect on the output of the SUM function. It returns 18,345 - the total of all points on the table. If there were a ranking mechanism (or even if I could do sub-queries) I could put that in the WHERE clause and reduce the rows counted into the aggregate. Otherwise I don't see any way to do it. --pcj (talk) 15:24, 14 December 2017 (UTC)


 * Okay, I get it - you want (in this example) to get the combined points of only the 3 highest-scoring players. It's true that subqueries would make this a lot easier. What I would recommend is to create a Lua module, if you know how to do that (it looks like you have Scribunto installed on that wiki) that calls a Cargo query to get the top 3 players - then have Lua cycle through those values to calculate the sum. It's the easiest solution I can think of. Yaron Koren (talk) 18:23, 14 December 2017 (UTC)


 * If LIMIT in Cargo would work like in the MySQL documentation (where the first, optional number is an offset, and the other number is the limit) the then you could (I think) do three Cargo queries including,   and   then use #expr to add them up like  . As I don't know Lua I'd find that simpler :-) At the minute Cargo requires a single integer for LIMIT. Jonathan3 (talk) 21:09, 14 December 2017 (UTC)


 * That's true, you could also do that if Cargo had an "offset" option. Yaron Koren (talk) 21:40, 14 December 2017 (UTC)


 * I've not looked at the code to see what would be easier, but I like the Cargo philosophy of making things as like SQL as possible, which would favour the LIMIT x,y possibility :-) Jonathan3 (talk) 00:43, 15 December 2017 (UTC) Just noticed this: "For compatibility with PostgreSQL, MySQL also supports the LIMIT row_count OFFSET offset syntax." - which weakens my argument a bit... Jonathan3 (talk) 00:45, 15 December 2017 (UTC)

Sometimes googlemaps marker clustering does not work
Usually it works fine, and has the numbered clusters, but sometimes it shows up as dozens of tightly-packed red markers and also leaves the Javascript drop-down menus not working. Here's the error:

Thanks Jonathan3 (talk) 01:24, 15 December 2017 (UTC)


 * I'm starting to look into this. It ceases to be a problem when I log out of Mediawiki, or when I use the Vector skin instead of Foreground. Jonathan3 (talk) 00:41, 20 December 2017 (UTC)

UPDATE a specific field
Hi,

I might not have read properly the documentation (though I spent quite some time on it), but is there a way to explicitely update a given field for a recording. (I wanted to have a column for multiple references to a given cargo recording, and to be able to update it, when I make a 2nd reference to that element… Now I don't see any other way I can do that but by creating a new table just for that field).

--Loizbec (talk) 18:37, 15 December 2017 (UTC)


 * Sorry, I don't understand. What's a recording, and what's a reference? Yaron Koren (talk) 22:09, 15 December 2017 (UTC)
 * Sorry, a recording would be a literal translation of "enregistrement", which would correspond to a line in a data table. And multiple "references" to a cargo "line" would mean that two cargo_store command would be issued to affect a given line. For instance, I have a model that identifies the sounds I'm using in an application. And each time I'm using the sound in a certain section I'm making a new reference to this sound. Now I want to have a field that counts the number of times a sound is used. So basically I want to do an . --Loizbec (talk) 18:16, 17 December 2017 (UTC)
 * Okay, now I get it. (For "recording" I would instead say "record", or "row".) No, there's no way (within Cargo) to modify an existing Cargo row - it can only be deleted and re-created. But if "useCount" is based on a template parameter, then you can update that parameter within the page, and the new value will get stored. If you're using the Page Forms extension, the #autoedit parser function may be useful for this. Yaron Koren (talk) 21:57, 17 December 2017 (UTC)

Querying a specific page that is a redirect to the proper page
Trying to think of a way to query the data from a specific page which has been moved to another page without having to double-update pages when moving them. Putting WHERE _pageName = '(origin page)' doesn't work because the Cargo table stores the proper name of the page that it has been moved to, not the redirect. In SMW you could query the redirect page with #ask and it would return the results from the target. Any ideas? --pcj (talk) 14:03, 17 December 2017 (UTC)


 * Yes, Cargo, unlike SMW, doesn't allow for querying on redirect pages. That's in part because a redirect doesn't necessarily equality. I'm not sure what you mean by "double-update", but what I'd suggest is updating the page name throughout the wiki after a page move. You can do that fairly easily with the Replace Text extension. Yaron Koren (talk) 14:21, 17 December 2017 (UTC)
 * Yeah that's what I'm referring to, the extra page maintenance is a bit unsustainable. --pcj (talk) 14:38, 17 December 2017 (UTC)


 * It would be good if Replace Text could add to the page move process - identifying links to the old page and suggesting renaming them throughout the website. Just an idea - I don't know how feasible this would be. Jonathan3 (talk) 14:43, 28 December 2017 (UTC)


 * That's a very interesting idea, I never thought of. I, too, don't know how easy it would be to implement, but it's worth looking into. Yaron Koren (talk) 02:21, 29 December 2017 (UTC)


 * This could maybe be used: Manual:Hooks/SpecialMovepageAfterMove. A message could be shown like “There are x pages linking to old title...” including a link to the Replace Text special page with the old title and new title already there. If I have time to try this out in the new year I’ll mention it on the Replace Text talk page. Jonathan3 (talk) 11:23, 29 December 2017 (UTC)

List in Fields returns only the match from the WHERE clause?
We're trying to write a query to join on a reference table with canonical names for pages to work around the redirect issue above...one issue we've run into is a list only returns the match from the HOLDS. I.E., this query:

only returns "Fear (Keefe Pascual)" itself, which is a bit useless (even though of course the page has other values as well). Any suggestions? --pcj (talk) 15:15, 17 December 2017 (UTC)


 * Yes - change "fields=NameList" to "fields=NameList__full". Yaron Koren (talk) 16:32, 17 December 2017 (UTC)

Using 'The Siblings' as a field value breaks
On a page that uses a template to store information multiple times with only two values, a grouping name and then a list of potential pages in that group, when the group was named 'The Siblings', we got this error Error: operator for the virtual field 'Characters.siblings' must be 'HOLDS', 'HOLDS NOT', 'HOLDS LIKE' or 'HOLDS NOT LIKE'. Changing the name made the error go away. Page in Question, the template. --Cody3647 (talk) 14:47, 23 December 2017 (UTC)


 * That does seem bad... there's some sort of bug there in the query parsing. I wouldn't be surprised if simplifying the query in the template would make the problem go away, though. Looking at the query, it doesn't seem like the helper table "Plot_Characters__characters" needs to be included, and it looks like some of the stuff in "fields=" can be simplified too. Maybe it's worth trying. Yaron Koren (talk) 02:14, 29 December 2017 (UTC)


 * The helper table gets the individual character names from the list and then makes it possible to get information from the character information table, otherwise the only information available is the name, not the picture (and everyone likes pictures). Everything in the fields= is used to make the gallery.  So I'm not sure I can get the same functionality with a simplified query.  --Cody3647 (talk) 09:04, 29 December 2017 (UTC)


 * It seems to me that replacing the first five lines of the query with the following would produce the same results:

{{#cargo_query: tables=Plot_Characters,Characters
 * join on=Plot_Characters.Characters HOLDS Characters._pageName
 * where=Plot_Characters._pageName="{{FULLPAGENAME}}" AND group_name="{{{groupName|}}}"
 * fields=Characters._pageName=name,avatar, Characters._pageName=cPage, full_name=full name
 * ...though I haven't tested it. (My guess is that you didn't know about "HOLDS" within "join on".) Yaron Koren (talk) 13:26, 29 December 2017 (UTC)

API cargoquery crossed wires between GROUP BY and HAVING
Trying to use the api  parameter in Cargo 1.5 seems to translate into a   statement in the resulting SQL, while the api   parameter translates into an SQL   statement. Image attached where providing an intentionally invalid field name to  shows the SQL query (with  ) in the error message. Also tested the other way around. Possible to work around by simply using the  parameter for   functionality and vice versa, but very confusing.

Is this a known bug, or perhaps some configuration issue? SaschB (talk) 00:38, 30 December 2017 (UTC)


 * Yikes! Neither - although I suppose it's known now that you've reported it. Those two values were indeed switched in the API code. I just checked in a fix. Sorry about that. Yaron Koren (talk) 04:18, 30 December 2017 (UTC)


 * When I go to a table with many rows, for instance Item_poe1, scroll to the bottom of the page and click the "More..." link, I get a database error message. Is that related to the above bug? Curiously, I didn't get it on Bestiary_poe1. —Pangaearocks (talk) 08:26, 10 January 2018 (UTC)


 * That's a different error - it looks like the problem is because you have a field named "unique", which is an SQL keyword. There's a check in the Cargo code to try to prevent SQL keywords from getting used as field names, but it looks like I forgot to add "unique" to the list. So I would recommend renaming the field to something else, then recreating the table. Yaron Koren (talk) 18:06, 10 January 2018 (UTC)


 * Thanks for the info and change. I wasn't are of this, and used "unique" as field name as it was the obvious choice. I've adapted the templates (using "is_unique" now) and am running a bot job to change all the related item pages. Hopefully another null-edit bot job on the whole lot later will take care of missing/duplicated rows. --Pangaearocks (talk) 05:19, 11 January 2018 (UTC)

Delimiter for |format=template
Is it possible to set some delimiter when |format=template? It was possible in SMW, and there's some queries I'm trying to switch over to Cargo that were set up like that. --RheingoldRiver (talk) 16:18, 7 January 2018 (UTC)


 * No - I haven't seen a good reason for it yet. What do you want the delimiter to be? Yaron Koren (talk) 21:28, 7 January 2018 (UTC)


 * In some cases &amp;ensp; (although this one is generally fine to just append to the template, sometimes the template is used in multiple places and I don't always want the leading/trailing space), often &#8226;. Sometimes a break between table cells, this is the template. There were probably other cases that I used a separator in SMW with template format but these are the ones that come to mind. --RheingoldRiver (talk) 23:06, 7 January 2018 (UTC)


 * Is the current behaviour to insert a space between result item templates, or is it a newline? Or nothing? (I know I could check, but the couple of places I'm using it it seems to be a newline, but I'm not sure.) Sam Wilson 01:40, 11 January 2018 (UTC)


 * Okay, that makes sense. I checked in a "delimiter" parameter for the "template" format earlier today - hopefully it will get to you before too long. Sam - by default it doesn't put in anything. Yaron Koren (talk) 03:13, 11 January 2018 (UTC)
 * Thanks! --RheingoldRiver (talk) 12:19, 11 January 2018 (UTC)

Jobs issue
MW 1.30, Cargo 1.30.

If $wgJobRunRate > 0, records are sometimes stored to DB twice when you save a page. --StasR (talk) 21:21, 12 January 2018 (UTC)


 * Sorry, what version of Cargo are you using? Yaron Koren (talk) 22:45, 12 January 2018 (UTC)


 * Git branch REL1_30 — v. 1.4 (1798b60) 13.09.2017 --StasR (talk) 23:03, 12 January 2018 (UTC)


 * Oh - you should upgrade to version 1.5, which is when the duplicates problem was fixed. Or just use the very latest code. In general, the "REL" branches are not useful for any of my extensions, and possibly not for any non-WMF extensions. Yaron Koren (talk) 23:31, 12 January 2018 (UTC)


 * Thanks, it helped! --StasR (talk) 00:03, 13 January 2018 (UTC)

Underscore inconsistency Part II
Previously I [https://www.mediawiki.org/wiki/Extension_talk:Cargo/Archive_August_to_September_2017#Underscore_inconsistency? reported] there is inconsistency with use of underscores in queries and templates. For some reason underscores must be replaced with spaces when referring to fields in templates. (Of course, if this is changed, many templates must be redone, as they now use the workaround of space instead of underscore).

It looks like I came across another issue related to this one (unless "type" is another banned SQL keyword?). In for instance Enchantment_poe1, try to click on one of the "next 250" and such links, and an error message pops up (notice the lack of underscore): Table "Enchantment poe1" not found in Cargo database. –Pangaearocks (talk) 09:08, 13 January 2018 (UTC)


 * Wow - it's surprising that no one saw, or at least reported, that error until now. My guess is that it's in part because the "view next pages" functionality in Special:Drilldown is rarely used, since the page is meant to winnow down options to a manageable size. Anyway, I just checked in a fix for the problem; sorry about that. Yaron Koren (talk) 14:01, 15 January 2018 (UTC)

Missing tail end of record set when calling same #cargo_store template multiple times from the same article
I have certain pages for which I want to store multiple rows of data in the same table for a single page, I'm doing this by calling the same #cargo_declare/#cargo_store template multiple times from the page, with different variable values. While this does work, I have noticed that random pages often experience missing rows. Upon examining this, I've noticed that it's always the tail end of the records that go missing. Consider a page trying to store 5 rows with the values 1-5 to some table, using a template, and five separate calls like this:

When this works, five records are created like

When rows are missing, it's always an uninterrupted set from the end, leaving the table in a state like for example

Different rebuilds of the same data results in different pages missing a different amount of records (from the end), so I'm ruling out malformed input, as a template call that sometimes works does not always work. I have experimented with recreating the data into a replacement table, checking for missing rows, null-editing all pages calling the template, checking again for missing rows, null-editing again, etc. and each set of null-edits has a chance to add some of the missing rows, but not always, and not necessarily all of them (causing a page to still be missing the tail end of records, but a lower amount of them).

Do you have any insight into what might be causing this behavior? Worth mentioning is that I'm using a Scribunto module to calculate the values which are to be set, and using the Lua  for each call, which may or may not prevent the Cargo data from being properly stored on page re-cache, as pages with missing records do not get them added after the incomplete replacement table has been switched in, even through null-edits or scheduled page cache refreshes. SaschB (talk) 05:09, 14 January 2018 (UTC)


 * My guess is that the jobs are timing out before completion, possibly due in part to the complexity of these storage calls. Is your "job run rate" set to something higher than 1? If so, I would try bringing it down to 1 (or even less). Or, if you have command-line access, you could try running the cargoRecreateData.php script, instead of recreating from the web interface - that should work better. Yaron Koren (talk) 14:04, 15 January 2018 (UTC)


 * This is on Gamepedia, so the admin/config is handled by other people, I asked them about the job run rate, and it's apparently set to 1. A separate issue is that, while newly created pages (with less complex call structures) are picked up fine, any edits to pages which already have some data stored do not seem to update the live tables. This means I have to use the web interface to recreate tables (and subsequently null-edit all category members, as it were) after any edit I want immediately usable. I believe these recreates might not be strictly necessary, but in many cases I can't wait 24 hours for the tables to refresh, and null-edits or purges of pages do not update live tables - only replacement tables (additionally, null-edits to pages which store data in replacement tables delete that page's rows from the live "read-only" version of that table, which is yet another problem).


 * All of this could probably be solved if a null-edit or purge of a single page would properly update the live table, since this would eliminate the need for recreating tables after every edit. Is this something you recognize as an issue, and might know what it could be caused by? (Under SMW, purging a page did properly update the values) SaschB (talk) 19:24, 15 January 2018 (UTC)


 * While a replacement table exists, a "live" table is read-only, as you note. (Or at least, it's intended to be that way.) Given that, there is (or is supposed to be) no way to modify the "live" tables that still have a replacement table - it has nothing to do with caching or purging. Maybe that's an error in design - I wasn't taking into account the possibility that recreating a table could become a very lengthy, complex process, as it is for you, due to (I think) your combination of complex template calls and lack of command-line access. Yaron Koren (talk) 13:31, 16 January 2018 (UTC)


 * They're not strictly read-only though, as any edits to pages storing data in them deletes rows from them (I consider deletion a write-operation). Even so, the live tables are effectively read-only even while there are no active replacement tables, as page edits aren't stored in them unless the table is recreated. SaschB (talk) 16:32, 17 January 2018 (UTC)


 * Oh... if the "live" tables are getting modified, that sounds like a bug. Yaron Koren (talk) 20:50, 17 January 2018 (UTC)