Extension talk:Cargo

Delimiter for |format=template
Is it possible to set some delimiter when |format=template? It was possible in SMW, and there's some queries I'm trying to switch over to Cargo that were set up like that. --RheingoldRiver (talk) 16:18, 7 January 2018 (UTC)


 * No - I haven't seen a good reason for it yet. What do you want the delimiter to be? Yaron Koren (talk) 21:28, 7 January 2018 (UTC)


 * In some cases &amp;ensp; (although this one is generally fine to just append to the template, sometimes the template is used in multiple places and I don't always want the leading/trailing space), often &#8226;. Sometimes a break between table cells, this is the template. There were probably other cases that I used a separator in SMW with template format but these are the ones that come to mind. --RheingoldRiver (talk) 23:06, 7 January 2018 (UTC)


 * Is the current behaviour to insert a space between result item templates, or is it a newline? Or nothing? (I know I could check, but the couple of places I'm using it it seems to be a newline, but I'm not sure.) Sam Wilson 01:40, 11 January 2018 (UTC)


 * Okay, that makes sense. I checked in a "delimiter" parameter for the "template" format earlier today - hopefully it will get to you before too long. Sam - by default it doesn't put in anything. Yaron Koren (talk) 03:13, 11 January 2018 (UTC)
 * Thanks! --RheingoldRiver (talk) 12:19, 11 January 2018 (UTC)

Jobs issue
MW 1.30, Cargo 1.30.

If $wgJobRunRate > 0, records are sometimes stored to DB twice when you save a page. --StasR (talk) 21:21, 12 January 2018 (UTC)


 * Sorry, what version of Cargo are you using? Yaron Koren (talk) 22:45, 12 January 2018 (UTC)


 * Git branch REL1_30 — v. 1.4 (1798b60) 13.09.2017 --StasR (talk) 23:03, 12 January 2018 (UTC)


 * Oh - you should upgrade to version 1.5, which is when the duplicates problem was fixed. Or just use the very latest code. In general, the "REL" branches are not useful for any of my extensions, and possibly not for any non-WMF extensions. Yaron Koren (talk) 23:31, 12 January 2018 (UTC)


 * Thanks, it helped! --StasR (talk) 00:03, 13 January 2018 (UTC)

We are running 1.5 and still get duplicates. I don't think it is triggered by page save though. For example, I recreated a table about an hour ago because it was full of duplicates. The recreate populated about 450 rows (should have been more like 1500). After an hour, it has 1,330 rows, but many of them are duplicates. Nobody has been editing the pages with dupes, so where are these duplicates coming from? ... Update: Just ran recreate on it again. 430 Rows. Edited my test "Michelle" page, now 867 rows, many dupes but the "Michelle" rows are not duplicated. Since I have been typing, there are now 1266 rows, more duplicates. And for sure zero edits on any page on the wiki.

I have seen this behavior of duplicates appearing in other templates also, but can't figure out what triggers it since sometimes there are none then you come back a few days later and new duplicates have appeared. The dupes are from pages that have not been recently edited.

--2angelic1 (talk) 09:52, 12 February 2018 (UTC)
 * This is the table: https://futuramaworldsoftomorrow.gamepedia.com/Special:CargoTables/Actions
 * This is the template: https://futuramaworldsoftomorrow.gamepedia.com/Template:Action
 * This is one of the pages that inserts data into the Actions table: https://futuramaworldsoftomorrow.gamepedia.com/Michelle


 * This is very odd - and what makes it more odd is that the "page values" displays don't seem to show any of the values - like here. I'm guessing that these two problems are related, but I don't know how. When recreating, are you populating values into a "replacement table"? If not, I would suggest doing that - maybe it will work better. Yaron Koren (talk) 16:09, 12 February 2018 (UTC)


 * At the moment you looked at it, the Characters template and table were out of synch due to a template change. Running recreate data fixed what you were seeing there with missing data. It is interesting though, that while the Character template was broken, it was easier and faster to reproduce the dupe issue on the Actions template. It makes me wonder if the dupe issue is something to do with a process erroring out mid-way, causing dupe rows to be added when it is run again. Also, my other thought.. Recreate doesn't actually seem to recreate all the data on the Actions table. It will say jobs="0" but all the rows won't be there. I wonder if that process breaking mid-way has any impact. And yes, I usually do a replacement table, check it out and then switch. For the Actions table, I have tried both ways to see if it affected dupe issue and doesn't seem to matter. (2angelic1 (talk) 6:03, 14 February 2018 (UTC)


 * Let me also add that there's no reason to have all those #if calls in the #cargo_store call in that template, as far as I can tell. I doubt that's the cause of any of these problems, but it would at least simplify things to remove them. Yaron Koren (talk) 21:00, 13 February 2018 (UTC)


 * Stripped out all but one if statement in the Actions template (it was an if/then and made sense). Not going to impact issue with dupes, but agree those weren't needed there. (2angelic1 (talk)6:07, 14 February 2018 (UTC)


 * Alright. This is odd - though less odd than before, thankfully. Is it correct that the "Actions" table is currently the only one with duplicates? If so, could it be due to the recent-ish changes to the template - not any specific changes, just the fact that it was changed. Could you try recreating that table one more time? Perhaps this time will work better. Yaron Koren (talk) 14:08, 15 February 2018 (UTC)


 * This duplicates issue has been happening since at least November, and happens to other tables. I just used Actions as an example since it seems to do it the most frequently so easiest to see/reproduce. I recreated yesterday and see dupes back in there today. Other wiki admins on the gamepedia platform have the same issue and some have written some ugly custom queries to work around this problem. We think it is either an issue with the source or the configuration on the gamepedia servers. They have a ticket in and have also been looking into it but no result so far. Thanks for your time. (2angelic1 (talk) 12:56, 16 February 2018 (UTC)


 * Well, I thought it was just "Actions" because I skimmed through the other tables and couldn't see any duplicates. As far as I know, for other wikis (on Gamepedia and elsewhere) that are having problems with duplicates, it's just because they're using versions of Cargo older than 1.5 - though I could be wrong about that. Clearly there's still a problem here, though I can't figure out why. By the way, I see that you're now using Cargo 1.6 - do you know when that upgrade was done? Yaron Koren (talk) 16:51, 16 February 2018 (UTC)


 * All Gamepedia wikis are on the same version. Not sure when they upgraded to 1.6 though. In the futurama wiki, you can also see some dupes currently in the Unlock_Materials and Goal_Parts tables. I periodically go in and recreate data on tables with dupes to try and clean them up. The only problem with it is it doesn't always recreate all the data. Not sure if related to the dupes issue or not but I think that issue with the recreate process has been reported elsewhere in here. (2angelic1 (talk) 20:35, 16 February 2018 (UTC)

Oh, good to know all Gamepedia wikis are running the same versions - I didn't know that. Anyway, based on what you just said, my guess is that you are running a recreate before the previous recreate has finished, which is leading to the duplicate rows. Why the duplicate-checking code in Cargo is not catching it, I don't know, but that's my theory - so I would recommend just waiting longer between one recreate and the next. Yaron Koren (talk) 21:30, 16 February 2018 (UTC)


 * I can confirm that running a recreate prior to the previous one finishing definitely causes a ton of dupes (or at least did in the past). And if the earlier recreate didn't populate all rows initially so I blank edited all pages, then recreated relatively soon after that, it still made a bunch of dupe rows, so even though it was done populating in that everything showed up, Cargo didn't think it was done. The tables I was recreating were pretty big (by our standards at least), ~3k (non-dupe) rows or so. I haven't reproduced this in a while (probably just because I'm not rebuilding tables that much anymore) but I can try to reproduce when I get a chance, though it might be a bit complicated because I don't want to break stuff on our live wiki. --RheingoldRiver (talk) 23:06, 16 February 2018 (UTC)


 * Yes, recreating data and running recreate again (or editing pages using that template) before it is done seems to make dupes worse. However, this isn't the source of the dupes problem. Running recreate was an attempt to clear the dupes. Instead of helping it seems to make the problem worse. It is also usually weeks between me running a recreate to clear dupes. Every wiki seems to have a different strategy to deal with the dupes. I've been trying to recreate to clear them. Others have been doing group by queries so the page results just don't show the dupes. (2angelic1 (talk) 11:32, 19 February 2018 (UTC)


 * Well, I don't know, then. Gamepedia wikis are still the only ones I know of where data duplication is a problem, post-version 1.5. And the code in Cargo that tries to prevent storing duplicate rows seems pretty foolproof to me. Could it be that there are some local changes to the Cargo code on Gamepedia that are preventing this code from working correctly? That's my new theory. Yaron Koren (talk) 00:03, 20 February 2018 (UTC)

Just in case it helps someone troubleshooting this issue. I just watched the duplicate problem happen. It occurred when I null edited a page. Instead of replacing the existing rows, it added a new set of rows. This page calls the same template 25 times and another template 1 time. It was the template with 25 items to update that duplicated the rows. Doing another null edit right after updated the same data correctly, clearing the dupes. So it happens often but not every time. Same thing with recreate. Adds some rows then quits, but not consistent as to which row it glitches on and quits. (2angelic1 (talk) 12:01, 1 March 2018 (UTC)


 * There have been some improvements to the Cargo storage code in the last month that may solve this issue - I don't know. I'm hoping to release a new version of Cargo soon. Yaron Koren (talk) 04:33, 2 March 2018 (UTC)

Underscore inconsistency Part II
Previously I [https://www.mediawiki.org/wiki/Extension_talk:Cargo/Archive_August_to_September_2017#Underscore_inconsistency? reported] there is inconsistency with use of underscores in queries and templates. For some reason underscores must be replaced with spaces when referring to fields in templates. (Of course, if this is changed, many templates must be redone, as they now use the workaround of space instead of underscore).

It looks like I came across another issue related to this one (unless "type" is another banned SQL keyword?). In for instance Enchantment_poe1, try to click on one of the "next 250" and such links, and an error message pops up (notice the lack of underscore): Table "Enchantment poe1" not found in Cargo database. –Pangaearocks (talk) 09:08, 13 January 2018 (UTC)


 * Wow - it's surprising that no one saw, or at least reported, that error until now. My guess is that it's in part because the "view next pages" functionality in Special:Drilldown is rarely used, since the page is meant to winnow down options to a manageable size. Anyway, I just checked in a fix for the problem; sorry about that. Yaron Koren (talk) 14:01, 15 January 2018 (UTC)

Missing tail end of record set when calling same #cargo_store template multiple times from the same article
I have certain pages for which I want to store multiple rows of data in the same table for a single page, I'm doing this by calling the same #cargo_declare/#cargo_store template multiple times from the page, with different variable values. While this does work, I have noticed that random pages often experience missing rows. Upon examining this, I've noticed that it's always the tail end of the records that go missing. Consider a page trying to store 5 rows with the values 1-5 to some table, using a template, and five separate calls like this:

When this works, five records are created like

When rows are missing, it's always an uninterrupted set from the end, leaving the table in a state like for example

Different rebuilds of the same data results in different pages missing a different amount of records (from the end), so I'm ruling out malformed input, as a template call that sometimes works does not always work. I have experimented with recreating the data into a replacement table, checking for missing rows, null-editing all pages calling the template, checking again for missing rows, null-editing again, etc. and each set of null-edits has a chance to add some of the missing rows, but not always, and not necessarily all of them (causing a page to still be missing the tail end of records, but a lower amount of them).

Do you have any insight into what might be causing this behavior? Worth mentioning is that I'm using a Scribunto module to calculate the values which are to be set, and using the Lua  for each call, which may or may not prevent the Cargo data from being properly stored on page re-cache, as pages with missing records do not get them added after the incomplete replacement table has been switched in, even through null-edits or scheduled page cache refreshes. SaschB (talk) 05:09, 14 January 2018 (UTC)


 * My guess is that the jobs are timing out before completion, possibly due in part to the complexity of these storage calls. Is your "job run rate" set to something higher than 1? If so, I would try bringing it down to 1 (or even less). Or, if you have command-line access, you could try running the cargoRecreateData.php script, instead of recreating from the web interface - that should work better. Yaron Koren (talk) 14:04, 15 January 2018 (UTC)


 * This is on Gamepedia, so the admin/config is handled by other people, I asked them about the job run rate, and it's apparently set to 1. A separate issue is that, while newly created pages (with less complex call structures) are picked up fine, any edits to pages which already have some data stored do not seem to update the live tables. This means I have to use the web interface to recreate tables (and subsequently null-edit all category members, as it were) after any edit I want immediately usable. I believe these recreates might not be strictly necessary, but in many cases I can't wait 24 hours for the tables to refresh, and null-edits or purges of pages do not update live tables - only replacement tables (additionally, null-edits to pages which store data in replacement tables delete that page's rows from the live "read-only" version of that table, which is yet another problem).


 * All of this could probably be solved if a null-edit or purge of a single page would properly update the live table, since this would eliminate the need for recreating tables after every edit. Is this something you recognize as an issue, and might know what it could be caused by? (Under SMW, purging a page did properly update the values) SaschB (talk) 19:24, 15 January 2018 (UTC)


 * While a replacement table exists, a "live" table is read-only, as you note. (Or at least, it's intended to be that way.) Given that, there is (or is supposed to be) no way to modify the "live" tables that still have a replacement table - it has nothing to do with caching or purging. Maybe that's an error in design - I wasn't taking into account the possibility that recreating a table could become a very lengthy, complex process, as it is for you, due to (I think) your combination of complex template calls and lack of command-line access. Yaron Koren (talk) 13:31, 16 January 2018 (UTC)


 * They're not strictly read-only though, as any edits to pages storing data in them deletes rows from them (I consider deletion a write-operation). Even so, the live tables are effectively read-only even while there are no active replacement tables, as page edits aren't stored in them unless the table is recreated. SaschB (talk) 16:32, 17 January 2018 (UTC)


 * Oh... if the "live" tables are getting modified, that sounds like a bug. Yaron Koren (talk) 20:50, 17 January 2018 (UTC)


 * This was indeed a bug, and I believe it's fixed now. Yaron Koren (talk) 22:02, 21 January 2018 (UTC)

Query continuation
Hi !

I've started to use the Cargo API, but I'm facing an issue. I want to make a request that would yield massively over 500 results, but the server limits the results to the first 500 first elements. Is there a way to "continue" a request, to get the next results ?


 * For some reason, I never added an "offset" parameter to #cargo_query, or to the "cargoquery" API action. That was probably a mistake. There might be another way to do this, though: you can use the page Special:ViewData as an API (its "csv" and "json" formats basically function like an API), and it does allow an "offset=" query string parameter. Yaron Koren (talk) 02:18, 21 January 2018 (UTC)

Option to store integers without commas
Is there any way to do this, or if not could it be added? We would like to be able to use #expr: on results queried that have type Integer, and also store years (2017, 2018, etc) as type Integer and have them print without commas. Right now we just #replace all of the commas with nothing but it would be nice to be able to skip this step. --RheingoldRiver (talk) 22:44, 20 January 2018 (UTC)


 * To be clear: the values are stored without commas; the commas are added when the values are displayed. I didn't know that #expr failed if there were commas in the numbers - that's too bad. You may be able to get rid of them just by putting "CONCAT" around the field name within the #cargo_query "fields=" parameter, so it's "CONCAT(MyIntField)" instead of just "MyIntField". But as far as I know, there's no reason to store years as type Integer - I think it's always better to store them as type Date, which will get rid of the commas problem there. Yaron Koren (talk) 02:24, 21 January 2018 (UTC)


 * We have a very weird reason to store years as Integer type, which is that our 2011 and 2012 seasons in competitive League of Legends esports were called Season 1 and Season 2, and then after that it switched to "2013 Season" etc. So seasons are 1, 2, 2013, 2014, etc. That said we could probably just store as String. I'll check if CONCAT works to get around the other problem, thanks. --RheingoldRiver (talk) 22:57, 21 January 2018 (UTC)

Case Sensitivity in HOLDS
Sorry, one more question - when we move our game stats from SMW to Cargo in the future, I want to join a table that has a column listing all the different names player has played as to the table of game data and use HOLDS to get all of the games that the player played in, regardless of name at the time. Is there any way to make this query case-sensitive? We don't always have disambiguation links for players that had the same name but with different casing (for example ILLusion and illuSioN), especially when the player with unusual casing is very obscure. --RheingoldRiver (talk) 22:51, 23 January 2018 (UTC)


 * No problem, keep the questions coming. I don't think you can do that with "HOLDS" directly. Here you can see the longer query that a "HOLDS" query corresponds to - I think you'll have to go with the longer query, and put something like "LOWER" around both sides of the comparison. Yaron Koren (talk) 14:54, 24 January 2018 (UTC)
 * Sorry, that's the opposite of what I want - ILLusion and illuSioN should be distinct entries so that if I query one I only get his games and none from the other. Is that how it works by default? --RheingoldRiver (talk) 19:44, 24 January 2018 (UTC)
 * Oh... never mind. Yes, HOLDS is case-sensitive. Yaron Koren (talk) 19:53, 24 January 2018 (UTC)
 * Are you sure? Illviljan tried here and it seems to be insensitive. - https://pathofexile.gamepedia.com/User:Illviljan/cargo#Case_sensitivity_in_HOLDS --RheingoldRiver (talk) 20:42, 24 January 2018 (UTC)
 * Oops again! Somehow I forgot that the '=' operator in MySQL is case-insensitive. I just looked it up, and thankfully there's an easy solution: change "HOLDS" to "HOLDS BINARY" (or "=" to "= BINARY"). Yaron Koren (talk) 22:10, 24 January 2018 (UTC)
 * Yep that works! Thanks --RheingoldRiver (talk) 23:58, 24 January 2018 (UTC)

Feature Request: Parameter for adding styles/classes to display formats
Would it be possible to add a parameter for styling the results of a cargo query? I understand that this can be done with templates but since there are already a bunch of nice display formats available it would be great to be able to specify |class=red to make colored versions of the tables, for instance.


 * Right, there's no "class" parameter for any of the formats. You can accomplish the same thing, though, by wrapping the query in a "div" or "span" tag, like '&lt;div class="red"&gt;'. Then the CSS needs to look like "div.red table ...". Hopefully this is only a little bit of a hack. Yaron Koren (talk) 15:21, 25 January 2018 (UTC)

Attaching to multiple tables
Hi! Is there a specific reason why you can only ATTACH to exactly ONE table within a template? While having multiple instances of #cargo_attach (on different tables each), only the last one is computed and applied during data recreation... --149.243.232.3 14:44, 26 January 2018 (UTC)


 * That sounds like a bug - I'll have to look into it. Sorry about that. Yaron Koren (talk) 17:17, 26 January 2018 (UTC)

False positives from HOLDS queries
Using a query like this sometimes produces results where in fact isn't part of FocusHeroes. (For example returning ) Am I doing something wrong? SaschB (talk) 03:56, 28 January 2018 (UTC)


 * The query looks correct - something strange is happening with the storage. I would recommend recreating that table - if you still get errors like that even after recreating it, this may be a bug that still exists in Cargo. Yaron Koren (talk) 05:00, 28 January 2018 (UTC)


 * Recreating the table seems to have worked, but if whatever caused it to happen is still in effect, it's bound to reappear in the future, in which case I'll come back to this post. SaschB (talk) 12:39, 28 January 2018 (UTC)


 * That's good. Yes, please let me know if you see this problem again. Now I'm wondering about a different problem, though, which is that the table you just recreated, SummoningFocuses, has a lot of duplicates - and it shouldn't have any, given that the Cargo code now includes a check to avoid duplication. I can't think of any reason why that's happening, and I couldn't replicate the problem on my system (using some of your data). Do you know if the Cargo code on your wiki exactly matches what's in the main Cargo code (at least, at some relatively recent point in time)? Or are there any custom changes? Yaron Koren (talk) 17:03, 28 January 2018 (UTC)


 * Couldn't tell you if there are custom changes, but you have some contact at Gamepedia, right? You should be able to ask them about it. I did recreate this table in my usual fashion: recreate with replacement table -> bot-null-edit all category members calling the cargo_store template -> switch in replacement table. This usually causes duplicates for the ~50% of rows the "recreate table" command manages to pick up straight away, since they are also null-edited a minute later, causing the duplicates. SaschB (talk) 17:46, 28 January 2018 (UTC)


 * Aha - the fact that these are bot edits may have something to do with it, since bot edits seem to cause problems for Cargo, at least in some systems. Why are you doing these null bot edits? If it's to help with recreating the data, they may not be necessary. Yaron Koren (talk) 23:05, 28 January 2018 (UTC)


 * Yeah, the bot null-edits are to ensure all pages are present in the cargo table "immediately", so that the replacement table can be safely switched in without causing missing data for an unknown period of time. The bot null-edits do cause duplicates, but at least they seem to totally prevent missing items. A "Recreate using replacement table" I performed just now seems to have stopped at 58 records while the current live table contains 90 unique _pageName records. SaschB (talk) 15:50, 29 January 2018 (UTC)


 * Well, it's at 81 rows now, although it looks like the jobs have all run (you can always check at this link). I don't know why 9 of the pages didn't get their data stored. In any case, I think you can safely call that null-edit bot now if you want, without fear of leading to duplicate data. Yaron Koren (talk) 17:42, 29 January 2018 (UTC)


 * That's a useful link I didn't know about, I'll keep that in mind when tinkering/troubleshooting SaschB (talk) 20:17, 30 January 2018 (UTC)


 * Haven't done any null-editing on the pages in this table/category yet, a couple of new pages have been added, so the total should now be 91 or 92. Table is up to 89 records, 72 hours after being recreated. SaschB (talk) 18:45, 31 January 2018 (UTC)

I guess there were remaining jobs after all, then, even though that statistics API link said there were none. Well, it's great that the table is getting fully populated, although it's too bad that it's taking so long. This is the kind of thing that makes me glad that the "replacement table" option exists, though. Yaron Koren (talk) 20:05, 31 January 2018 (UTC)


 * Most of the time, when adding new data, we can't wait several days, as information is often announced/made available mere hours before being needed (by users), so I guess I'll keep batch-null-editing to help it along, even if it causes duplicates. SaschB (talk) 08:28, 1 February 2018 (UTC)


 * Alright. In any case, you can always create a replacement table the "right way" afterwards, as you did here. Yaron Koren (talk) 15:41, 1 February 2018 (UTC)


 * I'll probably skip that step, as there's no simple way of knowing when the replacement table is finished (the one in this example wasn't finished after 72 hours, and I don't know when/if it would have been finished without manual intervention). I'd rather have a complete table full of duplicates than an incomplete table without duplicates. SaschB (talk) 17:21, 1 February 2018 (UTC)


 * A replacement table lets you have both, in a sense, until it gets filled completely and can be switched in... Yaron Koren (talk) 18:24, 1 February 2018 (UTC)


 * Well, yes, I see your point, but with new data being added with our frequency, we often have to rebuild the tables once every two weeks - so keeping track of a table for one week in order for it to lack duplicates (but work the same) for one week doesn't make sense to me. I know we're not supposed to have to recreate the tables when data is added/edited, and some of these issues have been solved, but I'm not sure if all of them have been (yet). SaschB (talk) 21:57, 1 February 2018 (UTC)


 * Yeah, on that note - sorry to belabor the discussion, but I didn't understand what you meant about new data being added. Are you just talking about more data, without any changes to the data structure? If so, what's the benefit of recreating the table? Yaron Koren (talk) 22:08, 1 February 2018 (UTC)


 * I am just talking about new data (INSERT or UPDATE), but with previous issues of pages not updating the live cargo tables correctly/immediately upon page edits, my workaround to ensure all new data was stored was the previous procedure of recreate -> replacement -> null-edit -> switch in. As of recently, it seems the storage is a bit more robust, so hopefully I won't have to in the future. If I have to, however, that's what I'll do. SaschB (talk) 12:12, 3 February 2018 (UTC)

It happened again: shows  as a result, even though the value isn't there. Also /+ and / SaschB (talk) 16:59, 3 February 2018 (UTC)


 * That's too bad that this storage problem is still there in the latest code, but of course it's good to know that it's there. I looked into this, and I think I figured out what was causing it. I just checked in a fix - hopefully it'll get to you soon, and hopefully it really does fix the problem. Yaron Koren (talk) 05:26, 5 February 2018 (UTC)

CSV output only exports 100 rows
The CSV output format only outputs the first 100 rows. Is there any way to get it to output more? --pcj (talk) 18:56, 29 January 2018 (UTC)


 * You can set the number with "limit=" - hopefully you can set it high enough to include all the rows. Yaron Koren (talk) 19:12, 29 January 2018 (UTC)

Error with Special:CargoTables after deleting attached templates
Hey there. I recently deleted a bunch of templates that were attached to a table and subsequently visiting the Special:CargoTables page gives me this: [94d2634e1606751386c8fc50] /a3/index.php?title=Special:CargoTables TypeError from line 145 of /home/sites/1b/e/e73edcb322/public_html/a3/includes/linker/LinkRenderer.php: Argument 1 passed to MediaWiki\Linker\LinkRenderer::makeLink must implement interface MediaWiki\Linker\LinkTarget, null given, called in /home/sites/1b/e/e73edcb322/public_html/a3/extensions/Cargo/includes/CargoUtils.php on line 971

Backtrace:


 * 1) 0 /home/sites/1b/e/e73edcb322/public_html/a3/extensions/Cargo/includes/CargoUtils.php(971): MediaWiki\Linker\LinkRenderer->makeLink(NULL, NULL, array, array, array)
 * 2) 1 /home/sites/1b/e/e73edcb322/public_html/a3/extensions/Cargo/specials/CargoTables.php(298): CargoUtils::makeLink(MediaWiki\Linker\LinkRenderer, NULL)
 * 3) 2 /home/sites/1b/e/e73edcb322/public_html/a3/extensions/Cargo/specials/CargoTables.php(353): CargoTables->tableTemplatesText(string, array, array)
 * 4) 3 /home/sites/1b/e/e73edcb322/public_html/a3/extensions/Cargo/specials/CargoTables.php(28): CargoTables->displayListOfTables
 * 5) 4 /home/sites/1b/e/e73edcb322/public_html/a3/includes/specialpage/SpecialPage.php(522): CargoTables->execute(NULL)
 * 6) 5 /home/sites/1b/e/e73edcb322/public_html/a3/includes/specialpage/SpecialPageFactory.php(576): SpecialPage->run(NULL)
 * 7) 6 /home/sites/1b/e/e73edcb322/public_html/a3/includes/MediaWiki.php(283): SpecialPageFactory::executePath(Title, RequestContext)
 * 8) 7 /home/sites/1b/e/e73edcb322/public_html/a3/includes/MediaWiki.php(851): MediaWiki->performRequest
 * 9) 8 /home/sites/1b/e/e73edcb322/public_html/a3/includes/MediaWiki.php(512): MediaWiki->main
 * 10) 9 /home/sites/1b/e/e73edcb322/public_html/a3/index.php(43): MediaWiki->run
 * 11) 10 {main}

I'm not sure if the deletion of the templates was actually the cause but I'm at a loss re: how to fix it. Viewing the table pages themselves seems to be fine and nothing else is borked as far as I can tell.

Rurikawa (talk) 13:46, 1 February 2018 (UTC)


 * Very odd - looking through the code, and your backtrace, it looks like the problem is that one of the "attaching" templates that you deleted did not get its "attached to a table" page property deleted from the page_props table - so the Cargo code sees the property, goes to display a link to that template, then crashes because the template doesn't exist. I didn't know that could happen. Anyway, I just checked in what is hopefully a fix, so that the code checks whether a page is null before trying to link to it. Yaron Koren (talk) 15:29, 1 February 2018 (UTC)


 * That fixed it! Thank you for the quick response. Rurikawa (talk) 19:14, 1 February 2018 (UTC)

Data not updating.
Hello,

I have installed Cargo, and added a couple of templates (http://crewsgenealogy.com/wiki/index.php?title=Template:Infobox_person). I belive I have added the code correctly, cargo doens't produce any errors when I set up the tables, but it doesn't seem to be saving any of the information. I have tried the maintenance script cargoRecreateData.php which ran without errors, but still nothing. I have several pages with the template, so I would think something would be saved. If it makes any difference it seems to be stuck with the message "Note: One or more of these tables are currently being populated, via the job queue." even after I ran the script.

Any suggestions?


 * I see the #cargo_declare which defines the table itself, but I can't see a #cargo_store, which defines which data to store in the table. Might be as simple as that? SaschB (talk) 12:18, 3 February 2018 (UTC)
 * Example:

Automatic |no html added when query used in page forms?
I have this query in a form, and the query works but it seems like it's adding a |no html to it. This is how the same query looks embedded in a page. Is this a Cargo issue, and if so any way to fix it? Thanks! --RheingoldRiver (talk) 17:35, 3 February 2018 (UTC)


 * There are a number of complicated aspects to that query you have - maybe the most relevant one is that the query is itself calling a template, in order to make the table display more customized. If it's not too much work, could you try testing to see if a simple table display of the query, instead of that template display, results in that same error? If not, there may be some workarounds (though ideally your current query should work as well). Yaron Koren (talk) 14:36, 4 February 2018 (UTC)


 * That table actually has no fields that directly have markup stored into them, but I made this instead and the markup is showing properly. We also have a number of query forms in SMW with relatively complicated "line" templates such as this one that I was hoping to be able to switch over to Cargo in the next couple months. --RheingoldRiver (talk) 21:56, 4 February 2018 (UTC)


 * Something that may be related is that on occasion we will see similar behavior on an actual wikipage, i.e. it looks like this after saving. As soon as another user opens the page, it displays normally [edit - actually this may not always be the case, after a comment from someone on another Gamepedia wiki that they had the issue persist on another user viewing the page]. This is the page that screencap is from, though it can happen on a number of pages, and this problem isn't unique to the LoL wiki, a bunch of Gamepedia wikis have experienced the same thing. In the case of that page I changed "Smeb" to "Smeb2" and back, repeatedly. Whether the bug happens or not is relatively inconsistent, but I was playing around with user rights on my bot account, and it seemed to happen more often when I had Admin than when I didn't. --RheingoldRiver (talk) 22:49, 4 February 2018 (UTC)


 * It sounds like you're saying that the Special:RunQuery bug only happens when the "template" format is used in the query. Which is still bad, but not as bad as if happened for every query. Until this bug is actually fixed, I can think of two possible workarounds: (1) switch to the "table" format, and replicate as much of the logic as possible using MySQL functions, and (2) do the whole query display in a Lua module. (I suppose there's a third option, which is to use SMW for all these queries, but I'm leaving that aside for now. :) ) Would either of these be an option?
 * As for the second issue - this is the first I've heard of this. My only thought is that this is related to whether the page is being cached or not; although I can't see the error on that page in (I think) either state. If you can figure out some way to reproduce it consistently, that would be very helpful - though that's probably obvious. Yaron Koren (talk) 01:20, 5 February 2018 (UTC)


 * Yeah that should probably be an option for this template, though we definitely have some that I wanted to convert to Cargo where it wouldn't be possible. I'll let you know if I see any (Cargo-related) bugs with it when trying to change it to just a table query.


 * I tried for about an hour to find a combination of user rights that led to the problem happening consistently, and sadly I couldn't. This error has been around for a while, but it's always been super intermittent; and it's never happened when I'm logged in as myself (in Firefox fwiw, while my bot is in Chrome). And since our last issue with saving pages was a conflict with another Gamepedia extension it's also possible that that's what this is. AFAIK this is also only happening when |format=template (maybe suggesting the two are related?), but nearly all of my queries are |format=template so that's not necessarily the case. I'll ask around and see if anyone had the same issue occur with another format. --RheingoldRiver (talk) 07:57, 5 February 2018 (UTC)


 * Oh also, let me know if you want Editor on our wiki ever since we have FlaggedRevs enabled. --RheingoldRiver (talk) 07:59, 5 February 2018 (UTC)

Preventing cargo_store on a transcluded page
I'm trying to use Cargo for a blog, and am using cargo_query to transclude blog-post pages onto a front page (using format=template, and the template then transcludes the full page with ).

The problem I've got is that I don't want the cargo_store directives to be processed on the main blog page (i.e. where the most recent 10 posts are listed). To this end, I tried wrapping the cargo_store in a conditional statement:

But this for some reason this means that the cargo_store is never processed, on any page. Running cargoRecreateData.php reports the correct number of pages that call the template, but no rows are stored in the table.

Does anyone have a suggestion of what I'm doing wrong? Thanks! Sam Wilson 02:40, 10 February 2018 (UTC)


 * Are you 100% sure you have no typo in the cargo store vs the cargo declare field names? If there's a single field in the declare that doesn't exactly match a field in the store it won't save any row. --RheingoldRiver (talk) 12:43, 10 February 2018 (UTC)


 * Yep, good point though. It does work though as soon as I remove the wrapping, so I'm assuming there's something odd going on with the PAGENAME magic word. Anyway, I should probably be looking at doing this a different way, and perhaps storing the text of the blog posts as template parameters or maybe using _pageData table. I'll keep experimenting! Sam Wilson 03:20, 11 February 2018 (UTC)


 * The syntax of the ifeq is wrong in your example on this page:  has an extra pipe at the start, and should be  . That might be it? The example you provided compares PAGENAME to [blank], evaluates to false, skips the true value "Blog", to render the false value [blank]. SaschB (talk) 08:20, 11 February 2018 (UTC)


 * Gosh, I feel very silly! :-) That was indeed the problem. Thank you. I think I'd stared at it for far too long. Sam Wilson 00:49, 12 February 2018 (UTC)


 * Let me just add that it would be slightly simpler to use "format=embedded" instead, which would hopefully have the same output. Yaron Koren (talk) 02:35, 12 February 2018 (UTC)

Page edits breaking Lua queries?
Hi, me again! Example page, invokes a Lua module which performs a Cargo query: local ctables = 'Heroes' local cfields = '_pageName,WeaponType,MoveType,Rarity' local cquery = { groupBy="_pageName", limit="500" } local heroQueryResult = cargo.query(ctables, cfields, cquery) and then sorts through the results, displaying them in a custom table.

Whenever one of the pages returned as a result by this query is edited, it disappears from the page, presumably due to no longer being returned by the query. The edited page still shows all the data in the Cargo table, but it disappears from the module-generated table on this page, and others like it.

Missing unit, not yet missing unit. Both from the same cell (Shuriken column, 3-4* row).

Any ideas why? SaschB (talk) 07:24, 11 February 2018 (UTC)


 * My only thought is that maybe these newly-saved pages are now exceeding the query limit, because they've moved to the end of the results. I know you set a limit of 500, which should be more than enough, but maybe it didn't "take", for one reason or another. Is that possible? Yaron Koren (talk) 18:52, 11 February 2018 (UTC)


 * I'm not sure I follow, the table itself only has 225 records total, at the moment, so the limit shouldn't be a problem, right? Are you saying I should edit the  part to something else? SaschB (talk) 09:58, 12 February 2018 (UTC)


 * I guess so - I'm saying, perhaps the "limit" value is not getting sent correctly. It doesn't seem that likely, but it's my only theory. Yaron Koren (talk) 15:42, 12 February 2018 (UTC)


 * To be clear, it doesn't have to be an edit to the #cargo_store-calling template, just adding some text to a different part of the page also results in their data disappearing from the lua-generated table, but not from the Cargo tables. Ever since this edit to the "Lyn" article, "Lyn" has been missing from the page I linked initially. Do page edits like this also cause the records to move to the bottom of the SQL tables? SaschB (talk) 16:23, 12 February 2018 (UTC)


 * Yes. Every edit to a page, no matter how small, wipes out the Cargo data for the page and recreates it. Yaron Koren (talk) 16:30, 12 February 2018 (UTC)


 * Okay, I figured it out. A few weeks ago, an anonymous user made an invisible (but actual) edit to one of the deeper templates used to generate the values stored in the Cargo table. This change only updates the values once each page is edited, but doesn't produce visible changes to the value, however it does break the Lua code. Simply put, this isn't indicative of any Cargo issue at all, false alarm! SaschB (talk) 17:06, 12 February 2018 (UTC)


 * phew That's great to hear! Yaron Koren (talk) 18:48, 12 February 2018 (UTC)

API improvements
Hey :) I was just playing around with the Cargo api and noticed a few things I'd like to bring to your attention: --178.2.47.251 20:09, 22 February 2018 (UTC)
 * Field names starting with an underscore (_pageName, _pageID, ...) only work if a display name is set (e.g. ).
 * The results key is always title (Something like the row id might be more useful here).
 * Some way to get a list of all tables and all fields in a table might be a nice addition to the api.
 * Not limited to the api, but is there some way to use  in Cargo?


 * Hi - my responses:
 * Are you sure about that? If so, what version of Cargo are you running?
 * I don't understand this one.
 * That's an interesting idea; what would it be useful for?
 * Unfortunately, no, but it would be nice to have. Yaron Koren (talk) 17:11, 23 February 2018 (UTC)


 * Thanks for your answer!
 * Yeah I'm sure I just tested it again (See &fields=_pageName and &fields=_pageName=someName). I'm on version 1.6.
 * I meant that the json output always has this format (with "title" being static instead of something specific like the row id):

{ "cargoquery": [ {           "title": { "someField": "someData", "someOtherField": "someOtherData", }       },    ] }
 * Having a way to get all cell names would make it possible to simulate  and easily get all data in the table without having to specify the field names in the script and having to change it whenever the table changes. For the table names I'm not quite sure how useful it is and just felt like it was strange there was no way to get them through the api. --178.2.47.251 21:38, 23 February 2018 (UTC)
 * Another thing I forgot about yesterday is the lack of an offset option which would be useful due to the results limit.


 * Oh, I didn't realize you were talking about the API here too. That sounds like a bug; I'll have to look into it.
 * If you want nicer-looking JSON, your best bet is probably to use the "json" result format, instead of the API - it functions as its own sort of API.
 * That's true - being able to get the list of field names would be helpful, especially when calling from Lua or the like.
 * Yes, I agree that an "offset" parameter would be good too. Yaron Koren (talk) 21:53, 25 February 2018 (UTC)


 * Some of these issues are now resolved in the new version 1.7: having a "_" at the beginning of a field alias leads to an error message in API queries, and an "offset" parameter was added. Yaron Koren (talk) 19:10, 9 March 2018 (UTC)

Data types with incorrect capitalization are being stored as varchar
If I create a "float" field then Cargo is creating the SQL column as varchar(300) instead of float. "Float" operates correctly. Please add some case-insensitivity handling to the data types. --pcj (talk) 18:38, 23 February 2018 (UTC)

QOL Improvements for page values page
Hey, two related requests about the ?action=pagevalues view. 1 - could you add a table of contents at the top of the page listing all tables that are included on the page? And 2, could each heading link to the table? (Or these could be combined as a list at the top with anchor & table link listed for each table.) Not super important but would be a nice QOL thing. Thanks! --RheingoldRiver (talk) 01:30, 24 February 2018 (UTC)

timeline causing error
When using the following cargo query

I get the error on the page:

Caught exception: SyntaxError: expected expression, got '<'

and no timeline is displayed. Not sure what could be the problem. Using cargo version 1.6 (5a44fdc)


 * This doesn't seem to happen for me - I'd recommend upgrading to the latest version, 1.7 (released today!), to see if maybe that was a bug that's been fixed. Yaron Koren (talk) 19:52, 9 March 2018 (UTC)


 * Thanks for your reply, updated today but the issue persists. I noticed the error also occurs for me on this page from the Cargo example page: http://discoursedb.org/wiki/Virginia_United_States_Senate_election,_2006


 * Oh yeah, now I remember... this sometimes happens due to page caching, for reasons I can't explain. I need to look into, but I think you can get around the problem by installing the MagicNoCache extension, and adding " __NOCACHE__ ", on a line by itself, to any page (or better yet, template) that calls the "timeline" format. It's not an ideal solution, but it should work for now. Yaron Koren (talk) 14:04, 12 March 2018 (UTC)


 * I see, one of the example pages had __NOCACHE__ but I didn't know what it meant. I have updated Cargo, added the magicnocache extension and added __NOCACHE__ but now there is a different error "TypeError: v1 is NULL"

Table with more than 70 columns will not create
Hello, we're trying to work with a table that has 84 columns, but our tests are showing that if it has more than 70 columns it will not create. https://dota2.gamepedia.com/Template:User:Litzsch/Cargo_test is the test table currently. There isn't any data going in it yet, just trying to define the table. It creates super quick at 70 columns (up to buff4 in the table) but if I add just one more column to the declare it just gives a spinning wheel with no action. Any ideas? --pcj (talk) 13:22, 11 March 2018 (UTC)


 * I'm not sure why it's failing for you - these 70 rows appear to be all of type "String", which in Cargo translates to "VARCHAR(300)", which should be somewhere around 21,000 bytes, which is significantly less than MySQL's limit of 65,535 bytes per row.
 * However, it seems to me that what you're trying to do would better be accomplished with multiple-instance templates. Do you know about those, from Page Forms? Here, you're trying to store three different two-dimensional tables of data, within each row of this table. I think switching to multiple-instance templates would make for better storage, easier editing, and easier querying. Yaron Koren (talk) 14:17, 11 March 2018 (UTC)

how do I get plain numbers from aggregat functions
hi,

when I do a #cargo_query and select one integer field I get the integer back as a result – e. g. 5000 –, but when I query using MAX( integer field ) the result is a formatted number like 5,000.0. It's the same when using SUM. I just wonder why this happens and if it's possible to suppress this formatting?

Doing the same thing using direct sql within pma produces plain numbers for all queries. So is it a cargo or mediawiki issue? Or am I just doing something wrong? --~o)sabine(o~ (talk) 15:10, 15 March 2018 (UTC)


 * It's a Cargo issue - some helpful formatting around Integer and Float fields. If you want to get rid of all the formatting, the easiest way to do it is to put "CONCAT" around the whole thing, which will turn it (in Cargo's eyes) into a String field. If, on the other hand, you just want to switch around the the "," and ".", you can use the global settings $wgCargoDigitGroupingCharacter and $wgCargoDecimalMark for that. Yaron Koren (talk) 15:54, 15 March 2018 (UTC)


 * Thank you for the fast answer. I have to get plain numbers for calculations with #expr. All went well until the highest number reached 1000. After that my form didn't work any longer…
 * I apologize for bothering you, because when I read your answer I remembered reading this somewhere or at least something similar.
 * And well, I wrote this earlier – maybe a year ago –, but thanks again for this great extensions, especially Cargo and PageForms make my work a lot easier :) --~o)sabine(o~ (talk) 21:00, 15 March 2018 (UTC)


 * I'm glad to hear it! And no problem. Yaron Koren (talk) 14:50, 18 March 2018 (UTC)

Querying a table on the same page and writing the result to a field
I've known about this limitation for a while but I figured it was too specific to ask about, but it's come up a couple other times on other wikis since, that said I don't really expect this to be possible to fix but figured I'd bring it up anyway just in case - If you have row 1 of Table 1 stored on a page, and on the same page you make a query of Table 1 row 1 and attempt to save the value to Table 2, the order in which page content is saved makes it so that you don't actually get the queried value stored. Recreating the table will make everything show up as intended, but then further edits to the page break it again.

Also, you can't store a #var_final, so this isn't an option for saving a value that depends on a template lower on the page.

Any way to get around either of these, or is the current page save order impossible to touch? --RheingoldRiver (talk) 07:21, 18 March 2018 (UTC)


 * If you want to use a value earlier on the page than where it's set, that's going to be tricky. Out of curiosity, why do you need to store this value in two different tables? Maybe there's an easier way to structure this? Yaron Koren (talk) 14:48, 18 March 2018 (UTC)


 * Actually in the case today, he wanted to store in the same table (fighting moves evolve from each other, and he wanted to input only the "to" and compute the "from"), but he ended up just doing one single input for all instances on the page in order to do the computation without Cargo, and I don't remember what the second use case was. The reason I wanted to was: In PUBG, most games are played with teams of 4 players each, and that's how teams standardly are, but sometimes there's events with 2 players. In these 2-player events there's a mix of standard teams and pick-up teams. So in some cases I will be identifying a squad of two players by their proper team name, and sometimes by a generic "unaffiliated" tag with a page-specific unique ID to refer them together. (Unique ID is manually entered by the editor and I want it to be mandatory only in the case that it's actively disambiguating.) I have one table that associates players to teams, along with the team's seed etc, which is defined in the team rosters template, and a second one for tournament placements that does team + prize + placement etc (but no rosters). Results are higher on the page than rosters. Normally in queries I just join on the team field, but this case is more complicated, so in the results template I wanted to query the rosters table to get the participants and store directly in order to avoid more complicated queries when I use the data elsewhere, but still be able to enter players only one time. I ended up just giving up on this, and I've been doing the more complicated where clauses where it looks for team if unique id isn't defined. --RheingoldRiver (talk) 15:29, 18 March 2018 (UTC)


 * Now that I think about it I could probably have unique ID default to just be the team variable in this case since there will be an ID iff team *isn't* unique. So that's an even easier workaround. But it would still be nice if var_final could be made to work. --RheingoldRiver (talk) 15:31, 18 March 2018 (UTC)

Multiple HOLDS checks on a single field?
Does HOLDS not work for seeing if a list has multiple specific values? Looking at _pageData._categories specifically. I want to know if the page is in two different categories but it returns no results, contrary to expected behavior. Or am I doing something wrong?
 * Query
 * Tables.

Thanks --pcj (talk) 00:06, 22 March 2018 (UTC)

Can I get the hierarchical structure as a result?
Hi,

I'm doing my first steps in using a hierarchy instead of a list of allowed values and I just wondered, if it's more of a visual help when having a long list of values or if it's possible to get a part of the hierarchical structure as a result.

I have a structure of categories and a page that's part of category C. So it should automatically be part of A and B, too. But can I get all three categories as a result in this case or do I have to handle this in the template e. g. with a case trap? --~o)sabine(o~ (talk) 13:18, 22 March 2018 (UTC)
 * A
 * B
 * C
 * 1
 * 2
 * 3


 * See 'The "WITHIN" command' - is this the sort of thing you're asking about? Though it doesn't work with MediaWiki categories, unlike SMW - you have to define the hierarchy in #cargo_declare. Yaron Koren (talk) 13:29, 22 March 2018 (UTC)


 * No, it's not the solution I was looking for. With the WITHIN command I can get the subcategories, but I need to get the supercategories of a subcategory. --~o)sabine(o~ (talk) 17:08, 22 March 2018 (UTC)


 * Oh... now I get it. No, it's not possible, but this is something I've actually wanted to add to Cargo for a while - the current plan is to have a parser function called #cargo_hierarchy_values, that takes in the parameters "table", "field", "root", "level" and maybe "direction", and displays a tree. It's good to know that there's an interest in it. Yaron Koren (talk) 20:23, 22 March 2018 (UTC)


 * Sounds great. I'm looking forward to it. But if I could vote for a new function, it would be child tables over this. I'm using a lot of them and it would be lots more, when handling them would be easier :D
 * But I don't want to push you in one direction, I know almost all programmers are short on time while having too much to work on. And as far I can see you're doing a really great job and good programmers have even more to do :) --~o)sabine(o~ (talk) 21:09, 22 March 2018 (UTC)


 * Thanks! And both of those are good to know. I'm trying to set the priorities now for future development, so it's good timing to be discussing this kind of thing. Yaron Koren (talk) 21:40, 22 March 2018 (UTC)