Extension talk:Cargo

Where are errors defined for creating a new page with a property that is set to be unique?
I'm trying to create short id's that are more user friendly for use in a url. However, since the likelihood of a collision is more of a concern than with RFC compliant UUID's. I need to handle that error but I'm not sure how the error is expressed. How can I check, after creating a page with a potential collision, what the error array looks like?


 * I would say that the easiest and most foolproof way is to just check the value of that field and see if it's null or not - assuming it would only be null in the event of an error. Yaron Koren (talk) 02:29, 1 June 2018 (UTC)

== What would be the best way to keep track of the page's revision id such that I don't overwrite the page when my extension's local revision id is less than the page's current revision id. Magic words don't work. ==

I'd like to get the revision data when I do my initial  so I can reduce the overall number of api queries if possible. However, if it's not doable I can just make a revisions api request on page load.


 * Nice subject line. :) Is this a Cargo question? It sounds like a general MediaWiki question. Yaron Koren (talk) 16:40, 3 June 2018 (UTC)


 * Well there is this  but I wasn't sure if that was the only approach towards determining potential conflicts. Also it says it's a date but is that actually a datetime?


 * Yes, it's actually a datetime. Yaron Koren (talk) 18:15, 4 June 2018 (UTC)

Updating Cargo table's columns causes "no database exists" error
I'm using Cargo with the PageForms extension. After making my forms I decided to add some fields. After modifying the template associated with a Cargo table and recreating the table from the template's page, I can see the new fields' columns in the Cargo table in PhpMyAdmin. However, when I try to create a new page using a form referencing the template tied to the updated Cargo table, I get an "Error: No database table exists named x" message. Here's the odd part: When I go into PhpMyAdmin and manually change the Cargo main_table's name to "cargo__x", the form works! However, when I try to access the table using the Special:CargoTables page on my wiki, I get this error: "This table is registered, but does not exist! (not defined by any template)." How can I fix this error?


 * Could the problem be with the #cargo_store call in the template? Is the "_table" value set to the same thing in #cargo_declare and #cargo_store? If that's not the issue - could there be something unusual in the table name, like non-Latin characters, or punctuation? Yaron Koren (talk) 20:27, 3 June 2018 (UTC)


 * Strangely enough it went away and works fine now. May have been a caching error.

Why does HOLDS implement RIGHT OUTER JOIN?
I'm utilizing the very helpful HOLDS command, but I've having confusion in why it uses RIGHT OUTER JOIN as opposed to LEFT OUTER JOIN because when I try to join with an empty array of IDs I would just expect it to still return the object from the first table.

I have a table Action_Items and I'm joining it with Updates because Action_Items can have n updates. However, if I do HOLDS with Action_Items.updates that is empty the overall result is an empty array as opposed to the originally queried Action_Items information.

UPDATE: I changed it to a   by just editing   and got the result I expected.

_pageData (and other tables) not storing values for a page
Hello, we're running into an issue with this page in that no Cargo tables are being stored for it, even _pageData. Tried null editing and even deleting and recreating the page with no change. Meanwhile other pages are working fine. Any ideas on how to resolve? --pcj (talk) 17:38, 6 June 2018 (UTC)


 * Is it only that page for which the problem is happening? If I had to guess, I would guess that the massive amount of data in that page is causing the database connection to time out while the code tries to insert all the values. Or are there pages that are succeeding with even more data? Yaron Koren (talk) 01:32, 7 June 2018 (UTC)


 * Haven't found any other page that has this problem (yet). We have a page where 793 rows are stored in a single table, which is much more then this particular page should create (which is (besides page data) 1 row in items, 1 row in skill gems, 40 rows in skill_levels, 4 in skill_stats_per_level, 1 row in item_purchase_costs, 1 row in item_sell_prices, 1 row in skill).
 * There are other pages with the same template that work as well.
 * On the other hand, it seems that it somehow stops somewhere in between, which would speak for the timeout thing, since it creates various rows and doesn't prune the old one, which is probably due to it not really finishing the process (so I assume anyway) see this for example
 * Not sure if it is related, but I've also noticed that the intermediate tables for list fields suffer from the duplication/data not being pruned problem that was fixed in cargo a while ago. Perhaps that is somehow related? OmegaK2 (talk) 11:41, 7 June 2018 (UTC)


 * Alright. I didn't know about the problem with duplication in the helper/intermediate tables - that's very good to know. I don't think that's the cause of this issue, but who knows. What I would suggest is removing parts of that problem page until you can get it to save its data, to try to isolate what exactly is the cause of the error. I wouldn't be surprised if it's some character or other unusual value. Sorry to make you do that work, if you end up doing it - I can't think of a better approach. Yaron Koren (talk) 16:16, 7 June 2018 (UTC)


 * It seems I might have figured the cause of this particular issue. There is an unique field for one of the tables - if I remove the data and then add it back in it works. I'm guessing the underlying problem here might be that it tries to insert into the table where a row with the unique already exists on the same page before deleting the old row and as a result gets some kind of error and stops all subsequent actions. I'm not sure why deleting/restoring the entire page didn't work though (I assume it should have nuked all the entries in tables for the pages). OmegaK2 (talk) 22:34, 7 June 2018 (UTC)


 * Oh... that's a great find. It's not totally surprising: there was a fairly recent change in Cargo's data storage to make it one big DB transaction, and that's probably causing unrelated inserts to not get run when one insert runs into a "unique" problem. I'll have to look into that. Yaron Koren (talk) 23:21, 7 June 2018 (UTC)

Issues querying fields with extended characters in the name?
We're running into some issues querying data from a table with fields with extended characters in the column name. For this table, querying for funções or "jogabilidade_de_heróis.descrição" fails while just "descrição" or jogabilidade_de_heróis.complexidade succeeds. The issue is perhaps more apparent when used on this page. Any thoughts on a resolution or workaround? --pcj (talk) 13:50, 7 June 2018 (UTC)

Special:CargoExport JSON output has footer warning
Back with another weird error. I'm using Cargo's Special:CargoExport feature to get JSON output for a javascript widget. Ever since I deleted some tables via PhpMyAdmin, when I try to manually get the JSON output using variables in the URL, I get the JSON output, but then also this bit at the end:

It's annoying because I've been using a $.getJSON to convert the output to JSON and that doesn't work anymore. I know I could use a javascript workaround (i.e. convert the weird output to a string and strip away everything from the  onward) but I'd rather treat the cause and not the symptoms to avoid other weirdness. Any ideas?


 * Is there any other error message, or is that it? Usually that "headers already sent" error comes after something else has been printed on the screen. Yaron Koren (talk) 23:30, 7 June 2018 (UTC)


 * No, the JSON prints perfectly right before in the [, { bracket formats. It's just these annoying break tags and error after that. Another update: The problem seems to be caused by the number of Cargo fields included in the export. When I remove one of the fields, the error described above goes away and the JSON loads smoothly. When I try to add back a field, it breaks down again.


 * Oh, that's very interesting. Is it one specific field, or does it happen if you remove any field? If it's the latter, how many fields are you querying? Yaron Koren (talk) 23:21, 8 June 2018 (UTC)


 * It seems to be 4-5 max, unless I include a field that contains a lot of text -- then the length doesn't matter, I get the error regardless. The issue seems to be the length of the text inside the field, or possibly the special characters inside. Is there a way to increase this limit?


 * I'm guessing that this is somehow due to one or more specific values... is this on a public site? Yaron Koren (talk) 18:29, 10 June 2018 (UTC)


 * Unfortunately not -- it's an internal site for a team. I made a test Cargo table to try to recreate the problem. With a string length of about 120 words (700 characters), I could only export one field at a time -- any more would produce the error. Could this be an export limitation from within MediaWiki itself?
 * Went with the workaround, using $.get and then stripping away the Cargo errors. If anyone finds a solution in the future I'd be grateful.

Parsing email addresses and dates
Hello all

I'm using Cargo tables to store users' email addresses, and dates. I'd like to parse these to show the email addresses and links, and to change the format of the dates from YYYY/MM/DD. Is this possible?

This is my query showing email addresses:

and for the dates, it's simply this:

| Date

Any help or pointers would be much appreciated! --Melat0nin (talk) 12:52, 10 June 2018 (UTC)


 * For emails, how do you want to change their display? And for dates, are you talking about displaying them in a query, or just displaying the template parameter (as you have in your example)? Yaron Koren (talk) 18:03, 10 June 2018 (UTC)


 * Thanks for the reply. I managed to implement what I was looking for using the CONCAT and DATE_FORMAT SQL commands. Now I'm trying to implement a recursive LEFT OUTER JOIN...--Melat0nin (talk) 13:52, 11 June 2018 (UTC)

PostgreSQL and replacement tables
When the checkbox to use a replacement table is enabled:

I think there may be concurrency errors or attempts by Cargo to manually assign IDs over using sequences.

Jun 12 06:31:02 db postgres[90423]: [7-1] ERROR: duplicate key value violates unique constraint "cargo_tables_template_id_key" Jun 12 06:31:02 db postgres[90423]: [7-2] DETAIL: Key (template_id)=(9908) already exists.

New __NEXT tables are created in the cargo database: public | cargo__ships__NEXT        | table | azurlane_wiki public | cargo__ships__NEXT___files | table | azurlane_wiki

But Cargo throws exceptions:

[exception] [4ff8ae73bec7d8b9f2d2807b] /Special:CargoTables/ships?_replacement  MWException from line 196 of /var/www/azurlane.koumakan.jp/w/extensions/Cargo/includes/CargoUtils.php: Error: Table ships__NEXT not found.

The only way to fix this is to manually drop these __NEXT tables as Cargo is unable to do anything with the table in question. Trying to delete the table and recreate data without dropping these NEXT tables obviously fails.


 * Sorry for the delay. What version of Cargo are you running? I would try switching to the very latest code, if you're not running it already. Yaron Koren (talk) 18:04, 14 June 2018 (UTC)

Cargo Admin User Class?
Would this make sense to have the extension create? To grant  and any other right that Cargo only gives to sysops currently. --RheingoldRiver (talk) 11:42, 13 June 2018 (UTC)


 * I don't think it makes sense for MediaWiki extensions to define their own user groups, except in rare cases, but you could do it yourself on your wiki (or have the admins do it) by just adding something like the following to LocalSettings.php:


 * That by itself is enough to define a new user group in MediaWiki. Yaron Koren (talk) 16:15, 13 June 2018 (UTC)

unique field not working?
I was trying to recreate a table on Gamepedia with a unique param, and instead of recreating it just got hung up on the spinning wheel indefinitely. I tried recreating on a sandbox wiki here, and same thing - regardless of how small we set the size it still won't create. You can check revision history to see things we did, basically every iteration without a unique param recreated no issue, but with the unique it failed with infinite loading circle. Do you know what might be going on? Thanks --RheingoldRiver (talk) 16:03, 13 June 2018 (UTC)


 * It should be "size=", not "size:" - that might be the issue. Hopefully it works with a smaller field size... Yaron Koren (talk) 18:08, 14 June 2018 (UTC)
 * Same thing is still happening. I tried . --RheingoldRiver (talk) 23:02, 14 June 2018 (UTC)
 * Okay, I just tried this out for myself, and now I see the issue. The problem is that the field in question, "UniqueLine", is of type "Text" - and the "unique" keyword doesn't work at the moment with "Text" fields, since they get stored as BLOBs, which have special handling for uniqueness. (I never thought to test it with "Text".) I guess there are three options: add support for "unique" with "Text" (which would involve just checking some initial substring of the value for uniqueness), ignore "unique" if it's a "Text" field, or display an error message. (The second two are more or less the same.) Any thoughts on this? Yaron Koren (talk) 03:33, 15 June 2018 (UTC)
 * I think we need unique to work with fields of type "Text" in order to avoid duplication. The last of our SMW that isn't in Cargo yet is our (huge) database of game stats + (many) templates that query it, and I need to do queries with a bunch of fields with SUM and COUNT, so if there's dupes all of this will give the wrong results; and my current plan for setting a unique field is to do, where N increments with each line that's stored. If you can think of an alternative solution for this issue then I'd be happy to consider something else, but I haven't been able to think of any other option. --RheingoldRiver (talk) 11:19, 15 June 2018 (UTC)
 * Ah, pcj reminded me that text and string aren't the same in Cargo as they are in SMW. I can probably use string instead --RheingoldRiver (talk) 11:48, 15 June 2018 (UTC)
 * Alright - I think I'll add an error message for "unique" with "Text". Let me know if you decide you need "unique" to be able to work with "Text", though. By the way, "Text" and "String" in Cargo are modelled after their equivalents in SMW, so they should work more or less the same, though there are some differences in the implementation. Yaron Koren (talk) 13:25, 15 June 2018 (UTC)
 * Some SMW version a couple years ago made string and text equivalent right? --RheingoldRiver (talk) 14:00, 15 June 2018 (UTC)
 * That's true... they're stored the same way in the database now, although they're still handled differently in Page Forms and Semantic Drilldown (with "String" intended to represent shorter strings). So I guess it depends on how you look at it. Yaron Koren (talk) 14:21, 15 June 2018 (UTC)

Table of Contents on Page Values page
I think I may have brought this up before but I don't remember any response to it, so apologies if you saw this already. Would it be possible to add a table of contents (ideally a flatlist or aligned right) to the "page values" page? We have some pages with a lot of tables/rows stored, and it would be a nice QOL change over using ctrl+F to navigate. Alternatively a system message that could be used to add a TOC if we want (if this possible to work). Thanks! --RheingoldRiver (talk) 11:26, 15 June 2018 (UTC)


 * Yes, you did bring it up before - it's good to keep reminding me of stuff, so feel free to re-post things if I don't respond. You make a good point that this would be useful for pages with lots of rows; I just checked in an addition to add a TOC to this page. Yaron Koren (talk) 19:52, 18 June 2018 (UTC)

Using Cargo data for reusing file data
We are using Cargo for storing featurd image data on pages. We would like than to load this data on tiles leading to these pages. It seems that we are missing somthing, because this information is not parsed. We are trying to make this work:

This is what we have tried: What can we do about it?
 * 1) Storing the data as "File" will not work with external image repo. Also it does not allow us to create the tile with the specific settings we need.
 * 2) Storing the data as "String" will not parse - we are getting the markup displayed as text (i.e. displaying: [[file:Query-image.jpg|other image settings]]
 * 3) Storing the data as "Wikitext" is parsed in a very strange manner and does not produce the intended behavior. We even tried storing it with "[[file:" appendix.

Also, is there's a way to use Cargo quering for with parserFunctions logic? i.e. - if we want to use a boolean variable in order to hide or show content like this:

Wess (talk) 20:01, 16 June 2018 (UTC)


 * It's possible that adding "|no html" to both #cargo_query calls will fix the problem. Yaron Koren (talk) 19:59, 18 June 2018 (UTC)

Error giving a table an alias when one of the fields queried has type list
Here is the example. In the first query, I'm only querying fields that have Integer types. In the second query one of the fields has a list type and the query fails; the third query doesn't have any kind of join condition, just an alias, and still fails. The final query has no alias and queries the same fields as the 2nd and 3rd, but works fine because there is no alias. Do you know what's going on? --RheingoldRiver (talk) 05:06, 18 June 2018 (UTC)
 * Well I guess I figured out a solution? It works fine if I add . Still this seems weird but no longer something that's in the way of the code I'm working on. --RheingoldRiver (talk) 10:13, 19 June 2018 (UTC)

Adding an _isRedirect field to _pageData
Is it be possible to add an  or similar field to the   table to easily exclude redirect pages from queries? --92.217.24.175 11:39, 23 June 2018 (UTC)


 * That's a very good idea - I just added a column called "_isRedirect" in to the code. Yaron Koren (talk) 16:34, 25 June 2018 (UTC)


 * Thanks! --94.216.183.244 21:28, 25 June 2018 (UTC)

Row separated by a comma
Hi, how to make a row separated by a comma. - 212.80.62.177 06:30, 25 June 2018 (UTC)
 * I found. - 212.80.62.177 06:47, 25 June 2018 (UTC)

Make `mw.ext.cargo.query` raise Lua errors instead of exceptions
Currently the  raises exceptions for most issues apart from missing arguments. There are two major problems with this behaviour: These issues make modules exposing the  function to users non-ideal. I've even noticed cases where such a query "bricked" an entire page to a degree where only deleting and restoring everything but the affected revision would fix it (I'm not sure how to reproduce this though).
 * 1) Exceptions are not catchable using Lua's   function, making error handling impossible.
 * 2) Exceptions prevent the entire page from being saved. While this is usually fine, this can cause some major issues with the entire page not being displayed in some specific situations (e.g. a table/field name changing/being deleted, module changes, …)

Instead, the  function should raise a Lua , which would solve those problems. --Litzsch (talk) 12:09, 27 June 2018 (UTC)


 * This sounds like a good idea, though I don't think I have enough Scribunto/Lua knowledge to do it myself. Would it be possible for you to create a patch that fixes this problem? Yaron Koren (talk) 14:33, 27 June 2018 (UTC)


 * I don't think there's a way to catch the exception in Lua, so you'd have to to it in PHP which I have zero experience in. From what I can see you can just wrap the actual function call using try catch and then throw a  (see here for how Extension:ParserFunctions does it). --Litzsch (talk) 17:13, 27 June 2018 (UTC)


 * Here's some more examples:, , --Litzsch (talk) 17:20, 27 June 2018 (UTC)


 * Okay, I think I get it. Could you try editing the file CargoLua.library.php, in the main Cargo directory, and change these lines:

$query = CargoSQLQuery::newFromValues( $tables, $fields, $where, $join,                       $groupBy, $having, $orderBy, $limit, $offset ); $rows = $query->run;
 * ...to:

try { $query = CargoSQLQuery::newFromValues( $tables, $fields, $where, $join,                               $groupBy, $having, $orderBy, $limit, $offset ); $rows = $query->run; } catch ( Exception $e ) { throw new Scribunto_LuaError( $e->getMessage ); }
 * ...and see if that fixes it? Yaron Koren (talk) 18:35, 27 June 2018 (UTC)


 * Yes it works perfectly. Thank you so much for the quick fix :) Just one more small thing: The  function automatically adds "Lua error: " at the beginning and a full stop at the end of the error message, making Cargo's error message Lua error: Error: Table test not found... You might want to strip the "Error: " and the "." to avoid repetition. --92.217.25.96 19:41, 27 June 2018 (UTC)


 * That's great to hear. I just checked in this fix. Getting rid of the double "Error" text is more complicated than it sounds, unfortunately, but I hope it gets done eventually. Yaron Koren (talk) 02:41, 28 June 2018 (UTC)


 * Wouldn't something like this work:

 $errorMessage = substr($e->getMessage, 7, -1); throw new Scribunto_LuaError( $errorMessage );
 * Or is the  format not consistent? --Litzsch (talk) 12:16, 28 June 2018 (UTC)


 * The main problem is that the string for "Error:" will not always be in English - some of these error messages are hardcoded, but others are translated. Yaron Koren (talk) 13:58, 28 June 2018 (UTC)


 * Maybe using  would work?

 $errorMessage = substr( $e->getMessage, 0, -1 ); $errorMessage = explode( $errorMessage, ": " )[1]; throw new Scribunto_LuaError( $errorMessage );
 * That should work in all languages. --Litzsch (talk) 14:05, 28 June 2018 (UTC)
 * Not necessarily - some languages might not use colons. Yaron Koren (talk) 14:17, 28 June 2018 (UTC)


 * Ah damn :( Well still thanks for the fix itself the message is just a cosmetic thing anyways. --94.216.148.152 14:34, 28 June 2018 (UTC)

Still problems with HOLDS when aliasing table names, also duplicates with HOLDS
So, the workaround of using  that I posted last time I had this issue is now failing for two reasons:
 * 1) I actually want the not full field list in this particular query
 * 2) I'm running into the same issue in my WHERE condition when I'm trying to do an OR.

Here is the query I want to run: (So the not  fields are to see which team banned the champion in question.) But it will not work if I alias   and try to use that instead (and this is important in the case of wanting to self join a table).

Here is the above query on a page both with and without the alias.

Could you see if there's any possible fix for this?

Separately, I'm wondering why I'm getting duplication here -  is forced to be unique. So I think this is another issue with HOLDS?

I would switch to using LIKE, but there are champions whose names are fully contained in other champions names (for example one called Vi and another called Viktor). Ok I'm using RLIKE as a workaround for now but I'd really like to use HOLDS for it so that I can know which field contains the value in question.

Thanks --RheingoldRiver (talk) 17:53, 30 June 2018 (UTC)


 * Sorry about that - unfortunately, HOLDS parsing still doesn't work correctly when there's more than one HOLDS condition. But I think you can get around the problem by using the actual table names, even though that makes things a little awkward. In your case, I think you can change the relevant parts of the query to:

tables=ScoreboardGame, ScoreboardGame__Team0Bans, ScoreboardGame__Team1Bans
 * join on=ScoreboardGame._ID=ScoreboardGame__Team0Bans._rowID, ScoreboardGame._ID=ScoreboardGame__Team1Bans._rowID
 * where=ScoreboardGame__Team0Bans._value = "LeBlanc" OR ScoreboardGame__Team1Bans._value = "LeBlanc"
 * Please let me know if (a) that worked, and (b) you still see the duplication problem. Yaron Koren (talk) 13:39, 2 July 2018 (UTC)
 * Sorry for delayed response. I'm not entirely sure what to put for the fields in this example, but I tried this as the full query:

I'm still getting duplicates, you can see the result here. I was able to alias fields if I only queried the __full values, but I couldn't alias ScoreboardGame as SG and query SG.Team0Bans, only SG.Team0Bans__full. --RheingoldRiver (talk) 04:38, 5 July 2018 (UTC)


 * Actually I'm not sure if the duplicates in this case are a bug even - it's not what I want but maybe it makes sense for them to be there. --RheingoldRiver (talk) 04:40, 5 July 2018 (UTC)


 * Sorry, I should have included the corresponding "fields" value as well - it should be something like:


 * fields=ScoreboardGame__Team0Bans._value=Team0Bans,ScoreboardGame__Team1Bans._value=Team1Bans,ScoreboardGame.Team0Bans__full,ScoreboardGame.Team1Bans__full,ScoreboardGame.UniqueLine}}
 * The duplication problem looks pretty bad... but I guess, one thing at a time. Yaron Koren (talk) 15:11, 5 July 2018 (UTC)


 * That seems to work, yeah. It's a bit nicer that in the HOLDS version, it's either empty string/nil or the value, but that's not a big deal, I can just check if the strings are the same instead of if one is empty. Unfortunately...there are now WAY more duplicates present. Somehow there's 49 lines for this one game, which seems really weird - when it was 6 per it almost made sense if it was giving me one line per entry in one of the bans fields, but this seems totally random. Result is here. --RheingoldRiver (talk) 18:34, 5 July 2018 (UTC)

How to structure Cargo Query for subset of possible fields
I'm working on the template for a Form for RunQuery. I have a single table with 5 possible fields, of which, I want to be able to query on 2 of the fields, or 3 of the fields etc... How do you implement a multiple WHERE to include fields that are filled in, but ignore fields that have no entries?

I've tried single cargo queries for each field - that yields separate tables for each field completed.

I've tried joining all into one cargo query with AND for each field. But that requires all fields to be completed or it gives a no result. May214 (talk) 11:05, 2 July 2018 (UTC)


 * I haven't tried it, but you could potentially have a "where=" clause like " ( =  OR MyField = '') AND ... ". Yaron Koren (talk) 13:54, 2 July 2018 (UTC)


 * Thanks, Yaron. I ended up using the #if parser inside the where clause, terminating each with AND. Then I ended the where clause with a _pageID > '0'.  Seems to be working so far. Looks like this:

May214 (talk) 15:03, 2 July 2018 (UTC)


 * That works too. Yaron Koren (talk) 15:17, 2 July 2018 (UTC)

Problem with some dates in Cargo tables
Hi,

I have a problem with some dates like 1967-01-01 wich appears to be registered as 1966-01-01 in the Cargo table. Some dates are registered as I expect, so I really don't know what I am missing.

Examples :
 * 1967-01-01 -> 1966-01-01
 * 1966-01-01 -> 1966-01-01
 * 1932-01-01 -> 1931-01-01
 * 1933-01-01 -> 1932-01-01
 * 1933-01-02 -> 1933-01-02

I'm using Mediawiki 1.30.0 and Cargo 1.4. I have the same behaviour on another wiki with the same versions.

Best regards, --Ludovic Strappazon (talk) 12:03, 2 July 2018 (UTC)


 * Sorry about that problem - it's due to a known issue with date display in PHP, for some January 1 dates. I just checked in a fix for it. Yaron Koren (talk) 15:18, 2 July 2018 (UTC)
 * Ok, thanks for your answer. Best regards. --Ludovic Strappazon (talk) 07:16, 3 July 2018 (UTC)