Extension talk:Cargo

From MediaWiki.org
Jump to navigation Jump to search

Querying data in cargo with the "HOLDS" command combining with "AND" clause [RESOLVED][edit]

Hi, How can I modify the "HOLDS" command in this example to use an "AND" clause? (Version I'm using now: MediaWiki 1.32.1 Cargo 2.1.1)

Example what I'm trying to achieve (lets pretend the Authors wrote among others one book together and Authors is decleared as a List of Strings):

{{#cargo_query: tables=Books |fields=_pageName=Book |where=Authors HOLDS 'Leo Tolstoy' AND Authors HOLDS 'Dostoevsky' }}

It works fine with 'OR' but sadly no results for 'AND'.

I asked a similar question in January - Use HOLDS with multiple values - and the suggestion then was to query the actual Cargo tables. I didn't pursue it so can't give you a full answer. Jonathan3 (talk) 13:38, 14 May 2019 (UTC)
Just tried to query the actual Cargo tables but with the same result. It works just fine with 'OR' but no sign of result with 'AND'.
{{#cargo_query:tables=Books,Books__Authors|join on=Books._ID=Books__Authors._rowID|fields=_pageName=Book,Books.Authors__full=Authors|where=Books__Authors._value = 'Leo Tolstoy' AND Books__Authors._value = 'Dostoevsky'}}
Should I assume there is still no any solution for this issue? Grombol
Update: The issue is solved! Here is the explanetion Grombol

Importing from XML Pagename delimited by ~[edit]

Got a problem. Creating a Cargo table "Sources" (and category/form/template) that has a field "Repertoire" that has multiple values (list ~ of Page) where each page is from Category=Repertoire. Each entry in the Source.Repertoire is a pageName from category Repertoire. I've uploaded a scad of new "Source" pages via XML upload that has the correct Pagenames in the Repertoire field delimited correctly. After the XML upload process, all the pages were recognized, and I can click the link in the newly created Source Page/Cargo table, and follow those links to the correct already exiting page (in category "Repertoire"). Looks like the upload worked great.

But, in the individual repertoire pages, I have a cargo query that returns a list of all sources that HOLD the pagename in Sources.Repertoire.

Here's the query in the Repertoire Template

{{#cargo_query:
tables=Source
|fields=_pageName
|where=Repertoire HOLDS '{{PAGENAME}}'}}

None of the pages uploaded by XML show correct query results. However, if I go into Form for Sources, and add individual pages in the Repertoire field. , it works. The Cargo Source table shows an entry (from the XML upload) and another entry (from the FORM). They are the same page. If I follow the XML generated link, edit it, and open the Form generated link, it's definitely the same page.

Stumped.May214 (talk) 21:03, 2 April 2019 (UTC)

Also tried
'{{FULLPAGENAME}}'
May214 (talk) 21:04, 2 April 2019 (UTC)
Just to be clear - is the problem in the data storage, or the data querying, as far as you know? Yaron Koren (talk) 23:55, 2 April 2019 (UTC)
My assumption is storage. If I add an individual entry using the Form (rather than a mass upload by XML) then the query works. THe XML loaded entries work functionally - I can click and follow links to existing pages. May214 (talk) 12:37, 3 April 2019 (UTC)
If you resave one of the imported, non-working pages (making a minor change first), does it start to show up in queries? Yaron Koren (talk) 02:24, 11 April 2019 (UTC)
No. If I go to Cargo Tables, and view the table for each entry, all the correct pages show up on the table viewer. I can click and it will open the correct page. In order for the query to work, I have to enter the page in the Form. The field in the Form is set up to autocomplete on category. When I add using the Form the query will work. But I'm adding the same page that is already showing up in the Cargo table. The exact same page. I can open the page from the cargo table, make a minor edit and save it. The query doesn't pick it up. Then if I add the same page in the Form (now I have duplicate entries of the same page in the table) and open the Form added page, the minor edit shows up (proving I don't have duplicate pages) and then the query works. Also, If I add the exact text of the page title without clicking on a page that shows up from the autocomplete options, the query doesn't work. There's something about selecting the page from the autocomplete options that makes the query work. But it shouldn't matter,right? I mean the page title is currently held in the Cargo Table and functions. May214 (talk) 19:07, 18 April 2019 (UTC)
Update. I can't copy paste text into the form either. I have to start typing the title, and then select the autocomplete option that works. May214 (talk) 19:16, 18 April 2019 (UTC)
Don't you have to include the Repertoire field in your field= line? Jeremi Plazas (talk) 19:28, 19 April 2019 (UTC)
Well, I want the query to return the pageNames in the source tables that hold a certain item in Repertoire. And it works, when I add each page by autocomplete using the Source Form. But I'll try a couple variations to see if including Repertoire in the fields work. If I just replace _pageName with Repertoire, it spits out the name of the repertoire, and not the source that holds it in repertoire. 204.124.167.5 15:51, 23 April 2019 (UTC)
So, no other suggestions? Any idea why it will work with an autocompleted entry, but not with a straight text entry? May214 (talk) 20:28, 1 May 2019 (UTC)
Sorry - I read through your explanation from April 18 about five times, but I still don't understand it. Each time you say "form" and "table", I don't know if you're talking about Source or Repertoire. This might be easier for me to understand with an example (even a fake one). Yaron Koren (talk) 00:53, 2 May 2019 (UTC)

Date and "Start date"[edit]

I'm planning to merge two tables. Table A has a Date field. Table B has Start date and End date fields. I am considering converting Table A's Date field to Start date (with no End date), and having Start date and End date for Table B's data. Would this work? Is Start date, when it's there on its own, treated as if it were Date? Thanks. Jonathan3 (talk) 23:05, 10 April 2019 (UTC)

I think having a "Start date" field without an "End date" field is simply not allowed. Yaron Koren (talk) 02:18, 11 April 2019 (UTC)
I’ll check it out but what about this? “Start date, End date - similar to Date, but are meant to hold the beginning and end of some duration. A table can hold either no Start date and no End date field, or exactly one of both.” Jonathan3 (talk) 08:08, 11 April 2019 (UTC)
That's right. Yaron Koren (talk) 15:50, 11 April 2019 (UTC)
I see what you meant now! I would end up with both a start date and end date field but usually with no value for end date. I was wondering if a start date would be as good as having a date for those entries with no end date value. Jonathan3 (talk) 16:04, 11 April 2019 (UTC)
Oh, I see. Well, either or both values can be null. Out of curiosity, why would the end date usually be null - is it because the end date is unknown, or because the concept of an end date is invalid for those pages? If the latter, it may be worth restructuring the data. Yaron Koren (talk) 17:12, 11 April 2019 (UTC)
Table A relates to documents (publication date) and table B relates to consultation documents (publication=start date, and end date). I want both together so that, for example, it’s easy to drill down on author. It seems there are two choices:
  1. Use a Start Date for every document and leave End Date empty unless required.
  2. Retain “publication date” as a Date, and when necessary ALSO use both Start Date (same value as the publication date Date field) and End Date.
Which seems preferable? Thank you. Jonathan3 (talk) 21:23, 11 April 2019 (UTC)
Okay, now I think I fully get it. Merging these two seems awkward... I know that it would make drill-down easier, but it still seems strange. If you're planning to do it, though, the first approach seems better. Yaron Koren (talk) 22:22, 11 April 2019 (UTC)
Thanks. I'll probably adopt a third option - keep publication date as an ordinary Date field, and add an End/Closure date also as an ordinary Date field. If I decide I need any of the special Start Date & End Date stuff (e.g. lines instead of dots on timeline format) then I can change the table definition later. Jonathan3 (talk) 13:56, 12 April 2019 (UTC)
That approach makes sense too. Yaron Koren (talk) 14:20, 12 April 2019 (UTC)

How does one exclude something in a Cargo query? [RESOLVED][edit]

Forgive my noobiness...

How does one exclude something in Cargo? I've got a query of the type:

{{#cargo_query:
	tables=LtrhfFiles
	|fields=_pageName,Caption,AssetPage
	|where=AssetPage HOLDS '{{PAGENAME}}' AND _pageName NOT '{{{FeaturedImagePage| }}}'
	|format=template
	|template=TinyGallery
}}

Obviously i get an error, but would like to accomplish the following: find all pages in that table where the AssetPage field contains {{PAGENAME}}, but excluding the pagename {{{FeaturedImagePage| }}}.

--Jeremi Plazas (talk) 18:46, 19 April 2019 (UTC)

I can give you half an answer. For most fields, change your NOT to <> and it'll work. I can't get it to work with _pageName fields though. Jonathan3 (talk) 21:03, 19 April 2019 (UTC)
P.S. It works fine. I just forgot that the pages in my test query had a namespace (it works when adding the namespace, or instead using _pageTitle). Jonathan3 (talk) 21:11, 19 April 2019 (UTC)
Fantastic! That did it thank you. Jeremi Plazas (talk) 04:02, 20 April 2019 (UTC)

ORDER BY CASE is broken with PostgreSQL and general Pg issues[edit]

Cargo seems to quote the ORDER BY clause, making it a column name:

Apr 20 20:13:25 db postgres[62078]: [41-1] ERROR:  column "CASE WHEN "Rarity" = 'Normal' THEN '1' WHEN "Rarity" = 'Rare' T" does not exist at character 183
Apr 20 20:13:25 db postgres[62078]: [41-2] STATEMENT:  SELECT /* CargoSQLQuery::run  */  "Name" AS "Name","Rarity" AS "Rarity","5-4note" AS "5-4note"  FROM "cargo__shipDrops"    WHERE "5-4" IS NOT NULL OR "5-4note" IS NOT NULL  ORDER BY "CASE WHEN ""Rarity"" = 'Normal' THEN '1' WHEN ""Rarity"" = 'Rare' THEN '2' WHEN ""Rarity"" = 'Elite' THEN '3' WHEN ""Rarity"" = 'Priority' THEN '4' WHEN ""Rarity"" = 'Super Rare' THEN '5' WHEN ""Rarity"" = 'Ultra Rare' THEN '6'  ELSE ""Rarity"" END" DESC,"ID" LIMIT 100

Does it pass the quotation marks when used via the API?

Another note: GROUP BY clause parsing:

ERROR:  column "artist" does not exist at character 90
HINT:  Perhaps you meant to reference the column "cargo__ships.Artist".
STATEMENT:  SELECT /* CargoSQLQuery::run  */  "Artist" AS "Artist"  FROM "cargo__ships"     GROUP BY Artist ORDER BY "Artist" LIMIT 500

If "Artist" is put in double quotes for groupBy, it escapes that which results in

ERROR:  column ""Artist"" does not exist at character 90
HINT:  Perhaps you meant to reference the column "cargo__ships.Artist".
STATEMENT:  SELECT /* CargoSQLQuery::run  */  "Artist" AS "Artist"  FROM "cargo__ships"     GROUP BY """Artist""" ORDER BY "Artist" LIMIT 500

--IijimaYun (talk) 11:30, 20 April 2019 (UTC)

Storing Cargo in Scribunto pages[edit]

I'm not sure if this is a Cargo thing or a Scribunto thing but it seems like data won't store on any page with Scribunto content model - if that's a cargo thing could it be changed or added to a config setting? I want to store some cargo data about a module on its documentation, and I'd prefer to have that be an in includeonly and executed on the page itself instead of on the doc page for consistency/clarity. --RheingoldRiver (talk) 08:42, 5 May 2019 (UTC)

(This is the data I'm showing, I'd like to have it immediately update when the module page is edited rather than needing some other location blank edited as well) --RheingoldRiver (talk) 10:16, 5 May 2019 (UTC)

Well I guess I could do this without Cargo at all & instead do a DPL query to get category members and then pull data by directly transcluding everything here with mw.loadData. But it would still be nice to have Cargo able to store from Scribunto pages --RheingoldRiver (talk) 10:47, 5 May 2019 (UTC)

Are you saying that pages in the Module: namespace aren't storing Cargo data? If so, that's not too surprising, since modules can't call templates... or can they? Yaron Koren (talk) 14:42, 5 May 2019 (UTC)
Yeah, but the documentation on module pages does transclude templates (that's how you add categories to Lua modules etc), so I would expect Cargo transcluded from the documentation subpage to store properly even in that content model --RheingoldRiver (talk) 01:25, 6 May 2019 (UTC)
What's an example of such a module? Yaron Koren (talk) 03:08, 6 May 2019 (UTC)
Everything in the category I linked above, and here's a direct link to one in particular --RheingoldRiver (talk) 09:47, 6 May 2019 (UTC)
Okay, so you linked to the module, but the documentation page for it, which is a standard wikitext page, is here. But how is the category getting set for the actual module? Somehow the /doc page is doing it, but I don't see how. Is that some Scribunto feature? Yaron Koren (talk) 14:29, 6 May 2019 (UTC)
The doc page is just transcluded at the top of the module. I did weird things with titleparts on that particular one to make the display on the doc subpage completely different from the display on the module page. Also I just realized that I deleted all Cargo from this within an hour of setting it up because I did change to DPL, whoops. Maybe this is a better example of how doc pages are transcluded to module pages. --RheingoldRiver (talk) 09:37, 7 May 2019 (UTC)

Calendar display that lists different text than page name but still works as a link[edit]

I am trying to set up a calendar that displays something other than a page name in the event, but that can still link to the event page. I tried generating a link with something like

CONCAT('[[', workdays._pageName, '|', person, ' - ', Location, ']]') = name

but that just shows the []s, it doesn't use them to designate a link. I could not find an alias to use for this in the calendar documentation, and the "link field" available for the gallery etc don't seem to work. Is there a way to do this? Thanks! Tenbergen (talk) 05:52, 8 May 2019 (UTC)

I think the following will work, though I'm not sure: "fields=workdays._pageName,CONCAT(person, ' - ', Location)=name". The documentation could be better, I admit... I'm not sure I understand it myself. :( Yaron Koren (talk) 03:23, 10 May 2019 (UTC)
Thanks Yaron, that does the trick; so does that mean the first field listed will always be taken as the "link" value, and there is no named alias this, and the name as "what yous see"? I am trying to think how I would update that in Extension:Cargo/Display_formats#calendar. Tenbergen (talk) 22:45, 22 May 2019 (UTC)

Group or Order by YEARWEEK (or by anything...)[edit]

I can get Cargo to display the following: |fields=YEARWEEK(_pageData._modificationDate) - but can't get it to work with |group by= or |order by=. Is this possible somehow? Thanks. Jonathan3 (talk) 13:13, 16 May 2019 (UTC)

The SQL produced has an ORDER BY but only based on the order the fields are listed in the |fields= line, and there is no GROUP BY in the SQL at all. Just tried a really simple query and the same applies, i.e. |group by= and |order by= are being ignored. Jonathan3 (talk) 13:20, 16 May 2019 (UTC)

"Group by" works elsewhere on my website so it must be something stupid I'm doing... ignore the question for now :-) Jonathan3 (talk) 22:13, 16 May 2019 (UTC)

Correct storage of values of multiple instance templates only after Recreate Data on main cargo table[edit]

Hello everybody,

may I ask you for some time and expertise on an issue related to cargo storage of values of a multiple instance template (MIT).

Here some details:

In the form of an indicator database I use the functionality of multiple instance templates to enter current measurementvalues of the corresponding indicator. It is implemented in this way within the form of the indicator database:

{{{field|current|holds template}}}

{{{for template|assessment|multiple|add button text={{int:add-assessment}}|embed in field=indicator[current]}}}
;{{int:date}}:{{{field|assessment date|input type=datepicker|disable input field|week start=1|highlight days of week=0|date format={{int:date-format-form}}|size=8}}}
;{{int:current}}:{{{field|assessment current|size=10|default=0}}}
{{{end template}}}

All values of the MIT are stored in an additional cargo table called "assessment".

Based on the most recent currentvalue of that table, I try to calculate the current progress of the indicator and store it to the cargo-table "indicator". The formular of the progress calculation is like that:

(current - baseline) / (target - baseline) * 100

Baseline and target can be entered in the form of the indicator database, while the current value(s) are entered via MIT as stated above

The corresponding store command in the indicator template looks as follows:

{{#cargo_store: 
_table=indicator
|progress={{#expr: ({{#cargo_query:
                      tables=assessment
                     |fields=TRIM(current)
                     |where=_pageID="{{PAGEID}}"
                     |limit=1
                     |order by=date DESC
                     |more results text=
                     |no html
                    }} - {{{baseline|}}}) / ({{{target|}}} - {{{baseline|}}}) *100}}
}}

The problem is now, that every time I add a new instance with a recent current value, cargo stores "0" to the progress column. First of all I was assuming that it is maybe an data update problem and a null edit on the indicator would solve the problem. But I was mistaken. The only way to fix the problem is to recreate the indicator cargo table. After recreating the table the progress column is being calculated correctly. But as soon as I add a new instance or edit an existing instance of the MIT progress is again "0" and will only be calculated correctly after recreating the (indicator) table once again.

Anybody an idea, what could cause the problem of false storage of the current progress and how it can be resolved?

Thanks in advance. Axel

It's not surprising that this is not working - the page is querying itself, and it's storing the computed data at the same time that the original data is being stored. So everything depends on the timing - which templates get called first. I don't know what the best solution for this is. Would it be possible to not store the "progress" at all, but instead query it when needed? Or would that lead to queries that are too complex for #cargo_query? Yaron Koren (talk) 16:58, 21 May 2019 (UTC)

Text-file output display format[edit]

The T223886: Add a bibtex format page on Phabricator reminded me of something...

Would it be possible to add a text-file Export display format, with the text being formatted using a template in the same way as with the normal Template display format?

I'd like to be able to offer people a link to a RIS file for import into Endnote (reference management software). And maybe other types of file too.

Thanks. Jonathan3 (talk) 22:06, 23 May 2019 (UTC)

I wonder if it'd be cool to make a way of exporting to any other format in that fashion, e.g. a filename=foo.txt parameter would give text, and filename=foo.gpx would give XML (both coming from a template, as you say, via the |format=template |template=foo parameters). Maybe there would be weirdnesses that would arise with that though; it's not always wise to generalize things too much. Sam Wilson 12:24, 26 May 2019 (UTC)
Or maybe, in an ideal world, have all output formats default to displaying on screen (including csv etc), and have a new “destination” parameter to override that eg destination=txt, destination=csv to save as file, or destination=link to display link to display in new tab/window.
This could be in conjunction with format=template and template= (eg for RIS format) or without that (eg for CSV or other formats that Cargo itself can create).
It would also allow new formats to be added to Cargo in future (without then needing format=template any more).
I haven’t fully thought this through and maybe (even if it makes sense) it would require too much of a change to Cargo code to be worth it. Jonathan3 (talk) 14:39, 26 May 2019 (UTC)
The basic idea of being able to customize an export format is very clever - it makes sense to do this for obscure formats, rather than requiring a Cargo result format to be created every time. Though something widely-used like BibTeX seems like it should have a true result format. Maybe I'm just saying that because SMW has a "bibtex" format, though. I had never heard of RIS before - do you happen to know how popular it is compared to BibTeX?
Potentially this new query format could be called "export template", or just "export" - or maybe even just "template", with another parameter like "export" to indicate that this is an export format. The last of those seems unnecessarily confusing, though.
The "filename" parameter idea I'm not sure I understand - I mean, it's useful to allow for a "filename" option, but I don't see how the file suffix could dictate what the file type would be. It could be possible to have "file type=" parameter, to specify that manually, though, assuming that doesn't pose any security risk.
Jonathan3 - any chance that you would be willing to create this query format? I guess I could ask the same question of Sam Wilson... Yaron Koren (talk) 02:42, 27 May 2019 (UTC)
As far as I know, RIS format is used primarily in Endnote, which many academics use to store resources and create footnotes and bibliographies, though other software like Zotero can use the format too. I get the impression that there is some overlap with Bibtex but the latter is used more in publishing rather than just storing a personal database of resources/writing articles. Not sure.
If I were to create a template for the RIS format it would just take named parameters like AU=Shannon, Claude E. | TI=A Mathematical Theory of Communication - or AU={{{Author|}}} | TI={{{Title|}}} - to create text like in the example record on the Wikipedia page.
Let's say you had Bibtext as a Cargo-supported format and RIS was just a template, you could have:
  • Bibtex: format=bibtex
  • RIS: format=template | template=ris
I mentioned a destination parameter, and I'm not sure what the best solution would be, so here are some random ideas...
  • I guess currently (using that idea) format=csv, format=excel and format=json are effectively what would be destination=file. The RIS example above, and any format=template format, would currently just display on screen, so we would need a way to know to show a download link instead. This could be destination=file, or (as suggested above), just filename=....
  • Incidentally, it may be useful to be able to show the CSV/JSON (and Excel?) within the wiki page, in which case destination=screen might work.
  • The "Export citation" link on this randomly-selected academic article does different things for the various formats (which include Bibtex and Endnote, but not RIS as it happens). Some display on a new web page and some prompt you to download a file. I wonder whether there could be destination=newpage (link opens new page) and destination=file (link downloads file) or, somehow, the possibility of offering both options beside each other.
  • Alternatively, if the link were to view on a new web page, maybe a tooltip could explain that right-clicking will allow file download.
All the best, Jonathan3 (talk) 22:12, 27 May 2019 (UTC)

HTML Entities special export[edit]

READ: this might be just a problem of my setup, sharing it just in case someone is facing a similar issue..

Version: Cargo 2.1.2

Problem: Values containing characters which are also HTML Entities (in my case the char &) are properly stored as plain text in cargo tables, but when exporting them as csv either with or without "parse values" param, the result contains HTML encoded value, consequently causing the row to be wrongly displayed and splitted in multiple columns when opened with excel.

NOTE: I found out that parse values parameter needs to be declared with a value (e.g. |parse values=something) in order to be encapsulated and properly evaluated as WebRequest parameter. Alternatively it will not be taken in consideration..

With parse values parameter in the query, the value will be initially properly decoded as text but then parsed by the function smartParse in includes/CargoUtils.php, resulting in the csv containing an & and consequently causing the row to be splitted in multiple columnd if opened with excel.

Without parse values parameter in the query, the value will still be parsed but not be decoded, having as result the same unwanted result as above.

My dirty solution, in "specials/CargoExport.php":

// Added html decoding function for all values
function decodeHTML( $queryResults ) {
	$parsedQueryResults = array();
	foreach ( $queryResults as $rowNum => $rowValues ) {
		$parsedQueryResults[$rowNum] = array();
		foreach ( $rowValues as $colName => $value ) {
			$value = htmlspecialchars_decode( $value );
			$parsedQueryResults[$rowNum][$colName] = $value;
		}
	}
	return $parsedQueryResults;
}
// Call decodeHTML function id not set $parseValues
function displayCSVData( $sqlQueries, $delimiter, $filename, $parseValues ) {
...
	if ( $parseValues ) {
		$queryResults = $this->parseWikitextInQueryResults( $queryResults );
	}else{
		$queryResults = $this->decodeHTML( $queryResults );
	}
...

Built-in UCFirst?[edit]

Could you add a built-in ucfirst function? Since pagenames are first-letter case-insensitive it's not great to have to workaround this currently. --RheingoldRiver (talk) 00:26, 26 June 2019 (UTC)

Where would you want to call this function? I'm guessing in a #cargo_query call within Special:RunQuery, but maybe it's something else. Yaron Koren (talk) 02:52, 26 June 2019 (UTC)
Actually it's all over the place, the issue I have is players rename constantly, and often change case when they rename. So I have a table called PlayerRedirects that has all of the pages that redirect to the player's overview page, and I join that between Players table & any other table you might query with player information, with a BINARY condition to prevent dupes that would otherwise occur due to the case changing. In the past, I stored the player's display name & link as different fields in case of disambiguation, but yesterday I realized this setup is silly since I can do a pattern match to replace the disambig part of a link to get the display, so I may as well store only the link. BUT then I realized this still doesn't work, because if the display is a lowercase first letter, then the BINARY condition will fail. And I'd rather not double the size of PlayerRedirects, already a pretty big table. And since I'm publishing docs on how to access my API I don't want to do a workaround with looking at part of the name at a time to compare, I'd rather be able to tell people "just do a UCFIRST() in the BINARY condition."
One possible workaround to this would be to store only one single capitalization to the redirects cargo table, and kill the BINARY condition completely, but then either I'm further complicating by adding another where condition and column ("IsUniqueCase" which is true for exactly one capitalization per set of redirects) or removing some redirects from Cargo completely, and that might come with its own issues. But regardless the situation I have doesn't seem TOTALLY absurd for wikis that use a lot of Cargo, so I think it might be worth adding in general, as a kind of "title-ify for MediaWiki" functionality. --RheingoldRiver (talk) 12:56, 26 June 2019 (UTC)
Oh... you're talking about an SQL-style function. I thought you were talking about a parser function. You could always go into the database and create one (well, get a DBA do it)... or you could have a workaround like the one described here. Would either of those be a good enough solution? Yaron Koren (talk) 14:19, 26 June 2019 (UTC)
Oh, yeah sorry. I thought it would be similar to HOLDS, and had to be added to the extension directly. I do want something that's done server-side with a friendly syntax so I don't have to make people using our API do the workaround in the query itself, but if this can be aliased somehow without a direct edit to Cargo that would definitely be enough - I'll talk to one of our sysadmins and get back to you --RheingoldRiver (talk) 16:54, 26 June 2019 (UTC)

_pageData table stores category of pages/files only after second saving of page[edit]

I am currently switching former SMW-content to a cargo-based enviroment. In the SMW-environment we used categories massively and queried for files/pages within these categories via ask-queries.

Querying for categories directly is, as I understood the cargo documentation, not possible anymore via cargo queries. For that reason, I try to query for categories via the _pageData cargo table.

Queries like this work fine:

{{#cargo_query:
  tables=_pageData
 |fields=_pageName
 |where=_categories HOLDS "Category X" 
 |group by=_pageID
}}

The problem I face is, that the category of any page or file is only being stored to _PageData after the second saving of the page/file. Is there maybe any parameter I am not aware of, which forces the _pageData table to store the category in the first run? Or is it a bug?

I had a similar problem - queries that should have worked didn't, though I never worked out that it required a second save to make it work - and I ended up using DPL3 on this part of my website, even though I'd have preferred to use Cargo. I'd be interested to hear the solution. Jonathan3 (talk) 13:35, 1 July 2019 (UTC)
I don't think I knew about this problem before - it sounds like a bug, and, thinking about it now, maybe an intractable one. The problem is that, in order to get the category information, the code queries the "categorylinks" DB table, then stores the information in its own _pageData table. So when the page is first saved, the "categorylinks" information doesn't exist yet, so it can't be queried. The only suggestion I can think of is to... use categories less? If you store the relevant data as template parameters instead of categories, it's easier to query anyway - and easier to edit via forms. Yaron Koren (talk) 13:16, 2 July 2019 (UTC)
Maybe Cargo could use a MediaWiki hook to add/remove the relevant information in the _pageData table when a page is saved. If this would slow things down it could be optional. Jonathan3 (talk) 22:42, 6 July 2019 (UTC)
That seems like it could work, yes. Yaron Koren (talk) 22:26, 9 July 2019 (UTC)

Table X not found[edit]

Hi, I'm new to Cargo and still trying to figure out how to use it. Templates that contain tables claim the tables have not been created yet, and whenever I try to query data, I get this error. I have tried following advice from an earlier topic on this thread to create the tables manually with no luck. What might I be doing wrong? Disgustedorite (talk) 18:25, 9 July 2019 (UTC)

This section might help... Yaron Koren (talk) 19:47, 9 July 2019 (UTC)
It worked, thank you! Disgustedorite (talk) 20:03, 9 July 2019 (UTC)

Wrong Lua error when join conditions are reversed[edit]

I'm doing a relatively complicated query with a bunch of joins, and I had the following error:

Lua error: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? Query: SELECT `PR`.`_pageName` AS `_pageName`,`LG`.`LongName` AS `LeagueGroup`,`IT`.`StandardName` AS `Tournament`,`SP`.`Team` AS `Team` FROM `cargo__ScoreboardPlayer` `SP`,`cargo__LeagueGroups` `LG` LEFT OUTER JOIN `cargo__PlayerRedirects` `PR` ON ((`SP`.`Link`=`PR`.`AllName`)) LEFT OUTER JOIN `cargo__Tournaments` `IT` ON ((`SP`.`OverviewPage`=`IT`.`OverviewPage`)) LEFT OUTER JOIN `cargo__LeagueGroups__Leagues` `L` ON ((`LG`.`_ID`=`L`.`_rowID`)) WHERE (LG.Leagues__full IS NOT NULL AND PR._pageName="Bjergsen" AND IT.IsPlayoffs="0" AND IT.IsQualifier="0") ORDER BY PR._pageName,LG.LongName,IT.StandardName,SP.Team LIMIT 5000 Function: CargoSQLQuery::run Error: 1054 Unknown column 'SP.Link' in 'on clause' (esports1_cluster.aurora.local.curse.us:3306) .

However the actual error is that the last join condition is written in the wrong order: LG._ID=L._rowID instead of L._rowID=LG._ID; the column SP.Link does exist and when I set up the join properly it works fine.

I'm not sure how the errors are generated so maybe it can't know what the actual issue is, but maybe add a hint that perhaps it's the order of tables in the join conditions? --RheingoldRiver (talk) 22:51, 13 July 2019 (UTC)

I don't understand - what does the order of those two columns have to do with the error? Yaron Koren (talk) 01:39, 14 July 2019 (UTC)
Yeah exactly haha, that's the error that I got when I had the order of columns in the join listed the other way. I have no idea why it's printing this error, it seems completely unrelated to the actual problem (if there even was a problem?) If you want I can recreate the error on a page, but I'm not sure how much that will help, the only thing I did to fix it was reorder that join condition I listed before --RheingoldRiver (talk) 01:45, 14 July 2019 (UTC)
Oh, now I get it. Very strange! I have no idea... this seems like a question for an SQL expert, which I am not really. Yaron Koren (talk) 15:12, 14 July 2019 (UTC)

Duplication or missing rows in tables still an issue?[edit]

Looks like I'm about to get involved in another SMW to Cargo transition. Previously we had quite some issues with duplication of rows in cargo tables, and at other times of rows not getting added to the table when creating or recreating cargo tables. Is this still an issue, or are there new and better ways to handle it now from users on the template side? Any tricks to prevent the problem?

Secondly, is it possible to get the full image showing when using table output, instead of the thumb version, without having to use a template? It would make things much easier when you have for instance small logos/icons in games, plus various data that fits well in the table format. Pangaearocks (talk) 11:49, 14 July 2019 (UTC)

That's great! I haven't heard of duplication problems in a long time, so I'm guessing that this problem is over. As for some data not getting added - this seems to be less of a problem when tables are recreated via the command line rather than the web interface, so if it's a problem I'd try using the command line script. And to show the full image, it might work to replace the field name in the "fields=" clause with "CONCAT('[[File:', FieldName, ']]')". Yaron Koren (talk) 17:02, 14 July 2019 (UTC)
Thanks, that sounds very promising. It was quite the pain before, so I'm glad it shouldn't be an issue any more :) Cheers for the image tip as well. Once we get things set up, I'll try that on a test table and see if I get it working. Of course, we may need template output anyway, it depends on what we need, but we'll cross that bridge if we get there. Pangaearocks (talk) 02:48, 15 July 2019 (UTC)

Composer installation problems[edit]

The admins on the Rimworld Wiki have tried to install Cargo, but I'm told there were some problems related to Composer. The update.php script said Composer needed to be updated. They updated it, but update.php still says Composer needs to be updated. I've not tried to install Cargo myself, nor am I an admin, so I don't know exactly how the process goes, or what the problem is here. However, can Yaron or others with installation experience try to guide us a bit? From the Version page, Cargo is installed, but with version 2.0rc1. When I click on Cargo tables, this fairly generic error message pops up:
[XS33UrP9mRLsm6nBFo-jAwAAABE] 2019-07-16 16:12:02: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"

No tables are actually created yet, if that is an issue here, as I don't have the right to do that (must be admin, I think). View Data works, but not Cargo tables or Drilldown.

SMW is currently installed, and as this will be a transition process, both need to be installed for quite a while. Pangaearocks (talk) 16:15, 16 July 2019 (UTC)

I don't use Composer, and I don't know much about it. Does it have to be used for Cargo? Yaron Koren (talk) 17:23, 16 July 2019 (UTC)
Hopefully not, but that is the error message they apparently got when they tried to install Cargo, or rather when they ran the update.php script afterwards. I pointed him in this direction, so hopefully he shows up when time allows. He can be more specific about the problem. I'm just an intermediary and trying to help to get things in order so we can get started properly. Pangaearocks (talk) 19:45, 16 July 2019 (UTC)