Extension talk:Cargo

From MediaWiki.org
Jump to navigation Jump to search

Contents

User group right for viewing Special:Drilldown[edit]

Would it be possible to add a user right for viewing Special:Drilldown? We had some slowness on Leaguepedia that was caused from some IPs repeatedly pinging expensive queries and would like to restrict access to logged-in users. --RheingoldRiver (talk) 07:46, 8 January 2019 (UTC)

That sounds like a reasonable idea (though it's sad to hear that Special:Drilldown can cause performance issues). What do you think about a single permission that would encompass both that page and Special:ViewData? Yaron Koren (talk) 17:43, 9 January 2019 (UTC)
TBH I was unaware of Special:ViewData until now, I always make sandbox pages when I want to run temporary queries, hah - I'll probably start using it now. That sounds like a good idea to combine, then. --RheingoldRiver (talk) 11:05, 10 January 2019 (UTC)
Okay, that sounds good. I'm glad you learned about Special:ViewData! I admit its interface is quite primitive, but hopefully it's enough for your purposes. There's actually a planned project to make it more useful and user-friendly. Yaron Koren (talk) 03:54, 11 January 2019 (UTC)
I just checked in a new permission, 'runcargoqueries', that governs access to those two special pages. By default it's true for everyone. Hopefully it'll work out for you. Yaron Koren (talk) 22:48, 15 January 2019 (UTC)

Use HOLDS with multiple values[edit]

I'd like to do something like this:

|where=fieldname HOLDS 'value1' AND fieldname HOLDS 'value2' AND fieldname HOLDS NOT 'value3'

Is this possible, and if so what's the syntax? Thanks. Jonathan3 (talk) 12:56, 13 January 2019 (UTC)

That's possible in theory - I guess it's not actually possible at the moment due to a bug in query parsing. The only way to do it right now, I think, is to actually query on the underlying tables, instead of using the "HOLDS" syntax - see here for somewhat of a hint on how to do that. Yaron Koren (talk) 17:11, 13 January 2019 (UTC)
"So you're telling me there's a chance!" :-) I'll give it a go and put the answer here if I find it. Jonathan3 (talk) 17:17, 13 January 2019 (UTC)

Custom tab for Special:Drilldown[edit]

Would it be possible to allow a custom tab on Special:Drilldown? I'd like to be able to include a template (which would provide a summary of each page in the Drilldown results).

I proposed a patch to allow something like this ages ago, before you had tabs, but didn't persuade you to include it. You did agree to add a hook but I never followed this up.

A separate custom tab per table would be good. It could maybe be set in the table definition (the relevant template could be named there). Alternatively, Drilldown could check for a custom template (which would have to follow some naming convention).

Also it would be good if the custom tab could optionally be set as the main tab.

What do you think? Jonathan3 (talk) 13:13, 13 January 2019 (UTC)

You can already do that, if I understand the question correctly, with "_drilldownTabs" - see here. Yaron Koren (talk) 14:54, 13 January 2019 (UTC)
Yes, that's perfect! Thanks for your quick reply. Jonathan3 (talk) 16:34, 13 January 2019 (UTC)

MW categories and Cargo hierarchy[edit]

I'm currently migrating my plain pages to Cargo template pages. I've been using categories and would like to continue to use these (at least for the time being) so have a Cargo field List (,) of String which uses {{#arraymap...}} to allocate the pages to categories. Would I be able to have this field as a hierarchy to mirror the current category structure (so that it autocompletes on the relevant existing main category and and sub-categories, and perhaps also only allows existing category names as values)? Sorry if I've missed something obvious in the documentation again. Jonathan3 (talk) 17:14, 13 January 2019 (UTC)

That sounds like a Page Forms, rather than Cargo, question. Check out the tree input type with "top category" - it may be helpful here. Yaron Koren (talk) 23:50, 13 January 2019 (UTC)
That form setting worked perfectly, thank you. I can't understand the Cargo "hierarchy" fully, though:
It looks like the allowed values have to be set using bullet points in the table definition. Is there any way of linking this (automatically) with the category names and structure?
What is the benefit of having a hierarchy? I guess it has an effect on Drilldown, so that if you click a heading name it is as if you clicked on all its sub-headings too. Is that right?
Thanks Jonathan3 (talk) 01:05, 14 January 2019 (UTC)
Hierarchies can be defined within a Cargo declaration, within a form input, or by using an existing hierarchy from the category structure. I don't know what it would mean to link a Cargo field to a category structure, given that the set of categories can change at any time. Cargo-defined hierarchies get special handling within forms, drilldown, and regular queries (via the "WITHIN" keyword). Yaron Koren (talk) 02:02, 14 January 2019 (UTC)
Thanks. Do you know any way to convert a category structure into bullet-point format? If I were to put that into a template, would I be able to use that template within #cargo_declare to define the allowed values, and recreate the table when the category structure changes? Is there anyway to change the allowed values of the field without having to recreate the entire table, rows and all? Jonathan3 (talk) 14:27, 23 January 2019 (UTC)
Why do that - why not just use the existing category structure, using "top category="? Yaron Koren (talk) 18:32, 23 January 2019 (UTC)
Is it possible to define a field as a hierarchy in Cargo without listing the allowed values? I thought “top category” was a PF parameter for the tree input type, which wouldn’t lead to the benefits of having a hierarchy in Cargo (queries, Drilldown). I was considering using both (ie top category in PF and somehow defining the Cargo hierarchy allowed values to be the same hierarchy as the categories. Jonathan3 (talk) 19:11, 23 January 2019 (UTC)
That's true - it would be a Page Forms-only solution, and it wouldn't let you do the hierarchy handling in queries and drilldown. But having it defined in two places sounds like a hack. Let me ask you this: what about removing those categories altogether, and only using a Cargo hierarchy field? Yaron Koren (talk) 20:41, 23 January 2019 (UTC)
I've got about 2000 pages to move across to Cargo. These will probably take 5-10 mins each, as I'm adding data as well as just copying across old data, so can't just automate it... until that's all done I need to maintain the categories :-) After that then, yes, I could probably do without categories, I think. The category pages are like dead-end Drilldown pages, especially now that I am able to add a tab to Drilldown the same as I had for the category pages.
Is it possible to use a template to set the allowed values of a Cargo hierarchy field? If so then I can look into how to turn the category structure into the bullet-point format required. Jonathan3 (talk) 21:25, 23 January 2019 (UTC)

I see. What about just hardcoding it in #cargo_declare? If the category setup is just a legacy system, then presumably it won't change much more? Yaron Koren (talk) 21:49, 23 January 2019 (UTC)

That would be possible (not much more work as - you're right - there won't be many changes, and any will involve other work anyway). Am I right to infer from your answer that it's not possible to use allowed values={{template}}? Jonathan3 (talk) 22:13, 23 January 2019 (UTC)
@Yaron Koren: Good news, I think! I have created a template using DPL3 which takes the name of a category as a parameter and outputs the category tree in the required * and ** format, then added allowed values={{template name}} to the Cargo template page. When I add a new category page to the wiki, I need to (1) do a null edit of the template page so that the "tree" input type on the form discovers the new category, and (2) "recreate data" so that the new category name is able be saved into the table by the relevant page's template.
It would be good if it were possible to change the table structure without recreating the data (saving time with large tables). I guess a problem with changing allowed values that the user might try to remove one that already appears in the data, but that could be checked for. This feature could also be useful for, for example, adding a table column. Is it something you would consider adding? Jonathan3 (talk) 14:37, 28 January 2019 (UTC)
There certainly are specific cases where a Cargo table could be modified without requiring recreating the entire table - adding additional allowed values to a field, adding a still-empty field/column, removing a field/column. (That last one is probably the least risky.) It's conceivable to do, and I thought about adding in that kind of functionality, but I don't know if it's worth the effort, given that it would only be useful in a pretty specific set of cases, and the code to enable it might have to be pretty complex. Ever since the "replacement table" functionality was added, I think long recreate-data times have been less of a major issue. But if you think it's worthwhile, and that there's an easy way to do it, let me know. Yaron Koren (talk) 19:58, 28 January 2019 (UTC)
If it's complicated (or could take ages to work out whether it's complicated) then I agree it's not important :-) Jonathan3 (talk) 23:37, 28 January 2019 (UTC)

Make selected filter field collapsible rather than collapsed on Drilldown page[edit]

At the minute it says "(Click arrow to add another value)". It would be more intuitive it the whole thing were visible on page load, maybe with a message like "(Click on another value to add it)".

This is particularly the case if the Drilldown page (with filter selected) has been reached via a direct link rather than clicking on a value.

Is there any way of changing this? Thanks. Jonathan3 (talk) 17:41, 13 January 2019 (UTC)

Not at the moment, I don't think. Yaron Koren (talk) 23:51, 13 January 2019 (UTC)
This worked for me, in MediaWiki:Common.js, without needing to change the Cargo code (I went further and hid the the arrows and message entirely):
/* Special:Drilldown */
/* Show filter values even when value selected; hide "Click arrow to add another value" message; hide arrow icons */
$( function() {
  $('.drilldown-filter-values').show();
  $('.drilldown-filter-notes').hide();
  $('.drilldown-values-toggle').hide();
} );
Jonathan3 (talk) 03:49, 27 January 2019 (UTC)

Drilldown tab headers[edit]

On the iPhone when the Drilldown page is loaded in portrait mode (if that's the word) the headings appear as an unordered list, then disappear instead of becoming tabs.

When the page is loaded in landscape mode, they appear as normal, then disappear when you rotate the screen to portrait, and don't reappear when you return to landscape.

It would probably be better to keep them even in narrow screens, either as tabs or (if that's not possible) as an unordered list. Would that be possible? I guess there's some CSS that could be changed somewhere. Jonathan3 (talk) 18:53, 13 January 2019 (UTC)

Hierarchy/grouping of fields[edit]

Do you have any plans to implement something like the following? Using your Books table as an example, a group of fields could be called something like "Contributor", and within that there could be fields like "Author", "Editor", "Illustrator". On the Drilldown page the entries for these fields could (optionally) be merged, so that if you click on "John Smith" you get to see all the books he has contributed to (written, edited, illustrated or whatever). Thanks, Jonathan3 (talk) 14:33, 14 January 2019 (UTC)

No. I can think of two ways to do something like that, if that's what you want: one is to create a query form in Page Forms, to run that exact query on any person the user selects; the other is to define contributors using a multiple-instance template, so instead of having different fields for "Author", etc., you would have a "Contributor" template that lets you input each person and what they did. Then you could drill down based on that, using "parent tables". Yaron Koren (talk) 17:31, 14 January 2019 (UTC)

Storing categories in page data table[edit]

I have a feeling that this isn't working right. I have a Cargo infobox template A that assigns the page to a category. Another template B uses a Cargo query to work out which category the page is in (as a different further Cargo query is required depending on that answer) - this template B never gets it right for a new page (doesn't realise it's in the assigned category) until I run setCargoPageData.php, after which it works fine. Am I doing something wrong? Jonathan3 (talk) 14:41, 20 January 2019 (UTC)

P.S. I have switched template B (which checks the category of the page) to use DPL3 rather than Cargo, and it works fine now [i.e. it's a problem with my use of Cargo]... Jonathan3 (talk) 15:37, 20 January 2019 (UTC)

Join several tables and order by _creationDate [RESOLVED][edit]

Would it be possible to devise a Cargo query to join four tables, all of which have a "Summary" field but which are otherwise different, and display them in order of page _creationDate (most recent first)? I guess I'd need to use SQL on the actual database tables for this. Jonathan3 (talk) 14:48, 22 January 2019 (UTC)

@Jonathan3: For that, you'd probably need to be able to use the union operator, which I don't think is possible with Cargo. Another way would be to modify each of the four table's templates so that they also add data to a fifth table, one that contains only 'date_created' and 'summary', and then query that one. Sam Wilson 14:57, 22 January 2019 (UTC)
@Samwilson: Thanks. The separate table sounds ideal. It's for automatically creating updates to the website, so it would be good to have a separate updates table for a historical record, and also to be able to add to it manually if required. Jonathan3 (talk) 14:23, 23 January 2019 (UTC)
@Samwilson, Yaron Koren: Looking into this now - what is the best way to do this? Do I define the new ("Updates") template, then put a #cargo_attach into the existing templates? If the existing and new tables share field names, will #cargo_attach send these automatically to the new table? Thanks. Jonathan3 (talk) 00:15, 25 January 2019 (UTC)
@Jonathan3: I don't think you even need to attach it, just call cargo_store twice in one template (one for each table) and pass the same parameters to both. e.g. 'Template:Photo' might have 'date' and 'description' fields, and 'Template:News' 'date_and_time' and 'summary', so in 'Template:Photo' you call both {{#cargo_store: _table=photos |date={{{date|}}} |description={{{description|}}} }}{{#cargo_store: _table=news_items |date_and_time={{{date|}}} |summary={{{description|}}} }} with the end result being that every photo would have an entry in the 'news_items' table, along with all the other non-photo news items. If that makes any sense. :) Sam Wilson 01:18, 25 January 2019 (UTC)
That does make sense, thank you. What does this part of the documentation mean? “You do not actually need this call [#cargo_attach] in order for a template to add rows to some table; a #cargo_store call placed anywhere, via a template or otherwise, will add a row to a table (assuming the call is valid). However, #cargo_attach lets you do the "Recreate data" action for that template - see "Creating or recreating data", below.” Does it have an impact on templates that call #cargo_store twice, on separate tables? Jonathan3 (talk) 10:19, 25 January 2019 (UTC)
I've tried this out. When I added a new "News" template/form/table, and added #cargo_store to an existing template: (1) the #cargo_store worked fine when a page is saved, but (2) recreation of new and/or old tables did not re-add the row to the News table. Jonathan3 (talk) 14:03, 25 January 2019 (UTC)
Adding #cargo_attach to the existing template gives: "Error: #cargo_attach must be called from a template page." Any more ideas? Thanks. Jonathan3 (talk) 14:13, 25 January 2019 (UTC)
I haven't really followed the full discussion here, but - did you put #cargo_attach in the <noinclude> section of the template? Yaron Koren (talk) 14:27, 25 January 2019 (UTC)
You're right. Now that it's in the right place the error doesn't appear... But now that the old template is attached to the News table, how can I get the old template to save anything to the News table? I've tried having field names which are the same in both tables but it doesn't do anything... I must be missing something simple... Jonathan3 (talk) 14:51, 25 January 2019 (UTC)
@Yaron Koren: I wonder if it would be possible for cargoRecreateData.php to change $templatesForThisTable = array_merge( $templatesThatDeclareThisTable, $templatesThatAttachToThisTable ); somehow also to include templates that store to the table? Jonathan3 (talk) 14:36, 25 January 2019 (UTC)
The answer seems to be that: (1) a second #cargo_store does the work of storing data from template A to table B, and (2) #cargo_attach is what lets cargoRecreateData.php know to check Template A when recreating data for Table B. It works fine now with both #cargo_store and #cargo_attach included in template A :-) Thanks to both of you for your help! Jonathan3 (talk) 15:02, 25 January 2019 (UTC)

Case Sensitivity in stores[edit]

I think there's an issue where if two rows on the same page are exactly identical up to case sensitivity, only one of the rows is stored. This came up for me on this page - https://lol.gamepedia.com/index.php?title=PiPiXuan&action=pagevalues#.26quot.3BPlayerRedirects.26quot.3B_values - there should be one line for each of the pages that redirects (visible here - https://lol.gamepedia.com/PiPiXuan#Redirects). However since there are two sets of lines that are identical up to casing, for those sets only one is stored. A workaround would of course just be to add another column that's a unique identifier of the line (which I'll probably do now that I've identified the problem) but maybe this is unintended behavior. --RheingoldRiver (talk) 09:44, 23 January 2019 (UTC)

I see - you're trying to store both "XuanXuanPI" and "XuanXuanPi". That's a bug, although it's not clear to me how big an issue this is. Why do you need a separate table for redirects? Why not just use the existing "IDList" field value, which seems to have that same information? Yaron Koren (talk) 18:31, 23 January 2019 (UTC)
Often players change names, and sometimes even mid-event, and redirects reflect all of the name changes. So I'll do a series of 3 joins across 4 tables to group players together - the data table (e.g. Team Rosters from a tournament), two copies of PlayerRedirects, and InfoboxPlayer, to get data about a player. So I join on the AllName field of PR and then _pageName, and in the where I add BINARY conditions to avoid getting extra results when case insensitivity is a problem. Example module - https://lol.gamepedia.com/Module:TournamentPlayerInformation, and example use of the module - https://lol.gamepedia.com/Special:RunQuery/TournamentPlayerInformation?TPI%5Bpage%5D=LVP%20SuperLiga%20Orange/2018%20Season/Summer%20Season&pfRunQueryFormName=TournamentPlayerInformation (the redlinked coach "Gil" as no page so that's caught with an OR IS NULL in the where). It's definitely possible there's some better way to do all of this, but if so I'm not sure how.
Also, HOLDS still seems kinda buggy sometimes (though it's been a while since I used it) so some wikis at Gamepedia (at least me & the Path of Exile admins) try to avoid it overusing fields of list type, unless they're for display only and never querying on. --RheingoldRiver (talk) 22:43, 23 January 2019 (UTC)
That said this definitely isn't a huge issue since adding a UniqueLine field (or even an N field) is a very easy workaround for this --RheingoldRiver (talk) 23:20, 23 January 2019 (UTC)
Alright. I think the case-sensitivity thing depends on the MySQL configuration, since I believe the check is done in SQL itself. But I do hope you try out HOLDS again - maybe it works better now than last time you tried it. Yaron Koren (talk) 16:15, 24 January 2019 (UTC)

Problem recreating tables[edit]

I am trying to recreate a table from a template. When the new table is generated and I choose to replace the old table, I get an error saying the old table doesn't exist. If I try and look at the existing table I get the same message. Also, the table doesn't show up on the special page that lists Cargo tables. However, if I go to the template page it thinks the table does exist (e.g. it only offers me the option to recreate data, not to create data). If I look in the database itself, the table does exist with the prefix 'cargo_' as I'd expect. I've tried the maintenance script to recreate all of the cargo tables but it ignores this one. I tried using the same script to recreate just this table which went through without error (after I removed the '_NEXT' table left from trying to recreate the table) but I'm still getting the same problems.

It seems like I'm in a catch 22 here, any suggestions how I can get around this issue? All my other tables seem just fine?

Many thanks, Duncan (24 Jan 2018)

Sorry about that. It sounds like the table exists but doesn't have an entry in the "cargo_tables" DB table. The code needs to be better about handling both cases - the table exists but doesn't have an entry, or the table has an entry but doesn't exist. I would try manually deleting that table from the database - hopefully that will fix the problem. Yaron Koren (talk) 16:19, 24 January 2019 (UTC)
Hi Yaron, thanks that did indeed solve the problem. Duncan, 26 Jan 2019

Using Template Format in Cargo Query gives an error when a field has data type WikiText[edit]

I have a query that formats data in a cargo query using a template, as follows:

{{#cargo_query:
  |tables=
    Journals,
    Page_properties 
  |join on=
    Journals._pageName=Page_properties._pageName 
  |fields=
    DATE_FORMAT(Journals.Journal_Date,'%D %M %Y')=Date,
    CONCAT( '[[', Journals._pageName, ']]' )=Title,
    CONCAT( '[{{fullurl:Special:FormEdit/Article/}}', Journals._pageName, '&returnto={{FULLPAGENAME}}#tab=Knowledge_Catalogue Edit]')=Edit,
    Page_properties.Description=Description,
    Page_properties.Content=Content
  |where= (({{Holds Journal Groups|GroupNames={{{GroupNames}}} }}) AND (DATEDIFF(Journals.Journal_Date,NOW()) <= 0))
  |group by=Journals._pageName
  |order by=Date DESC
  |format = template
  |template = Display Journal Article
  |named args = yes
  |max display chars=100000
  |limit = 1000
}}

The 'Content' field has a datatype of WikiText. For the example below, the 'Content' field contains the following:

<ol>
<li><strong>[[Template:Display All Groups|Template:Display All Groups]]</strong> - added 'delete' to table rows.  Can this be combined with Display Groups template?</li>
<li><strong>[[Template:News|Template:News]]</strong> - correction to in template documentation</li>
<li><strong>[[Template:Group|Template:Group]]</strong> - changed tab back to news from journal and added journal tab</li>
<li><strong>[[Form:Sub News|Form:Sub News]]</strong>- reverted from Journal and made selective on News Groups</li>
<li><strong>[[Form:Sub Event|Form:Sub Event]]</strong>- made selective on Calendar Groups</li>
<li><strong>[[Form:Sub Assignment|Form:Sub Assignment]]</strong>- made selective on News Groups</li>
<li><strong>[[Form:Sub Article|Form:Sub Article]]</strong>- made selective on Interest Groups</li>
<li><strong>[[Form:Sub Journal|Form:Sub Journal]]</strong> - new form for inputting journal items</li>
<li><strong>[[Template:Journal|Template:Journal]]</strong> - new template or processing journal items</li>
<li><strong>[[Template:Display Journal|Template:Display Journal]]</strong> - new template for displaying journal information</li>
<li><strong>[[Template:Holds Journal Groups|Template:Holds Journal Groups]]</strong> - new template for use in Display Journal template</li>
<li><strong>[[Form:Article|Form:Article]]</strong> - included Journal option on 'additional processing' tab</li>
<li><strong>[[Template:Display Journal Article|Template:Display Journal Article]]</strong> - Template to format journal for Display Journal template</li>
</ol>

The resulting output shows as follows, where the template doesn't seem to be recognised and so has failed to be parsed correctly.

{{Display Journal Article|Date=26th January 2019|Title=List of changed templates and forms|Edit=Edit|Description=These should be updated to the live system. They are listed in the order they have been changed.|Content=
      Template:Display All Groups - added 'delete' to table rows.  Can this be combined with Display Groups template?
      Template:News - correction to in template documentation
      [[Templa }}

It looks a bit like the problem is because the Content field was being truncated, so I increased 'max display characters' to a very large number but that didn't change the output. I do find, however, if I remove the 'Content' field the query works as expected. I have also tried formatting the 'Content' field as Text but then the wikitext is displayed un-parsed.

I wonder if there's something else I could try?

Many thanks, Duncan 26 Jan 2019

Do you think this might be a similar problem to the one in #Text is not placed in a table above? The solution there was to increase the size of the Wikitext field by using Wikitext (size=5000). Jonathan3 (talk) 03:19, 27 January 2019 (UTC)
Great thanks Jonathan, that does look like it solves the problem. I agree also with Yaron, perhaps he should make Wikitext a blob so that users doesn't have to worry about setting an arbitrary size parameter? That gets my vote
Duncan, 28th Jan 2019
Actually, this is already done. In another still-unannounced change, about a month ago I changed the Wikitext type to be stored in the same way as Text, and added a new type, "Wikitext string", which is stored like a String. If you get the latest code, and recreate the relevant table(s) this should work better. Sorry, I didn't realize that this was the issue. Yaron Koren (talk) 20:02, 28 January 2019 (UTC)

Order by "DESC" not working [RESOLVED][edit]

After updating Cargo (I think that's the cause) existing queries containing |order by= _creationDate DESC gives the error Error: 1054 Unknown column '_creationDate DESC' in 'order clause'. When I remove "DESC" it works fine, though obviously isn't in the order I want... Is this something to do with now saving the time (as well as date) in _creationDate, or something I'm doing wrong? Thanks. Jonathan3 (talk) 17:45, 26 January 2019 (UTC)

Looks like the problem is in CargoSQLQuery.php:
	function setOrderBy( $orderByStr = null ) {
		if ( $orderByStr != '' ) {
			if ( strpos( $orderByStr, '(' ) === false && strpos( $orderByStr, '.' ) === false && strpos( $orderByStr, ',' ) === false ) {
				$this->mOrderByStr = $this->mCargoDB->addIdentifierQuotes( $orderByStr );
			} else {
				$this->mOrderByStr = $orderByStr;
			}
The problem seems to be that in the first case above it adds backticks to the whole thing, e.g. `_creationDate DESC`. I can work round it by using |order by= _pageData._creationDate DESC so that the . avoids the problem. Jonathan3 (talk) 20:59, 26 January 2019 (UTC)
Sorry about that - it was due to a fix from a week ago. I just checked in a fix for this fix... hopefully everything works now. Yaron Koren (talk) 03:36, 28 January 2019 (UTC)
No problem - works fine again now, thanks. Jonathan3 (talk) 13:38, 28 January 2019 (UTC)

Display field as plain text or wikitext [RESOLVED][edit]

I'd like to use #cargo_query (in one place) to be able to display a field to display as plain text (e.g. [[links]], ''italics'') and (in another place) to be able to display it rendered as wikitext (e.g. links, italics). What's the best way to do this? Thanks. Jonathan3 (talk) 23:42, 26 January 2019 (UTC)

What seems to work is to save the field as Text (previously I'd been using Wikitext). Then in a normal query (no format specified) it comes out as plain text, but in with |format=template it comes out formatted as wikitext. Jonathan3 (talk) 23:53, 26 January 2019 (UTC)

Yes, what Cargo needs is a way to declare the type of any field in a query. Right now the code just tries to figure out the type, which usually works but doesn't always, clearly. I actually added in a fix for this particular case about two months ago - it's not publicized yet, because there hasn't been a new release since then. But if you're using the latest code, you can wrap a TRIM() function around the field name - Cargo now treats TRIM() as being of type String, as opposed to Wikitext. So I would change the type back to Wikitext, then try that TRIM() hack. Yaron Koren (talk) 03:42, 28 January 2019 (UTC)
That works fine for me on the latest version of Cargo, thanks. Jonathan3 (talk) 14:50, 28 January 2019 (UTC)

Hiding some tables from Drilldown tabs[edit]

I'd like to hide some tables, e.g. "All pages". I see they're within <li class="tableName"> elements. Perhaps if they were to have an id equal to the table name, then I could use CSS to hide them? Would you be willing to do that? It wouldn't necessitate any major Cargo code changes. Or maybe it's possible already? Thanks, Jonathan3 (talk) 23:07, 28 January 2019 (UTC)

Behaviour of hierarchy in Drilldown[edit]

There seems to be an inconsistency:

  • Normally when a value is selected for a field, the page reloads with "(Click arrow to add another value)" for that field - the user is allowed to broaden the search by selecting further values.
  • However, when a value is selected for a hierarchy, the "(Click arrow to add another value)" message does not appear – the user is not allowed to broaden the search by selecting further values. The user is stuck with the selected value - if there are values lower down the hierarchy, he can then pick one but is then similarly stuck with that choice.

Is there a way to change this? I think the normal way is preferable. It would be good to be able to select two or more values from the same level of a hierarchy, or even from a different part of the hierarchy altogether. Maybe I've messed something up by hiding stuff using CSS/JS... Jonathan3 (talk) 23:37, 28 January 2019 (UTC)

cargoRecreateData.php and replacement tables[edit]

It would be nice if the script when used with --replacement offered, at the end, to switch in the new table... not important, but if not too complicated it would be good to have. Jonathan3 (talk) 23:44, 28 January 2019 (UTC)

Hide the form on Special:Viewdata [RESOLVED][edit]

Is it possible to create a link, containing a query, to Special:Viewdata and tell the page not to display its table? Jonathan3 (talk) 23:47, 28 January 2019 (UTC)

I don't understand - if you include a query in the query string of Special:ViewData, the form doesn't get shown. Yaron Koren (talk) 03:04, 29 January 2019 (UTC)
Sorry. I’ve been relying on this page too much instead of experimenting more. Jonathan3 (talk) 07:58, 29 January 2019 (UTC)

#if within where= parameter [RESOLVED][edit]

Museum of Lincolnshire Life, Lincoln, England - DSCF1724.JPG

Is it possible to have something like |where={{#if:{{{parameter|}}}|x=y}} in a Cargo query. I'm sure it should be but can't get it to work. Thanks. Jonathan3 (talk) 14:24, 29 January 2019 (UTC)

Yes it's possible but you still need the quotes, e.g. x="y". --RheingoldRiver (talk) 10:05, 31 January 2019 (UTC)
Cheers. I’ll check whether that’s the mistake I was making! Jonathan3 (talk) 11:16, 31 January 2019 (UTC)
That wasn't it, unfortunately. Maybe there's something up with my wiki. Maybe it's not parsing the {{{...}}} and is searching for that literal text.
When I have |where = {{#if: {{{parameter|}}} | _pageName="{{{parameter|}}}" | _pageName="Page B" }} it goes for Page B whether I use {{templatename|parameter=Page A}} or just {{templatename}}. In the same template I have put {{#if: {{{parameter|}}} | {{{parameter|}}} | B }} and it correctly either displays the parameter or "B".
When I try something simpler like |where=_pageName="{{{parameter|}}} it displays no results, wheras when I use |where=_pageName="Page A" it correctly selects Page A. Jonathan3 (talk) 13:21, 31 January 2019 (UTC)
I had the Cargo query within a tag extension which meant it wasn't parsing things properly - sorry! Jonathan3 (talk) 13:30, 31 January 2019 (UTC)
Adding image :-) Jonathan3 (talk) 13:43, 31 January 2019 (UTC)
Nice image, although it's not really necessary. :) Yaron Koren (talk) 23:45, 31 January 2019 (UTC)

Adding limit as template parameter not working any more [RESOLVED][edit]

I have a query in a template which contains |limit={{{limit|}}}. It used to work but now no limit is applied at all. Do you know what's up? I'm now on Cargo 2.0.1 (9c4aef9). Thanks. Jonathan3 (talk) 00:22, 30 January 2019 (UTC)

Does it work for you when it's not a template parameter? Yaron Koren (talk) 16:30, 30 January 2019 (UTC)
Yes, it works when I change the template to have limit=1 or any number. Jonathan3 (talk) 16:39, 30 January 2019 (UTC)
I had the Cargo query within a tag extension which meant it wasn't parsing things properly - sorry! Jonathan3 (talk) 13:31, 31 January 2019 (UTC)

Cargo and filecache[edit]

I am thinking about migrating from SMW to Cargo. On my site with the latest version of SMW, the rebuildFileCache script reports most pages "not cacheable". Are pages with Cargo data or queries cacheable with filecaching enabled? (I am aware of potential issues with stale data in the cached files, but I am the only contributor to the site and have a CRON script that runs rebuildFileCache, so this is not a big problem in my case). Henryfunk (talk) 22:39, 31 January 2019 (UTC)

I haven't heard of problems with rebuildFileCache.php and Cargo, but then again I haven't heard of problems with rebuildFileCache.php and SMW either... Yaron Koren (talk) 23:53, 31 January 2019 (UTC)
Thanks. I will certainly try Cargo then. FWIW: The caching problem started after upgrading to SMW 3.0 and MW 1.31. Filecaching works OK without SMW loaded. Henryfunk (talk) 11:58, 1 February 2019 (UTC)

How to query for raw field values with Cargo[edit]

Does anybody know whether there is a way to query for raw field values of integer fields (e.g. _pageID).

In a project database I try to connect sub projects to that project by this field

{{{field|belongs to subproject|input type=dropdown|values={{#cargo_query:tables=webmo_sub_project|fields=_pageID|where=belongs_to_project="{{#var: project id}}"|no html}}|mapping template=webmo_mapping sub project_id to name short }}}

If now the _pageID is e.g 1234, the thousand separator of the _pageID (1,234) causes the attached mapping template to search for _pageIDs 1 and 234 instead of 1234.

In a SMW environment I just used # after the correspondig property, like

{{#ask: [[Category:My Database]]
        |?My number property#
}}

Is there something similar possible in Cargo queries?

This gets rid of the comma for me: |fields=TRIM(_pageID). Jonathan3 (talk) 14:35, 1 February 2019 (UTC)

Thanks a lot. Solved my problem.--Shuitavsshente (talk) 16:36, 1 February 2019 (UTC)

Error: 1267 Illegal mix of collations on cargoRecreateData.php[edit]

This is probably a problem with my database setup but I wanted to record it here anyway in case it affected anyone else. If I'd not been using the command line cargoRecreateData.php I'd not so easily have known why the table wasn't recreating properly. It turned out to be a character that I'd copied and pasted from a Wordpress site into the Description field: U+200E or &# 8206;

Query: SELECT  COUNT(*)  FROM `..._cargo__...`    WHERE `_pageID` = '8980' AND `Description` = '...'

Function: Wikimedia\Rdbms\Database::select

Error: 1267 Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '=' (...)

Jonathan3 (talk) 14:25, 1 February 2019 (UTC)

Start date/End date and Timeline display format[edit]

This works fine for pages with both set, but a page with an empty end date doesn't appear at all in the timeline format. In contrast, the calendar format shows it as normal (i.e. being one day long). I wonder if this could be changed, so if the end date is not specified then timeline format assumed it to be the same as the start date. Thanks. Jonathan3 (talk) 14:58, 2 February 2019 (UTC)

That's a good idea - I just made this change in the Cargo code. Yaron Koren (talk) 03:28, 5 February 2019 (UTC)
I've just got the latest Cargo code but Special:Drilldown's timeline tab doesn't show the item with the Start date and no End date on my test wiki. Jonathan3 (talk) 13:38, 6 February 2019 (UTC)
It should appear as just a thin vertical bar. Yaron Koren (talk) 14:09, 6 February 2019 (UTC)

Cargo equivalent to PF restricted parameter[edit]

In Page Forms there's the parameter:

"restricted - Specifies that this field will be editable only by admins/sysops and disabled for all other users."

But I guess anyone could still edit the page and add |Field=Value. Is there anything in Cargo that would prevent the field being saved to the database in this situation? I guess something like "allowed values" but in relation to the type of user saving the page.

Or should I go the whole hog and start using Extension:Approved_Revs? Thanks. Jonathan3 (talk) 09:03, 6 February 2019 (UTC)

No, Cargo doesn't do any checks on the user, or comparisons between the old and new value. Yaron Koren (talk) 14:10, 6 February 2019 (UTC)

Cargo and $wgUseFileCache[edit]

@Yaron Koren: I've added a query on the Mediawiki Support desk (here, and in italics below) and wonder if there is anything Cargo-specific I need to know about this setting.

I've recently set $wgUseFileCache = true. Would it be possible to have pages cached for a maximum of 24 hours by setting $wgCacheEpoch = date('YmdHis', strtotime('-24 hours'))?

I have Cargo query pages which I would probably be happy to have cached for that limited period. I know alternatives are to $wgUseFileCache = false again, or use MagicNoCache's __NOCACHE__</nowiki> on the Cargo templates. Are there other options? Thanks.

Thanks. Jonathan3 (talk) 14:37, 11 February 2019 (UTC)

I don't know of anything. Yaron Koren (talk) 21:21, 11 February 2019 (UTC)
Thanks. Jonathan3 (talk) 16:01, 18 February 2019 (UTC)

LEFT JOIN [NOTFIXED / WORKAROUND][edit]

tl;dr To recap the massive discussion bellow, Cargo queries using join on cannot do full conditionals, which makes LEFT OUTER JOINS less useful.

Workaround: Create a dummy template to declare a new table that only holds the subset of rows you wanted in the join. cargo attach it from the main storage template and only store the subset of data you wanted. You can then join your main table with WHERE, with your new dummy smaller table that is pre-reduced. This only works if you only need full conditionals on one table. If you need it one more than one joined table, a bug with cargo_attach currently stops you from doing this. By the time you read this it may be fixed.
Bug 1: You cannot use more than 1 cargo_declare and 1 cargo_attach in a template, only the last one is respected when it comes to populating rows in the tables. You can get around this for cargo_declare by putting it in another template and attaching, but this only works for a maximum of 2 tables per template.
Bug 2: If you try to use a Cargo Query to populate a field in Cargo store, you may get garbage UNIQ--item data. There may be some bigger chicken and egg problems here even if this was fixed.
Thanks for all the help Joanthan3 and Yaron --Finlay Beaton (talk) 18:41, 15 February 2019 (UTC)

I have two tables where I want to display all items in TableA that are not in TableB... except I want to specify exclusion criteria for rows in both tables. In SQL SELECT tableA.col1 FROM tableA LEFT OUTER JOIN tableB ON tableA.col1 = TableB.col2 AND tableA.col2 = 'somevalue' AND tableB.col2 = 'anothervalue' WHERE tableB.col1 IS NULL but in Cargo the |join on= gives a Table and field name must be specified in somevalue error. I can't put it in the WHERE because then the tables would match every row on col1 and I would never have any nulls in tableB. So they need to be in the ON which is valid MySQL. I think this check might be a DDoS protection to ensure all tables have matching columns, but clearly I am doing that. I just want additional ON. I know if I installed Lua I could do the processing in Lua, but that would be inefficient and I would much rather my DB to do the work. I could use your ExternalData extension to query the cargo db directly, but I would lose the formatting niceness of Cargo. Any other ideas or recommendations? --Finlay Beaton (talk) 19:36, 13 February 2019 (UTC)

I don't understand what you mean by "exclusion criteria" for TableB - is the set of results you want to show a mix of pages that are and aren't in TableB? Yaron Koren (talk) 17:00, 14 February 2019 (UTC)
DB Fiddle clearly showing the problem with example tables. I don't want to come across as patronizing, just trying to clear up misunderstanding of the problem. About my example above: AND tableA.col2 = 'somevalue' is an inclusion/exclusion criteria. It specifies which rows in TableA to include for the join. Unlike with INNER JOIN, with LEFT OUTER JOIN you get different results if you put this in the WHERE or the ON. This is because if you put it in the WHERE you won't get NULLs if there is a row match BEFORE the exclusion criteria, it will just remove the whole matched rows. If instead you do it in the ON (while valid SQL, Cargo doesn't allow it with the error above) then you get NULL rows from the left outer join because those rows were removed before being joined (ie the rows never existed). The fiddle really does demonstrate it well IMHO --Finlay Beaton (talk) 19:00, 14 February 2019 (UTC)
Right, okay. Maybe the simplest solution is to store the name of the "captain" in the "groups" table, instead of storing that information in the "members" table. That would simplify queries, and it would also allow you to guarantee that a group has no more than one captain defined for it. Would that work? Yaron Koren (talk) 19:43, 14 February 2019 (UTC)
Yup that would work too, however since all I need for the groups is the _pageName in reality all I have is the members table. Since people populate the group page with all the members, I don't want them to have to mark the captain twice, so I'd have to make a groups table, and use a Cargo query to then populate its captain field based on the members table. It seems a bit hackish given there is a perfectly proper way to do the query in SQL, just Cargo joins don't support it... I could also maybe turn on the "table of all pages" and join against that to reintroduce the nulls, but again, why do that if there is perfectly valid SQL? If there's no intention in Cargo to support full conditionals in the |join on clause (anything you can put in |where= should be valid here), then I'll certainly look at doing one of those other workarounds. It might be nice for the docs to mention |join on doesn't support full conditionals. Thank you for taking the time to look at the problem and consider the feature. --Finlay Beaton (talk) 22:05, 14 February 2019 (UTC)
Sorry, I still don't understand. Is there a Cargo table for groups? If not, what tables are you joining? Yaron Koren (talk) 02:27, 15 February 2019 (UTC)
DB Fiddle I'm sorry, I guess the reality is more complicated then my example. My example showed how Cargo doesn't support full conditionals in the join and how that could be useful, but not my particular use case to the fullest. You can always restructure things into more or different table viewpoints, but I was trying to talk more about functionality and less about my particular use case. Restructuring would have different tradeoffs that are hard to express, like making other reports or queries more complicated, maintenance, etc. We have projects that have contacts listed on the page, with a mix of users and team contacts. One team is the "Owner Team", and one user is the "Owner User". I want a report that tells me which projects owned by a given team are missing an owner user. The fiddle I've added shows our problem more accurately, but introduces a lot of complexity I had hoped to avoid in my previous examples. Full conditionals in the join make this query trivial, hence my feature request.--Finlay Beaton (talk) 03:00, 15 February 2019 (UTC)
Alright. It's true that there will always be use cases for which special handling is needed, but still, since you're asking about a specific need, I feel compelled to drill down. Does this example more closely reflect your situation, with a table being joined to itself? If so, it still seems to me like there's a simpler option, which is to replace the "contacts" table with a "projects" table, which has fields like "owner team", "member teams" (a list), "owner user" and "member users" (another list). Or, if there's already a "projects" table, add those four fields to it. I don't see the need for this (I'm guessing) multiple-instance template setup you have. Unless, again, I don't know the full story. Yaron Koren (talk) 03:34, 15 February 2019 (UTC)
I do in fact have a projects table (generated by the projects infobox) for all the project pages. Then I have a ==Contacts== section where people can edit it one or more Template:Contact rows and it display a nice table of all the contacts for that project, their roles (fully customisable with a few preset like owner and owner team) and any special notes about them. It also cargo query displays the phone number / email / cell information from the relevant user/team page fields (user table, team table). So I could absolutely add a owner user and owner team to the project table. Since I don't want people to have to edit the same information in two places, I could use a Cargo query on the contacts table to autofill this entry in the projects table. Because of the extra fields in the contact table, I don't think adding all the members to the project table is realistic. I mentioned this workaround solution as possible in a previous reply too. I will probably do this. This did feel like a hack (using Cargo queries to populate cargo columns), and adding redundant fields just to satisfy a specific report seems like a bad pattern, but I could be wrong here. Especially since valid simple SQL exists to generate the report, just Cargo SQL doesn't support it. Thanks for taking so much time to understand my use case, and suggest possible solutions (even if they aren't the one I want), it's above and beyond, so thank you. User support is thankless work and despite this thread I'm very happy with Cargo and the transition from SMW. --Finlay Beaton (talk) 03:50, 15 February 2019 (UTC)
Set out to add the workaround this morning. Realised we have contacts on many different types of pages (not just projects), and really on any wiki page, not just ones with infoboxes. So adding the fields to the infobox not doable. Figured it was ugly but I would just add a ContactsActiveTeamOwners and ContactsActiveUserOwners tables to my contact template, and store each contact in 3 places using parserfunctions ifs based on their role. Then I discovered a template can only define a single cargo table (it uses the last one you define)... so that's out too. Tried to make sub-templates to define the other tables and call it from the first template, but of course Cargo doesn't work like that. Still searching for a workaround for cargo's limitations to join syntax. I'll update if I figure anything out.--Finlay Beaton (talk) 15:03, 15 February 2019 (UTC)
What a rabbit hole this has been. So because my contact lists are tables (and not say bulleted lists), I realised that I have another template on all pages that have contacts, a Template:Contacts head template. So I can make it store a new table called PagesWithContacts and add a ActiveOwnerTeam and ActiveOwnerUser fields to do the workaround you suggested. All good so far, and the table is created. To populate it, I would like to use a carge_query on the contacts table. Still good, and I get values in the columns where I should. The problem is that the value it stores is not the result of the query (a format=list, with default=0, which works perfectly when called outside the template), but text like ?'"`UNIQ--item-198--QINU`"'?. This seems like a bug, unless you're not supposed to use cargo queries to populate fields in cargo store. I could potentially see a chicken and egg situation when re-creating all tables here, depending on creation order (say if my PageWithContacts was populated before my Contacts table was). I could potentialy use Page Forms to make the user fill in the owners manually, but they already did using the Contacts template so this seems like forcing them to repeat work and would likely result in people forgetting to update it. So I'm a bit stumped again and looking for other options. I may just move the cargo tables to a separate db, then enable ExternalData extension to get proper sql join syntax on it. That will limit the security risk of exposing all wiki data via SQL. I lose the formatting and other query protections from cargo though. --Finlay Beaton (talk) 16:06, 15 February 2019 (UTC)

I’ve not read everything but noticed this: “Figured it was ugly but I would just add a ContactsActiveTeamOwners and ContactsActiveUserOwners tables to my contact template, and store each contact in 3 places using parserfunctions ifs based on their role. Then I discovered a template can only define a single cargo table (it uses the last one you define)... so that's out too.” Have you tried defining those two extra tables in separate templates that just never get called, and saving to those extra tables using your main template? You’d also need to use cargo_attach so that the data can be recreated again properly in future if needed. Adding fuller SQL functionality might be a good GSoC project, as students would be all over that sort of thing... Jonathan3 (talk) 17:33, 15 February 2019 (UTC)

Hey thanks for chiming in! I'm super new at Cargo and still learning a ton. You were right I did need to add a cargo_attach to make my contact template store the info. Then I used the dummy ContactsActiveUserOwners template to declare a new cargo_declare. In doing so with ContactsActiveTeamOwners I noticed a bug in Cargo though... my contact template only attaches to the LAST declared attach. So if I have 2 attaches, it only uses one. I can have a declare, and 1 attach, and that works great. But as soon as I add a 2nd attach only the last attach declared works for creating/storing data. Maybe I'm going crazy as I didn't make a full PoC bug reproduction. In my case I was able to eliminate the ContactsActiveTeamOwners table completely, and just join my Contacts table (with a WHERE to only display active team owners) with the new ContactsActiveUserOwners table. This all seems like a huge hack to get around the lack of full conditionals in joins... but at least my reports work now. Thanks for all the help Joanthan3 and Yaron. --Finlay Beaton (talk) 18:41, 15 February 2019 (UTC)
You’re welcome. I hope the only-one-attach problem is something Yaron can change relatively easily. It probably hasn’t occurred to anyone before. How about your main template storing to (and attached to) extra table A, and template A storing some of those fields to (and being attached to) extra table B? Though, re-reading your message on the phone here it seems you don’t need to try that. Jonathan3 (talk) 21:00, 15 February 2019 (UTC)
Thinking through it again, to have one template save to three tables, you'd maybe need actually to call the extra template with all the extra fields as parameters, and that extra template could either save to its own table or one attached to it... Jonathan3 (talk) 15:03, 18 February 2019 (UTC)
huh! I wouldn't have thought this would work. But it seems to? Thanks for the further workaround! I also learned something new about Cargo! I'd still rather have full conditionals as it doesn't require a ton of extra templates on a per-query basis, but I'll be using this until then. --Finlay Beaton (talk) 17:46, 18 February 2019 (UTC)

Fatal exception of type "Wikimedia\Rdbms\DBUnexpectedError"[edit]

I'm trying to recreate a table after adding two new fields to the Page Schema, Form, and Template.

Is this possible or do I need to recreate everything?

I go to "recreate table" from the Template:Nameofcategory and it gives me the new table to choose from, but when I hit "keep," I get this error:

A database query error has occurred. This may indicate a bug in the software.
[XGi-KUrQOn4AAJJi6CcAAAAb] 2019-02-17 01:55:53: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"

I also got this error, too, separately. I think I was trying to delete the table (and start over)

Fatal exception of type "Wikimedia\Rdbms\DBUnexpectedError"
I usually get the 'Wikimedia\Rdbms\DBQueryError' error when I didn't run the update script. Maybe it would solve your issue?
php maintenance/update.php
--Extarys (talk) 07:30, 17 February 2019 (UTC)

Using the data provided in the template inside a page[edit]

I'm currently adding basic data using Cargo but I would like to avoid using the template to display the data inside a page because the values are needed in different places and would make editing of articles harder.

I call the template at the top of the article. Is there an efficient way of accessing those values inside the same article?

Extarys (talk) 07:31, 17 February 2019 (UTC)

Would this work? Create a template for each value you want to use in the page, e.g. Template:Show_field1, and in that template have a Cargo query returning just that field |where=_pageName={{PAGENAME}}. Jonathan3 (talk) 15:54, 18 February 2019 (UTC) ...
Just checked - this works fine... you could do a generic template which also takes the field as a parameter e.g. {{show|fieldname}}, or even also the table if you need to do this for more than one, e.g. {{show|tablename|fieldname}}... Jonathan3 (talk) 22:05, 18 February 2019 (UTC)
Thanks Jonathan3 I'll try that, it's a good idea. Extarys (talk) 04:19, 19 February 2019 (UTC)

Importing csv[edit]

I'm currently using WikiDB to store large tables, and Cargo obviously seems like a possible improvement. But in order to use it, I must be able to do a batch import of a few thousand records. Given that I have a lot of content already, I dont want to go back and re-create tons of content using elaborate forms. But Cargo still seems like a nice upgrade from WikiDB. What's the easiest way to do a batch import, so that the records are later easily editable? thanks Osishkin (talk) 13:00, 17 February 2019 (UTC)

I've had importing data from elsewhere on my to do list for a while, and have been looking at using Extension:External Data to do this. Jonathan3 (talk) 16:00, 18 February 2019 (UTC)
You should probably use the Data Transfer extension, and specifically the page Special:ImportCSV, for this. Yaron Koren (talk) 22:58, 18 February 2019 (UTC)
I’m sorry, that’s the one - I should have checked before typing. Jonathan3 (talk) 08:55, 19 February 2019 (UTC)
Excuse my newbie ignorance but how exactly would that work? would you be willing to provide, say, an example csv file for the Books table which shows how the csv would be formatted in order to be automatically parsed into Cargo table data upon loading? also, can I later update records if I upload an updated version of records in a later csv, or will that only create duplicates? thank you Osishkin (talk) 14:56, 19 February 2019 (UTC)
The documentation gives one example - does it make sense? Yes, you can later modify the CSV and re-import it, overwriting the previous pages. Yaron Koren (talk) 20:11, 19 February 2019 (UTC)
Perhaps it does, but not to me. I dont understand how mediawiki knows to parse the imported records as ones that pertain to a specific Cargo table. I'm obviously missing something very basic here. This may be entirely my own ignorant fault, but if you're willing to shed a light on it, I'd appreciate itOsishkin (talk) 20:39, 19 February 2019 (UTC)
Data Transfer - and Page Forms - just deal in wikitext: they create pages that include template calls. It's the templates being called that handle the Cargo stuff, through their use of #cargo_declare and #cargo_store. Does that make sense? Yaron Koren (talk) 04:49, 20 February 2019 (UTC)
Sounds like I just need to toy around with it (once I get my tables working), and come back with more informed questions if I still dont get it. Thanks Osishkin (talk) 06:46, 20 February 2019 (UTC)

Tables not created[edit]

My tables are not being created, getting an error "table X not found". (from CargoUtils.php line 231);". I've seen people getting this error before. I'm using the latest version, and I tried the Books table example (so there's no issue with probelmatic field names). Any idea what might be the problem? MW 1.30.1, php 5.5.9, MySQL 5.7 Osishkin (talk) 14:56, 19 February 2019 (UTC)

You have to create each table yourself, either from each template page or all at once via the command-line script cargoRecreateData.php. Could that be the issue? Yaron Koren (talk) 04:49, 20 February 2019 (UTC)
I tried both. The above error is displayed after trying to view the table data after creating it from the template page. The php script then runs without any messages being displayed (perhaps because the table was already created?)
UPDATE - the php script does nothing even if run first after anew template declaring a table is created. Anyway, looking at the the relevant mysql DB shows that all cargo tables ARE created, but cargo_tables and cargo_pages have no records (i.e., "select * from cargo_tables" comes up empty)Osishkin (talk) 11:26, 20 February 2019 (UTC)
I’d something similar a while back. I just deleted everything from the database using SQL and started from scratch, which solved it. Jonathan3 (talk) 17:54, 20 February 2019 (UTC)
Tried it. Dropped all tables, restarted apache, ran update.php again, recreated table. Same error when trying to see table Osishkin (talk) 18:40, 20 February 2019 (UTC)
Maybe the database user hasn’t got the right permissions? Jonathan3 (talk) 20:14, 20 February 2019 (UTC)
It's the root user. And the tables do get created, they just cant be viewed. And I'm guessing that has to do with the fact that cargo_pages and cargo_tables remain empty Osishkin (talk) 21:14, 20 February 2019 (UTC)

I have no idea why that would be happening. What version of Cargo are you running? Yaron Koren (talk) 17:49, 21 February 2019 (UTC)

2.1.1 Osishkin (talk) 18:43, 21 February 2019 (UTC)
What happens if you create a template with nothing but <noinclude>{{#cargo_declare:_table=Test|Fieldname=String}}</noinclude> in it? Jonathan3 (talk) 23:35, 21 February 2019 (UTC)
Same error message Osishkin (talk) 05:42, 22 February 2019 (UTC)
BTW, I cant think how exactly this matters, but my wiki is an upgrade from MW 1.18. Could this somehow be the issue? notably, after some debugging I realized that the problem is indeed that indeed inserting an entry to cargo_tables in includes/CargoUtils:createCargoTableOrTables() doesnt work. I tried to create the cargo_tables entry manually from the command line, and that made the error go away.Osishkin (talk) 07:48, 22 February 2019 (UTC)
Okay, that's good to know. I don't think the former version is the issue. There's clearly some problem with the API call. Is this a public wiki, by any chance? Yaron Koren (talk) 15:02, 22 February 2019 (UTC)
It's not public, sorry Osishkin (talk) 17:59, 22 February 2019 (UTC)
Alright. Can you access the API on your wiki? It should look like the URLs that have index.php in them, but with index.php replaced with api.php. If you can - and it looks correct - please go to "api.php?action=cargorecreatedata&template=TemplateName&table=CargoTableName" on your wiki. An example would be "api.php?action=cargorecreatedata&template=Book&table=Books". What do you see when you go to that URL? Yaron Koren (talk) 21:14, 22 February 2019 (UTC)
{"success": ""} Osishkin (talk) 06:37, 23 February 2019 (UTC)

Do Cargo tables get indexes in the database or is every query a table scan?[edit]

I've been running the `slow log` on our mysql instance and 8 of the top 10 slow queries are all Cargo related, and they all show 100% table scan which would explain their slowness. So is there a way to create indexes with Cargo?

As of version 2.0, most Cargo DB fields are supposed to get indexed automatically - everything except "Text", "Wikitext" and "Searchtext" fields, I think. Do you know what types the fields are that are slowing down the queries on your site? Yaron Koren (talk) 18:10, 21 February 2019 (UTC)

where clause with special character #[edit]

I just ran into this problem from Extension_talk:Cargo/Archive_November_to_December_2015#where_clause_on_page_names_with_special_characters Dec 2015. Error in "where" parameter: the string "#" cannot be used within #cargo_query.. I tried to look in common problems and tried using the same trick as for quotes with a \# but that didn't work. Is there another workaround? --Finlay Beaton (talk) 16:15, 22 February 2019 (UTC)

I don't think there's any workaround at the moment. Why do you need to use '#'? Yaron Koren (talk) 16:28, 22 February 2019 (UTC)
The user specifies a page link to store in a cargo table. That link can be a dedicated page about the subject, or enterprising users started putting in links to a section on another page, ie Page#Section. Said value is sometimes searched from another template in a cargo query. Given the template that inputs the cargo data is using numbered parameters, if I added a new dedicated field just for subsections I'd have to go and edit every single use of the template. --Finlay Beaton (talk) 21:29, 22 February 2019 (UTC)