Extension talk:Cargo

[SOLVED] Show all items from yesterday and today, but at least three items from any date
I'd like to do a Cargo query that will:
 * return all items dated "today/yesterday" (maybe 10 or so, depending on the day)
 * if the above would return fewer than 3 results, display older results to make up the number.

I wonder whether I need something like

where the second query returns the max of 3 or the number of items from today/yesterday.

If I find time to work it out then I'll put it here. Otherwise I'd be grateful for any pointers.

Thanks. Jonathan3 (talk) 00:15, 25 January 2021 (UTC)


 * With logic that specific, I would think the only real way to accomplish it is with the Scribunto extension... Yaron Koren (talk) 16:02, 25 January 2021 (UTC)


 * Challenge accepted :-) Jonathan3 (talk) 17:03, 25 January 2021 (UTC)


 * I seem to have got this to work:




 * Hopefully I've included all the relevant parts above. Jonathan3 (talk) 21:52, 25 January 2021 (UTC)


 * Impressive! Does that really accomplish the full set of criteria? Yaron Koren (talk) 15:13, 26 January 2021 (UTC)


 * Thanks. Yes - it does what I wanted it to, as far as I can tell - though maybe I didn’t set it out perfectly above. Jonathan3 (talk) 15:29, 26 January 2021 (UTC)

Cargo and PostreSQL Port
I upgraded from MW 1.31 to MW 1.35 (1.35.1 (13dc141)) and Cargo is also the latest version (2.7.1 (28b4351)). I'm connecting to PostgreSQL 12.4 (Debian 12.4-1.pgdg90+1) with a custom Port set in the LocalSettings.php. The Mediawiki is correctly connecting to it! But it seems Cargo doesn't respect this setting because I get the following error message when I do a query:

Cheers and thank you Brabi81 (talk) 11:08, 26 January 2021 (UTC)


 * Found a fix that worked for me. Hardcoded the port in the open function in the includes->libs->rdbms->database->DatabasePostgres.php of MW. It seems that $this->port is not correctly set or something like when called from Cargo (it returns 0 ...). Brabi81 (talk) 13:36, 1 February 2021 (UTC)


 * I didn't know about that variable before. I just checked in a fix so that the $wgDBport value gets used with Postgres databases. Ideally there should also be a $wgCargoDBport variable as well, but that doesn't exist yet. Yaron Koren (talk) 17:10, 18 February 2021 (UTC)


 * The fix works as expected. Now I can remove that part again :) Thanks alot! Brabi81 (talk) 07:09, 19 February 2021 (UTC)

Table can't be created because it was "not declared" despite being declared
On our client's wiki, they have made a template and declared a table at Template:Factory, but the Create/Recreate data tab doesn't appear on the page. Manually navigating to it leads to an empty page.

Then we tried running  to no avail.

Is there a specific maintenance script that we need to run in order to fix this issue? We've already tried running update.php and runJobs.php.

Thank you in advance,

--MyWikis-JeffreyWang (talk) 09:57, 29 January 2021 (UTC)


 * This is very strange. The empty page is actually fine, in a sense - it just means that Cargo doesn't think that template contains a #cargo_declare call. (The display should obviously be better - right now it always does look like an error.) Yes, the template page says that the table is defined, but that's a separate set of code from the code that really determines whether the table has been declared. I don't know why it's failing, though. It might because the table name is lowercase, though I doubt it. I would recommend trying to re-save the page a few times (with at least a small change every time) - maybe it's just a temporary glitch. Yaron Koren (talk) 16:14, 29 January 2021 (UTC)

Cargo Order by
I upgraded from MW 1.31 to MW 1.35 (1.35.1 (13dc141)) and Cargo is also the latest version (2.7.1 (28b4351)). Same as above on a PostgreSQL Database. Another "bug" is now visible that I didn't saw immediately. It now makes a difference in the ordering of the fields for the "Order by" clause. I.e. if a query has the fields:

it will NOT get sorted by

Whereas if I rearrange the fields order to

it will be sorted as expected with

I'm totally sure that this was working in the old MW 1.31 Version I was running. Cheers Brabi81 (talk) 11:32, 15 February 2021 (UTC)


 * There was definitely a problem with the handling of ordering in Cargo; I think it was actually fixed a few weeks after the version you have now. If you get the now truly latest version of the code, this problem will hopefully go away. Yaron Koren (talk) 23:00, 15 February 2021 (UTC)


 * Okay my fault ... With the Verison  this problem doesn't exist anymore :) Sorry and thanks. Brabi81 (talk) 07:36, 16 February 2021 (UTC)

Search form [SOLVED]
What's the best way of adding a search form that would allow the user to add text, ideally with autocomplete?

For instance, with the book example:

Date: [...........]

Author: [..............]

Title: [...........]

I suppose the form should end up on the relevant Drilldown page. Thanks. Jonathan3 (talk) 22:54, 24 February 2021 (UTC)


 * Could you explain this desired interface in more detail? I don't understand how you can have searching and editing at the same time. Yaron Koren (talk) 19:42, 25 February 2021 (UTC)

Sorry, ignore “add text” - I just meant the user would type search terms into the form. Jonathan3 (talk) 20:09, 25 February 2021 (UTC)


 * Oh, I get it. You should use the Page Forms extension - see Creating query forms. Yaron Koren (talk) 22:23, 25 February 2021 (UTC)


 * Sorry - I'd seen that ages ago and forgotten. Can it do autocomplete on Cargo fields or is that really a question for the PF page? Thanks Jonathan3 (talk) 23:09, 25 February 2021 (UTC)


 * That's a PF question, but yes. Yaron Koren (talk) 23:12, 25 February 2021 (UTC)


 * Thanks again. Jonathan3 (talk) 23:36, 25 February 2021 (UTC)

[SOLVED] Create an on-wiki to do list
I'd quite like to have a to do list on my wiki and have buttons/links beside each item:
 * Edit the item (simple enough - link to the PF form).
 * Mark the item as done.
 * Delete the item entirely. This could link to the  page but could it be done without that intermediate step?

Would this be possible? Thanks.

Maybe the answer instead is just to display the items as a normal list (from a to do list Cargo table) on the page, and to use Special:MultiPageEdit for marking as done or deleting. Jonathan3 (talk) 23:35, 25 February 2021 (UTC)


 * These again sound like Page Forms questions. Yaron Koren (talk) 02:05, 26 February 2021 (UTC)


 * Thanks. I'll ask on the talk page there. Jonathan3 (talk) 10:47, 26 February 2021 (UTC)

[SOLVED] Order by date when some dates are Month/Year and some are Year
If I use the MySQL  in Cargo's   parameter, I think   orders it alphanumerically rather than by date.

Do I need to use the date as it comes from the database (which I think has defaults to 1/Jan for blanks) - and the  field in a template to work out whether to display MMM YYYY or just YYYY?

Thanks Jonathan3 (talk) 10:16, 1 March 2021 (UTC)


 * I've tried and  works, but I wondered whether there is another way. Jonathan3 (talk) 10:24, 1 March 2021 (UTC)


 * "__precision" is the way to go, yes. Yaron Koren (talk) 14:20, 1 March 2021 (UTC)


 * Great, thanks :-) Jonathan3 (talk) 14:44, 1 March 2021 (UTC)

[SOLVED] Use date's __precision field within _drilldownTabs parameter for Special:Drilldown
Is this possible? When I try (for a field called Date)  or just   I get an error. I could turn on debugging to tell you what error, though it might be something already known not to work. Thanks. Jonathan3 (talk) 22:33, 1 March 2021 (UTC)


 * Why would you want that? Yaron Koren (talk) 23:28, 1 March 2021 (UTC)


 * For a template that distinguishes between dates with month/year and dates with only year. Jonathan3 (talk) 23:32, 1 March 2021 (UTC)
 * Cancel that - the standard "Table" tab works fine for dates like that. Sorry. Jonathan3 (talk) 00:04, 2 March 2021 (UTC)

Formatting for "more results text"
I'm able to format "default" text but not "more results text".

Produces:


 * ''No results

Whereas

Produces:

*''More...'

Where am I going wrong? Thanks. Jonathan3 (talk) 13:54, 23 March 2021 (UTC)

Error: 'Table "..." is not declared in any template.'
I just deleted a template and our nightly cron that calls cargoRecreateData.php now throws an error 'Table "..." is not declared in any template.' I searched the talk page for this extension and it turns out I asked about this before: Extension_talk:Cargo/Archive_August_to_October_2019. The solution Yaron suggests in there works. I just wonder if it would be possible to update the error message to provide the info how to solve this. E.g. change "'Table "..." is not declared in any template. If the template was deleted and the table is no longer needed it can be deleted in Special:CargoTables.' Or something like that that sends the user off in the right direction. Thanks! Tenbergen (talk) 16:09, 23 March 2021 (UTC)


 * Clearly I needed a reminder. :) I just checked in a fix, so that the message is only printed in non-"quiet" mode. By the way, I wouldn't really call it an error message, and I wouldn't say it gets "thrown" - it's just a line that's printed on the screen. I changed the message slightly, so now it reads "Table "..." is not declared in any template; skipping.". I don't think all that instruction is needed, because this is not necessarily an error, from the perspective of this script; it's just a table that the script can't handle. Yaron Koren (talk) 22:30, 23 March 2021 (UTC)

List of Strings and commas
Hello! First of all, I would like to thank you for this incredible extension, it's helping me so much. And I'm not an expert in Cargo, so sorry for my noob-ness.


 * I have a query in a template and one of the declared values is a . The thing is, when I use this query with another table (using other fields as well), the list of strings doesn't show the delimiter (it shows ItemA,ItemB,ItemC... instead of ItemA • ItemB • ItemC...). Is there a way I can make this work with the delimiter? Using   doesn't work (on the query) doesn't work as well, it doesn't even show the links for some reason.
 * Another question: is it possible to set each item on a list of string to be a value on a template? Let's say I have two queries, one of them with simple and plain fields, and other querying a list of string. On the list, each item would use a parameter on the template. So Row1 has just ItemA, so in the template the parameter will be set to ItemA, Row2 has ItemA and ItemB, so in the template there would be two of the parameters, one with ItemA and the other with ItemB, etc.

Sorry if it's confusing. :P &mdash;Lakelimbo (talk) 16:47, 23 March 2021 (UTC)


 * You could probably use Extension:Page_Forms/Page_Forms_and_templates to solve both problems. Jonathan3 (talk) 20:58, 23 March 2021 (UTC)

Tabs for _pageData and _fileData Drilldown tables
Is is possible to add tabs for these tables? I'd like to add a custom tab. Jonathan3 (talk) 22:59, 23 March 2021 (UTC)

[SOLVED] Error on full text search using " or ' characters
When I search for  on Special:Drilldown/_fileData it lists 5 results, fine.

When I sarch for  (or anything like   it correctly states "Showing below up to 5 results in range #1 to #5." but then


 * A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? Query: SELECT `_fileData_alias`.`_pageName` AS `_pageName`,`_fileData_alias`.`_fullText` AS `File text` FROM `wm_cargo___fileData` `_fileData_alias` WHERE MATCH(_fileData_alias._fullText) AGAINST ('" IN BOOLEAN MODE) needle"' ORDER BY _fileData_alias._pageName,_fileData_alias._fullText LIMIT 250 Function: CargoSQLQuery::run Error: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ORDER BY _fileData_alias._pageName,_fileData_alias._fullText LIMIT 250' at line 1 (localhost)

Should I just upgrade to the latest Cargo or is this an known problem? Jonathan3 (talk) 23:10, 23 March 2021 (UTC)


 * I've just upgraded but it's the same. Jonathan3 (talk) 23:28, 24 March 2021 (UTC)

P.S. If the answer is "There are no results for this report." then there is no error shown (e.g. searching for . Jonathan3 (talk) 00:04, 24 March 2021 (UTC)


 * Sorry about that; there was a problem with Cargo's parsing of "MATCHES" queries. I just checked in an improvement; the parsing is still not perfect, but this query, at least, should work now. Yaron Koren (talk) 15:19, 8 April 2021 (UTC)


 * Thanks. The query (using double quotation marks around a phrase) now works without any errors, but the "results in context" shows seemingly random text (instead of showing some words before/after the emboldened search term). Jonathan3 (talk) 16:18, 8 April 2021 (UTC)


 * How strange! I looked into it - Cargo uses MediaWiki's own code to find the "search context", and by default the MW code is pretty bad, apparently for performance reasons. If you add the line "$wgAdvancedSearchHighlighting = true;" to LocalSettings.php, it works a lot better. Maybe Cargo should be changing the value of this variable; I'm not sure. Yaron Koren (talk) 16:37, 8 April 2021 (UTC)


 * I changed Manual:$wgAdvancedSearchHighlighting to true and it started to work properly! Thank you. This is something that could be noted in the Cargo manual somewhere. Jonathan3 (talk) 08:57, 9 April 2021 (UTC)

[SOLVED] Context of full text search term in query
On the Drilldown page there is a nice feature in which the words surrounding the search term are shown.

Is it possible to replicate this in a #cargo_query? Thanks. Jonathan3 (talk) 23:12, 23 March 2021 (UTC)


 * Sorry for the delay. This is already done by default for queries that use the MATCHES keyword - see here. Yaron Koren (talk) 20:14, 29 March 2021 (UTC)


 * That's great, thanks. Jonathan3 (talk) 16:18, 8 April 2021 (UTC)

[SOLVED] Ability to search full text for phrase rather than separate words
Currently if you search for Joe Bloggs, you'll get pages with Joe and pages with Bloggs. Is it possible to restrict it to the exact words "Joe Bloggs"? I guess if the error with the quotation marks gets fixed it might allow for this - would that be right? Thanks. Jonathan3 (talk) 23:33, 24 March 2021 (UTC)
 * Yes. Yaron Koren (talk) 15:20, 8 April 2021 (UTC)


 * Thanks. But see the problem with the lack of "context" results mentioned in the topic above this one. Jonathan3 (talk) 16:19, 8 April 2021 (UTC)

Unable to get Drilldown working
I'm struggling to get the Drilldown function working. I have a table with data in it, but every filter but one shows "(There are no values for this filter)". The SQL query executed by Cargo that returns no results is:

For the only filter that appears to work, the  clause is omitted, and, indeed, the above query returns values when the   clause is removed.

Perhaps related:   recreates the table just fine, but clicking the "Recreate data" button from the template page returns a replacement table with 0 rows.

What am I doing wrong?

Thanks so much!


 * What version of Cargo are you running? Yaron Koren (talk) 14:47, 4 April 2021 (UTC)


 * Sorry -- Cargo 2.7.1, Mediawiki 1.34.0 (but the problem first manifested with Cargo 2.5, and I upgraded to the latest version as part of my efforts to debug). php7.3 and a PostgreSQL backend in case that might be relevant.  Any hints where I should be looking?  Thank you!  Wilo108 (talk) 16:02, 4 April 2021 (UTC)


 * I have no idea - I haven't heard of that particular problem happening before. I'm guessing that it's somehow due to the use of PostgreSQL, but I'm not sure about that either. What are the other queries getting called? There should be various queries, not just that one. Yaron Koren (talk) 20:52, 4 April 2021 (UTC)


 * hmm, that's concerning. There are other queries, but none of the cargo queries are being logged when using   or in the debug toolbar, so I've been writing   to a debug log directly in an attempt to work out what's going on.  I'm also seeing problems with simple queries like


 * failing because (e.g)  isn't qualified or quoted, so PostgreSQL throws  .  (Query works fine when quoted appropriately and executed directly against the PostgreSQL server.)  Aggregating queries that use   also return empty columns, but the same queries generate the expected results when executed directly against PostgreSQL (my development server is running PostgreSQL 11.11 on Debian 10.9, fwiw.)
 * Has there been any testing with PostgreSQL? Would I be better just seeing if I can get this moved to MySQL/MariaDB?


 * That's pretty bad. Testing on PostgreSQL has been quite limited; I haven't tested directly on it in a long time, so testing is basically confined to whatever bugs people report (I do try to fix whatever bugs I hear about). Switching to MySQL would most likely get rid of these problems in the short run, though, yes. Yaron Koren (talk) 15:18, 5 April 2021 (UTC)


 * Okay, no worries. Do you have a bug tracker somewhere, or is this it?  Also, what's the state of the test suite?  I'm not really a php guy, but migrating this to MySQL/MariaDB properly looks like it will take more work than I'd first imagined (doable, but not trivial by any means), and if that time could be better spent fixing SQL queries so that they execute properly against PostgreSQL... Wilo108 (talk) 16:53, 7 April 2021 (UTC)
 * There is a bug tracker of sorts on the Wikimedia Phabricator; see here. The testing code for Cargo is pretty limited; just some unit tests. I don't think any of them involve DB operations, unfortunately. I do want this problem to get fixed, one way or another. Yaron Koren (talk) 02:09, 12 April 2021 (UTC)

Irrelevant results and $wgAdvancedSearchHighlighting
I set Manual:$wgAdvancedSearchHighlighting to true, which fixed a problem above where search terms within double quotation marks were not highlighted in Cargo "MATCHES" results.

Strangely, it leads to another problem.

I've got a Cargo query which joins a normal Cargo table, the _pageData table, and the _fileData table (joined when the filename matches a File field in the normal Cargo table).

I use it to search both _pageData._fullText and _fileData._fullText for the search term.

In a situation where the search term is found in the file but not the page, the term is highlighted in the _fileData._fullText cell but:


 * With $wgAdvancedSearchHighlighting = false, the _pageData._fullText cell in the results table is blank (as expected)


 * With $wgAdvancedSearchHighlighting = true, the _pageData._fullText cell in the results table irrelevantly contains the first part of the Cargo template call's wikitext (|Field=contents|Field=contents etc).

Is there any way round this? Thanks. Jonathan3 (talk) 09:16, 9 April 2021 (UTC)


 * P.S. When the search term is found in both the page and the file, the correct parts are highlighted for each.


 * And when the search term is found in the page but not the file, the _fileData._fullText cell is blank (as expected). Jonathan3 (talk) 09:18, 9 April 2021 (UTC)

Fixing the plaintext image HTML glitch without disabling ConfirmEdit?
Hi! So I’m aware this is an issue that has been addressed before a couple of times (here and here) and I’m also aware that it seems to be linked to the extension ConfirmEdit. However, I’d prefer not to disable that extension if I can help it. Is there no other way to fix this issue? (And yeah, I’m aware the second link included a lua workaround, but... well, I can’t understand lua, so.) --Kiwibasket (talk) 20:20, 10 April 2021 (UTC)