Extension talk:Cargo

Changes to Cargo content via XML Import won't update tables?
We used the MediaWiki Import/Export special pages to allow mass changes to some pages holding Cargo content, but related tables didn't update afterwards unless we recreate data (or save individual pages) manually. Is this expected behavior, or have we done something wrong? Thanks!

--198.49.180.254 14:26, 4 May 2017 (UTC)


 * That sounds like a bug in Cargo - my guess is that Special:Import doesn't call the hook that Cargo relies on to know when a page has been re-saved. So maybe it's really a bug in MediaWiki; I don't know. Yaron Koren (talk) 14:47, 4 May 2017 (UTC)

Hiding the "where" column(s) (SOLVED)
Hi there. I'm trying to run a query to show a list of events - a personal timeline - on each character sheet, based on the person's name showing up in the event.

The events show up fine, but the last column is redundant, and I'd like to hide it. Is there a way to have the output table not show a column that is needed only for the where clause? --Brilligtove (talk) 17:44, 12 March 2017 (UTC)


 * Just don't include it in the "fields" parameter - you can have fields in "where" that aren't in "fields". Yaron Koren (talk) 21:40, 12 March 2017 (UTC)
 * Blimey, that was easy. I definitely tried that at one point but had other syntax errors to deal with and didn't go back to try it again. Thanks!--Brilligtove (talk) 17:31, 13 March 2017 (UTC)

Creation of tables fails on last column
There is an SQL-error apparently when trying to create the last column defined:

Software:

Cargo 1.3, MW 1.27.1, PHP 5.6.30, MariaDB 10.0.29

Template:

Error: php cargoRecreateData.php --table Knowledge_Bar_Foo Recreating data for Cargo table Knowledge_Bar_Foo in 5 seconds... hit [Ctrl]-C to escape. Deleting and recreating table... A database query error has occurred. Query: CREATE TABLE `MyPrefix_cargo__Knowledge_Bar_Foo__Knowledge_HasInfos` ( `_rowID` Int, `_value` Varchar(300) ) Function: DatabaseBase::query Error: 1103 Incorrect table name 'MyPrefix_cargo__Knowledge_Bar_Foo__Knowledge_HasInfos' (localhost) This operation also fails via the GUI. Help appreciated. --91.65.247.224 16:26, 17 March 2017 (UTC)


 * That's odd... I have no idea. Especially not why it fails on that table/column, and not on the previous ones. Could you try removing that "HasInfos" line from #cargo_declare, and see if it works without it? Yaron Koren (talk) 17:13, 17 March 2017 (UTC)


 * Turns out that we ran into multiple issues here.


 * Underscores cannot be used to define a field so the template must be like this and things roll:


 * So the script probably already failed on the lines before but silently because of the second issue we got a visible error:


 * The second issue was that on the same template #cargo_store did not have a line for the KnowledgeHasInfos field so I got

PHP Notice: Undefined index: KnowledgeHasInfos in /extensions/Cargo/parserfunctions/CargoStore.php on line 293 Notice: Undefined index: KnowledgeHasInfos in /extensions/Cargo/parserfunctions/CargoStore.php on line 293
 * Same for line 315 of that file.


 * It seems to be working now. Thanks for the suggestion which helped track down the issue. --91.65.247.224 09:09, 19 March 2017 (UTC)


 * Still out of luck. Now I get

[7450cee4a6402af82ba10421] /wiki/Special:CargoTables/Knowledge_Bar_Foo MWException from line 526 of /extensions/Cargo/CargoSQLQuery.php: Error: no field named "" found for any of the specified database tables.

Backtrace:


 * 1) 0 /extensions/Cargo/specials/CargoTables.php(101): CargoSQLQuery->setDescriptionsForFields
 * 2) 1 /includes/specialpage/SpecialPage.php(479): CargoTables->execute(string)
 * 3) 2 /includes/specialpage/SpecialPageFactory.php(576): SpecialPage->run(string)
 * 4) 3 /includes/MediaWiki.php(282): SpecialPageFactory::executePath(Title, RequestContext)
 * 5) 4 /includes/MediaWiki.php(735): MediaWiki->performRequest
 * 6) 5 /includes/MediaWiki.php(509): MediaWiki->main
 * 7) 6 /index.php(43): MediaWiki->run
 * 8) 7 {main}


 * --91.65.247.224 09:16, 19 March 2017 (UTC)


 * Removing all the blancs from #cargo_declare and #cargo_store does not help either. I also created a second template KnowledgeFooBar but this special page still fails. --91.65.247.224 09:24, 19 March 2017 (UTC)


 * Actually, field names can contain underscores. Can you run regular #cargo_query calls on this table, or do those fail as well? Yaron Koren (talk) 22:52, 19 March 2017 (UTC)


 * Turns out I told only half of the fun regarding "#cargo_declare" and "#cargo_store". Both functions contained invalid characters like umlauts when declaring fields. Now after only using characters MySQL can handle in names things seem to work. Three wishes if possible from what I have learned now: 1. The user or at least the sysop should get a warning that invalid characters were used, 2. the user/sysop should get a warning that the fields specified with both functions should match in name and number and 3. cargoRecreateData.php or a new script should have an option to delete tables if no longer used on the wiki. --91.65.247.224 15:38, 27 March 2017 (UTC)


 * Ah - that makes sense. #1 is a very good idea - I should have added that check in a long time ago. #2 I'm not sure is possible, unfortunately. For #3 - you should be able to delete any table at the page Special:CargoTables. Is this table not showing up there? Or would you prefer to do this from the command line in any case? Yaron Koren (talk) 15:54, 27 March 2017 (UTC)


 * I figured that 2 might be pretty tricky. For 3 yes, I prefer to have a command line option. If there is a listing option for cargo tables this will be nice too. --91.65.247.224 16:04, 27 March 2017 (UTC)

Okay, I looked into this, and it turns out that MySQL - and thus Cargo - can handle accented characters in table and field names. There was just a bug in the code that caused the error message you saw. I just checked in a fix for it, so if you get the latest code, you may be able to go back to the field names you used to have. Yaron Koren (talk) 02:17, 28 March 2017 (UTC)


 * Sorry that it took a while. No the issue caused by e.g. umlauts is still there using current master (d54c207) of Cargo. --91.65.247.224 17:30, 7 April 2017 (UTC)


 * That's not good. What's the exact error message you see? Yaron Koren (talk) 18:38, 7 April 2017 (UTC)


 * It is basically the same error but from another line:

[203dd3ab9e2c92b6eab6cd24] /wiki/Special:CargoTables/Knowledge_Bar_Foo MWException from line 565 of /.../w/extensions/Cargo/CargoSQLQuery.php: Error: no field named "" found for any of the specified database tables.

Backtrace:


 * 1) 0 /.../w/extensions/Cargo/specials/CargoTables.php(115): CargoSQLQuery->setDescriptionsForFields
 * 2) 1 /.../w/includes/specialpage/SpecialPage.php(479): CargoTables->execute(string)
 * 3) 2 /.../w/includes/specialpage/SpecialPageFactory.php(577): SpecialPage->run(string)
 * 4) 3 /.../w/includes/MediaWiki.php(282): SpecialPageFactory::executePath(Title, RequestContext)
 * 5) 4 /.../w/includes/MediaWiki.php(735): MediaWiki->performRequest
 * 6) 5 /.../w/includes/MediaWiki.php(509): MediaWiki->main
 * 7) 6 /.../w/index.php(43): MediaWiki->run
 * 8) 7 {main}
 * --91.65.247.224 16:52, 25 April 2017 (UTC)


 * That's too bad. This is actually the same line, unfortunately - it's just that other parts of the code have moved around. What is the name of the field that is causing the problem, do you know? Yaron Koren (talk) 19:33, 25 April 2017 (UTC)


 * A bit sad. I just run cargoRecreateData.php after upgrading to 1.3.1 and it ends without issues. The issue when trying to access the special page on wiki remains the same. Withing the template I do not really see what could cause the issue. No special chars etc. --91.65.247.224 14:15, 17 May 2017 (UTC)


 * Could you email me the template definition - or pastebin it and link to it here? Yaron Koren (talk) 16:06, 17 May 2017 (UTC)


 * Here we are. --91.65.247.224 16:33, 17 May 2017 (UTC)

Aha! That's very helpful. My fix for the previous problem with non-ASCII characters ended up causing a problem with fields with numbers in their name. I just checked in a fix, so hopefully now everything works - ASCII and non-ASCII characters alike. At least, I hope so. Yaron Koren (talk) 17:40, 17 May 2017 (UTC)


 * Great that this was of help. Sometimes the small things can be troubling. Affirmative, current master of Cargo makes the issue go away. Thank you for the fix! --91.65.247.224 15:43, 18 May 2017 (UTC)

Pre- and post-nominal letters for surnames in List of String
I have several pages, relating to court cases, each of which was decided by one or more judges.

The judges have pre-nominal (e.g. DJ Bloggs or HHJ Bloggs) or post-nominal (e.g. Bloggs J or Bloggs LJ) letters.

I would like to display the full title for the judge on each case page, but use Special:Drilldown on the surname.

The reason for this is that, although the surname is unique, the letters may change over time (e.g. in a 2000 case it may be Bloggs J, but the same judge in 2010 may be Bloggs LJ).

What would be the best way to implement this?

I had thought of using three fields, all Lists of String, for pre-nominal, surname, post-nominal, but I don't think there is any way to manipulate the individual results of Lists (in order to display pre-nominal + surname + post-nominal in a template).

Alternatively I could maybe use a list of wikitext like " Bloggs J, Jones J " and in the judge surname pages have a template query using HOLDS LIKE on the surname, but this wouldn't help with the Special:Drilldown interface.

Any ideas would be very welcome :-) Jonathan3 (talk) 22:58, 20 April 2017 (UTC)
 * Are there going to be pages for the judges? Or would you have that only if it's necessary for filtering? And why filter on, say, "Jones", instead of "Jones, J"? What if there's more than one Jones? Yaron Koren (talk)


 * Third attempt at replying. The mobile site crashed on my iPad...


 * I don't plan to have a page per judge. Ideally I'll write that code for you to have Miga-like links as an option for lists of strings etc. But more limited functionality could be provided by a judge page which lists each relevant case page. Perhaps automatically-created?


 * A judge's pre- or post-nominal letters may change over time so it would be much better (easier, less prone to error) to be able to filter on name only. Or have a way of linking each one so that filtering on one automatically includes the others. For example, HHJ Jones, Jones J and Jones LJ may be the same person.


 * At each level a more junior judge uses his Christian name (in addition to surname) as disambiguation if there is another judge with the same surname. There may be ambiguities between levels - I am not sure - but I have never seen this, and would be happy to proceed on the basis that it's unlikely and I could add Christian names for disambiguation on my website if it became necessary.


 * Perhaps the simplest solution/workaround would be to use name only in the list of string (for filtering etc) but have an extra text field for the [name with] correct pre- or post-nominal letters for the judge(s) on each individual case page. That has its advantages - mainly flexibility, e.g. strictly "Jones and Bloggs JJ" is correct whereas "Jones J, Bloggs J" is wrong - but it's still not my preferred option. Jonathan3 (talk) 10:18, 21 April 2017 (UTC)


 * Alright. It seems to me that the right answer is a variant on your idea at the end - have a Text field for the exact wording of the authors list, and have a separate field, holding a list of Strings, for however you want to store the information as data, whether it's last names only or anything else. It's redundant, but it seems like the easiest approach. Yaron Koren (talk) 12:28, 21 April 2017 (UTC)


 * Thanks. I'll give that a go. Jonathan3 (talk) 21:04, 23 April 2017 (UTC)

Hierarchies and categories
I have some plain wiki pages which I would like to convert to using Cargo. I am using four sets of categories per page. Three of these can easily be changed to Cargo fields (e.g. year). But the fourth category, in relation to the page subject matter, is part of a hierarchy which can be browsed by Extension:CategoryTree. I see from Extension:Cargo/Known bugs and planned features that a hierarchical structure for fields is not currently possible, but what could I do in the meantime?

Would it be possible to have a field containing a List of String, corresponding to the existing subject matter categories, which could be used in a template to put pages into categories? If so, it would have the twin benefits of allowing the categories to be browsed by CategoryTree, and allowing for them (via the field) to be browsed using Cargo's drilldown page. What would be the best way of implementing this? Thanks in advance. Jonathan3 (talk) 21:03, 23 April 2017 (UTC)


 * Yes, that's possible - use the tree input type to select the values within the form, and then use #arraymap in the template to create a category tag for each value (see the "For category names" subsection in that link). Yaron Koren (talk) 23:48, 23 April 2017 (UTC)


 * Thanks. The tree input type example on DiscourseDB briefly flashes up on my iPad Safari normally (with a category structure) but then ends up showing empty check/radio box inputs. Otherwise it looks like what I need and I guess it works on a "real computer". Jonathan3 (talk) 21:19, 24 April 2017 (UTC)


 * Oh, I had no idea that "tree" failed on the iPad - I'll have to look into that. Yaron Koren (talk) 01:43, 25 April 2017 (UTC)


 * The same thing happens when I try on various browsers on the PC. Maybe I am selecting the wrong option. The URL is: http://discoursedb.org/wiki/Special:FormEdit/Category_input_test/Testing Jonathan3 (talk) 00:01, 27 April 2017 (UTC)


 * Oops! I made a change to the code last week that clearly broke the "tree" input - sorry about that. I just checked in what I think is a fix. Yaron Koren (talk) 02:41, 27 April 2017 (UTC)

Cargo query on Drilldown page
It would be good if it were possible to add a Cargo query (or template containing one) to the Drilldown page. For instance, for one table I'd like it to show a map, and for another table I'd like it to show the "summary" field from the relevant pages. Is this something that would be possible in future? Jonathan3 (talk) 21:55, 26 April 2017 (UTC)


 * It would be great to show a map if the table has a Coordinates field - it's actually one of the planned features to have that done automatically. Allowing additional display fields is an interesting idea - Semantic Drilldown has that feature, so it's not inconceivable. Yaron Koren (talk) 02:57, 27 April 2017 (UTC)


 * I have added a proposed patch for this on Phabricator at . I hope you find it useful! :-) Jonathan3 (talk) 01:03, 30 May 2017 (UTC)

timeline display format displays blank
I'm using a Referata wiki with Cargo. The timeline display format used to work, but now displays blank. (The calendar display format does work, but is not useful for my purposes, since I want to be able to scroll quickly over a span of at least 60 years.) Please confirm that timeline should work (it does work on the example given in the Extension:Cargo/Display formats page), and if so, suggest what I might try to get it to work again. I tried each Skin option and each Date format option, with the same blank result in each case.--Egnatoff (talk) 16:32, 29 April 2017 (UTC)


 * What's the URL with the problem? If you don't want to share the URL, what's the query? Yaron Koren (talk) 14:10, 30 April 2017 (UTC)
 * Here's the URL of the query page: Even Timeline and here's the query:

Concerts by Date
--Egnatoff (talk) 01:35, 2 May 2017 (UTC)


 * Oops! It looks like "timeline" has been broken for a while. I just fixed it, I think, on Referata (that's the nice thing about having access to the server where the problem is). I also plan to check in this fix. Yaron Koren (talk) 03:09, 2 May 2017 (UTC)
 * Hurray! Thank you very much. I will have several thousand events so the timeline display will be a very important browsing mechanism.--Egnatoff (talk) 02:32, 3 May 2017 (UTC)

Cargo table list coordination problem
Example: Music concert database

Problem: Coordinating two lists from different tables—ensuring they match on data entry, and displaying them nicely. In particular, instruments are listed in the Pieces table and performers are listed in the PiecePerformances table. They need to match.

Tables in question: Field1 (type), Field 2 (type),. ..

Pieces: Page, Composer (page), Title,. . . (where Page is calculated: Title—Composer)

PiecePerformances: Page, Piece (page), Concert (page), Performers (list of page) (where Page is called: Piece performed at Concert)

Concerts: Page, SeriesTitle (page), Title, ConcertDate, ConcertTime, etc.(where Page is calculated: SeriesTitle—Title—ConcertDate)

A concert page displays all the PiecePerformances for that concert.

Here are the two things I haven’t figured out how to do:

1. A pieceperformance page should display the performers along with the instrument(s) that each performer plays, something like this:

performer1 (instrument1) - performer2 (instrument2) -. ..

2. I would like the data entry form for a piece performance to take the entered name of the piece and then get and display for that piece each instrument, with a box for entering the corresponding performer. That would seem to be a two-step process.--Egnatoff (talk) 03:24, 30 April 2017 (UTC)


 * If I understand the problem, you want to define some piece as a duet for flute and bassoon, and then, every time someone fills out a form for some performance of that piece, you want to have pre-set fields for "Flute:" and "Bassoon:". If that's correct, unfortunately there's no easy way to do that. The only approach I can think of involves the Scribunto extension, and dynamically-generated forms. I would think it's better, though, to just have editors enter the instruments and musicians every time themselves (using a multiple-instance template). It's more work for editors, but then again it's a more foolproof system. What if someone decides to perform that piece on glockenspiel instead? It's still the same piece, isn't it? Yaron Koren (talk) 14:28, 30 April 2017 (UTC)
 * That's certainly a more direct solution to the main problem, and would accommodate cases where instruments are changed, and whether or not there is a conductor. Point taken, thanks.--Egnatoff (talk) 18:48, 30 April 2017 (UTC)

Hidden parameter
The documentation at Extension:Cargo/Storing_data states, in relation to the "hidden" parameter:

"hidden - takes no value. If set, the field is not listed in either Special:ViewTable or Special:Drilldown, although it is still queriable."

I added "(hidden)" to some fields in an existing table and recreated the data. This prevented those fields from appearing in Special:Drilldown, but they still appear in Special:CargoTables. To be honest, this is what I would like to happen, but it does not match the documentation. Maybe this is just something which applies only when an existing table is recreated. Incidentally, I think "Special:ViewTable" should be changed to "Special:CargoTables". Jonathan3 (talk) 23:05, 1 May 2017 (UTC)


 * Those were indeed some bugs in the documentation. I think I just fixed them. Thanks! Yaron Koren (talk) 02:06, 2 May 2017 (UTC)

Duplicates Every Time
I updated to the latest version via git this morning to get rid of an error with getNamespace, and now every thing seems to make two entries.

Templates defined on the page multiple times, as well as templates that are just defined once per page. Example

Clearing the table and then re-populating does seem to clear up the issue.--Cody3647 (talk) 20:41, 2 May 2017 (UTC)


 * I don't know what's causing this; it could be some difference in storage between your old version and the current one. I would suggest just recreating all the tables; if that doesn't fully solve the issue, let me know. Yaron Koren (talk) 20:57, 2 May 2017 (UTC)


 * I have a similar problem using Cargo to store external data. Any new entry added to the database is fine and gets stored once. As soon as I re-save an existing data record by editing the page, the database starts to hold multiple copies of the entry (sometimes 2, sometimes 3). As if the old saved versions are not deleted. Using 'recreate data' through a browser cleans up the multiple entries of any records and it's fine again. However, I don't want to 'recreate data' every time a entry get's modified. Any idea what might go wrong? Used PageForm to get external data: http://csdms.colorado.edu/wiki/Form:Reference-auto1. --Albert Ke (talk) 16:48, 8 May 2017 (UTC)
 * (Cargo version: latest (installed May 1st); MW 1.28.1; PHP 5.6.30(fpm-fcgi); mysql 5.6.23-log)


 * Does this happen for every form, or just this one? And does it happen on every page save? Yaron Koren (talk) 19:54, 8 May 2017 (UTC)


 * I only made 1 pf using cargo, so not sure. Will do a test on that. Happens for every entry after I re-save that entry. --Albert Ke (talk) 21:42, 8 May 2017 (UTC)


 * Sorry this took a while. I created a very basic Page Form example using cargo to define & store just 1 field and I'm having the same issue: I can create a new entry for the database and it will be stored once. However, editing this page again using 'Edit with form' and this entry results in 3 data rows to the same page. I did test to see if it matters using instead of 'Edit with form' the wiki 'edit', and it does, now cargo will keep it to 1 data row. The used form: http://csdms.colorado.edu/wiki/Form:TestCargo The used template to define, store and show cargo data: http://csdms.colorado.edu/wiki/Template:HoldName. Any idea what I'm doing wrong? --Albert Ke (talk) 22:18, 12 May 2017 (UTC)
 * SOLVED (at least for me). Thank you Yaron for solving this problem. I had included a 'namespace' and 'query' statement in the '#formlink' definition and both shouldn't be there. Now data just get stored once, also after editing using the 'Edit with form' option. Many thanks Yaron!!! --Albert Ke (talk) 15:07, 14 May 2017 (UTC)
 * It's great that that fixed it! Although my guess is that it was actually the removal of both namespace settings in the form's page name formula, in this edit, that solved the problem. That's just a hunch, though. It's worth looking into this further, at some point. Yaron Koren (talk) 01:01, 15 May 2017 (UTC)

I have just installed Cargo and Page Forms on a new offline wiki and seem to be getting random duplicate table rows. This is happening across two of the three Cargo tables I've defined so far. The first time I create the Cargo table from the template page every record has a duplicate. I have then tried deleting the table and creating it again. Subsequent table creations result in no duplicates (for a while), then randomly a duplicate will occur. Any suggestions on how to troubleshoot? --Nha4601 (talk) 00:20, 8 June 2017 (UTC)


 * I don't know. I wish I knew why it was happening - or why it seems to happen a lot more for some people than others. If anyone having this problem could give me access to their system, it might help a lot. Yaron Koren (talk) 15:08, 8 June 2017 (UTC)


 * I also have the problem… When I'm almost done with modifications, I launch the recreate data script and modify ASAP the indexes to the table so that the _pageID is unique. I haven't tested yet whether it writes all recordings or misses some, but that's the only way I found (and it works only because there is one and only one template call in each page). But I'm thinking that if the bug is hard to correct, letting the user manually specify a "unique" attribute to the appropriate columns could be a failsafe (the question remains about the update of entries, if I have time to check that aspect, I'll post about it). --Loizbec (talk) 14:54, 19 June 2017 (UTC)

Importing data (e.g. from Access)
I have been sent a database which would be too big to enter into Cargo manually. Is there any way to import automatically? I guess I would need an extension/bot to create the necessary wiki pages. Jonathan3 (talk) 22:30, 3 May 2017 (UTC)


 * Yes - use the Data Transfer extension. Yaron Koren (talk) 00:49, 4 May 2017 (UTC)

Storage of lists: how to store and query for the order of values
I later need to sort and query by the order in which the template instances are appearing in a wiki page. In other words, the order of appearance represents the priority, and it is easy to implement drag-and-drop with Extension:Page_Forms.

Now I wonder how to get the ordering information into Cargo's database for later query.


 * The templates are hopefully stored in the database in the same order in which they appear on the page, so you may be able to accomplish this by just sorting on the "_ID" field. If that doesn't work, you may need to install the NumerAlpha extension, and add a new field to that multiple-instance template, setting the field to the value of #counter within #cargo_store. (Let me know if that doesn't make sense.) Yaron Koren (talk) 01:33, 8 May 2017 (UTC)

"Illegal mix of collations" error [RESOLVED]
Here is an error from Cargo:

A database error has occurred. Did you forget to run maintenance/update.php after upgrading? See: https://www.mediawiki.org/wiki/Manual:Upgrading#Run_the_update_script Query: SELECT `Description` AS `Description` FROM `wm_cargo__Event` WHERE _pageName="‎ " ORDER BY `Description` LIMIT 100 Function: CargoSQLQuery::run Error: 1267 Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation '=' ()

Here's what happened next:
 * I changed the title of the event page, to something purely alphabetical, which made the error go away.
 * I then changed it back to the original title, and the error came back.
 * Finally, I changed it to a slightly different version of the original (just by changing a small c to a capital C) and the error went away again.
 * Even now, having deleted the page with the original title, a Cargo query on the original title returns the same error, which I guess this because MediaWiki has saved the page title as a deleted page.

I can't think what caused this problem, which has not happened before. I wonder whether it may be has to do with the two changes necessary to the the auto complete work (which you suggested above). In that case I suppose the answer would be to update Page Forms properly.

My Cargo database tables are all latin1_swedish_ci but my MediaWiki database tables have ended up being a mixture of latin1_swedish_ci and utf8_general_ci. I asked about this here before and the answer was, sensibly enough, along the lines of "if it ain't broke don't fix it"... But now I wonder whether I should perhaps look into this properly.

Do you think this might be a Cargo thing, or a particular problem with my installation? Thanks in advance. Jonathan3 (talk)


 * It's not surprising that the query fails regardless of whether the page exists or not - this query is just accessing a single Cargo table, and doesn't look at the core MediaWiki data at all. So everything has to do with the page name, which I don't know. Though it could well be that replacing the double quotes in the query's "where" clause with single quotes would fix the issue also - I would definitely try that. Yaron Koren (talk) 14:30, 9 May 2017 (UTC)


 * The name was alphabetic except for a colon and a hyphen (never caused a problem before and other pages are similar without errors). When I do the same query with a page which never existed it says No results but gives no error. I'll try to look into the problem today or tomorrow. Jonathan3 (talk) 17:36, 9 May 2017 (UTC)


 * I am completely stumped! I can't do anything to make the error appear today. Jonathan3 (talk) 12:50, 10 May 2017 (UTC)


 * I think have found the source of the error :-) When I cut and paste the page title from the page itself, or from Special:Recentchanges the query works fine. The error only appears when I cut and paste the page title from Special:Newpages - when I put a page title from this source into an online Charset Decoder it looks fine as UTF-8 but there is an extra â€Ž before the page title in ANSI. Jonathan3 (talk) 01:03, 23 May 2017 (UTC)


 * Ah, an invisible character... very annoying, and very hard to debug. I'm glad you were able to find this one. Yaron Koren (talk) 02:06, 23 May 2017 (UTC)


 * Cheers. It happens every time a page name is cut and pasted from Newpages, including from Wikipedia itself. Jonathan3 (talk) 18:40, 23 May 2017 (UTC)

Wikitable syntax exported from Page Schemas generated templates
I'm using Page Schemas to setup the schema for a new site and generate the starter templates. I noticed that when using the option "display if not empty" the resulting wikitable syntax in the form doesn't work quite right. The syntax itself seems to be correct but due to some odd line breaks it's not showing the data in each cell and is displaying a "-" instead and not breaking into rows. A partial example of code is below

Removing the extra space from the end of the If statement and making a new row with the next one seems to solve the issue.

A second issue is that the option for "Hide" is not actually hiding the data in the resulting wikitable. The label is hidden but the value is displayed in the previous cell.

add map type variable for googlemaps
Is there a way to add the maptype variable or other common variables that manage display types for Google Maps such as whether to show the zoom controls, map switcher, etc? I want to set the default map to Terrain type. In the SMW Maps extension I can add type=terrain, but that doesn't seem to work under Cargo.


 * Yes, I haven't added many parameters for the "googlemaps" format because I wasn't sure which ones people actually use. The SMW/Maps equivalent has about 50 parameters - which of these would you want? Yaron Koren (talk) 13:35, 4 June 2017 (UTC)


 * I'd be interested in experimenting with the resizable and enablefullscreen parameters. Jonathan3 (talk) 07:55, 8 July 2017 (UTC)


 * What about zoom controls, etc.? Yaron Koren (talk) 13:42, 9 July 2017 (UTC)


 * They're there by default and I can't see why anyone wouldn't want them :-) Jonathan3 (talk) 23:17, 9 July 2017 (UTC)


 * P.S. What would be helpful, but I don't think is even one of the options for (Semantic) Maps, would be to have numerals on the markers which could correlate to a list of locations displayed beneath the map. Sometimes you find this on "store locator" websites etc. Jonathan3 (talk) 23:20, 9 July 2017 (UTC)


 * What was the original question about, then? And for the last thing - there has long been a desire for some kind of interface that makes it easier to find a map location given its name - when I was still working on Semantic Maps, we wanted to add something like that too. I agree that it's a great idea. Ideally it could also involve highlighting or otherwise isolating a point on the map when its name is hovered over, or clicked, in the list. Yaron Koren (talk) 14:42, 10 July 2017 (UTC)


 * The original question wasn't mine :-) It sounds like he is interested in using map type etc and I'm interested in trying out the full screen/resize parameters. It would be fantastic if that multiple-marker functionality were to be added. Jonathan3 (talk) 18:03, 10 July 2017 (UTC)


 * Ah! That explains it. Yaron Koren (talk) 19:04, 10 July 2017 (UTC)


 * I've submitted a patch to Phabricator about a fullscreen parameter, as I would find this useful. I would be grateful if you could consider it. Jonathan3 (talk) 23:12, 10 July 2017 (UTC)

Do you definitely want this feature, or do you just want to experiment with it? And if you do want it, can you explain how it's useful? Yaron Koren (talk) 02:57, 11 July 2017 (UTC)


 * Yes I'd definitely like it (and have experimented with it now). When you have lots of markers it looks great to view on full screen. Jonathan3 (talk) 09:22, 11 July 2017 (UTC)


 * Can't you just set "width=100%"? I thought the big advantage of "fullscreen" is that it lets users toggle between the two modes, but I don't know how useful that is. Yaron Koren (talk) 11:56, 11 July 2017 (UTC)


 * Width=100% would make it the width of the current div. Yes, with full screen there's a button to press to enable it and you press escape or click a button or click on a marker's wiki link to exit it. I just think it looks great which I hope does make it "useful" :-) Jonathan3 (talk) 13:48, 11 July 2017 (UTC)


 * Interesting. What about enabling it all the time, then? Yaron Koren (talk) 14:05, 11 July 2017 (UTC)


 * That would suit me fine. One line of code would do it. It's quite an unobtrusive icon (top right of the map) so I don't think anyone would mind. Jonathan3 (talk) 14:24, 11 July 2017 (UTC)


 * PS I wonder whether it would answer the first question to have a general parameter for the googlemaps format which takes comma separated values within round brackets, which could contain any of the standard Google maps controls - so it would be possible to disable or enable anything, and you'd only need to write code to cover one extra parameter? Jonathan3 (talk) 14:30, 11 July 2017 (UTC)


 * That's an option, and I've thought about having something like that, although a general "other options" parameter may be too much of a hack. Another possibility is to have a "controls" parameter which lets people specify which controls they want. SMW has such a thing, although for some reason "fullscreen" isn't one of the allowed values - it gets a separate parameter. But "controls" would enable a lot of functionality without cluttering things too much. What do you think? Yaron Koren (talk) 03:53, 12 July 2017 (UTC)


 * Sounds like a plan. That would answer part of the original question and would suit me. Of the various controls, I'd probably enable "scale" too, as well as full screen. Jonathan3 (talk) 22:09, 12 July 2017 (UTC)


 * Alright, cool. I think it makes sense to enable all of them, if that parameter is getting added. Yaron Koren (talk) 15:44, 13 July 2017 (UTC)


 * I agree they all should be enabled in that sense (I meant I'd choose to enable scale etc in my maps) Jonathan3 (talk) 17:21, 13 July 2017 (UTC)


 * Oh... now I get it. Yaron Koren (talk) 18:44, 13 July 2017 (UTC)

External data not always saved
Hi Yaron,

Looks like external retrieved data not always gets stored by Cargo. I'm getting external reference data through rest API from [1] : (using the following template: [2]). Parts of data gets stored using cargo in [3]. Below 2 examples: a) data get stored; b) data doesn't get stored:

GETS STORED: a) http://csdms.colorado.edu/wiki/Reference:Reference-000779 Data is retrieved from [1] using DOI (so [5]) and displayed on the page and stored in Cargo [4] (close to the bottom of the table).

DOESN'T GET STORED b) http://csdms.colorado.edu/wiki/Reference:Reference-000780 Data is retrieved from [1] using DOI (so [6]) and displayed on page but not stored in Cargo [4] (Reference:Reference-000780 is missing from the table)

Turns out that data from a specific publisher (Wiley) doesn't get stored by Cargo, although I couldn't find any difference in the external data json file. Any idea why some data from 1 publisher doesn't get stored?

Thanks, and sorry for the many web links, --Albert Ke (talk) 16:44, 5 June 2017 (UTC)

[1]: https://en.wikipedia.org/api/rest_v1/data/citation/zotero/ [2]: http://csdms.colorado.edu/mediawiki/index.php?title=Template:CSDMS_reference_template_cargo&action=edit [3]: http://csdms.colorado.edu/wiki/Template:RefsExt2 [4]: http://csdms.colorado.edu/mediawiki/index.php?title=Special:ViewData&limit=500&offset=0&tables=RefsExt2&fields=_pageName%3DPage%2CDOIentry%3DDOIentry%2CItemType%3DItemType%2CTitle%3DTitle%2CPublicationTitle%3DPublicationTitle%2CBooktitle%3DBooktitle%2CJournalAbbreviation%3DJournalAbbreviation%2CVolume%3DVolume%2CIssue%3DIssue%2CUrl%3DUrl%2CDOI%3DDOI%2CPagenumbers%3DPagenumbers%2CPublicationdate%3DPublicationdate%2CAccessDate%3DAccessDate&order_by=%60_pageName%60 [5] https://en.wikipedia.org/api/rest_v1/data/citation/zotero/10.1080%2F02626667.2017.1307571 [6] https://en.wikipedia.org/api/rest_v1/data/citation/zotero/10.1002%2F2016GL069262


 * Is it definitely all of the Wiley pages, and only the Wiley pages, whose data is not stored correctly? Yaron Koren (talk) 17:52, 5 June 2017 (UTC)
 * Yes, all the Wiley published papers I have tried so far (~100 references) don't get stored. Unsure why as their json file from [1] is identical with other journals references ([5] vs [6]). Total mystery to me. --Albert Ke (talk) 18:15, 5 June 2017 (UTC)
 * Yaron, I should mention that references from Wiley publisher were for some time not available through [1]; see also: https://phabricator.wikimedia.org/T165105. It seemed like [1] was blocked by the publisher although this changed recently, as you see by e.g. [6]. Could this be related, although external data is able to get data? --Albert Ke (talk) 18:33, 5 June 2017 (UTC)
 * That's odd indeed. The previous Wiley issue doesn't seem connected, but who knows. Looking through the templates, I notice some settings that should probably be changed in the Cargo declarations... again, I don't know if any of these are connected, but they seem like a good idea to change regardless. Here, I think that "Firstname" and "Lastname" should just be a String, not a list of Strings; and here, I think that "Url" should be of type "URL", and "Publicationdate" should be of type "Date". I would recommend changing them all - it wouldn't hurt, and who knows, maybe it'll fix this issue. Yaron Koren (talk) 18:48, 5 June 2017 (UTC)
 * Thank you Yaron. I made the 4 suggested changes ("Firstname", "Lastname", "Url" and "Publicationdate"), updated the cargo tables (php ./cargoRecreateData.php) but no luck; Wiley references won't show up unfortunately.... --Albert Ke (talk) 21:11, 5 June 2017 (UTC)
 * Alright - well, that was probably an improvement regardless. I tried to replicate this myself, and saw the same problem, and I may have found the issue: I just checked in a small change to External Data that seems to solve the problem, at least for me, though I'm not sure why. But please get the latest ED code, and let me know if that fixes anything. Yaron Koren (talk) 17:31, 6 June 2017 (UTC)
 * The issue is fixed but not sure if your change was part of it. Actually, now that I'm think about it, I cloned the latest Cargo, not ED (will still do tomorrow). The problem was the following; references from Wiley publisher were not available for some time. So when pages tried to collect data an error message would show. This was fixed recently by citoid. I repopulated the Wiley pages initially using cargoRecreateData.php, (so not editing each reference that contained data from Wiley). Individual pages were again showing external Wiley reference data (so error message was gone) but this data wasn't actually saved in Cargo. To get data in Cargo tables, each page that contains Wiley reference data needed to be edit for that. This is now done and all looks good after running cargoRecreateData.php. It does bring back an older problem of when saving external data page after it's first creation, it will save it multiple times. See for example Cargo table: http://csdms.colorado.edu/wiki/Special:CargoTables/RefsExt2 and than search for "Reference:Reference-000168". It is saved 3 times. You have to run cargoRecreateData.php to remove the 2 extra edits. --Albert Ke (talk) 01:43, 7 June 2017 (UTC)

order by integer…
Hi,

I have carefully created a cargo table with integer columns in order to be the sort key later on, without performing any test… And now my table is rich with over 100 entries I realize that cargo sorts integers as strings…

Below is

the SQL query to confirm that the problem is in cargo
SELECT * FROM `wiki_cargo__sons_LUCIOLE` WHERE 1 ORDER BY mission, etape ASC, fonction DESC

the SQL query output


I'm not sure there's anything to do, but I'm welcoming any advice to sort this out ;) --Loizbec (talk) 12:43, 19 June 2017 (UTC)


 * Yes, that's an issue with the "dynamic table" format - it does its own custom sorting, and doesn't use the sorting from the database, because of the way the JavaScript is written. I'll have to look into that. Yaron Koren (talk) 15:31, 19 June 2017 (UTC)


 * Sorry, I'm only looking at this now, and now I realize that I don't know what the problem is. What should the order of those three rows be? Yaron Koren (talk) 03:49, 17 July 2017 (UTC)


 * Sorry I did not see your reply. To me the expected output is the same as in the SQL query “18,19,20” instead of “18,19,2”… --Loizbec (talk) 15:35, 29 August 2017 (UTC)


 * I can't replicate this problem, for what it's worth - ordering works correctly for me. In any case, this particular ordering is done by a 3rd party JavaScript library (DataTables), so I don't know if I could do anything to fix the problem, if there actually is one. Yaron Koren (talk) 03:49, 5 September 2017 (UTC)

Maybe a problem with HOLDS
I have a query with

In the output, the "About" field only has "Example" listed as values, which is wrong as I know that "Example" is only one of the values in each list.

What do you think is up? Thanks Jonathan3 (talk) 22:20, 3 July 2017 (UTC)


 * A field that holds a list of values, like "About" in your case, is a "virtual field" - there's no DB field directly called "About", but rather two different fields that together represent its contents. The code has to figure out which one to use in any circumstances; in this case, it's clearly making the wrong decision. You can get around this bug by replacing "About" in the "fields" parameter with "About__full", or "About__full=About", etc. Yaron Koren (talk) 00:32, 6 July 2017 (UTC)


 * Using  worked fine, thanks. Jonathan3 (talk) 19:29, 6 July 2017 (UTC)

Maybe a problem with List of Integer
When I declare a table with  only the first item in the list is displayed on Special:CargoTables. The $fieldValue variable at that stage of things is the full list delimited by spaces (e.g. "123 1234 12 2 3" (rather than one of the list items) so I guess the list was stored to the database with spaces instead of commas. What can I do to fix this? Thanks. Jonathan3 (talk) 22:56, 5 July 2017 (UTC)


 * Try using a delimiter other than a comma, like maybe ";". The commas may be confusing the code, which thinks that they are thousands separators. Perhaps commas (and periods) should simply not be allowed as delimiters for integers and floats... Yaron Koren (talk) 00:34, 6 July 2017 (UTC)


 * Good thinking. Changing to  fixed the problem. The normal use of commas as thousand separators works fine too. I guess you're right that commas shouldn't be allowed as delimiters - or save the numbers without thousand separators and only recreate them when displaying the numbers. Can't see any easy way round the problem that full stops as delimiters would cause. Jonathan3 (talk) 19:39, 6 July 2017 (UTC)

Maybe a problem with List of Integer on Special:Drilldown page
I get the following error when I click on any value relating to a List of Integer:

I was just creating the List of Integer field to test something else, so this isn't a real problem for me, just something I wanted to point out in case there is something wrong. Jonathan3 (talk) 19:52, 6 July 2017 (UTC)

Correct way to alter structure of existing table
I always seem to have difficulties with this. What exactly is the correct procedure? From memory, I think this should work, all from the web interface:


 * 1) Edit table definition (and the query if necessary) in template page
 * 2) Change form page if necessary.
 * 3) Delete existing table
 * 4) Create new table
 * 5) Recreate data

Is there anything else?

Does it matter at what stage the existing table is deleted?

Are there any points at which it is necessary to use runJobs.php? Thank you. Jonathan3 (talk) 19:20, 7 July 2017 (UTC)


 * Well, steps 3-5 are all just one action, from the web interface... but you probably knew that. It's not necessary to run runJobs.php, but it would probably make the new table get filled up more quickly. Yaron Koren (talk) 21:07, 7 July 2017 (UTC)


 * Thanks. Yes I think recreate data drops and recreates the table. I've had such trouble with it all (no doubt something I don't understand about how my databases are set up) that I wondered if there was a belt-and-braces approach! Jonathan3 (talk) 07:46, 8 July 2017 (UTC)


 * What kind of trouble have you had? Yaron Koren (talk) 13:42, 9 July 2017 (UTC)


 * I've tried to describe it here before (unsuccessfully) but will try again at some point in future if it crops up again. Jonathan3 (talk) 23:16, 9 July 2017 (UTC)

I'm also having the same issue, I've added a column to an existing table in the template, deleted the original cargo table, Clicked on recreate data and the new table is empty. Am I missing something? Cheers.


 * Have you tried using runJobs.php? Assuming the new table has the correct columns. Jonathan3 (talk) 14:49, 18 July 2017 (UTC)

Cargo and Elastic Search
Yaron, We have set up our wiki and are running 1.27.3, for the long term support, we are using Cargo ver 1.3 for all of our data. Now we want to set up search and have installed elastic search with Cirrus search extension. Our question is how do we get elastic to include the data in our Cargo tables? We have been told by two different people that elastic and Cargo cannot work together see link: https://www.mediawiki.org/wiki/Help_talk:CirrusSearch, we thought we would contact you to see if that is indeed true, that we cannot use Cargo and Elastic search. If it is true, do you know of any other search option that will work with the Cargo extensions.


 * They're right, unfortunately - there's no integration of the two extensions at the moment. What you can do instead is have Cargo store the full text of every page - see here for how to do that. Then, within Special:Drilldown, you'll see a text search input for every table; and you can also run queries that include a full-text search, using MATCHES. Yaron Koren (talk) 19:55, 19 July 2017 (UTC)

Treatment of blank date fields
When I don't enter a date for a date field it defaults to the current date when it is displayed in the template page, but there is blank when it is displayed in a dynamic table query. This is only a minor issue but something I thought I would mention for you. I'm not really sure whether it's a Page Forms or Cargo thing. Jonathan3 (talk) 00:08, 5 August 2017 (UTC)


 * I'm not sure it's either. What's the template page? Yaron Koren (talk) 01:38, 7 August 2017 (UTC)


 * You're right. It's neither. The template uses  and   defaults to today's date (e.g. now it displays 7/8/17). I guess I need to check for an empty date field, or just not leave it blank... Jonathan3 (talk) 20:00, 7 August 2017 (UTC)

Cargo Boolean should be 0/1 or yes/no by definition - how is this done, if database column type created is boolen?
I am not able to figure out the problem with Cargo fields of type Boolean. So if I declare field of type Boolean, when data is recreated, a column of type Boolean is created in the database table (I use PostgreSQL). How does this correlate to Cargo/Storing data definition: "Boolean - holds a Boolean value, whose value should be 1 or 0, or 'yes' or 'no'"? When defining a form, I declare a field by using cargo field= and form correctly displays checkbox for Boolean field. When data is stored, the Boolean value in the database is also correctly stored as true or false. However, I have issues when viewing or drilling down the table via Special pages:Cargo tables options. The table shows only Yes values for all Boolean fields and similar issue occurs with Drill-down - only None and Yes counts, no No counts. How to fix this? Blazm (talk) 19:00, 24 August 2017 (UTC)


 * That's odd. What versions of Cargo and MediaWiki are you using? Yaron Koren (talk) 20:41, 24 August 2017 (UTC)


 * Here is info about my system and samples of code for template and form:

Software Versions:
 * OS: Alpine Server Linux
 * Mediawiki: 1.29.0
 * PHP: 7.1.7
 * PostgreSQL: 9.6.4
 * SMW: 2.5.4
 * Page Forms: 4.1 (bb3328e)
 * Cargo: 1.3.1

Template:

Form:

In the Postgre SQL a table cargo_itequip is created which among other contains columns ipv4 of type character varying(300) and dhcp of type boolean.

If I check the Cargo table, I see: NOTE: DHCP for the SERVER in the database is set to FALSE

If I check Drilldown: Blazm (talk) 7:04, 25 August 2017 (UTC)
 * ipv4:
 * None(x) * 192.168.1.254(1)
 * dhcp:
 * None(x) * Yes(9)


 * What do you mean by "check the Cargo table"? And what is the value of that field in the "SERVER" page's wikitext? Yaron Koren (talk) 13:55, 25 August 2017 (UTC)


 * "check the Cargo table" > Special pages;Cargo;Cargo tables and then use either View or Drilldown. "SERVER" page will show the correct information - DHCP: No. Blazm (talk) 19:40, 25 August 2017 (UTC)


 * Alright, thanks. So Cargo is able to store boolean values correctly, but not read them again. I'm guessing that this is a PostgreSQL issue, but I don't know how exactly - and unfortunately there's no easy way for me to test directly with PostgreSQL. I do have a theory, though. In the drilldown interface, do those 9 "Yes" values represent 9 pages with "Yes", or 9 pages with "No", or a combination? Yaron Koren (talk) 11:54, 27 August 2017 (UTC)

Template/doc example stored in Cargo table, and other questions
I'm a new admin at the Pillars of Eternity Wiki, a reasonably big gaming wiki. It's running SMW, but recently we've installed Cargo to see about migrating. For now I've made a template using Cargo, which has its data here. If you look at that page, you'll see that there are two rows of a creature called "Example", which is from the Template page, and Template/doc page. Naturally I don't want these stored in the table, but I'd also like to have a proper example in the documentation, so people can see what it looks like. I hope more people will contribute because of it.

Firstly, how can I solve this, while keeping the the example in the documentation, but preferably without having to use e.g.: , which I think also means I need to use | ahead of every line in the #cargo_store.

Secondly, since I assume this is practice in many wikis, would it be possible to alter the extension so that it doesn't store data from the Template namespace unless we specifically ask it to? Unless it's possible to avoid this behaviour with a parameter I don't know about ofc.

Then I have a few more questions, related more to formatting.


 * 1) I've made some examples of output format, and like the table format best, as it fits design on the site, and it's easy to read. However, is it possible to make these tables sortable? I don't like the extra fields in the dynamic table output format, like search and page numbers, and it also appears to be on the slow side from what I've seen so far.
 * 2) Is it possible to put more than one variable in one cell in such a table? For instance, it would be preferable to be able to put all 6 attributes in one cell, to save width space.
 * 3) Three of the variables are of the   type. This seems to work well, I can click on the page in Drilldown. But in the tables I just get e.g. ". . . . . " (see here for instance). What must I do to get these values to show up in tables? Do I need a sub-query for instance? Could you show me an example of how to do this?

Thank you for an excellent extension. I hope it's possible to migrate the whole Wiki to it, or at least more parts in due time. I may have more questions as I test and learn more about it :) Pangaearocks (talk) 17:44, 29 August 2017 (UTC)


 * Hi - I haven't been asked about disabling #cargo_store before, but it certainly makes sense. I'll have to think about the best way to implement that in the code; for now, the #ifeq approach you suggested seems like the best bet. I don't think you would need to replace pipes in the call.
 * 1. It's not possible to make the regular table format sortable, though that does make sense as a feature, especially for larger tables - like more than 20 rows.
 * 2. I think you could put multiple fields together by having one field be a "CONCAT" call, like "CONCAT(FieldA, ', ', FieldB, ', ', ...)=MyCompoundField".
 * 3. I think the issue with those list fields is that the values are already in bracketed links, and Cargo expects the page names to just be strings. So you should either remove those square brackets, or change the type from Page to String (or maybe even Wikitext).
 * I'm glad you like Cargo so far! Yaron Koren (talk) 20:33, 29 August 2017 (UTC)


 * In relation to question 1, it's not very neat but you can use the "template" output format to make sortable tables.




 * And in the template page:




 * You could modify this to answer question 2 also.


 * For question 3, I'll guess and say I think the best answer is just to remove the and , as that should allow the Page type to work properly - and stop the  and  from appearing in the Drilldown page too. If you use String any  and  would show as plain text in queries too, I think. Jonathan3 (talk) 22:27, 29 August 2017 (UTC)


 * Thanks a lot for the swift answers. Then I hope this will be possible in future versions of Cargo :) I'd love a sortable table output format, because we'll have at least 100 rows on multiple CargoTables (including the bestiary, which I'm currently working on).


 * Haven't had time to test the code you replied with yet, I broke my leg yesterday so things haven't exactly gone quite to plan. But I will later. However, I have another query, about Redirects. Some of the creature names in Pillars of Eternity have funky names, and one of them is the Mênpŵgra. Naturally that required a redirect from the 'clean' name. However, this means that I have two versions of Mênpŵgra in the CargoTable (from here and here). How can I best solve this?


 * As for the and , I use those because I want the page links in each creature page, in the infobox, so people can quickly go to them. But maybe List (*) of Wikitext would work better? (instead of Page). The asterisk is so that the wiki automatically turns them into bulletpoints (ul). See an example here for instance. Pangaearocks (talk) 20:08, 30 August 2017 (UTC)


 * I could do with sortable tables as an option too, but until Cargo changes the code above produces the same result.


 * Instead of a redirect how about the DISPLAYTITLE magic word? Though I think that only has an effect when viewing the page itself (not in the Cargo query). Or find/write an extension to replace every instance of the plain word with the fancy version.


 * The Page type will create hyperlinks so there's no need to use square brackets.


 * Hope your leg ends up all right too... Jonathan3 (talk) 06:49, 31 August 2017 (UTC)


 * Pangaearocks - I'm sorry too to hear about your leg! I think there's one error in what Jonathan3 wrote, which is that, as you note, links indeed do need to be added manually for display within the page itself. I would recommend using #arraymap (defined in the Page Forms extension) for that - it lets you automatically add links around every value within the field. In fact, you can use #arraymap to add the bullet points as well - so that the template field just looks like "MyField=A, B, C", while it gets displayed as a set of bulleted links. I urge you to check out #arraymap if you haven't heard of it. Yaron Koren (talk) 17:18, 31 August 2017 (UTC)


 * Thanks for explaining that about the Page type. I hope this is relevant enough to belong here - could you set out the differences between String and Page types? Jonathan3 (talk) 00:44, 1 September 2017 (UTC)


 * The difference comes when they're displayed in a query - Page values get linked. Yaron Koren (talk) 01:40, 1 September 2017 (UTC)

Thanks for the help so far, and the well-wishes :) After some trial and error, I managed to convert the template and use #arraymap to create the bulletlists in the pages. Ended up using, which worked quite well.

Now I wonder more about formatting again, and how best to approach it. It looks a bit tricky to format each column in the table generated by cargo_query the way I'd like, and make the table itself sortable, so would it therefore be better to make a template with a table, and let output format from the queries be template? Do you have an example of how this might be done, besides what Jonathan3 wrote above? Unfortunately I'm not all that savvy with code, although I'm trying to learn. Would I be able to use normal template syntax, like to format the table, or am I mixing up cargo syntax with normal wiki syntax too much? Here is the current test page: https://pillarsofeternity.gamepedia.com/User:Pangaearocks/Antelopes

Btw, I couldn't get HOLDS LIKE to work. A long SQL error message was thrown back about syntax (can it only be used for lists?). But LIKE worked, which is what I'm currently using. Pangaearocks (talk) 02:27, 3 September 2017 (UTC)


 * A follow-up to this, as I sort-of got it working through Jonathan3's example. As he said, it's not pretty, but it works. One of the motivations of potentially changing to Cargo, however, was to easier fetch this data and format it. This way isn't dynamic, and with data split between template and page. It would have been far preferable if we could do something similar, but get variables AND values from cargo, then format it in a table, and send it back. That way, we might be able to make one 'formatting' template, which then adapted to amount of columns. I hope something akin to this will be possible in the future. Or better yet, making the table display format sortable, and making it easier to customise formatting of it (just making it sortable would be a great help, though). For now, the splitting up approach is a decent enough workaround, albeit not ideal. Pangaearocks (talk) 23:33, 3 September 2017 (UTC)


 * Yes, "HOLDS LIKE" is only for list fields. And yes, adding a "sortable" parameter to the "table" format seems like a good idea. Yaron Koren (talk) 01:22, 4 September 2017 (UTC)

Three database glitches (duplicate rows, rows not being added, table giving errors)
First off, I absolutely love Cargo -- thanks so much for your hard work on it.

I am, however, experiencing some problems with it on a wiki that I'm trying to set up. The three problems are:


 * One of my tables is returning errors when I try to view it via Special:CargoTables. You can see this here: This happens no matter how many times I delete/recreate the table. The template that creates this table is located at: . However, this problem is mostly unrelated to the following two (more troublesome) problems:
 * I am also having the "duplicate rows" problem that others have reported. It is particularly noticeable in the Trophies table, which is created by the Trophy template. (I will explain how to reproducably make a duplicate row shortly)
 * I have made a bot that creates some pages on the wiki automatically. The first time those pages are created, their entries are (correctly) added to the Trophies table (linked above). However, if I then "Recreate Data", those rows will *not* be added back into the table until a human edits them.

For example, you can test this weird behaviour as follows: right now, the Trophies table is missing many rows (from pages created by my bot). If you do a null edit on this page, it will be added to the Trophies table (likely twice). If you then "Recreate Data" for the Trophies table, that double row will be corrected, but it will *remove* numerous other rows (e.g., the row with _pageID equal to Game/dark-souls/trophy/1 will be removed).

Thanks for your help, and let me know if there's any other information that I can provide that would help. NathanielJ (talk) 18:59, 1 September 2017 (UTC)


 * For the first issue - if you add "$wgShowExceptionDetails = true;" to LocalSettings.php, it should display a more helpful error message. For the issue of pages getting ignored during "Recreate data" - do you have a job run rate higher than 1? If so, that might be the problem. In either case, running cargoRecreateData.php from the command line might fix it. Yaron Koren (talk) 22:16, 1 September 2017 (UTC)


 * Thanks for your help Yaron. I set $wgShowExceptionDetails = true; so that a more detailed error message appears here. Unfortunately, I do not understand what it means though.


 * For the recreate data issue, the job run rate is set at exactly 1. I ran cargoRecreateData.php, but that didn't help -- the Trophies table only has data for the 4 pages that use that template that have been edited by a human, but not the dozens of pages that use that template that were created by my bot. NathanielJ (talk) 22:35, 1 September 2017 (UTC)


 * As a follow-up to the previous message, I ran the maintenance script refreshLinks.php and then ran cargoRecreateData.php again, and that fixed the Trophies table. NathanielJ (talk) 22:53, 1 September 2017 (UTC)


 * Ah - that error message is very helpful, although it doesn't look like it. That was a bug with Cargo's handling of fields whose name contains a number, which was fixed on May 17, soon after version 1.3.1 came out. I would recommend upgrading to the very latest Cargo code, if you have Git on your system. If not, I'm actually planning to release the next version of Cargo, 1.4, in the next week (now that the hierarchy feature has been added), so you can just upgrade then.
 * As for your call to refreshLinks.php - that's very good to know too. I had just assumed that it was a coincidence that the bot-created pages were not being handled, but now it makes sense: the bot-created pages did not have an entry in the database linking them to the template that they call, and Cargo uses those links to know which pages need to have their data stored. Now I'm curious: what does your bot look like? Specifically, I'd like to know the line or lines of your script in which the actual edit call is done.
 * Finally, there's the data duplication problem - I'm less worried about that one, because I'm actually planning a fix for it. Though I admit that it's annoying at the moment. Yaron Koren (talk) 13:44, 3 September 2017 (UTC)


 * Thanks again for the reply. I'll just wait for Cargo 1.4 to fix the Games database issue (the site isn't public yet so I'm in no rush). The bot that I'm using runs in PHP via Chris-G's botclasses.php script. Then the line of code that creates a wiki page just looks like (for example) $wiki->edit('Game/game-name-here', 'Content of the wiki page.', 'Summary for the revision history.'); If that's not specific enough, I can e-mail the exact code to you. NathanielJ (talk) 21:13, 3 September 2017 (UTC)


 * Okay, that's very good to know, and that's specific enough. I still have no idea why the links are not getting refreshed when the edit is called from a script, but clearly this is an issue; I'll have to look into it. Yaron Koren (talk) 01:26, 4 September 2017 (UTC)

Cargo 1.4 changes
In a previous comment I see that there are plans to release an update to Cargo, Cargo 1.4, in the near future. In that update, will the fixes/changes to doubling up of data/rows and making the table display format sortable be included? I must admit it's rather frustrating to see the cargoTable all out of shape again, after I had to make a change to the template behind it, and all connected pages. Roughly half the creatures (rows) are now doubled up, and the other half missing, which messes up all creature pages to a bigger or lesser extend since they are querying for data as well now. Therefore it would be fantastic if this issue could be solved in the 1.4 update. --Pangaearocks (talk) 09:57, 5 September 2017 (UTC)


 * There's nothing in the new version that deals with either duplicate or missing rows, unfortunately. I'm more concerned about the missing rows - have any more appeared yet? If not, I would suggest running another recreate - my guess is that you may have run two or more in a row, and that they overlapped, i.e. one started before the previous one finished. Yaron Koren (talk) 14:39, 5 September 2017 (UTC)


 * I had a small hope and had to ask ;-) No new rows had been added, so I followed your advice and recreated the table (did indeed do that twice last time, as the first attempt only resulted in few rows, maybe 15%). Not everything showed up now either, but 96 out of 126 is a better ratio, and I hope the rest shows up over the coming hours, or at least a day. Still some doubling up, but not nearly as much as last time. I asked the Gamepedia staff if we had access to runJobs.php somewhere, but all those things are locked out (probably due to security), so I'll have to wait until the behind-the-scenes stuff do their thing. Pangaearocks (talk) 16:33, 5 September 2017 (UTC)

Underscore inconsistency?
Am working on adapting display format from cargoTable via a template, and noticed something odd. Some of the variables have underscores in the name. Naturally I used underscore when listing fields in the cargo_query, and in the template (with named args set). Yet for some reason these values did not show up. However, when I keep the underscore in fields, but remove it (for space) in the template, the values correctly show up in the displayed table. Had noticed earlier that underscores didn't show for these variables in the CargoTable (view CargoTables), but didn't think much of it. However, maybe there is an oddity in the code? It would make more sense if we had to use undescore in the variable when working with them in templates too, as afterall the variable name actually contains underscore. In my case, some of them are damage_lh, dr_base, dr_burn.

Template, where I had to swap underscore for space
(Naturally this is WIP, and I need some colspan and other formatting)

The pages in question is this for output, and this for the template. In the SpecialPage CargoTable, the underscores are gone too. Pangaearocks (talk) 04:20, 6 September 2017 (UTC)


 * Are you possibly confusing between template fields and Cargo fields? They tend to match each other and have the same names, but their names are distinct from one another - Cargo's require underscores but are displayed with spaces, while template field names can have either. Or am I misunderstanding the question? Yaron Koren (talk) 13:26, 6 September 2017 (UTC)


 * Don't think so. The only place I am now using  or   is in the template that deals with putting data from a cargo_query into a sortable table. Just to make sure, I searched for both terms in mainspace and template with Replace Text. Hits only for that page (plus its user testpage variant). Tried to revert back to using variables with underscore, and then the data is removed. For some reason I must use spaces instead of underscores, otherwise I can't locate the data in the output format. See two pictures for examples. First is with space (as in the rolled out versions), and second is with underscore (as the variables are called everywhere else). With space and With underscore Pangaearocks (talk) 06:32, 7 September 2017 (UTC)


 * Ah, now I get it. Yes, that seems like a bug. Yaron Koren (talk) 16:37, 7 September 2017 (UTC)