Extension talk:Cargo/Archive January to February 2017

Recreation doesn't work

Hi Yaron,

first of all a Happy New Year and all the best for you in 2017.

I have a big problem with Cargo recreation. I doesn't work on any way: wether the recreation button nor the call of runJobs.php nor the call of cargoRecreateData.php has any result on any table.

I use

• MediaWiki 1.23.9
• PHP 5.5.9-1ubuntu4.19 (cgi-fcgi)
• MySQL 5.5.44-0ubuntu0.14.04.1

• Semantic MediaWiki 2.2.1 => $smwgEnabledCompatibilityMode = true;
• Semantic Forms 3.6
• Cargo 1.2

and some other extensions.

Changes to the tables work, I proofed with phpAdmin. But i have to save any page containing a #cargo_store manually to fill up the table after recreation; purge doesn't work, too.

No SQL errors, no exceptions.

Here the result of cargoRecreateData.php:

Recreating data for Cargo table Beispiel in 5 seconds... hit [Ctrl]-C to escape.
Deleting and recreating table...
Handling template that adds to this table: Cargo Beispiel
Saving data for pages 1 to 0 that call this template... 

=> There is one Page using cargo_store => the counter counts "... pages 1 to 0 ..."

Recreating data for Cargo table Test in 5 seconds... hit [Ctrl]-C to escape.
Deleting and recreating table...
Handling template that adds to this table: TestCargo
Saving data for pages 1 to 0 that call this template...

=> There are two pages using cargo_store => the counter counts "... pages 1 to 0 ..."

So, what can i do?

Addition

same problem on an other server

the table "cargo_pages" is empty after recreation

Could you include here, or link to, the text of any of the relevant template pages? Yaron Koren (talk) 00:21, 2 January 2017 (UTC)

Here a small one; the admin has all rights on MySQL except 'grant'

 <noinclude>{{#cargo_declare:
 _table=Test
 |Feld1=string
 }}</noinclude><includeonly>
 {{#cargo_store:
 _table=Test
 |Feld1={{{1}}}
 }}</includeonly>

and a bigger one; at this one the #cargo_store is used in another template.

<noinclude>{{#cargo_declare:
_table=firmen
|Bearbeitungsnummer_1=string
|Bearbeitungsnummer_2=string
|FirmenID=string
...
|zuletzt_geaendert_von=string
|Kategorie=string}}</noinclude>

--2003:72:2F23:8E00:8A1:CA60:AACD:E382 Klaus from Dresden

I couldn't replicate this problem, with that first template - it worked fine for me. Just to be clear - if you save a page calling that template, the Cargo data is stored correctly? But then if you call cargoRecreateData.php, the data disappears again? And what happens if you recreate the data from within the web interface? Yaron Koren (talk) 04:38, 3 January 2017 (UTC)

Q1: Yes, storing works fine.
Q2: Only if i change or add fields and save the template
Q3: Web interface doesn't work anyway.
Today i installed a MW 1.27.1 (virgin with no extensions except Cargo) => recreation with phpRecreateData.php works, web interface not.
Then i installed a MW 1.23.9 (virgin with no extensions except Cargo) => recreation with phpRecreateData.php works, web interface not.
After that i used an existing MW 1.23.9 with Cargo and other extensions, rebuilding the template (now with #cargo_store in the template with #cargo_declare inside), resaving data content pages => recreation with phpRecreateData.php works, web interface not.
Using this Wiki again, rebuilding the template (now with #cargo_store outside of the template with #cargo_declare), resaving data content pages => recreation with phpRecreateData.php doesn't work, web interface too.
=> seems that the #cargo_store has to be in the same template as #cargo_declare. --2003:72:2F2D:1400:CCDE:82FC:7AD7:2085 Klaus from Dresden

Well, this sounds like it might be three different problems. For the last one - if a template calls #cargo_store but not #cargo_declare, it needs to also call #cargo_attach - that could be the issue. For the web interface not working - do you have Joomla running on that server? (See here.) Otherwise: what do you see when you press the "Recreate" button? And for the original issue: under what circumstances does cargoRecreateData.php not work for you, do you know? It sounds like it now works for you in most cases. Yaron Koren (talk) 14:32, 3 January 2017 (UTC)

Thanks, so i will put #cargo_attach into templates without #cargo_declare
Joomla: no, i only host wikis on this server
seems that cargoRecreateData.php doesn't work in cases with templates with #cargo_store without #cargo_declare or #cargo_attach => solved;
Using web interface to recreate i get the information, that the datas were recreated and a link to view the table. When i follow the link, i get an empty table and see only the fieldnames.
last but not least: Yaron, you are always ready to help and make so useful tools for MW - what can I do for You? --2003:72:2F2D:1400:CCDE:82FC:7AD7:2085 Klaus from Dresden

Okay, now we're making some progress. It could be that the web-based recreate is actually working - it re-populates the table via MediaWiki jobs, so it could be that those jobs are still just waiting in the job queue and haven't run yet. Is that possible? As for helping me - that's nice of you. If you represent a company that uses this software and could potentially offer some funding, it would be great to talk about it; otherwise - just keep reporting problems as you encounter them. And of course, feel free to spread the word! Yaron Koren (talk) 18:06, 3 January 2017 (UTC)

Sorry, i had a heavy workload the last days. With #cargo_declare and #cargo_attach inside runJobs.php works fine.
So, the project i need Cargo for, is unfortunately a non-commercial for the Dresden Museum of Technik. --2003:72:E00:3400:C492:1FB9:2508:F5C Klaus from Dresden

That's great to hear. And that's also cool to hear that the software is being used at that museum - I wouldn't have known otherwise. Yaron Koren (talk)

Duplicate entries

Hello Yaron,

Happy New Year 2017 :)

I have been trying out this extension, which I like very much - thanks for creating it, I am grateful :)

However, I'm having a big issue with it. I've been testing on a clean install of Mediawiki. The issue is as follows: when an item is created/edited (using Page Forms), it usually ends up duplicated. This results in hard-to-read query output, since everything is duplicated. As a temporary fix, I've been forced to create a cron job that recreates the tables once a day. For now there is only one table and around 20 items, but I shudder to think what it will be like once the wiki gets big.

Searching through this discussion page, I came across another entry describing such a behavior. But in the other post, it seemed to only happen sometimes. In my case, it happens on almost every single creation or update of an item.

I tried to compare my template to the one on semantic discourse DB, to see if there might be some issue. I wasn’t able to identify any meaningful difference.

The setup: MediaWiki 1.28.0 PHP 7.0.8-0ubuntu0.16.04.3 (apache2handler) MySQL 5.7.16-0ubuntu0.16.04.1 ICU 55.1 Page Forms 4.0.2 Cargo 1.2 MagicNoCache 1.3.1 ParserFunctions 1.6.0 And some other extensions

Best, Michel

Update

In order to eliminate the possibility that this might be due to a mistake in the template or form page syntax, I used Page Forms to create a new template and form for a super-simple, single-field cargo table. The problem is still there.

The template and form (in case useful)

Template:

<noinclude>
This is the "DaysTodos" template.
It should be called in the following format:
<pre>
{{DaysTodos
|Todos=
}}
</pre>
Edit the page to see the template text.
{{#cargo_declare:_table=DaysTodos|Todos=List (,) of Page}}
</noinclude><includeonly>{{#cargo_store:_table=DaysTodos|Todos={{{Todos|}}} }}{| class="wikitable"
! Todos
| {{#arraymap:{{{Todos|}}}|,|x|[[x]]}}
|}

[[Category:DaysTodos]]
</includeonly>

Form:

<noinclude>
This is the "DaysTodos" form.
To create a page with this form, enter the page name below;
if a page with that name already exists, you will be sent to a form to edit that page.


{{#forminput:form=DaysTodos}}

</noinclude><includeonly>
<div id="wikiPreview" style="display: none; padding-bottom: 25px; margin-bottom: 25px; border-bottom: 1px solid #AAAAAA;"></div>
{{{for template|DaysTodos}}}
{| class="formtable"
! Todos: 
| {{{field|Todos|placeholder=Type the day's todos here...|values from category=Todo}}}
|}
{{{end template}}}

'''Free text:'''

{{{standard input|free text|rows=10}}}


{{{standard input|summary}}}

{{{standard input|minor edit}}} {{{standard input|watch}}}

{{{standard input|save}}} {{{standard input|preview}}} {{{standard input|changes}}} {{{standard input|cancel}}}
</includeonly>

That's not good. Do you have the VisualEditor extension installed? It has been known to cause this kind of problem. (That needs to be fixed.) If not, what other extensions are installed? Yaron Koren (talk) 14:40, 6 January 2017 (UTC)

I don't have VisualEditor. Here is the list of all the extensions I have: nuke, page forms, cargo, renameuser, cite, inputbox, magicnocache, parserfunctions, PDF handler, SpamBlacklist, Gadgets, Wikieditor. BTW I've noticed an issue with Page Forms: the createclass sheets only creates the category, not the template or form. (I've checked the output of runJobs.php to make sure.) Who knows, maybe the two are related? Could it be due to php7?

Michel

I don't know, then... those are all pretty standard extensions. I'm guessing that your two issues are related, especially since both involve the job queue. I doubt it has to do with the PHP version, but I could be wrong. Do you have any settings relating to the running of jobs, like a high or low job run rate? If not, I would try temporarily disabling all the extensions except for Cargo and Page Forms, and see if those two problems are still there when you do that. Yaron Koren (talk) 14:14, 9 January 2017 (UTC)

Sorry for the late response, I was (and still am) on a roadshow, and arriving back to the hotels exhausted each evening very late. I'll try that as soon as I get back. Michel (PS I left the job run rate at the default, though I'm planning to increase it at some point.)

Installation with Composer?

@Yaron Koren: Are there plans to permit installation via Composer? It works well for SMW. Sam Wilson 16:42, 10 January 2017 (UTC)

Cargo already has a composer.json file, so I believe this is already possible - I don't know what the exact line(s) that need to be added are, though (I know very little about Composer). Whatever it is, it should probably be added to the documentation. Yaron Koren (talk) 18:21, 10 January 2017 (UTC)

@Yaron Koren: No, the existing composer.json file is missing a few keys (mainly `name` but if you run `composer validate` it'll say more). Also, you'd need to register it on Packagist (which is pretty straight forward). Is this something you'd be willing do support? I could submit a patch if you like. My main reason for wanting it is for ease of upgrading. —Sam Wilson 19:13, 10 January 2017 (UTC)

Ah, okay. Sure - I'm definitely happy to approve a patch, or whatever else is required. Yaron Koren (talk) 20:07, 10 January 2017 (UTC)

Filtered display format?

Hi Yaron,

I am exploring the possibility of transitioning from an SMW-based wiki to a Cargo-based wiki. The Migration Guide is very helpful. One of my queries uses the SRF Filtered format - are there any plans for a similar format in Cargo?

Thanks! Deb Frost (talk) 14:49, 18 January 2017 (UTC)

That's great! Yes - Cargo already has the "exhibit" format, which I think is actually superior to "filtered". I haven't seen it tested in a while, but I believe it still works. Yaron Koren (talk) 15:28, 18 January 2017 (UTC)

Thank you! I tried it out and I like what I see so far. The Exhibit test on DiscourseDB was helpful too. I'm going to enjoy tinkering with this display. Deb Frost (talk) 15:14, 19 January 2017 (UTC)

In the tabular view, is it possible to display the results in the table as a link for those fields that are pages? I've tried to use CONCAT to produce a custom link text, but all I got back was blank space. My goal is to make it easier for the users to navigate to the results. Thanks! Deb Frost (talk) 21:21, 26 January 2017 (UTC)

Error: table "Books" not found.

On Azure, I've installed mediawiki-1.28.0 Cargo-1.2 PageForms-4.0.2

I've followed this getting started https://www.mediawiki.org/wiki/Extension:Cargo/Quick_start_guide RE: "For each template, go that template's page, select the "Create data table" tab option, then click the "OK" button." I do not see a "Create table" tab or button of any kind on the template's page

I don't see a "CargoRecreateData.php" file, I only see: D:\home\site\wwwroot\mediawiki-1.28.0\extensions\Cargo>dir *create* [...]

Directory of D:\home\site\wwwroot\mediawiki-1.28.0\extensions\Cargo

11/17/2016 04:25 PM 2,844 CargoRecreateDataAction.php

              1 File(s)          2,844 bytes

[...]

If I use action=cargorecreatetables, I get Success, but still no tables are created.

https://mywiki.azurewebsites.net/mediawiki-1.28.0/api.php?action=cargorecreatetables&template=Books {

   "success": ""

}

In MySQL: SELECT * FROM mywiki.cargo_tables; 0 rows(s) returned

How can I troubleshoot this further? Thanks.

That's a lot of problems! cargoRecreateData.php is contained in the /maintenance folder. Maybe just calling that will fix everything? Yaron Koren (talk) 16:18, 26 January 2017 (UTC)

D:\home\site\wwwroot\mediawiki-1.28.0\extensions\Cargo\maintenance>CargoRecreateData.php

process [1840] terminated! Press ENTER to start a new cmd process.

my fault; now did php CargoRecreateData.php

The command prompt returned with no messages

And I repeated all of above; I still see no tables.

Oh, okay. I'm guessing that there's a problem with the #cargo_declare call in your "Book" template (assuming you have such a thing). Yaron Koren (talk) 17:03, 26 January 2017 (UTC)

Requireonce blanks the wiki

ehy there, MW 1.27 on localhost semantic mediawiki actually not installed

I followed all the installation steps and when I add require_once( "$IP/extensions/Cargo/Cargo.php" ); it blanks my wiki. I gave a fast look to the archive but couldn't find a solution. maybe it's a problem of path where the extensions are stored?

thanks

Please see here for how to get the actual error message to show up on the screen. Yaron Koren (talk) 18:23, 1 February 2017 (UTC)

No permission to the folder "sudo chmod -R 0755 [folder of the extensions]" solved it. Thanks for everything.--Plasminhosantos (talk) 18:54, 1 February 2017 (UTC)

Database Error after update from 1.0 to 1.1 / 1.2

Hi Yaron,

after updating Cargo from version 1.0 to 1.1 (or 1.2) I get the following error for all cargo queries:

A database query error has occurred. This may indicate a bug in the software.

Query: SELECT `_pageName` AS Page,`Mn_model_item` AS Mn model item,`Mn_model_influence_on` AS Mn model influence on,`Mn_model_hypothesis` AS Mn model hypothesis FROM `cargo__webmo_hypothesis` ORDER BY `_pageName` LIMIT 100

Function: CargoSQLQuery::run

Error: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'model item,`Mn_model_influence_on` AS Mn model influence on,`Mn_model_hypothesis' at line 1 (localhost)

I did run maintenance/update.php and extensions/Cargo/maintenance/cargoRecreateData.php.

Any idea?

Thanks and regards, --Filburt (talk) 22:14, 1 February 2017 (UTC)

It's strange that those aliases (like "Mn model item") contain spaces and not underscores; that is what is causing the problem. I don't know why that's happening. Have you tried using the very latest version (via Git)? Yaron Koren (talk) 03:49, 2 February 2017 (UTC)

Thanks for the quick response. Here is the working query from Cargo version 1.0:

SELECT `_pageName` AS `Page`,`Mn_model_item` AS `Mn model item`,`Mn_model_influence_on` AS `Mn model influence on`,`Mn_model_hypothesis` AS `Mn model hypothesis` FROM `cargo__webmo_hypothesis` ORDER BY `_pageName` LIMIT 100

The difference are the quotes around the aliases. Thanks for support --Filburt (talk) 09:22, 2 February 2017 (UTC)

The field names in the #cargo_query call need to have underscores, not spaces - could that be the issue? Yaron Koren (talk) 15:02, 2 February 2017 (UTC)

After updating from MW 1.27alpha to MW 1.28 everything is working again. There was a change with the handling of the quotes. Problem solved for me - thanks for support. --Filburt (talk) 10:23, 3 February 2017 (UTC)

Show List of Strings as links to Special:Drilldown

I have a field which is defined as List (,) of String which shows up in the results as x · y · z. Is there any way of converting "x" (etc) into a link to the relevant Special:Drilldown page? I am able to use CONCAT to do this for String types (just not Lists of Strings).

A similar effect can be achieved using List (,) of Page though the link is not exactly what I want - I want to obviate the need for creating pages. To use your Book/Author example, I just want to have each author link to a Drilldown page with that author pre-selected (which will show all his books), and not need to create a separate page for each author.

Thanks in advance! Jonathan3 (talk) 14:36, 2 February 2017 (UTC)

I agree that that's a good idea, and the Miga JS library, which I wrote, and which was a big part of the inspiration for Cargo, takes that approach - if you see this page, for example, every string value - and every "red-linked" page - links to the drilldown interface with that value selected. It would be great if the system supported this, at least as an option - for both query results and the data display within the template. However, it's not supported within Cargo right now. You could probably accomplish in some way, maybe using a combination of SQL and JavaScript, but it wouldn't be easy. Ideally, such a feature could be added. Yaron Koren (talk) 15:59, 2 February 2017 (UTC)

Thanks for the reply. The page at that link is impressive, and it would be good if you could add that feature to Cargo. Jonathan3 (talk) 22:47, 2 February 2017 (UTC)

┌──────┘
I made some short changes to CargoQueryDisplayer.php to get this to work. At the minute, it only works for Lists of Strings, not for (a) Lists of anything else (the only other List I have is is of Pages, which show as links already), or (b) anything outside a List. Simple, similar changes could deal with these limitations. It is always on, so would need a new parameter to be added to make it optional, e.g. {{#cargo_query|...|add drilldown links}}?

				if ( $fieldDescription->mIsList ) { #existing line
...
						global $wgServer;
						if ($fieldType == 'String') {
							$text .= '<a href="' . $wgServer . '/Special:Drilldown/' . $tableName . '?' . $fieldName . '=' . $fieldValue . '">';
						}
						$text .= self::formatFieldValue( $fieldValue, $fieldType, $fieldDescription, $this->mParser ); #existing line
						if ($fieldType == 'String') {
							$text .= '</a>';							
						}

Before I do anything else, in case I'm barking up the wrong tree, what do you think? Jonathan3 (talk) 22:54, 4 February 2017 (UTC)

PS Just realised I probably don't need $wgServer at all, but should have used $wgScriptPath or something else (I have short URLs and my $wgScriptPath is empty) Jonathan3 (talk) 23:08, 4 February 2017 (UTC)

Thanks. I think it probably makes sense to have a separate icon or something, to indicate visually that this is not just a regular link - like the magnifying glass icon in Miga. But you don't have to do that part - I'm just explaining my thinking. Yaron Koren (talk) 00:01, 5 February 2017 (UTC)

I agree that the magnifying glass icon would be good. What would be the best way to implement this - a global array turning it on for the whole site, maybe an array of input types - or parameter for {{#cargo_query}}? What input types would it make sense to allow it for? (Incidentally, when should Text - "intended for longer values" - be used instead of String?) Jonathan3 (talk) 21:47, 14 February 2017 (UTC)

I don't know - maybe a query parameter makes the most sense, but I'm not sure. This might only make sense for fields of type String and Page, since those are the only two types for which it makes sense to look for exact matches. And maybe only String, since Page values, i.e. pages, can do their own aggregation. Though then again, some pages haven't been created yet. Maybe it makes sense to do it for String values and red-linked pages. As for String vs. Text - there are two big differences in the handling: String values get a drilldown filter, while Text values don't; and, in forms, Strings get a text input while Text values get (ironically) a textarea input. Yaron Koren (talk) 15:57, 15 February 2017 (UTC)

From the extension page: "Fields of type Text, Page, Integer, Float, Date and Datetime are turned into filters, with the input type used dependent on the field type; other field types, like URL, Coordinates, File and Wikitext, are not." I'd say Text yes, Page no (better to be treated normally and go to form/create page), Integer yes, Float don't know (never used it), Date and Datetime yes (e.g. book's year of publication).Jonathan3 (talk) 21:06, 15 February 2017 (UTC)

Oops, that text was outdated - thanks for pointing that out. For numbers and dates, it sounds like you're talking about linking to the relevant "bucket" surrounding each value, but that might be a challenge - to have it done right, it might require reconstructing the entire set of buckets from any page that links to Special:Drilldown, and that might have a performance impact. Yaron Koren (talk) 01:09, 16 February 2017 (UTC)

Change default namespace for Page input type?

Going back to you Books/Authors example, if you have separate "Books" and "Authors" namespaces, is it possible to get the wiki pages (initially, the red links) for each author to be in the "Authors" namespace (or, at least, the "Books" namespace) instead of the Main namespace? Sorry if I've missed something obvious. Jonathan3 (talk) 22:52, 2 February 2017 (UTC)

You just need to add the namespace to the #cargo_store call in the template, like "Author=Author:{{{Author|}}}". Or are you asking specifically about a field that holds a list of values? If so, I think you'd have to call #arraymap within the #cargo_store call. Yaron Koren (talk) 00:58, 3 February 2017 (UTC)

Thanks, |Authors={{#arraymap: Author:{{{Authors|}}} }} worked fine. Jonathan3 (talk) 22:01, 3 February 2017 (UTC)

That won't work if there's more than one author; the #arraymap call needs to be a little more complex. Yaron Koren (talk) 22:49, 3 February 2017 (UTC)

You're right. I'd be grateful if you could tell me how to get it to work, as I'm stuck! Jonathan3 (talk) 22:57, 4 February 2017 (UTC)

Something like this should work: |Authors={{#arraymap:{{{Authors|}}}|,|x|Author:x}} Yaron Koren (talk) 23:52, 4 February 2017 (UTC)

Add URL column to exhibit display format

If I use CONCAT to create an [[internal link]] or an [http://www... external link] the exhibit display format displays the unparsed wikitext. Other formats (e.g. dynamic table) are OK with this. Is there any way to achieve the same result using the exhibit display format? Thanks. Jonathan3 (talk) 23:08, 2 February 2017 (UTC)

I don't think so. Yaron Koren (talk) 00:59, 3 February 2017 (UTC)

Thanks Jonathan3 (talk) 10:58, 3 February 2017 (UTC)

Thanks that answers my question too! Deb Frost (talk) 14:55, 3 February 2017 (UTC)

Oh, sorry - I missed your question above. Yaron Koren (talk) 14:59, 3 February 2017 (UTC)

Get one list of all "Lists of ..." [solved]

In your Books/Authors example, is it possible to get a list of all the Authors from the Books table? So a list like:

• Joe Bloggs
• Joe Soap
• John Doe

Rather than (if there is joint authorship):

• Joe Bloggs
• Joe Bloggs · Joe Soap
• Joe Bloggs · John Doe

The one directly above is what I get from:

{{#cargo_query:
tables=Books
|fields=Authors
}}

I know I could get the authors from the Authors table, but only if there are pages for each author (no red links left).

Cheers. Jonathan3 (talk) 23:21, 2 February 2017 (UTC)

To answer my own question, in case it helps others, I just needed to add |group by=Authors. This works for Page and String input types, and I guess others as well. Jonathan3 (talk) 23:26, 2 February 2017 (UTC)

HOLDS NOT on a List

Is there a neater way of doing the following?

...
tables=Books,Books__Author
|join on=Books._ID=Books__Author._rowID
...
|where=Books__Author._value NOT IN  ("x", "y", "z")

Maybe a way to use HOLDS? Thanks. Jonathan3 (talk) 10:58, 3 February 2017 (UTC)

I think you can do something like "NOT (Author HOLDS 'x') AND NOT (Author HOLDS 'y') AND NOT (Author HOLDS 'z')". Whether that's more neat, I don't know. Yaron Koren (talk) 15:03, 3 February 2017 (UTC)

Not display namespace in List of Pages

Thanks for your replies above. This may be linked to more than one of them. How can I prevent the authors' pages appearing as "Author:Joe Bloggs" (e.g. in a dynamic table)? I'd prefer just "Joe Bloggs". Thanks Jonathan3 (talk) 22:05, 3 February 2017 (UTC)

If the idea is to have this still be a link, probably a combination of CONCAT() and SUBSTRING() would work. Yaron Koren (talk) 22:47, 3 February 2017 (UTC)

I can't get CONCAT to work with any List input types. Possibly for the same reason as not being able to get them to link to the Drilldown page. Or is there a way for something more simple like removing namespaces? Jonathan3 (talk) 23:04, 3 February 2017 (UTC)

Hm, that's true. i don't know. Yaron Koren (talk) 23:54, 4 February 2017 (UTC)

If we can get the "click on anything to go to the drill down page" feature added then I would be happy with that, as I don't really need the author page to include anything except for a list of the author's books. Jonathan3 (talk) 16:32, 5 February 2017 (UTC)

I've added a patch on Phabricator which (if you find it works correctly) adds an option to hide namespaces from displayed page titles in any query, including Lists of pages (see T157754). Jonathan3 (talk) 00:49, 11 February 2017 (UTC)

Changing layout of Special:Drilldown

I don't think the current format is that easy for people who aren't familiar with MediaWiki category pages etc.

One thing that could improve it (I think) is to have the results on a second column, rather than the very bottom of what might be a long page, so people at least know that they have found something, and can then more easily see it change as they use the filters.

Would it be possible to make this an option? Thanks. Jonathan3 (talk) 23:05, 4 February 2017 (UTC)

P.S. Alternatively, the results could go at the top of the page, perhaps under "full text search", and the filters go at the bottom. At the moment it really isn't that easy to see that there are any results. Jonathan3 (talk) 23:43, 4 February 2017 (UTC)

What evidence do you have that the current interface is confusing to users? Your suggested approach seems valid, but I don't know if there's a need for it. Yaron Koren (talk) 23:55, 4 February 2017 (UTC)

Someone who is smart and computer literate (but not accustomed to using databases etc) made several comments along the lines of "that looks like an error page", "that looks like a page that the normal user shouldn't be seeing", "you can't expect people to know what's happening on that page", "that's just a mess". I accept that this is only evidence from one source, but I trust it :-) Personally I think the page is really good, but I can see that the interface could be improved.

I think that if we could sort out the ability to click on anything to go to the drill down page, and sort out the interface on the drill down page itself, that the whole thing would be fantastic. Jonathan3 (talk) 16:30, 5 February 2017 (UTC)

Alright. That's still just one person's opinion, but it's good to know. Yaron Koren (talk) 18:20, 5 February 2017 (UTC)

Not display namespace in Special:Drilldown

Is there a way to do this? I guess not, but worth asking! At the minute there are three headings for the pages, "C", "C cont.", and "C cont.", as the na|Jonathan3]] (talk) 20:40, 5 February 2017 (UTC)

I think if you add a "DEFAULTSORT" call to each page, via the template, and have it use the namespace-less name, that may fix the problem. Yaron Koren (talk) 00:42, 6 February 2017 (UTC)

In category pages it works fine already, showing A - Z headings. "By default, a page is sorted within a category under the first letter of its name — without the namespace." (Help:Categories#Sort_key) It's just a problem in the Drilldown page, which seems to work differently. Unfortunately {{DEFAULTSORT:{{PAGENAME}}}} makes no difference. Jonathan3 (talk) 23:14, 6 February 2017 (UTC)

I've added a patch to Phabricator (see T157754) which I hope sorts this out. Jonathan3 (talk) 00:27, 10 February 2017 (UTC)

Thanks for the patch! I added it in, with slight modifications. Yaron Koren (talk) 19:40, 10 February 2017 (UTC)

The previous patch just fixed the A-Z headings, but the pages were still displayed with namespaces. I've submitted another patch (T157849) which allows Special:Drilldown to hide namespaces. Jonathan3 (talk) 00:53, 11 February 2017 (UTC)

I wasn't sure whether to respond here or on Phabricator, but here seems more comprehensive. To summarize: your patch defines a global variable for not displaying namespaces in Special:Drilldown, as well as a new parameter, "hidenamespace', that does the same thing for query results. The global variable approach seems too heavy-handed to me - first because it's all-or-nothing, and second because it seems to violate MediaWiki conventions, to have a general list like that exclude namespaces - that's not done in any of the other special pages. A query parameter seems more reasonable.

However, there's another option, which is to use the "DISPLAYTITLE" value for each page. I think Cargo doesn't do anything right now with "DISPLAYTITLE", but it could - it could potentially replace the page name in both query results and its special pages. And in turn, you could have each page in a namespace set its display title to the name without the namespace. It might take a little more work, but it might be a cleaner solution.

Let me ask, though, something that I should have asked before: why are you using this namespace? I tend to advocate against using namespaces just to distinguish content - see the last item here. Maybe it would be easier to just move all these pages to the main namespace? Yaron Koren (talk) 14:34, 12 February 2017 (UTC)

┌───────────────────┘
Thanks. It didn't let me indent any more...

Query results - it would be great if you could incorporate into the Cargo code an option to remove namespaces from query results.

Drilldown - The gloabal variable could be an array, set for one table at time, so not quite "all or nothing", but I see what you mean. With the DISPLAYTITLE possibility: the skin I'm using (Foreground) displays the namespace separate from the title, in a nice unobtrusive way which I wouldn't want to change on the pages themselves. Though any way that removes "Namespace:" from the Drilldown page would be an improvement for me, on balance.

Namespaces - I'm using a separate namespace because it's a part of the website with a completely different type of information, which I don't want turning up in the standard search results. It also gives me a way of changing the background colour of the pages using CSS (though maybe that is possible using categories too). On the other hand, it's not necessary for disambiguation. Jonathan3 (talk) 15:18, 12 February 2017 (UTC)

Ah, I didn't think about using the namespace to have different search behavior. (The appearance thing is true too, but there may be other ways to accomplish that.) You're right that the global variable could be an array - which gives me an idea. What about having a global variable like $wgCargoHideNamespaceName, which takes in an array of namespace IDs, and for each of those namespaces, hides that namespace in any page name that contains it? That array would then take effect within both query results and special pages, including Special:Drilldown. It might even make sense to have the array hold one value, NS_FILE, by default, since the "File:" namespace already gets this handling, more or less. What do you think of that idea? Yaron Koren (talk) 01:21, 13 February 2017 (UTC)

This is a much more elegant way of doing it, thanks. I've submitted a new patch (at T157849). I've not tested it with the "File:" namespace, but it works just as well as the first patch did with the other namespaces which I've checked it with.Jonathan3 (talk) 21:30, 13 February 2017 (UTC)

Great, thank you - I just checked in the patch. This one's quite a bit more efficient, which is a good sign. Yaron Koren (talk) 16:44, 15 February 2017 (UTC)

Thank you :-) Jonathan3 (talk) 21:08, 15 February 2017 (UTC)

Problem with recreating table

I decided to change the structure of a table, so deleted all pages with template calls, deleted the template and the form, then used the CreateTable and CreateForm pages. Recreating data from the template page gives no errors. I got an error when I tried to save a page using the form, and an error when trying to view the table using Special:CargoTables. Essentially, it's trying to get the OLD fields which no longer exist, and it giving an "1054 Unknown column ... in 'field list'" error. The command line version gave this: "Error: 1062 Duplicate entry '<table name>' for key 'cargo_tables_main_table'". The table_schema in the cargo_tables has the OLD fields. I missing something obvious? I can send you fuller error messages if necessary. Thanks. Jonathan3 (talk) 22:52, 15 February 2017 (UTC)

With that row in cargo_tables, I just added "xxx" to the end of main_table and "999" to the end of template_id, clicked "recreate data" on the template page, and it all worked from then on. I guess it was not being updated correctly by the software. Perhaps I should have deleted the table manually from the Special:CargoTables page?

Can I go ahead and just delete the "...xxx" row relating to the unwanted table? It is showing up on Special:CargoTables as "Table is registered, but does not exist!" now. Thanks. Jonathan3 (talk) 23:04, 15 February 2017 (UTC)

Sorry, I don't fully understand what you did, but it does sound like you should have deleted the table via the web interface. Yes, you should delete that row. Yaron Koren (talk) 01:11, 16 February 2017 (UTC)

I'll try to set it out: (1) Deleted Form:Book; (2) Deleted Template:Book; (3) Deleted all pages calling Template:Book; (4) Created new Template:Book using Special:CreateTemplate (I wrongly typed CreateTable above); (5) Created new form for it using Special:CreateForm; (6) Visited Template:Book and clicked "Recreate data", which gave no errors (it replaced the old Books tables in Cargo's database, but did not update table_schema etc in the cargo_tables table in the main MediaWiki database).

Special:CargoTables/Books gave the 1054 error because it was looking up the old column names (which wrongly still appeared in the cargo_table table).

It was php cargoRecreateData.php --table Books that gave the 1062 error (not sure why now, but I thought it made sense at the time...)

Next: (1) Removed the "Books" row in cargo_tables (by just appending "xxx"); (2) Visited Template:Book and clicked "Recreate data" (this time a new "Books" row in cargo_tables was created with the new table structure); (3) Special:CargoTables/Books worked fine.

Special:Drilldown stopped working as it was looking for the "Booksxxx" table so once I deleted that (thanks for giving me the go-ahead) it too worked fine.

Do you think maybe Cargo has permission to insert but not update fields in the main Mediawiki database? How should I check this on my website? Jonathan3 (talk) 11:00, 16 February 2017 (UTC)

Hm, maybe that's it. Have you been able to successfully recreate any table? Yaron Koren (talk) 14:35, 16 February 2017 (UTC)

Just did this: (1) Used Special:CreateTemplate to create Test template, Test table; (2) Created data from template page and viewed table with no problems; (3) Used Special:CreateTemplate again, for Test template, Test table, but with different field details, and recreated data; (4) Special:CargoTables/Test says this: "Table "Test" not found in Cargo database."; (5) Special:CargoTables/ says this: "Test - Table is registered, but does not exist!" Does this help? Jonathan3 (talk) 19:32, 16 February 2017 (UTC)

Not really. What happens if you call the cargoRecreateData.php script? Yaron Koren (talk) 19:59, 16 February 2017 (UTC)

Sorry for not being more helpful about this. I've tried it all again and it works fine. I really can't account for the difficulties I had earlier. Sorry again! Jonathan3 (talk) 21:13, 16 February 2017 (UTC)

Okay, great. There still may be some real issue there, but we'll see. Yaron Koren (talk) 02:09, 17 February 2017 (UTC)

Cargo data field size for user cargo tables

Installed software MediaWiki 1.27.1 PHP 5.6.30 (cgi-fcgi) MS SQL Server 10.50.6529 Cargo 1.2 Page Forms 4.0.2

When a cargo data table is created in the database, from the definitions you created, the filed lengths are all getting set as varchar(300) fot the text types (text,string ). My question is where can I set the length of the field? I can see that I can set the form display length, maxsize, textarea, etc. but I don't see where I can set the actual database table field length?

It's explained near the bottom of this section; you just need to add "(size=500)" or something after the type name. (Though, for "Text" types, the DB type should also be "Text" - make sure the type name is exactly "Text" and not "text" or something.) Yaron Koren (talk) 16:44, 17 February 2017 (UTC)

Using a separate database

Hello,

I want to use a separate database for cargo data. I was not able to do it. The cargo tables are always created in the mediawiki database. Could you provide a detailde way of achieving tis. Thans in advances. JMU

You should follow the instructions here. Yaron Koren (talk) 16:48, 19 February 2017 (UTC)

I have followed the instructions but I am unable to set up the separate database. I have a local installation with bitnami installer MW1.28. The bitnami_mediawiki database is installed with a bn_mediawiki user. I created a wikicargo database, with the specific user usercargo with same privileges as bn_mediawiki user. When I run php update.php the cargo_tables and cargo_tables are created in the bitnami_mediawiki database. I used port 3314 for the mysql database. The bn_mediawiki has following rights :

GRANT USAGE ON *.* TO 'bn_mediawiki'@'127.0.0.1' IDENTIFIED BY PASSWORD data deleted;
GRANT ALL PRIVILEGES ON `bitnami_mediawiki`.* TO 'bn_mediawiki'@'127.0.0.1';

The usercargo has following rights :

GRANT USAGE ON *.* TO 'usercargo'@'127.0.0.1' IDENTIFIED BY PASSWORD data deleted;
GRANT ALL PRIVILEGES ON `wikicargo`.* TO 'usercargo'@'127.0.0.1';

Th LocalSettings.php file contains

$wgCargoDBtype           = "mysql";
$wgCargoDBserver         = "127.0.0.1:3314";
$wgCargoDBname           = "wikicargo";
$wgCargoDBuser           = "usercargo";
$wgCargoDBpassword       = data deleted;

Regards JMU

Maybe it's actually working correctly? The cargo_tables and cargo_pages always go in the main database - those aren't the "real" Cargo tables. Yaron Koren (talk) 13:34, 26 February 2017 (UTC)

Thanks Yaron, I create a table and it is in the wikigargo database. I was confused by " it differentiates its DB tables from all the rest by starting all their names with "cargo_" in the instructions.

Ah, yes, sorry, that sentence is outdated (and not that clear) - I just improved it. Yaron Koren (talk) 21:37, 26 February 2017 (UTC)

Join the same table twice in a Cargo query

Hi I'd like to do something like this SQL query using Cargo Query.

SELECT 
    comment, 
    userFrom.username AS commentFrom,
    userTo.username AS commentTo
FROM comments 
JOIN users AS userFrom ON userFrom.ID = comment.commentFrom
JOIN users AS userTo ON userTo.ID = comment.commentTo

Where 'commentTo' and 'commentFrom' are fields on the Comments table containing the IDs of users on the Users table. I figure I probably have to alias the Join clauses in CargoQuery but I'm not sure if this is possible and, if it is, how to do it. If not, would it be possible by making AS an allowable MySql function for Cargo?

Many thanks

Duncan

Oh... I never thought about the possibility of joining the same table twice. That is indeed sometimes useful. This would require changes to the query syntax, as well as additional handling (at least for query validation). Presumably the best way to do this (as far as I can think) is to add an "=alias" option to the "tables" parameter, like it exists for the "fields" param. Yaron Koren (talk) 15:15, 20 February 2017 (UTC)

Hi Yaron - that was indeed the way I tried to do it when I was experimenting, so it does seem the 'natural' approach I think.

Kind regards

Duncan

Okay, I just added aliasing of table names to the latest Cargo code in Git, using that "=alias" syntax. If you can, feel free to try out the latest code and let me know if it works for you. Yaron Koren (talk) 19:42, 27 February 2017 (UTC)

Using {{DISPLAYTITLE}}

It would be great if Cargo would make use of this (you mentioned it before) - in my case, I'd like to be able to have italics in page titles. I guess it would be possible to include it in a template, using the contents of a Cargo field - but it would be good if it showed up in search queries and the Drilldown page also. Thanks in anticipation! Jonathan3 (talk) 20:24, 22 February 2017 (UTC)

It's good to know that this would be useful for you. Yaron Koren (talk) 22:42, 22 February 2017 (UTC)

List of all places with another place within x miles

Together with the names of those other places. Is there any way of doing this via Cargo? It's to help to find duplicate entries. Or would I need to do it directly via SQL? Thanks. Jonathan3 (talk) 23:44, 23 February 2017 (UTC)

Sorry, I don't understand. Yaron Koren (talk) 18:32, 24 February 2017 (UTC)

I've not quite worked it out myself, to be honest. Something like - for each place with coordinates, show the place name for every other place within 0.2 miles. Jonathan3 (talk) 22:48, 25 February 2017 (UTC) P.S. I've never used Scribunto/Lua but would it be useful here? Jonathan3 (talk) 23:05, 25 February 2017 (UTC)

Oh, that's a query within a query. Yes, Lua would make this easier. Potentially you could also do it with the "template" format too, and have the template call the 2nd query. Yaron Koren (talk) 13:35, 26 February 2017 (UTC)

Add map to Special:Drilldown

Would this be possible, say, for tables with one set of coordinates? Jonathan3 (talk) 00:23, 24 February 2017 (UTC)

Yes, this would be great to have, and it's among the planned features (third item). Yaron Koren (talk) 18:33, 24 February 2017 (UTC)

Job run rate

I would be grateful for more information on this:

'If a table is recreated via the web interface and the "job run rate" (set via $wgJobRunRate) is too high, it can lead to some pages not getting their data stored.' (Extension:Cargo/Known_bugs_and_planned_features).

Mine's set to 1 and I haven't had a problem but I wondered what "too high" is or might be. Thanks. Jonathan3 (talk) 23:09, 25 February 2017 (UTC)

I vaguely recall that a rate of 10 caused problems. Yaron Koren (talk) 13:36, 26 February 2017 (UTC)