Extension talk:Data Transfer

Error after installation
I have attempted to install the Data Transfer Extension, but when trying to go to the View XML page I get the following error: "Warning: Invalid argument supplied for foreach in C:\mediawiki\extensions\DataTransfer\specials\DT_ViewXML.php on line 189". This error is printed 7 times. The page looks fine below the errors. Is there an easy fix? Jascraig 12:17, 23 April 2008 (UTC)


 * Hi, thanks for the bug report. I think I know the problem - you have some pages that don't belong to any category, and Data Transfer wasn't expecting those. There's an easy fix, which I'll add to the next version of the extension. Yaron Koren 14:28, 23 April 2008 (UTC)


 * Yaron, Thanks, that fixed the problem. Now I'm having problems when I click Submit on the View XML page, instead of getting the XML I expect, I'm redirected back to my Wiki Main Page.  I am going to try and diagnose this locally, but if you have any ideas what could be causing this I would appreciate the information.  Thanks for the quick response. Jascraig 16:21, 23 April 2008 (UTC)

About dt_xml_* messages
I'm a translater in translatewiki.net. dt_xml_* software messages such as dt_xml_namespace are used in xml outputs as names of elements or attributes. They are, however, translatable. When you export xml files in some wiki, they can't be imported in different language wikis because of lacking portability of xml outputs. Is this intentional? --fryed-peach 15:01, 16 April 2009 (UTC)


 * Well, Data Transfer was never meant to help with transferring pages from one wiki to another, since Special:Export and Special:Import already handle that; it's meant only for transferring data with non-wiki systems; so the tags can say anything. Yaron Koren 16:19, 16 April 2009 (UTC)


 * Thank you for your quick response. The current documentation is not clear to me around interwiki-portability. But now I understand its design. --fryed-peach 06:03, 17 April 2009 (UTC)

CSV example needed
I am unclear on how to properly format a CSV file to have it accepted by this extention. I am attempting to import a CSV file exported from Excel or Open Office.

Without the proper headers, the import generates this error: Error: the column 0 header, 'Revised: June 23, 2009', must be either 'Title', 'Free Text' or of the form 'template_name[field_name]'


 * As the documentation says, actually it's pretty easy. All you have to do is preparing the header, in one line. IE, with the following CSV:


 * George, 22, 1977, He is a good guy
 * Julian, 24, 1979, He is a bad guy


 * The proper header would be:


 * Title, People[date], People[year], Free Text


 * You also have to create the Template with the proper variable to take it. In this case, would be Template:People, and in there you should have something like this "He is year old, was born in ". And that's it.

#ask queries treated as templates
Hi... the XML output seems to treat {{#ask queries as templates, so I get:


 *  TypeUserDescription 

Is that the expected behaviour? - Borofkin 04:39, 14 January 2010 (UTC)

Call to a member function getNamespace
I have a CSV-file with 5400 rows of information created in Openoffice Calc and saved as CSV. Just titles and a short free text on each row. When I try to import the CSV I get an error saying:

Fatal error: Call to a member function getNamespace on a non-object in /home/christoffer/public_html/wiki/includes/JobQueue.php on line 277

This error seems to have something to do with non-UTF-8 encoded characters, but as far as I know I don't have any "special" character and my files are always saved with UTF-8 encoding. If I take some random rows from the file it works, but the whole file won't work. I have also tried to import a file with several non-UTF-8 characters like åäö, lots of quotation marks, commas and so on, and it also works. Is it the size of the file that is the problem (to big) or is it something else? --Squall Leonhart 22:37, 8 April 2010 (UTC)


 * Hi - if possible, could you try splitting this up into smaller files, and try importing those? At the very least, it'll help determine whether the problem is the file size, or some specific row(s); and maybe help isolate the offending rows, if any. Yaron Koren 00:47, 10 April 2010 (UTC)


 * I tried that now. I think I found the error. Some of the rows did contain characters like [], {} and ~ in the title column. When I removed them everything worked like a charm! --Squall Leonhart 13:16, 10 April 2010 (UTC)


 * Okay, that's good to know, thanks. I'll try to fix the code so that it handles those characters more gracefully. Yaron Koren 21:20, 11 April 2010 (UTC)


 * This has been fixed now in SVN, and it'll be fixed in the next version of Data Transfer as well, FYI. Yaron Koren 05:32, 13 April 2010 (UTC)

Multiple templates & ImportCSV
One thing I've noticed - if you attempt to import a CSV file to create a page which has multiple instances of the same template, it silently just uploads the last one specified. eg in CSV:
 * Title, Template1[field1], Template1[field1]
 * Testpage, foo1, foo2

The workaround was to use ImportXML which does let you specify multiple versions of the same template. --92.236.50.225 17:05, 2 May 2010 (UTC)

EDIT: hmm... have also noticed that "&amp;amp;" in field values don't get re-imported by ImportXML, even if correctly escaped... --92.236.50.225 20:38, 2 May 2010 (UTC)


 * The hack I've used is to create various templates, then Renaming them by ReplaceText extension, like this:
 * Title, Template1[field1], Template2[field1]
 * Testpage, foo1, foo2
 * Template2 is exactly the same as Template1, and once you imported it you go to Special:ReplaceText and replace Template2 for Template1 (yeah, its a tiny dirty hack XD)


 * Ooh, that's not bad! Yeah, I can't think of another solution for it. Yaron Koren 16:44, 12 May 2010 (UTC)

Templates Inside of Templates
I run into a lot of XML parsing difficulties when it runs across a page with a template inside of a template. For instance, if the wikitext is:

Harmony
So we have ,

then the xml file gives

Home DemoMp3Aisumasen Home.mp3&#125;&#125;&#125;</Field>=Harmony==

So we have Bridge</Field>I</Field>ii</Field></Field>,

It then seems to interpret main body text as various fields, which runs into all sorts of weird problems from then on.

Douglas (May 25 2010)


 * That looks like a bug in the code. Though Data Transfer isn't really geared for templates-within-templates, so I'm not that surprised that it doesn't work. Yaron Koren 13:20, 26 May 2010 (UTC)

Template Calls with Special Characters
For instance, I sometimes have the line

as a parameter input. Example:

The XML then looks for a matching

Dunno if this is a bug in the code or a poor convention of mine.

Okay, I fixed this, so in case anyone wants to know: the convention between HTML and XHTML is that HTML permits the unclosed line break while XHTML requires that it be closed like

One more problem, however, arises with the use of

I use this in my template input all the time when I want a box, for instance, to display some terminal text. I'll write something to ignore equal signs when its inside angular brackets.

-Douglas (May 26 2010)

overwrites existing pages
hi,

how can I prevent overwriting existing pages with this import-mechanism?

Thx. --Rolze 08:27, 15 June 2010 (UTC)


 * Unfortunately, you can't - but that would be a nice feature to have. Yaron Koren 12:21, 15 June 2010 (UTC)
 * Ok, so I - as a faithful semantic mediawiki fan and user - request that this feature gets on the roadmap of this fine extension! And that it won't be forgotten on the nice2have-wishlist! ;) --Rolze 21:29, 15 June 2010 (UTC)


 * I think this would be a nice feature, too. Cheers --kgh 22:42, 15 July 2010 (UTC)

Bot flag is being ignored
Hi Yaron, so far the bot flag is being ignored by this extension. All the log actions of this extension are displayed on recent changes even though the importing user has a bot flag. This is not really a problem, but it would be nice if the extension accepts the bot flag in the future. Cheers --kgh 22:41, 15 July 2010 (UTC)

semicolons as separators
Hi, it is me again. :-) I would be very useful if semicolons are the separators of choice with this extension, since they are by far more rare in use within texts. So if texts containing a comma get imported, they are destined to ruin the respective imported page. Hmm, is it possible to use text markers like ' or " for texts. This should be a workaround. Cheers --kgh 22:48, 15 July 2010 (UTC)
 * Hi - well, CSV is a very standard format, unlike, say, "SSV". In CSV, values that contain commas are meant to be enclosed within quotes, which solves the problem. Yaron Koren 23:04, 15 July 2010 (UTC)
 * That is true and every user should stick to this to keep the format exchangeable without having to prepare the file for further use. Thank you and cheers --kgh 07:16, 16 July 2010 (UTC)

Enhancement to DataTransfer
Hi, a nice enhancement would be, if one is able to insert an individual description for the upload instead of the standard "CSV-Import". Another wish would be, that the name of the file used for import is piked up and displayed, too. To me the second request seems to be more important. Cheers --kgh 22:20, 19 July 2010 (UTC)


 * That's a good idea, and you asked at just the right time - I was in the middle of getting a new version ready for a release. This feature is now available (though only for CSV import right now, not for XML import) in the latest version, 0.3.6. Yaron Koren 16:31, 26 July 2010 (UTC)


 * Cool, thank you! Great work! I am very happy about this. I am mostly doing CSV imports, thus my urgent needs a fulfilled, which does not mean... :-) Cheers --kgh 08:51, 27 July 2010 (UTC)

SVN tag for 0.3.6 missing
The last SVN tag is 0.3.5 --RScheiber 22:40, 14 September 2010 (UTC)

Why does this extension exist?
RE: It should be noted that Data Transfer is not an ideal solution for backing up one's wiki, or transferring wiki pages from one MediaWiki site to another; for that, the much better solution is to use MediaWiki's built-in "Special:Export" and "Special:Import" pages.


 * So why have this extension at all if "Special:Export" and "Special:Import" is better? Adamtheclown 06:05, 24 November 2010 (UTC)


 * What do you do in case your data is not stored in xml? That is the reason for this extension. Cheers --kgh 11:28, 24 November 2010 (UTC)

Expected pages got mediawiki error
My error message: Since I don't see why this extension exists, since "Special:Export" and "Special:Import" already moves pages better, I will not try to get assistance to fix this problem. Adamtheclown 06:17, 24 November 2010 (UTC)


 * I think you've misunderstood the point of Data Transfer - it's for getting in data from any system other than MediaWiki. Here you seem to be using MediaWiki's XML, but Data Transfer has its own XML format that's different. Yaron Koren 15:03, 24 November 2010 (UTC)