Extension talk:Data Transfer

From MediaWiki.org
Jump to: navigation, search
An archive box Archives 

MWException[edit | edit source]


I have previously tried to solve this problem here and it has been suggested that I contact yourselves to see if I can get it resolved, Many thanks,

--Budstellabecks (talk) 15:03, 3 January 2014 (UTC)

My wiki is currently bugged. I think it may have something to do with the same info being submitted twice as a csv import.

As a side effect I'm unable to save pages as well as a few other things.

MediaWiki 1.20.4 PHP 5.3.3 (cgi-fcgi) MySQL 5.0.92-50-log

The error I get using $wgShowExceptionDetails = true; bring up the following,

Thanks in advance for any info

Internal error
Jump to: navigation, search

The Title object did not provide an article ID. Perhaps the page doesn't exist?


#0 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/parser/ParserOutput.php(390): LinksUpdate->__construct(Object(Title), Object(ParserOutput), true)
#1 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/WikiPage.php(1749): ParserOutput->getSecondaryDataUpdates(Object(Title))
#2 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/WikiPage.php(1617): WikiPage->doEditUpdates(Object(Revision), Object(User), Array)
#3 [internal function]: WikiPage->doEdit('{{Individual Mo...', 'Add more months...')
#4 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/Article.php(1820): call_user_func_array(Array, Array)
#5 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/extensions/DataTransfer/includes/DT_ImportJob.php(77): Article->__call('doEdit', Array)
#6 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/extensions/DataTransfer/includes/DT_ImportJob.php(77): Article->doEdit('{{Individual Mo...', 'Add more months...')
#7 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/Wiki.php(594): DTImportJob->run()
#8 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/Wiki.php(556): MediaWiki->doJobs()
#9 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/includes/Wiki.php(447): MediaWiki->restInPeace()
#10 /websites/123reg/LinuxPackage23/yo/ur/si/yoursitehost.co.uk/public_html/partymixers/manager/index.php(59): MediaWiki->run()
#11 {main}
Hi - I would delete the contents of your "job" database table - that should at least take care of these current problems. Then, if you're ready to do the import thing again, it might work better now that you've updated all the software... but if you get the same problems, please let me know. Yaron Koren (talk) 16:07, 3 January 2014 (UTC)

Import CSV error, line break SOLVED[edit | edit source]

In case you use Mac OS X, make sure you save your CSV file with Windows line break format. I have Excel and TextWrangler (just another text editor). I used Excel to edit the CSV file. Then I reopened the file in TextWrangler and save as, specifying that it uses Windows line breaks (CRLF) (as per the CSV specification linked to on the extension page), rather than Classic Mac (CR).

--Tommyheyser (talk) 22:19, 9 March 2014 (UTC)

That's very helpful! It's odd, a linux open source system using the DOS convention, but I suppose they do it deliberately.

In vi you need to type:


Incidentally, this isn't the right place to ask, but, is there a way to have vi instead of the editor mediawiki uses as a standard?

Fustbariclation (talk) 05:13, 10 March 2014 (UTC)

Tommyheyser - thanks for diagnosing that issue; I just added a note about that to the documentation. Fustbariclation - I believe CSV files saved with both Windows and Linux are handled fine; it's only Mac (apparently) that requires special treatment. (It could well be that the Data Transfer code could be modified to handle the Mac style as well; I haven't looked into it.) Yaron Koren (talk) 21:54, 10 March 2014 (UTC)

How to add text at the top the page while keeping current page content, ?[edit | edit source]


Nicolas NALLET (talk)

If you choose the "append" option, it will add content to the bottom of the page; there's no way to add content to the top of the page, as far as I know, but you could probably easily change the "append" behavior in the code. Yaron Koren (talk) 14:40, 25 March 2014 (UTC)
Ok Thanks Yaron it works, in the file DataTransfer/includes/DT_ImportJob.php, line 61

I have replaced

$text = $existingText . "\n" . $text;


$text = $text . "\n" . $existingText;

Nicolas NALLET (talk)

Extension breaking /mw-config/ [SOLVED][edit | edit source]

Hi Yaron,

Firstly, I must say that I really admire your work and use several of your extensions. Bravo!

Two years ago I used DataTransfer successfully on an earlier incarnation of my website. I was then using it with the Semantic bundle which is impressive, but the bundle was unsuited to my needs and I no longer use it.

Today I upgraded my site to Mediawiki 1.24 and coincidentally installed the DataTransfer extension soon afterwards. I have been trying to get Special:ImportCSV and Special:ViewXML to work but without success so far. Both just seem to hang. Import CSV responds with "Importing... 2 pages will be created from the CSV file" but it does not create either of the new pages. Two years ago Special:ImportCSV worked okay.

Later, when I visited the config page - eg. http://www.example.com/mw/mw-config/index.php - I found that DataTransfer did cause an error:

Warning: include_once(/languages/DT_LanguageEn.php) [function.include-once]: failed to open stream: No such file or directory in /home/mysite/public_html/mw/extensions/DataTransfer/DataTransfer.php on line 98

Warning: include_once() [function.include]: Failed opening '/languages/DT_LanguageEn.php' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/mysite/public_html/mw/extensions/DataTransfer/DataTransfer.php on line 98

Fatal error: Class 'DT_LanguageEn' not found in /home/mysite/public_html/mw/extensions/DataTransfer/DataTransfer.php on line 102

I am hopeful that the /mw-config/ related error may in someway be symptomatic of the reason that I cannot get the extension to work.

Thanks in advance for any advice.

Regards, Cruickshanks (talk) 09:20, 26 September 2014 (UTC)

Are you using the latest version of Data Transfer, 0.6? The line numbers make it seem like you're using an older version. Yaron Koren (talk) 12:45, 26 September 2014 (UTC)
Yaron, I have just found that the Download Snapshot, using the Special:ExtensionDistributor for Mediawiki Release 1.23, was downloading Version 0.5. I have now downloaded Version 0.6 using the adjacent (Git master) link in the Infobox. It is still breaking Config however, but with different line numbers now:
Warning: include_once(/languages/DT_LanguageEn.php) [function.include-once]: failed to open stream: No such file or directory in /home/mysite/public_html/mw/extensions/DataTransfer/DataTransfer.php on line 106
Warning: include_once() [function.include]: Failed opening '/languages/DT_LanguageEn.php' for inclusion (include_path='.:/usr/lib/php:/usr/local/lib/php') in /home/mysite/public_html/mw/extensions/DataTransfer/DataTransfer.php on line 106
Fatal error: Class 'DT_LanguageEn' not found in /home/mysite/public_html/mw/extensions/DataTransfer/DataTransfer.php on line 110
Special:ViewXML is now working with Version 0.6 installed. I have not tried ImportCSV as I have a lengthy BOT running at present, but my 'spider senses' guess that ImportCSV will probably work now.
By the way, I incorrectly said in my first post I had upgraded to 1.24; I meant to say I had upgraded to 1.23.4.
Regards, Cruickshanks (talk) 15:43, 26 September 2014 (UTC)
Oh, okay. I looked through my emails and realized that I had a an email discussion with someone about this very same problem a few weeks ago. We weren't able to figure out the underlying cause, but were able to fix it by just commenting out line 77 of DataTransfer.php, which reads "dtfInitContentLanguage($wgLanguageCode);". It's fine to do because the language stuff is only used for a few minor features, like XML grouping, that you're probably not using. Yaron Koren (talk) 16:40, 26 September 2014 (UTC)
Thanks Yaron, that fixed it. Regards, Cruickshanks (talk) 18:49, 26 September 2014 (UTC)

Multi-instance templates not preserved in output[edit | edit source]

If you import a file and update fields in the base template of a page it removes any multi instance templates embedded within and replaces them with the text "Array" this can be solved quite easily in DT_PageStructure.php

public function toWikitext() {
   if ( $this->mIsTemplate ) {
      $wikitext = '{{' . $this->mTemplateName;
      foreach ( $this->mFields as $fieldName => $fieldValue ) {
         if ( is_numeric( $fieldName ) ) {
            $wikitext .= '|' . $fieldValue;
         } else {
//----fix for internal multi-instance templates
            if (is_array($fieldValue)){
               $wikitext .= "\n|$fieldName=";
               foreach ($fieldValue as $arrayItem){
                  if (is_a($arrayItem, 'DTPageComponent')) {
                     $wikitext .= $arrayItem->toWikitext();
                $wikitext .= "\n|$fieldName=$fieldValue";
//--end fix
       $wikitext .= "\n}}";
       return $wikitext;
     } else {
        return $this->mFreeText;

XML Import Only Imports 2 Pages[edit | edit source]


I have an XML file coded to create 80 new pages.

When I try to do the import, I get a message saying:

2 pages will be created from the XML file.

How do I create all 80 pages at once?

Thank you,

--Patricia Barden (talk) 20:37, 6 January 2015 (UTC)

I'm guessing that there's some issue with the XML. Could you pastebin it or something? Yaron Koren (talk) 22:47, 6 January 2015 (UTC)
Here is a link to the XML file on Google Drive: https://drive.google.com/file/d/0BxAhvHwTqRzPTVRjbEFaVTVkRTg/view?pli=1
By the way, I'm using MediaWiki 1.23.6 and Data Transfer 0.6
Thank you so much for taking a look.--Patricia Barden (talk) 02:03, 7 January 2015 (UTC)
Ah, great. I'm guessing the problem is the quotes in the page title of the 3rd page - those need to be escaped, like with \" or something. Yaron Koren (talk) 02:10, 7 January 2015 (UTC)
I escaped the quote marks in the Page Title attribute by using "
E.g., <Page Title="Mary P. &quot;Prissy&quot; Hickerson">
After doing that for all Page Title attributes, the pages were created in a flash.
It was not necessary, however, to escape the quote marks in the Field Name attribute values. This worked just fine:
E.g., <Field Name="Name">Mary P. "Prissy" Hickerson</Field>
Thought you'd like to know. Thank you again! --Patricia Barden (talk) 16:49, 7 January 2015 (UTC)