Help talk:Extension:FileImporter

About this board

Central feedback page for FileImporter/FileExporter.

Forcing Import for duplicate file

3
Locke Cole (talkcontribs)

Not sure if this is beyond the scope of this extension, but would it be possible to detect when someone is attempting to upload an identical file from another Wikimedia project (like English Wikipedia) directly to Commons and change that action into a FileImport automagically? This would help keep revision histories intact, and stop editors who may be deliberately trying to "take credit" for work they didn't do from doing so (bonus points if it detects an attempt to upload a non-current version of an identical file from another project).

Thiemo Kreuz (WMDE) (talkcontribs)

I think this is technically possible. It can probably reuse the technology we can see at the bottom of Commons file description pages where usages in other wikis are listed. What makes it especially complicated is that there are so many ways to upload files. Some UX work is needed as well to let the user know what's going on. The team at Wikimedia Deutschland currently doesn't have the resources to work on this. But it might be wort a feature-request ticket on Phabricator.

Locke Cole (talkcontribs)
Reply to "Forcing Import for duplicate file"
Jonteemil (talkcontribs)
Christoph Jauera (WMDE) (talkcontribs)

Hej @Jonteemil!

There was an issue with FileImporter that triggered that error and might have broken some imports. [1]

It should be fixed now though. If you have a broken import on Commons now, please try to reach a Commons admin and ask for the file to be deleted there so you can import it again.

Thanks!

Reply to "Bug?"

CannotCreateActorException

2
Summary by Thiemo Kreuz (WMDE)
MGA73 (talkcontribs)

Hi! I tried to re-import ja:ファイル:Tojyo-kiha120.JPG so I deleted the file on Commons and tried. I get the message "[81d89d35-b45b-4150-9712-1c6331024e55] 2022-05-18 17:32:23: Fatal exception of type "CannotCreateActorException"". Any idea why? (I restored the file on Commons again so you will get a dupe warning if you try now. You can just revert on ja.wiki and try).

Jeff G. (talkcontribs)
Reply to "CannotCreateActorException"
Summary by Thiemo Kreuz (WMDE)

Added to the config page.

Silraks (talkcontribs)

Maybe I am missing something, but I think the license cc-by-sa-4.0 is allowed on Commons.

If that is so, then it seems FileImporter does not recognize it properly and does not allow importing of such images.


Maybe it does not affect all images with the license, but I had trouble with this file (and the reason of a incompatible license being given)

Thiemo Kreuz (WMDE) (talkcontribs)

One file revision returns a 404 error

4
Stefan2 (talkcontribs)

The first revision of w:File:Almakerek6.jpg returns a 404 error. It is possible to start the transfer process and I was able to press the edit button to add c:Category:Lutheran church in Mălâncrav-interior to the file information page, but when I press the final transfer button, the transfer fails because the tool can't find the file.

If a file can't be transferred, please warn before the user starts cleaning up the file information page so that time isn't wasted on this if the cleaned up information can't be used in the end. Also, please try to find a way to transfer files like this while preserving the upload log. If the file is gone from the server, then it can't be uploaded to Commons, but it's still useful to see that there was a file on Wikipedia at one point.

Jeff G. (talkcontribs)
Johanna Strodt (WMDE) (talkcontribs)

Thanks for reporting this issue! I've passed it on to our engineering team so they can have a look at it.

Thiemo Kreuz (WMDE) (talkcontribs)

There are many things that can go wrong during a file transfer. Most of them are checked before. But this is not always possible. We can't, for example, download megabytes worth of files just to check if it will work.

It might be possible to add a validation step that does a quick HEAD check for all file revisions. This will catch the described edge case. However:

  • What to do then? We don't want to skip broken revisions without leaving a trace, nor do we want to create broken database entries.
  • Implementing this takes up resources that can't be invested otherwise. Is this worth it? How many files like this exist?

In a situation like this the best solution might be to use one of the older transfer tools.

Reply to "One file revision returns a 404 error"

FileImporter is not recognizing {{PD-textlogo}} from Wikiversity

3
Ixfd64 (talkcontribs)

I tried to import a file tagged with {{PD-textlogo}} on the English Wikiversity, but FileImporter does not recognize it as having a valid license.

Tacsipacsi (talkcontribs)
Ixfd64 (talkcontribs)

Thanks. I've added the template.

Reply to "FileImporter is not recognizing {{PD-textlogo}} from Wikiversity"

FileImporter is not showing on all files on fa.wiki

21
MGA73 (talkcontribs)

Is it my settings that is messed up or is there something strange going on at fa.wiki?

On fa:File:Ertn.png the export tab does not show up unless I click edit (then it shows up).

On fa:File:Alidivandari-photo1.jpg the export tab shows up both with and without edit.

I tried both with and without "Use Legacy Vector".

Thiemo Kreuz (WMDE) (talkcontribs)

I had a look but don't understand what's going on. A bunch of criteria must be met, but they all are: You need to be "autoconfirmed". The file must be local (i.e. not already on Commons). For me, the button is always there. Might be some kind of caching issue?

Stefan2 (talkcontribs)

On fa:File:Ertn.png with "Use Legacy Vector", I see the "Export to Commons" link for half a second and then it disappears. The link also shows up for half a second if I use useskin=monobook. It doesn't disappear if I click on edit.

If I disable javascript (setting javascript.enabled in Firefox about:config), then I see the "Export to Commons" link properly.

Tacsipacsi (talkcontribs)
Jeff G. (talkcontribs)
MGA73 (talkcontribs)

When I click the link User:Jeff G. mention (safemode=1) then I see the tab.

Thiemo Kreuz (WMDE) (talkcontribs)

That helps a lot already. Thank you! Unfortunately I'm unable to do more tests myself. I don't have access to an account that is autoconfirmed on fawiki. Since MGA73 apparently doesn't have any custom JavaScript running, I believe either one of the gadgets or Beta features is causing the issue. Can one of you do the following test?

Try to do the same with the Beta features and let us know. Thanks!

MGA73 (talkcontribs)

I had no beta and even if I disable all gadgets it still do not work.

Tacsipacsi (talkcontribs)

I found it in Common.js (added by Huji), and it seems to be intentional: it’s hidden for non-free files. While I understand the intent, it turned out to be confusing, so maybe another approach should be considered (e.g. greying out the tab), or simply FileImporter should be set up in a way that it doesn’t allow importing non-free files.

MGA73 (talkcontribs)

FileImporter can allready prevent you from moving "bad" files. However it was not set up to do block non-free files. But since no non-free licenses is added it can't move then.

But I like the idea that FileImporter is grey if FileImporter can't move the file.

Thiemo Kreuz (WMDE) (talkcontribs)

Thanks for figuring this out.

I suggest to not remove or disable the button. Being able to click it is helpful: It gives the user an explanation. Hiding the button is just confusing, as we have seen.

MGA73 (talkcontribs)

I have added the category for non-free content but not the two other. I agree that files normally need a source and author but if it is a very old file or it is PD-ineligible then we do not need it. Or perhaps someone want to move the file to Commons and fix it there. Personally I find it very hard to edit on pages that do not edit left-to-right so I usually move to Commons and edit there.

Stefan2 (talkcontribs)

I think that it only is a waste of time to clean up a page on the source wiki since it's just going to be deleted shortly after the transfer. Say a PD-textlogo is tagged as w:Template:Non-free logo and you wish to move it to Commons. You need a correct file information page on Commons, but there is no need for an adjusted file information page on Wikipedia as it will simply be deleted shortly after it's transferred. It's much faster to clean it up on Commons only than to first clean it up Wikipedia and then clean it up again on Commons because the transfer tool doesn't take care of everything which need to be done on Commons. If files without source or author can't be moved without adding a pro forma template, this just increases the time to transfer files to Commons.

MGA73 (talkcontribs)

Good example Stefan. But in this case we have to balance between saving time and prevent users without knowledge of copyright to move lots of copyvios to Commons. I'm afraid that if we make it too easy then we will flood in bad files.

Huji (talkcontribs)
MGA73 (talkcontribs)

@Huji: Perhaps that will be confusing too. Now users can see the tab and click it but nothing happens. If you do not disable it and users click they will get a message like this:

"This file cannot be transferred to Wikimedia Commons, because it is marked as رده:تصویرهای با منبع نامعلوم. Wikimedia Commons does not allow such files. This might be resolvable, but most probably means the file is not compatible. Please consult the Wikimedia Commons community policy and talk pages about licensing."

That way users will be told what the problem is and they will have an idea how to fix it. Is there a way to make the tab grey but still allow users to click it?

Huji (talkcontribs)

@MGA73: done in fa:Special:Diff/32777121. Can you point me to where the ineligible category names are stored? I want to make sure the list is complete.

MGA73 (talkcontribs)

@Huji: The configuration is done at Extension:FileImporter/Data/fa.wikipedia.

You can add both bad categories and bad templates. If they are on the file FileImporter will refuse to move the file.

You can also add good templates. FileImporter will only move the file if at least 1 good template is found.

Jeff G. (talkcontribs)

@MGA73: Is it fair to assume that anything bad overrides any good templates?

MGA73 (talkcontribs)

Jeff Yes a bad category or template means full stop. End of game.

Huji (talkcontribs)

@Jeff G.: the logic seems to be defined here and the functions called there are defined here. The page must have at least one good template AND no bad templates AND no exclusionary categories.

Reply to "FileImporter is not showing on all files on fa.wiki"

Does FileImporter have API?

5
Edgars2007 (talkcontribs)

hi! so, yeah... what the title says :) I assume the answer is no, then next question is - are there any chance it is in plans? i assume, answer is also no (didn't find anything about this on Phab) :(

some time ago (i think then Fileimporter was in early stages or even before that) i was starting to build a tool for moving images (basically was trying to replicate functionality of FtCG in web). having fileimporter as API would save some time :)

Tacsipacsi (talkcontribs)

If there’s no Phabricator ticket for it, chances are pretty low that it will ever get implemented. If you create one, they’re a bit higher. 😉 (And if you write the API, they’re even higher.)

Thiemo Kreuz (WMDE) (talkcontribs)

I wonder what the benefit of such an API would be, in addition to what we currently have? I mean, it's not like one could run a bot that blindly moves an entire File namespace from one wiki to another. FileImporter is not a bot but an interactive tool for a reason. One would need to re-implement pretty much everything FileImporter does, e.g.:

  • Show all possible kinds of error messages.
  • Let the user edit bad or conflicting file names.
  • Ask the user to review the automatic edits made to the file description.
  • Let the user edit and e.g. translate the file description.
  • Direct the user to the target wiki to e.g. categorize the file properly.

We would like to understand the problem you want to solve. What do you try to do with your tool that can't be done with FileImporter?

Edgars2007 (talkcontribs)

"bot that blindly moves an entire File namespace from one wiki to another" - no, that's not on the table or something i'm trying to propose. but i do understand, that this concern would become reality no matter of my or your intentions.

well, why developers provide an API to their services? so that other developers can build some apps on top of that.

about the "what's the problem" part - currently one can mostly do it "randomly" (via articles) or going through some category, where there isn't guarantee, that user won't be presented with files, that somebody has already looked through or otherwise clutter not-yet-looked-through-images. ok, you can so some fancy things (intersections and so on) with Petscan, but I would bet, that many, many users don't know about that. And it happens in "old fashion" way - new filepage is opened in new tab/window etc.

the "proposed solution" (the tool) would contain list of images, that should be good to ex(im)port and presented in SPA-like environment, so that user won't need to go outside tool to make actions. mainly - workflow improvements for power users. something like this.

Thiemo Kreuz (WMDE) (talkcontribs)
Reply to "Does FileImporter have API?"

FileImporter can't move files with broken versions (and broken versions can't be deleted)

4
Summary by Thiemo Kreuz (WMDE)
MGA73 (talkcontribs)

Hi! FileImporter can't import files with broken versions. That have been partly discussed in one of the posts below. One solution could be to delete the broken versions but as reported in T244567 that is not always possible. Instead of trying to fix FileImporter would it be possible to assign someone to fix the problem that prevents us from deleting files?

Thiemo Kreuz (WMDE) (talkcontribs)

This might be one of the situations where properly fixing the issue would consume way more resources than manually fixing it when it happens. Do you have a recent example? How often have you seen this error so far?

MGA73 (talkcontribs)
MGA73 (talkcontribs)

I changed the title of this topic. I just realised that it did not make any sense. The point is that FileImporter can't MOVE files if there are broken versions. The easiest way to fix that is probably to fix the error that prevent us from deleting and restoring files mentioned in T244567. Hope it makes more sense now :-)

Reply to "FileImporter can't move files with broken versions (and broken versions can't be deleted)"

Importing files from the source wiki te.wikipedia.org

5
Wouterhagens (talkcontribs)

I have asked this in the Village pump and there it was suggested to ask it here. I want to added images from the te.wikipedia.org to a Commons category, but got the response: "Unfortunately, importing files from the source wiki (te.wikipedia.org) is not yet possible because there is no configuration for the wiki in the configuration list". Is there anyone who can make this possible? ~~~~

Michael Schönitzer (WMDE) (talkcontribs)

Hi, you can make te.wikipedia.org supported by creating an configuration "file" (a wiki site) here: Extension:FileImporter/List of configured wikis. The easiest is to copy-paste the configuration from an other project and then change the content where necessary. The configuration states how templates are to be replaced and what license-templates are compatible with Commons.

The format of the configuration is described here.

Wouterhagens (talkcontribs)

Thanks for the info. The problem is that I don't know the Telugu language to change the content where necessary in the copy-paste version.

Jeff G. (talkcontribs)
MGA73 (talkcontribs)
Reply to "Importing files from the source wiki te.wikipedia.org"