Help talk:Extension:FileImporter/2021

Central feedback page for FileImporter/FileExporter.

Importing files from the source wiki te.wikipedia.org

I have asked this in the Village pump [1] and there it was suggested to ask it here. I want to added images from the te.wikipedia.org to a Commons category, but got the response: "Unfortunately, importing files from the source wiki (te.wikipedia.org) is not yet possible because there is no configuration for the wiki in the configuration list". Is there anyone who can make this possible? ~ Wouterhagens (talk) 21:14, 11 January 2021 (UTC)

Hi, you can make te.wikipedia.org supported by creating an configuration "file" (a wiki site) here: Extension:FileImporter/List of configured wikis. The easiest is to copy-paste the configuration from an other project and then change the content where necessary. The configuration states how templates are to be replaced and what license-templates are compatible with Commons.

The format of the configuration is described here. Michael Schönitzer (WMDE) (talk) 21:19, 11 January 2021 (UTC)

Thanks for the info. The problem is that I don't know the Telugu language to change the content where necessary in the copy-paste version. Wouterhagens (talk) 09:43, 12 January 2021 (UTC)

@వైజాసత్య and Ranjith Sutari: Would one of you please help with this?   — Jeff G. ツ 15:20, 13 January 2021 (UTC)

I have made a configuration page on Extension:FileImporter/Data/te.wikipedia but local users should help improve it. MGA73 (talk) 17:11, 5 February 2021 (UTC)

FileImporter can't move files with broken versions (and broken versions can't be deleted)

Tracked in Phabricator
Task T244567

Hi! FileImporter can't import files with broken versions. That have been partly discussed in one of the posts below. One solution could be to delete the broken versions but as reported in T244567 that is not always possible. Instead of trying to fix FileImporter would it be possible to assign someone to fix the problem that prevents us from deleting files? MGA73 (talk) 16:52, 5 February 2021 (UTC)

This might be one of the situations where properly fixing the issue would consume way more resources than manually fixing it when it happens. Do you have a recent example? How often have you seen this error so far? Thiemo Kreuz (WMDE) 18:06, 5 February 2021 (UTC)

Hi! The files in ja:Category:FileImporter_error, en:Category:Wikipedia files that cannot be deleted and in c:Category:Deletion error/T244567 seems to have that problem. MGA73 (talk) 21:42, 5 February 2021 (UTC)

I changed the title of this topic. I just realised that it did not make any sense. The point is that FileImporter can't MOVE files if there are broken versions. The easiest way to fix that is probably to fix the error that prevent us from deleting and restoring files mentioned in T244567. Hope it makes more sense now :-) MGA73 (talk) 09:58, 3 April 2021 (UTC)

Does FileImporter have API?

hi! so, yeah... what the title says :) I assume the answer is no, then next question is - are there any chance it is in plans? i assume, answer is also no (didn't find anything about this on Phab) :(

some time ago (i think then Fileimporter was in early stages or even before that) i was starting to build a tool for moving images (basically was trying to replicate functionality of FtCG in web). having fileimporter as API would save some time :) Edgars2007 (talk) 19:27, 4 April 2021 (UTC)

If there’s no Phabricator ticket for it, chances are pretty low that it will ever get implemented. If you create one, they’re a bit higher. 😉 (And if you write the API, they’re even higher.) Tacsipacsi (talk) 00:47, 5 April 2021 (UTC)

I wonder what the benefit of such an API would be, in addition to what we currently have? I mean, it's not like one could run a bot that blindly moves an entire File namespace from one wiki to another. FileImporter is not a bot but an interactive tool for a reason. One would need to re-implement pretty much everything FileImporter does, e.g.:

• Show all possible kinds of error messages.
• Let the user edit bad or conflicting file names.
• Ask the user to review the automatic edits made to the file description.
• Let the user edit and e.g. translate the file description.
• Direct the user to the target wiki to e.g. categorize the file properly.

We would like to understand the problem you want to solve. What do you try to do with your tool that can't be done with FileImporter? Thiemo Kreuz (WMDE) 08:08, 6 April 2021 (UTC)

"bot that blindly moves an entire File namespace from one wiki to another" - no, that's not on the table or something i'm trying to propose. but i do understand, that this concern would become reality no matter of my or your intentions.

well, why developers provide an API to their services? so that other developers can build some apps on top of that.

about the "what's the problem" part - currently one can mostly do it "randomly" (via articles) or going through some category, where there isn't guarantee, that user won't be presented with files, that somebody has already looked through or otherwise clutter not-yet-looked-through-images. ok, you can so some fancy things (intersections and so on) with Petscan, but I would bet, that many, many users don't know about that. And it happens in "old fashion" way - new filepage is opened in new tab/window etc.

the "proposed solution" (the tool) would contain list of images, that should be good to ex(im)port and presented in SPA-like environment, so that user won't need to go outside tool to make actions. mainly - workflow improvements for power users. something like this. Edgars2007 (talk) 16:08, 19 April 2021 (UTC)

As said, you would need to re-implement pretty much the entirety of what the FileImporter extension currently does.

Can you talk a bit more about what in the workflow of FileImporter is currently not suitable for power users?

If you have your list of images you can have direct links to e.g. https://commons.wikimedia.org/wiki/Special:ImportFile?useskin=minerva&clientUrl=… and possibly open these in an iframe. Thiemo Kreuz (WMDE) 11:39, 21 April 2021 (UTC)

FileImporter is not showing on all files on fa.wiki

Is it my settings that is messed up or is there something strange going on at fa.wiki?

On fa:File:Ertn.png the export tab does not show up unless I click edit (then it shows up).

On fa:File:Alidivandari-photo1.jpg the export tab shows up both with and without edit.

I tried both with and without "Use Legacy Vector". MGA73 (talk) 15:20, 4 August 2021 (UTC)

I had a look but don't understand what's going on. A bunch of criteria must be met, but they all are: You need to be "autoconfirmed". The file must be local (i.e. not already on Commons). For me, the button is always there. Might be some kind of caching issue? Thiemo Kreuz (WMDE) 15:42, 4 August 2021 (UTC)

On fa:File:Ertn.png with "Use Legacy Vector", I see the "Export to Commons" link for half a second and then it disappears. The link also shows up for half a second if I use useskin=monobook. It doesn't disappear if I click on edit.

If I disable javascript (setting javascript.enabled in Firefox about:config), then I see the "Export to Commons" link properly. Stefan2 (talk) 22:22, 4 August 2021 (UTC)

Does it occur also in safe mode, i.e. on https://fa.wikipedia.org/wiki/%D9%BE%D8%B1%D9%88%D9%86%D8%AF%D9%87:Ertn.png?safemode=1 (with JS enabled)? If it doesn’t, the bug should be in a sitewide script, a gadget, or your m:User:Stefan2/global.js file. Tacsipacsi (talk) 22:32, 4 August 2021 (UTC)

My experience is the same as that of MGA73 above, but Tacsipacsi the tab does show on https://fa.wikipedia.org/wiki/%D9%BE%D8%B1%D9%88%D9%86%D8%AF%D9%87:Ertn.png?safemode=1 (with JS enabled). I am autoconfirmed on fawiki, I use the MonoBook skin, and my scripts are publicly available.   — Jeff G. ツ 23:04, 4 August 2021 (UTC)

When I click the link User:Jeff G. mention (safemode=1) then I see the tab. MGA73 (talk) 06:36, 5 August 2021 (UTC)

That helps a lot already. Thank you! Unfortunately I'm unable to do more tests myself. I don't have access to an account that is autoconfirmed on fawiki. Since MGA73 apparently doesn't have any custom JavaScript running, I believe either one of the gadgets or Beta features is causing the issue. Can one of you do the following test?

• Open fa:File:Ertn.png in one tab.
• Open fa:Special:Preferences#mw-prefsection-gadgets in another tab.
• Disable 1 gadget at a time, save your preferences, and see what happens when you reload the 1st tab.
• If nothing changed, enable the gadget again, and disable the next one.

Try to do the same with the Beta features and let us know. Thanks! Thiemo Kreuz (WMDE) 07:10, 5 August 2021 (UTC)

I had no beta and even if I disable all gadgets it still do not work. MGA73 (talk) 13:50, 5 August 2021 (UTC)

I found it in Common.js (added by Huji), and it seems to be intentional: it’s hidden for non-free files. While I understand the intent, it turned out to be confusing, so maybe another approach should be considered (e.g. greying out the tab), or simply FileImporter should be set up in a way that it doesn’t allow importing non-free files. Tacsipacsi (talk) 13:46, 5 August 2021 (UTC)

FileImporter can allready prevent you from moving "bad" files. However it was not set up to do block non-free files. But since no non-free licenses is added it can't move then.

But I like the idea that FileImporter is grey if FileImporter can't move the file. MGA73 (talk) 13:51, 5 August 2021 (UTC)

Thanks for figuring this out.

• The more recent edit is this one. Three categories are excluded: non-free content, images with unknown source, and images with unknown creator.
• FileImporter already checks for non-free files and doesn't allow moving them. However, for performance reasons this check is not done on the source wiki. The export button must be clicked to get told if the file can be exported.
• Unfortunately, the relevant configuration page for fawiki is was incomplete. The categories need to be added to the "bad categories" section.

I suggest to not remove or disable the button. Being able to click it is helpful: It gives the user an explanation. Hiding the button is just confusing, as we have seen. Thiemo Kreuz (WMDE) 14:07, 5 August 2021 (UTC)

I have added the category for non-free content but not the two other. I agree that files normally need a source and author but if it is a very old file or it is PD-ineligible then we do not need it. Or perhaps someone want to move the file to Commons and fix it there. Personally I find it very hard to edit on pages that do not edit left-to-right so I usually move to Commons and edit there. MGA73 (talk) 14:14, 5 August 2021 (UTC)

I think that it only is a waste of time to clean up a page on the source wiki since it's just going to be deleted shortly after the transfer. Say a PD-textlogo is tagged as w:Template:Non-free logo and you wish to move it to Commons. You need a correct file information page on Commons, but there is no need for an adjusted file information page on Wikipedia as it will simply be deleted shortly after it's transferred. It's much faster to clean it up on Commons only than to first clean it up Wikipedia and then clean it up again on Commons because the transfer tool doesn't take care of everything which need to be done on Commons. If files without source or author can't be moved without adding a pro forma template, this just increases the time to transfer files to Commons. Stefan2 (talk) 14:28, 5 August 2021 (UTC)

Good example Stefan. But in this case we have to balance between saving time and prevent users without knowledge of copyright to move lots of copyvios to Commons. I'm afraid that if we make it too easy then we will flood in bad files. MGA73 (talk) 14:32, 5 August 2021 (UTC)

@MGA73 and Tacsipacsi: I liked the idea of disabling rather than hiding the tab, so I implemented it in fa:Special:Diff/32772652 Huji (talk) 23:21, 5 August 2021 (UTC)

@Huji: Perhaps that will be confusing too. Now users can see the tab and click it but nothing happens. If you do not disable it and users click they will get a message like this:

"This file cannot be transferred to Wikimedia Commons, because it is marked as رده:تصویرهای با منبع نامعلوم. Wikimedia Commons does not allow such files. This might be resolvable, but most probably means the file is not compatible. Please consult the Wikimedia Commons community policy and talk pages about licensing."

That way users will be told what the problem is and they will have an idea how to fix it. Is there a way to make the tab grey but still allow users to click it? MGA73 (talk) 08:40, 6 August 2021 (UTC)

@MGA73: done in fa:Special:Diff/32777121.

Can you point me to where the ineligible category names are stored? I want to make sure the list is complete. Huji (talk) 17:04, 6 August 2021 (UTC)

@Huji: The configuration is done at Extension:FileImporter/Data/fa.wikipedia.

You can add both bad categories and bad templates. If they are on the file FileImporter will refuse to move the file.

You can also add good templates. FileImporter will only move the file if at least 1 good template is found. MGA73 (talk) 17:16, 6 August 2021 (UTC)

@MGA73: Is it fair to assume that anything bad overrides any good templates?   — Jeff G. ツ 11:55, 7 August 2021 (UTC)

Jeff Yes a bad category or template means full stop. End of game. MGA73 (talk) 12:57, 7 August 2021 (UTC)

@Jeff G.: the logic seems to be defined here and the functions called there are defined here. The page must have at least one good template AND no bad templates AND no exclusionary categories. Huji (talk) 12:10, 7 August 2021 (UTC)

FileImporter is not recognizing Template:PD-textlogo from Wikiversity

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

I tried to import a file tagged with {{PD-textlogo}} on the English Wikiversity to Commons, but FileImporter does not recognize it as having a valid license. Ixfd64 (talk) 19:04, 20 August 2021 (UTC)

Extension:FileImporter/Data/en.wikiversity#Good lists the known good licenses, you can add any missing ones there. Tacsipacsi (talk) 20:21, 20 August 2021 (UTC)

Thanks. I've added the template. Ixfd64 (talk) 20:27, 20 August 2021 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

One file revision returns a 404 error

The first revision of w:File:Almakerek6.jpg returns a 404 error. It is possible to start the transfer process and I was able to press the edit button to add c:Category:Lutheran church in Mălâncrav-interior to the file information page, but when I press the final transfer button, the transfer fails because the tool can't find the file.

If a file can't be transferred, please warn before the user starts cleaning up the file information page so that time isn't wasted on this if the cleaned up information can't be used in the end. Also, please try to find a way to transfer files like this while preserving the upload log. If the file is gone from the server, then it can't be uploaded to Commons, but it's still useful to see that there was a file on Wikipedia at one point. Stefan2 (talk) 17:54, 10 October 2021 (UTC)

@Stefan2: This appears to be the result of a database error in which https://upload.wikimedia.org/wikipedia/en/archive/4/49/20070817224239%21Almakerek6.jpg and https://upload.wikimedia.org/wikipedia/en/thumb/archive/4/49/20070817224239%21Almakerek6.jpg/120px-Almakerek6.jpg (and other thumbnail sizes) are missing.   — Jeff G. ツ 18:35, 10 October 2021 (UTC)

Thanks for reporting this issue! I've passed it on to our engineering team so they can have a look at it. Johanna Strodt (WMDE) (talk) 12:19, 12 October 2021 (UTC)

There are many things that can go wrong during a file transfer. Most of them are checked before. But this is not always possible. We can't, for example, download megabytes worth of files just to check if it will work.

It might be possible to add a validation step that does a quick HEAD check for all file revisions. This will catch the described edge case. However:

• What to do then? We don't want to skip broken revisions without leaving a trace, nor do we want to create broken database entries.
• Implementing this takes up resources that can't be invested otherwise. Is this worth it? How many files like this exist?

In a situation like this the best solution might be to use one of the older transfer tools. Thiemo Kreuz (WMDE) 13:13, 19 October 2021 (UTC)