Help talk:Extension:FileImporter

Central feedback page for FileImporter/FileExporter.

• 1) The import log should be added to the file info as a separate section or Information field. The information regarding to Who moved the file, from which project and date of move. Now it look like the file uploaded directly to Commons by original uploader. Some info must be given her.

• 2) If one of the imported files nominated for deletion. The original uploader gets notification in Commons. The user that moved the file does not get any notification. This is not healthy. The user that moved the file to Commons is the real uploader of the file to Commons. He must be notified as well. And also. The discussion about the deletion will be in Commons. Might be some problems with that. For example, part of the users from he.wiki active only in he.wiki and uploading files to he.wiki. Involving in DR discussion might be a problem for them. Language problem, not familiar enough with Commos rules etc. I have no instant idea to solve it but we must give it a thought.

• 3) I don't think it is correct to link the original uploader in Commons. See the screenshot of the file I have moved from he.wiki. See A in the field of the author I added the template: Commons Template user at project and also B Should link to the author in his own wiki - the wiki he uploaded the file firstly.

• 2) Any example for your claim? I know only a bug in the Commons mobile App phab:T232142

phab:T232142 is not about FileImporter

Yes it is more general. You are right, as you described the problem more detailed on Commons.

I created example personal template for files I imported from he.wiki https://commons.wikimedia.org/wiki/File:Shlomo_lorentz.jpeg I think that this kind of template whan importing, would be useful.

• 1) It's somewhat similar to moving a file within a wiki. Then file also wasn't uploaded directly to new location. I think it makes sense that import information is kept in page history and logs along with other information on who did what to a file. While the situation that file version wasn't uploaded directly to Commons is relatively new, it has been the case for page revision imports for quite some time that author necessarily didn't edit on the very same wiki. I think we'll get used to it regarding file versions too. :)
• 2) Notifying the importer might be something for maintainers of the gadget behind "Nominate for deletion" sidebar link on Commons to consider. Discussion being on Commons and possible language problem concern earlier files moved to Commons, too. Having FileImporter and cross-wiki notifications I'd say is definitely an improvement in this matter as it's easier to notify the original uploader. Wiki communities should probably give thought on how to handle these discussions better and then see if this leads to some specific technical feature request.
• 3) File version being attributed to local user on Commons to my understanding is deliberate. T209346 has some of the background. Editable links, including links to user pages, were already set to point back to source wiki automatically (T198584). As for "user at project" template I know that some people use it, but I've never felt comfortable with it. After all, author field is where author provides proper attribution in accordance with licensing terms, and we probably shouldn't alter this by adding something like "from Wikipedia" to it. Using this template probably isn't straightforward enough to include it for all FileImporter imports.

Hi, I'm curious to learn more on the decision not to transfer categories. With Wikidata, matching most categories that exist both on source and destination wiki should be fairly simple. Is there any specific reason why you're reluctant to use this method?

Hi, thank you on the input! We decided not to do this for the first version, since it wasn't foreseeable how hard this would actually be to implement and we'd rather have FileImporter out of beta without that feature now, than with that feature in several moths. That doesn't mean the feature can't be added in a future version. There are lot's of things that could be done to make moving files even easier, but we have to also consider both: the resource effort needed to implement and maintain in the future and the benefit for the users. We always try to focus on the things for which this ratio is best.

Make it non-beta. ~~~~

Looks like it is going to be a default feature in first wikis. See Help:Extension:FileImporter#Deployment_roadmap.

It will soon go out of beta. Thanks for the feedback!

When exactly?

We don't have an exact date yet. We hope that we can go out-of-beta in the first few wikis (de, fa, ar, ko, wikisourceswiki) still in August and then follow with the all other wikis in September. But as always it depends on whether we still find any important bugs or are blocked by anything.

Tracked in Phabricator
Task T222445

Is there a way for a local project to opt out of FileImporter? Some wiki project(s) are too laxed on copyright fraud. I believe FileImporter should be turned off for these projects with or without a local consensus.

I wonder under which circumstances the limitations encoded in the configuration pages at Extension:FileImporter/Data aren't enough to prevent unwanted uploads from happening? Can you please provide one or more specific examples of such problematic files?

It's not the problem of the config. Some wikis have very poor oversight of local files that copyvios are often tagged with free licences. I think such wikis should have FileImporter turned off.

As far as I know, on Commons, there is currently no tool to identify all files imported from a particular wiki. They are not put into a single category either.

I would like to rephrase my question and ask you to provide more information that allow us to understand the issue better. It appears you are talking about one specific wiki. Which wiki is that? Can you please provide examples of files that have been imported to Commons, but shouldn't? It sounds like the issue are different community standards. It even sounds like the community you are referring to is willingly violating standards that exist on all Wikimedia wikis. Is this the case? If so, wouldn't be the Wikimedia Foundations legal team in charge?

Would it help if FileImporter sets a category to mark the source of an import?

@Thiemo Kreuz (WMDE): "FileImporter sets a category to mark the source of an import" would definitely help.

The question remains, I'm afraid: help with what? Please provide all information you have to enable us to help.

@Thiemo Kreuz (WMDE): Help with copyright enforcement. Two projects that come to mind as potentially problematic with respect to copyrights are Arabic Wikipedia and Arabic Wikisource. We had a declined bot request for transferring files from those two projects to Commons early last year, which revealed the following quotes: "Different Wikipedias have different attitude toward copyrights and its enforcement." by EugeneZelenko; "Seems to be an overall lack of consensus on Arabian Wikipedia about the process for how this bot would work. One commented that they don't trust Commons because there files got deleted for no convincing reason, another mentioned copyright issues and perhaps images not being transferable due to licenses." by ~riley; and my "I have issues with whoever is selecting these files for transfer, and that person's attention to detail (or lack thereof)." 60% of the files selected as examples for transfer (which should have been the best ones) had copyright problems which made them unsuitable for Commons or any WMF project. Admittedly, five is a very small sample size, but 60% is a huge error rate.

Thanks. I created phab:T222445 to help us keep track of this request. It would be great if you could provide specific examples here or in the ticket.

I think it's reasonable request. Other factor which should be taken in account is availability of active administrators on Commons who knows particular language well, better because communication mat lead to better local review before enabling transfer or just reduce misunderstandings between Commons and other projects.

Actually, to turn it off is pretty simple: remove/comment out all acceptable templates in the config, and protect the config. What is missing is a formal process. The best is of course if the local wiki has a consensus of disabling it, but in the event of no consensus, or if small wikis insist on enabling it but do nothing to tackle problematically licensed files, the larger Commons community should be allowed to initiate/impose an opt-out on such small wikis, since FileImporter involves both the local wikis and Commons. meta:Non-free_content is a not so up-to-date monitoring of local files.

On the other hand, a maintenance category akin to c:Category:Files moved to Commons requiring review by source would be very useful. Currently a line of the format <!--This file was moved here using FileImporter from //xxx.wikipedia.org/wiki/File:Example.jpg--> is automatically prepended for each import. We'll just need to append for example a line [[Category:Files moved from xxx.wikiproject to Commons requiring review]], or a remodeled c:Template:BotMoveToCommons.

meta:Non-free_content/taking_stock#List_of_Wikipedias_having_local_media_files is the actual monitoring of files on each local wiki. It's not regularly updated.

It's reasonable to have transfer disabled by default and enable it after formal review. Historically I could refer to bot-assisted transfer organized in Polish Wikipedia where several Commons administrators were actively involved. Russian Wikipedia changed attitude toward copyrights and started to review files scrupulously ~ 4th year of existence.

Tracked in Phabricator
Task T213955

At Extension:FileImporter/Data/fa.wikipedia#Transfer, I have defined "اجازه‌نامه" and "اجازه نامه" and "اجازه_نامه" as "Permission" when converting local template to Commons template. However, only the last definition works. The first one contains ZWNJ and the second one a whitespace.

Here is an example: fa:پرونده:Woman with pears.jpg

Thanks for reporting this extra example. Alternative source parameter names for the same target parameter name are currently not supported, see phab:T213955. You can keep the extra parameters on the config page. Just make sure the most important one is listed last.

Thanks, I created a phabricator-ticket for this, so that our developers can look into it: T221959

[Update: I Answered simultaneously to Thiemo… See his answer]

I'm not able to import files with OTRS tickets from Persian Wikipedia (fa:رده:کارهای تایید شده توسط OTRS).

That's because you're not a global OTRS member. Yet. Please see m:OTRS/Volunteering.

Thanks.

You're welcome. I imported File:Raami-011.jpg, but I couldn't access the tickets of the rest.

Thank you very much. I will ask on Persian WIkipedia for help.

You're welcome.

It's up to the communities to decide which templates they put on the white-list. You can change the whitelists and blacklists here.

Currently images licensed as PD-Australia are rejected by the FileImporter but PD-Australia images are allowed on Commons.

Oppose, files with such tags on enwiki may still be subject to US copyright due to URAA and therefore are not automatically hostable on Commons. See also c:Commons:Village pump#"Export to Wikimedia Commons" from en.Wikipedia - licence problem.

The situations where there's multiple versions and a decent size to them are pretty much the exact situations where having bot-assisted movement is most useful. It might be mildly annoying to move a single 75 KB image over. It's bloody awful to move 12 images of 36 megabytes each. The maximum move size needs pushed WAY up.

As to how this situation arose: Long story short, I was doing a difficult restoration while copyright status was being worked out on Commons. I uploaded partially-done files as I went. Consensus came down in favour of Commons hosting the file, as the only thing that would give doubt to the status (a partial attribution in an unreliable source to a name that does not appear to have any other known works discoverable, nor any reliable sources that even cover his or her existence) was deemed trumped by the very reliable source (the BnF) that said it was both anonymous and public domain.

Oh, there's also a PNG version of the file. 6 files in the history, 87 megabytes each. What IS the limit, anyway?

Can you please provide a link to the specific file you are trying to import?

The current limits are 100 revisions, and 250 megabytes.

Sure, it's https://en.wikipedia.org/wiki/File:Cavalleria_Rusticana_-_Santuzza_and_Turiddu_outside_the_church.jpg and https://en.wikipedia.org/wiki/File:Cavalleria_Rusticana_-_Santuzza_and_Turiddu_outside_the_church.png

I realise there will probably be secondary blocks in the license check - En-wiki does not have the most relevant license, and I'd like to keep a local copy there anyway, just in case it turns out the unreliable source was somehow both correct and that it makes any difference to status - but it's not too hard to work around that.

The commons discussion as to why it's fine for there is at https://commons.wikimedia.org/wiki/Commons:Village_pump/Copyright#Unidentified_artist

And the end licensing should be as in https://commons.wikimedia.org/wiki/File:Cavalleria_Rusticana_-_Alfio_and_Turiddu_embrace.jpg

The file exporter essentially "copies" the original history from localwiki to commons. Hence (even though the original page will eventually be deleted) it duplicates contribs. Can an option be made, where a file can be "moved" in it's entirety from localwiki to another (thus no contribs remains on localwiki)? Is it possible to enable that function at least to power users (i.e. admins on one or both projects)?

Thanks a lot for the suggestion. It would certainly be nice to provide this as an option. Unfortunately, it is technically not possible. To explain this briefly: The source and target wikis run on two independent databases where page and revision IDs refer to different pages. It is technically necessary to create a new page on the target wiki.

Maybe phab:T196735 is an option for your use case?

Thank you for your reply, Thiemo. T196735 speaks about deletion of the original file, thus there is still some "duplicate" edit created, before being deleted for most users (users with privileged access can permanently see the duplicate edits).

Maybe this could be posted as a suggestion somewhere?

Tracked in Phabricator
Task T198584

Local wikilinks (mostly found in descriptions), like [[Belphegor's prime]] should be changed to [[:en:Belphegor's prime|Belphegor's prime]] during import so that they don't all turn into redlinks.

Support.