Manual talk:Pywikibot

Jump to navigation Jump to search

About this board

Archives 


Please use one of the communication channels listed on Manual:Pywikibot/Communication rather than using this discussion board. There is very little traffic here, so it may take a while before you get a response.

Massive image upload with pywikibot

2
Discasto (talkcontribs)

Hi all. In the last years, I've been created some tools to upload free images from (mainly) government webs in Spain (for instance, most of the images in :commons:Category:Images from Ayuntamiento de Madrid have been uploaded by me using said tools). I've developed them as Python notebooks that run either on PAWS or locally by using pywikibot. I'm thinking of turning them into a flickr2commons-like tool (that is, with a web front-end from the toolserver), but, at the moment, upload of images by means of pywikibot is a task that takes several seconds per image (I assume this is a constraint set by any kind of policy). Is there a way to speed image upload (flickr2commons massively upload images) but still using pywikibot? Any contact person or link I can use to understand how I can proceed? Thanks

TheAeroExperience (talkcontribs)

Hello. pywikibot has builtin support for compressed files. I have no clue if you have enough permissions/capabilities after the upload but it may help in general. A look in the source of similar tools could also help, for example: Manual:Pywikibot/imagecopy.py - Also remember that bulk uploading of images is a heavy task because of the following distribution, tagging, content checking, etc. workflow each image upload triggers (sooner or later).

Also take a look at the commons wiki if you haven't already: https://commons.wikimedia.org

Regards

Reply to "Massive image upload with pywikibot"
79brue (talkcontribs)

Python version: v2.7.13

Using: Windows 10


I am following the installation guide on Manual:Pywikibot/Installation. But when I type the command:

'python pwb.py generate_user_files'

I get an error saying:


Traceback (most recent call last):

  File "pwb.py", line 177, in <module>

    import pywikibot

  File "C:\pywikibot\pywikibot\__init__.py", line 25, in <module>

    from pywikibot.bot import (

  File "C:\pywikibot\pywikibot\bot.py", line 97, in <module>

    from pywikibot import config2 as config

  File "C:\pywikibot\pywikibot\config2.py", line 392, in <module>

    base_dir = get_base_dir()

  File "C:\pywikibot\pywikibot\config2.py", line 345, in get_base_dir

    home = os.path.expanduser('~')

  File "C:\Python27\lib\ntpath.py", line 311, in expanduser

    return userhome + path[i:]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xb9 in position 9: ordinal not in range(128)


What should I do to solve this error?

Dvorapa (talkcontribs)

Hello, I'm not sure as I am not Windows user, but I guess:

a) this is a bug in Python v2.7.13

Then please try to upgrade your Python to 2.7.15 or 3.7.2 and try again

b) there is a problem in your environment variables or username (perhaps in diacritics or special characters?)

Then you could try to set one of PYWIKIBOT_DIR or PYWIKIBOT_DIR_PWB variables. Probably easiest way to overcome could be to use -dir:"my pywikibot dir" parameter.

If nothing helps, please send us python pwb.py version output.

79brue (talkcontribs)

Thank you for helping me, but nothing helps this error.


The output is:


Traceback (most recent call last):

  File "pwb.py", line 177, in <module>

    import pywikibot

  File "C:\pywikibot\pywikibot\__init__.py", line 25, in <module>

    from pywikibot.bot import (

  File "C:\pywikibot\pywikibot\bot.py", line 97, in <module>

    from pywikibot import config2 as config

  File "C:\pywikibot\pywikibot\config2.py", line 392, in <module>

    base_dir = get_base_dir()

  File "C:\pywikibot\pywikibot\config2.py", line 345, in get_base_dir

    home = os.path.expanduser('~')

  File "C:\Python27\lib\ntpath.py", line 311, in expanduser

    return userhome + path[i:]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xb9 in position 9: ordinal not in range(128)

Dvorapa (talkcontribs)

It definitely seems to me like the setup fault, bad username, broken/outdated Python installation or broken PATH variable. Try to add your Pywikibot installation folder to PATH environment variable and then run only pwb.py generate_user_files or pwb.py version (without "python"). If this will not help too, maybe my Windows-experienced colleagues will know more than I do

79brue (talkcontribs)

I think it's bad username. Because my username was not in ascii. I changed my username, but I don't think it will solve the error.

Matěj Suchánek (talkcontribs)

I'm not a Windows expert but the stacktrace really suggests that the problem is in your environment (or Python version, depends on the point of view). So either update to Python 3, which handles non-ascii characters better or update your environment, so that the path to your home directory and your username don't contain them (PYWIKIBOT_DIR etc. seem not to be related to this specific problem).

Reply to "Error on installing pywikibot"
Angrydog001 (talkcontribs)

when I run welcome.py, it returns:

Traceback (most recent call last):

  File "pwb.py", line 264, in <module>

   if not main():

  File "pwb.py", line 257, in main

   run_python_file(filename, [filename] + args, argvu, file_package)

  File "pwb.py", line 120, in run_python_file

   main_mod.__dict__)

  File ".\scripts\welcome.py", line 880, in <module>

   main()

  File ".\scripts\welcome.py", line 859, in main

   bot.run()

  File ".\scripts\welcome.py", line 611, in run

   for users in self.parseNewUserLog():

  File ".\scripts\welcome.py", line 563, in parseNewUserLog

   yield pywikibot.User(ue.page())

  File "D:\Pywikibot\core-aoa-zh\pywikibot\logentries.py", line 122, in page

   self._page = pywikibot.Page(self.site, self.data['title'])

  File "D:\Pywikibot\core-aoa-zh\pywikibot\logentries.py", line 34, in __missing__

   raise KeyError("Log entry (%s) has no '%s' key" % (self._type, key))

KeyError: "Log entry (newusers) has no 'title' key"

<class 'KeyError'>

CRITICAL: Closing network session.

Angrydog001 (talkcontribs)

What should I do?

Xqt (talkcontribs)

File this bug to phabricator

Framawiki (talkcontribs)

Hi Angrydog001, please fill a ticket on phab phab:/project/view/87/, with the command you entered to start the script, the full traceback present in your last message and the output of `python pwb.py version` command.

Miles Fides (talkcontribs)

I rather have some questions than a problem.

  • Does anyone know how the pywikibot would handle a timeout?
  • Has anybody ever experienced a timeout with the pywikibot?
  • What is the default ttl and the default reaction to a timeout?

Thanks

Zbmath authorid (talkcontribs)

I would be interested in the answer to these questions...

Reply to "Timeout handling"

How to disable "Sleeping for N seconds" on PyWikiBot

5
Summary by Dvorapa

Sleeping for N seconds should not be disabled as it makes load to servers

95.188.216.190 (talkcontribs)

How to disable "Sleeping for N seconds" on PyWikiBot

Xqt (talkcontribs)

You shouldn't. This is necessary to throttle writing to the site high frequently and it is restricted on most wiki's.

Framawiki (talkcontribs)

Why you want to disable a security ? Not a good idea I think :)

Sergezolotukhin (talkcontribs)
Xqt (talkcontribs)

maxthrottle isn't the right parameter for that.

Deleting blank lines with replace.py inside templates

6
Theklan (talkcontribs)

Hello! I have been using replace.py to delete obsolete lines inside templates, but now I have some empty lines. I can't find a way to delete them... can someone help with this? Thanks!

Matěj Suchánek (talkcontribs)

Please share more information: What is your regex, what does your input look like, what do you expect?

Theklan (talkcontribs)
Matěj Suchánek (talkcontribs)

Don't you want to replace it with a single newline \n? The problem I can see is that there were some spaces between the newlines, which you can fix with \n *\n *\n.

Theklan (talkcontribs)

Thanks for the clue! Is there any other script to delete blank lines only inside templates?

Matěj Suchánek (talkcontribs)

None that I know.

Reply to "Deleting blank lines with replace.py inside templates"
Summary by Dvorapa

Everybody can use pywikibot, but please be careful to local community restrictions

SamiIslam (talkcontribs)

Hi,

Can a normal user use the pywikibot functionality from a wiki site to create a site from a text file? Or is that something only an admin would have to do on the server hosting the wiki?

Thanks!

Sami

Xqt (talkcontribs)

There is a script "pagefromfile.py" to do this but it is may be local community restrictions to allow or discard edits in this way.

Summary by Dvorapa

The combination of movepages.py and redirect.py can effectively move pages and update redirects. There is currently no script, which could do both actions at once

Jc86035 (talkcontribs)

Does movepages.py update redirects as well to avoid creating double redirects?

This post was hidden by Jc86035 (history)
Xqt (talkcontribs)

Currently not.

Jc86035 (talkcontribs)

Are there any scripts which do so?

XXN (talkcontribs)

redirect.py

Pagegenerator in different language.

3
Summary by Dvorapa

Pagegenerator for the Czech Wikipedia is -family:wikipedia -lang:cs

Walter Klosse (talkcontribs)

How i make pagegenerator for category on wikipedia in czech language?

Framawiki (talkcontribs)

Hello, do you want something like

python pwb.py replace "1" "2" -lang:cz -category:test

?

Dvorapa (talkcontribs)

correct is -lang:cs

Replacing template with text

4
Summary by Dvorapa

To work with templates there is a template.py script. It can easily get e.g. category of templates to process by -cat parameter

87.0.79.89 (talkcontribs)

Hi,

I need a script that:

1) Select pages that are in a specific category

2) Delete in those pages an old template with its parameters (that could also be sub-templates); the template is the same, but the content is different for all, so I can't specify manually the text

3) And replace it with a text (a new template with standard content for all)

It's possible? I see something in ''replace.py'' (point 1 and 3), but I'm not sure it can do the 2nd; and it seems that ''template.py'' can only replace the template name.

Glz19

Xqt (talkcontribs)

Try template.py script.

Glz19 (talkcontribs)

Thank you for the answer. It's possible with it selecting a category, for work only on pages in it? I read the documentation page but I can't see it.

Xqt (talkcontribs)

All scripts have a documentation with it. Just use -help option. In your case -cat:<categoryname> should do it.