Manual talk:Pywikibot

Jump to navigation Jump to search

About this board

Archives 


Please use one of the communication channels listed on Manual:Pywikibot/Communication rather than using this discussion board. There is very little traffic here, so it may take a while before you get a response.

Use upload script with wildcards

1
RedKhan16 (talkcontribs)

Couldn't find it anywhere in documentation, but is it possible to use upload script with wildcards?

If I use

upload /path/to/files/foo*

the scripts suggest all the full paths of foo* files as a description. Something like

/path/to/files/foo 1 /path/to/files/foo 2 /path/to/files/foo 3

which is, of course, is not what I want. It's the same with the -summary key.

In case of

upload /path/to/files/foo* -descfile:description.txt

I get an error "Both a description and a -descfile were provided. Please specify only one of those."

The workaround is to separately put all the foo* files to a separate folder and use -recursive.

upload -recursive /path/with/files -descfile:description.txt

works fine, but you have to go through the hassle of copying files to a seperate folder.

Reply to "Use upload script with wildcards"
Adithyak1997 (talkcontribs)

While trying to login to Pywikibot using the command python3 pwb.py login, I am getting the following error


Skipped '/home/user/pywikibot/core_stable/user-config.py': writeable by others.

family and mylang are not set.

Defaulting to family='wikipedia' and mylang='test'.

WARNING: No user is logged in on site wikipedia:test

Traceback (most recent call last):

  File "/home/user/pywikibot/core_stable/pywikibot/login.py", line 115, in __init__

   user = code_to_usr.get(site.code) or code_to_usr['*']

KeyError: '*'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "pwb.py", line 363, in <module>

   if not main():

  File "pwb.py", line 355, in main

   run_python_file(filename,

  File "pwb.py", line 74, in run_python_file

   exec(compile(source, filename, 'exec', dont_inherit=True),

  File "./scripts/login.py", line 192, in <module>

   main()

  File "./scripts/login.py", line 174, in main

   site.login(autocreate=autocreate)

  File "/home/user/pywikibot/core_stable/pywikibot/tools/__init__.py", line 1481, in wrapper

   return obj(*__args, **__kw)

  File "/home/user/pywikibot/core_stable/pywikibot/site/__init__.py", line 333, in login

   login_manager = api.LoginManager(site=self, user=self.username())

  File "/home/user/pywikibot/core_stable/pywikibot/tools/__init__.py", line 1481, in wrapper

   return obj(*__args, **__kw)

  File "/home/user/pywikibot/core_stable/pywikibot/login.py", line 117, in __init__

   raise NoUsername(

pywikibot.exceptions.NoUsername: ERROR: username for wikipedia:test is undefined.

If you have a username for that site, please add a line to user-config.py as follows:

usernames['wikipedia']['test'] = 'myUsername'

CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.NoUsername'>


Since the wiki is a third party wiki, I have actually created password.py file, added the family file and also created the user-config.py file. But still the error is showing

Xqt (talkcontribs)
Reply to "NoUsername error"

SSL Certificate Verification error

6
Adithyak1997 (talkcontribs)

I am currently trying to add the website https://schoolwiki.in/ into Pywikibot, so as to make use of the bot in Schoolwiki. While trying to run the code "pytho pwb.py generate_family_file" according to the instructions mentioned here, I am getting the error:

pywikibot.exceptions.FatalServerError: HTTPSConnectionPool(host='schoolwiki.in', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)')))

CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.FatalServerError'>

Is there any method to resolve this issue?

Edoderoo (talkcontribs)

Did you try to enter schoolwiki.in manually into your user-config.py ? This is just a text file, add a line like

usernames=['schoolwiki.in']='edoderoobot'

Xqt (talkcontribs)

This is about the familiy file not the user-config

Edoderoo (talkcontribs)

I tried to generate the family-file for edoderoobot, and got the same error. It might be an issue on the schoolwiki.in setup.

Adithyak1997 (talkcontribs)

Did you try any method to bypass the certification related error? I mean something like the ones mentioned here?

Reply to "SSL Certificate Verification error"
Andrew Campbell 67 (talkcontribs)

Hi there, I desperately need this tool to help groom 1500 imported SharePoint wiki pages in a private wiki. My setup is:

azureuser@aeran-linux-vm-002:~/pywikibot/core$ python pwb.py version
WARNING: Http response status 404
Pywikibot: pywikibot/__init__.py (, -1 (unknown), 2020/08/29, 01:01:30, n/a)
Release version: 4.2.0
requests version: 2.21.0
 cacerts: /etc/ssl/certs/ca-certificates.crt
   certificate test: ok
Python: 3.7.3 (default, Jul 25 2020, 13:03:44)
[GCC 8.3.0]
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB:
PYWIKIBOT_NO_USER_CONFIG: 2
Config base dir: /home/azureuser/pywikibot/core

When I attempt to initialise pywikibot I get:

azureuser@aeran-linux-vm-002:~/pywikibot/core$ python pwb.py generate_user_files.py
Traceback (most recent call last):File "pwb.py", line 360, in <module>
    if not main():
  File "pwb.py", line 355, in main
    file_package)
  File "pwb.py", line 74, in run_python_file
    main_mod.__dict__)
  File "./generate_user_files.py", line 473, in <module>
    main()
  File "./generate_user_files.py", line 443, in main
    local_args = pywikibot.handle_args(args)
  File "/home/azureuser/pywikibot/core/pywikibot/bot.py", line 878, in handle_args
    pywikibot.Site()
  File "/home/azureuser/pywikibot/core/pywikibot/tools/__init__.py", line 1434, in wrapper
    return obj(*__args, **__kw)
  File "/home/azureuser/pywikibot/core/pywikibot/__init__.py", line 1249, in Site
    _sites[key] = interface(code=code, fam=fam, user=user)
  File "/home/azureuser/pywikibot/core/pywikibot/tools/__init__.py", line 1517, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/azureuser/pywikibot/core/pywikibot/site/__init__.py", line 1705, in __init__
    BaseSite.__init__(self, code, fam, user)
  File "/home/azureuser/pywikibot/core/pywikibot/tools/__init__.py", line 1517, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/azureuser/pywikibot/core/pywikibot/site/__init__.py", line 720, in __init__
    if code.lower() != code:
AttributeError: 'NoneType' object has no attribute 'lower'
CRITICAL: Exiting due to uncaught exception <class 'AttributeError'>

Obviously something pretty fundamental with my install, but I'm at a loss.

Andrew Campbell 67 (talkcontribs)
165.225.86.65 (talkcontribs)

I have the same issues, I tried to debug code with no good result.


In file __init__.py in pywikibot/ at line 117 the code variable is set in None in

def Site(code=None, fam=None, user=None, sysop=None, interface=None, url=None):


but it's not manage None as value for code variable


I hope it will be solved quickly. :)

2806:106E:1E:1870:9124:3F37:C7C3:A07E (talkcontribs)

Hi. Pywikibot had been working fine. Then it started flagging errors, maybe from a python update. i updated pywikibot and python no longer flags errors. But I don't know why I can't log in. i set pywikibot all over again. any ideas? mediawiki 1.33 python 3.8.3


Traceback (most recent call last):

  File "pwb.py", line 360, in <module>

   if not main():

  File "pwb.py", line 352, in main

   run_python_file(filename,

  File "pwb.py", line 73, in run_python_file

   exec(compile(source, filename, 'exec', dont_inherit=True),

  File "./scripts/login.py", line 185, in <module>

   main()

  File "./scripts/login.py", line 167, in main

   site.login(autocreate=autocreate)

  File "/home/yutkeej/pywikibot/pywikibot/tools/__init__.py", line 1499, in wrapper

   return obj(*__args, **__kw)

  File "/home/yutkeej/pywikibot/pywikibot/site/__init__.py", line 1931, in login

   if login_manager.login(retry=True, autocreate=autocreate):

  File "/home/yutkeej/pywikibot/pywikibot/login.py", line 301, in login

   cookiedata = self.getCookie()

  File "/home/yutkeej/pywikibot/pywikibot/tools/__init__.py", line 1584, in wrapper

   return obj(*new_args, **new_kwargs)

  File "/home/yutkeej/pywikibot/pywikibot/data/api.py", line 3197, in getCookie

   and (response['messagecode'] == 'login-throttled'

KeyError: 'messagecode'

CRITICAL: Exiting due to uncaught exception <class 'KeyError'>

Dvorapa (talkcontribs)

Hi, this is an issue with CAPTCHA. Your bot account is not yet trusted by wiki administrators. You can either ask your wiki admins to assign your bot account a Bot flag, or a Confirmed account flag. The Confirmed account flag is assigned automatically when your account is 4 days old or makes more than 10 edits (en:Wikipedia:User_access_levels#Autoconfirmed_users).

Dvorapa (talkcontribs)

@Xqt Perhaps we should catch this new account captcha (and solve it) somehow too. Or we could just throw a better error message than login-throttled (which has nothing to do with captcha)

Xqt (talkcontribs)
Reply to "login key error"
Leyo (talkcontribs)

How can a ndash be inserted, e.g. using replace.py?

Dvorapa (talkcontribs)

Where do you want to insert it?

Matěj Suchánek (talkcontribs)

A right-click and "Paste" on terminal usually works. Or use user-fixes.py and either paste it using a text editor or type "\u2013" where you want it. In Python 3.8, you should be able to use \N{EN DASH} as well but I couldn't test this.

Leyo (talkcontribs)

Thanks for the replies. I would like to use the terminal to run replace.py and to define the replacements. I had tried to paste the ndash before, but it turned out to be converted into a hyphen minus. I had also tried \u2013, but it would have been inserted as text, not as ndash.

Reply to "ndash"

Error on installing pywikibot

9
79brue (talkcontribs)

Python version: v2.7.13

Using: Windows 10


I am following the installation guide on Manual:Pywikibot/Installation. But when I type the command:

'python pwb.py generate_user_files'

I get an error saying:


Traceback (most recent call last):

  File "pwb.py", line 177, in <module>

    import pywikibot

  File "C:\pywikibot\pywikibot\__init__.py", line 25, in <module>

    from pywikibot.bot import (

  File "C:\pywikibot\pywikibot\bot.py", line 97, in <module>

    from pywikibot import config2 as config

  File "C:\pywikibot\pywikibot\config2.py", line 392, in <module>

    base_dir = get_base_dir()

  File "C:\pywikibot\pywikibot\config2.py", line 345, in get_base_dir

    home = os.path.expanduser('~')

  File "C:\Python27\lib\ntpath.py", line 311, in expanduser

    return userhome + path[i:]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xb9 in position 9: ordinal not in range(128)


What should I do to solve this error?

Dvorapa (talkcontribs)

Hello, I'm not sure as I am not Windows user, but I guess:

a) this is a bug in Python v2.7.13

Then please try to upgrade your Python to 2.7.15 or 3.7.2 and try again

b) there is a problem in your environment variables or username (perhaps in diacritics or special characters?)

Then you could try to set one of PYWIKIBOT_DIR or PYWIKIBOT_DIR_PWB variables. Probably easiest way to overcome could be to use -dir:"my pywikibot dir" parameter.

If nothing helps, please send us python pwb.py version output.

79brue (talkcontribs)

Thank you for helping me, but nothing helps this error.


The output is:


Traceback (most recent call last):

  File "pwb.py", line 177, in <module>

    import pywikibot

  File "C:\pywikibot\pywikibot\__init__.py", line 25, in <module>

    from pywikibot.bot import (

  File "C:\pywikibot\pywikibot\bot.py", line 97, in <module>

    from pywikibot import config2 as config

  File "C:\pywikibot\pywikibot\config2.py", line 392, in <module>

    base_dir = get_base_dir()

  File "C:\pywikibot\pywikibot\config2.py", line 345, in get_base_dir

    home = os.path.expanduser('~')

  File "C:\Python27\lib\ntpath.py", line 311, in expanduser

    return userhome + path[i:]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xb9 in position 9: ordinal not in range(128)

Dvorapa (talkcontribs)

It definitely seems to me like the setup fault, bad username, broken/outdated Python installation or broken PATH variable. Try to add your Pywikibot installation folder to PATH environment variable and then run only pwb.py generate_user_files or pwb.py version (without "python"). If this will not help too, maybe my Windows-experienced colleagues will know more than I do

79brue (talkcontribs)

I think it's bad username. Because my username was not in ascii. I changed my username, but I don't think it will solve the error.

Matěj Suchánek (talkcontribs)

I'm not a Windows expert but the stacktrace really suggests that the problem is in your environment (or Python version, depends on the point of view). So either update to Python 3, which handles non-ascii characters better or update your environment, so that the path to your home directory and your username don't contain them (PYWIKIBOT_DIR etc. seem not to be related to this specific problem).

Goultard59 (talkcontribs)

Hello,

I get the same error when I try to generate_family_file for my website https://lagbt.wiwiland.net/ with python 3.

Traceback (most recent call last):

  File "generate_family_file.py", line 229, in <module>

   FamilyFileGenerator(*sys.argv[1:]).run()

  File "generate_family_file.py", line 55, in run

   w = self.Wiki(self.base_url)

  File "/home/goultard/piwikibot/core/pywikibot/site_detect.py", line 56, in __init__

   check_response(r)

  File "/home/goultard/piwikibot/core/pywikibot/site_detect.py", line 323, in check_response

   elif response.status == 200 and SERVER_DB_ERROR_MSG in response.text:

  File "/home/goultard/piwikibot/core/pywikibot/comms/threadedhttp.py", line 201, in text

   return self.decode(self.encoding)

  File "/home/goultard/piwikibot/core/pywikibot/comms/threadedhttp.py", line 185, in encoding

   raise self._encoding

  File "/home/goultard/piwikibot/core/pywikibot/comms/threadedhttp.py", line 167, in encoding

   self.raw.decode(self.header_encoding)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 16052: invalid continuation byte

CRITICAL: Exiting due to uncaught exception <class 'UnicodeDecodeError'>

Thanks you for your help

Dvorapa (talkcontribs)

Your issue is different. I can reproduce it on my own PC. Would you please report it on Phabricator?

Also check if you have MediaWiki database of your wiki set properly

Draco flavus (talkcontribs)

python3.8 pwb.py generate_user_files

Traceback (most recent call last):

  File "pwb.py", line 182, in <module>

   if not check_modules():

  File "pwb.py", line 149, in check_modules

   from setup import dependencies

  File "/home/wojtek/core/setup.py", line 35, in <module>

   from setuptools import setup

ModuleNotFoundError: No module named 'setuptools'


~~~~

Reply to "Error on installing pywikibot"
Sigbert (talkcontribs)

Hi, when I run "python pwb.py pagefromfile.py -force" to load pages from a file, its starts fast and a bit later says "Sleeping for 9.9 seconds". Is there a way to reduce the sleeping time safely to 5 seconds? --Sigbert (talk) 11:48, 16 June 2019 (UTC)

Xqt (talkcontribs)

The edit rate is restricted on most wikis. The default setting is put_throttle = 10 which allowes 6 edits per seconds. Some sites allows an edit rate up to 10 or 12 edits per minute which gives a put_throttle of 6 or 5. There is also a command line parameter: -pt:<value> (aliases are -put_thottle, -putthrottle)

Framawiki (talkcontribs)

"6 edits per seconds", by minute instead

Xqt (talkcontribs)

Sure, thanks.

Dvorapa (talkcontribs)

Yes, there is a config option in user-config.py called 'put_throttle'. However, if the server will be overloaded, it will increase the value to its needs automatically, so understand this value only as a recommendedation.

Geertivp (talkcontribs)

I have edited user-config.py to set noisysleep = 30.0 to get rid of most "Sleeping" messages. This means that the client will patiently wait for at least 30 seconds before complaining that the server has a delay in processing.

Reply to "Sleeping for xxx seconds"

BOT watchlist notification that triggers a PWB script

1
Revansx (talkcontribs)

The PWB bot user account on my wiki has a watchlist and receives notifications like any other user. Has anyone every written a pwb script that reads its un-read notifications and calls other scripts with properties based what pages are referred to in the un-read notifications?

Reply to "BOT watchlist notification that triggers a PWB script"

internal_api_error_Error_when_running_a_script

7
Summary by Revansx

problems were due to an experimental extension I should not have had loaded

Revansx (talkcontribs)

Hello, I have installed the latest pywikibot on a centos7 server (python3/pip3) and configured it to successfully log in to a bot account I have created on my private MW 34.x wiki (yay!), but when I try to run a basic script such as "add_text.py" I get an vague error message relating to the API. It's not a permissions error, I've solved that. Now it says, "ERROR: Detected MediaWiki API exception internal_api_error_Error". Can anyone please help me debug this? Here's the error:

[revansx@meza1 core_stable]$ python3 pwb.py login
Logged in on somewiki:en as MrBot.
[revansx@meza1 core_stable]$ python3 pwb.py add_text -cat:Some_Pages -text:'Hello World'
WARNING: API error internal_api_error_Error: [XqtN7LIvu8DlHjdVfQAmeQAAAAg] Caught exception of type Error
ERROR: Detected MediaWiki API exception internal_api_error_Error: [XqtN7LIvu8DlHjdVfQAmeQAAAAg] Caught exception of type Error
[errorclass: Error]; raising
Traceback (most recent call last):
  File "pwb.py", line 390, in <module>
    if not main():
  File "pwb.py", line 385, in main
    file_package)
  File "pwb.py", line 101, in run_python_file
    main_mod.__dict__)
  File "./scripts/add_text.py", line 360, in <module>
    main()
  File "./scripts/add_text.py", line 352, in main
    for page in generator:
  File "/home/revansx/core_stable/pywikibot/pagegenerators.py", line 1554, in CategorizedPageGenerator
    for a in category.articles(**kwargs):
  File "/home/revansx/core_stable/pywikibot/page/__init__.py", line 2983, in articles
    member_type=['page', 'file']
  File "/home/revansx/core_stable/pywikibot/tools/__init__.py", line 1790, in wrapper
    return obj(*__args, **__kw)
  File "/home/revansx/core_stable/pywikibot/site/__init__.py", line 4052, in categorymembers
    total=total, g_content=content, **cmargs)
  File "/home/revansx/core_stable/pywikibot/tools/__init__.py", line 1790, in wrapper
    return obj(*__args, **__kw)
  File "/home/revansx/core_stable/pywikibot/site/__init__.py", line 1943, in _generator
    gen = gen_class(type_arg, **req_args)
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 2921, in __init__
    QueryGenerator.__init__(self, **kwargs)
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 2456, in __init__
    self.site._paraminfo.fetch('query+' + mod for mod in self.modules)
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 395, in fetch
    self._init()
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 274, in _init
    self._fetch(self.preloaded_modules)
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 478, in _fetch
    result = request.submit()
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 2251, in submit
    self._data = super(CachedRequest, self).submit()
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 2040, in submit
    if self._internal_api_error(code, error, result):
  File "/home/revansx/core_stable/pywikibot/data/api.py", line 1910, in _internal_api_error
    raise e
pywikibot.data.api.APIMWException: internal_api_error_Error: [XqtN7LIvu8DlHjdVfQAmeQAAAAg] Caught exception of type Error
[errorclass: Error]
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.data.api.APIMWException'>
Framawiki (talkcontribs)

Hello Revansx, this is definitely an internal Mediawiki error (yes, it looks like you found a bug in mediawiki :), not in pywikibot itself)

You're logging at somewiki, you can ask admin of this wiki to update its mediawiki version to try to get the bug removed. If you're sure it's using up-to-date version, please put full console log of your command (if you've stripped in above paste) and the output of python3 pwb.py version.

Revansx (talkcontribs)

I'm the admin of the wiki as well. The wiki is 1.34.1. I'm running:

  • Python 3.5.6
  • pip 20.1 from /usr/lib/python3.5/site-packages/pip (python 3.5)

And my python3 pwb.pr version response is:

$ python3 pwb.py version
Pywikibot: [https] r-pywikibot-core.git (2cf6739, g1, 2020/04/05, 16:57:37, OUTDATED)
Release version: 3.1.dev0
requests version: 2.23.0
  cacerts: /usr/lib/python3.5/site-packages/certifi/cacert.pem
    certificate test: ok
Python: 3.5.6 (default, Nov 16 2018, 15:50:58)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB:
PYWIKIBOT_NO_USER_CONFIG: Not set
Config base dir: /home/revansx/core_stable
Usernames for family "somewiki":
        en: MrBot

Do you see anything specific I should update?

Matěj Suchánek (talkcontribs)

As you were told, this is beyond Pywikibot, it's an internal error of the wiki (despite the bot is triggering it). You may ask at Project:Support desk but you will need to provide the traceback of the exception thrown by the server engine.

Dvorapa (talkcontribs)

Perhaps you might try to reinstall MediaWiki as your first installation might be broken.

Revansx (talkcontribs)

I appreciate the responses. I'll do what has been recommended and reply back here with whatever the solution turned out to be. Thanks.

Revansx (talkcontribs)

problem solved. There was an permissions issue incurred by an experimental extension I have installed. I disabled the extension and my pwb worked as expected. Thanks everyone