Manual talk:Pywikibot/2023

Please use one of the communication channels listed on Manual:Pywikibot/Communication rather than using this discussion board. There is very little traffic here, so it may take a while before you get a response.

Replace script and fix option

The cause for the error messages was on user side: I mistakenly added a comma at a place where it does not belong.

@Xqt, I do not know whether this is a bug or an issue on my side, therefore here and not in Phabricator:

I’ve added this to user-fixes.py:

fixes['ampCode'] = {
    'generator': [
        r'-ns:0',
        r'-search:insource:"%26" insource:/%26/',
    ],
    'regex': False,
    'msg': {
        '_default': 'Bot-Änderung: überflüss. Kodierung für [[Und-Zeichen]] ersetzt, vgl. [[Spezial:Diff/233908956|Anfrage]]',
    },
    'replacements': [
        (r' %26 ', ' & '),
    ]
}

and then started Pywikibot with this (Windows) command line (user is pre-configured):

pwb replace -simulate -lang:de -fix:ampCode -log:fix-ampCode.log

This resulted in this error message:

2023-05-24 00:40:34            http.py,  123 in              flush: VERBOSE  Traceback (most recent call last):
  File "C:\Programs\Netz\pywikibot\pwb.py", line 39, in <module>
    sys.exit(main())
             ^^^^^^
  File "C:\Programs\Netz\pywikibot\pwb.py", line 35, in main
    runpy.run_path(str(path), run_name='__main__')
  File "<frozen runpy>", line 291, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 514, in <module>
    main()
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 498, in main
    if not execute():
           ^^^^^^^^^
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 485, in execute
    run_python_file(filename, script_args, module)
  File "C:\Programs\Netz\pywikibot\pywikibot\scripts\wrapper.py", line 147, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File "C:\Programs\Netz\pywikibot\scripts\replace.py", line 1113, in <module>
    main()
  File "C:\Programs\Netz\pywikibot\scripts\replace.py", line 1008, in main
    if not fix['replacements']:
           ~~~^^^^^^^^^^^^^^^^
TypeError: tuple indices must be integers or slices, not str
2023-05-24 00:40:34            http.py,  132 in              flush: CRITICAL Exiting due to uncaught exception TypeError: tuple indices must be integers or slices, not str

Used Pywikibot version is:

Release version: 8.1.2
setuptools version: 65.5.0
mwparserfromhell version: 0.6.3
wikitextparser version: 0.51.1
requests version: 2.28.1

With explicit command line options the task worked then, the call was:

pwb replace -simulate -lang:de -ns:0 -search:"insource:\"%26\" insource:/ %26 /" -summary:"Bot-Änderung: überflüss. Kodierung für [[Und-Zeichen]] ersetzt, vgl. [[Spezial:Diff/233908956|Anfrage]]" -log:fix-ampCode.log " %26 " " & " Speravir (talk) 00:30, 29 May 2023 (UTC)

CHATGPT generated Answer:

Hi there! It seems that there is an issue with the replacements field in your code snippet. The error message indicates that "tuple indices must be integers or slices, not str." To fix this, please replace the parentheses with square brackets in the replacements field to define a list of tuples. Here's the corrected code snippet:

fixes['ampCode'] = {
    'generator': [
        r'-ns:0',
        r'-search:insource:"%26" insource:/%26/',
    ],
    'regex': False,
    'msg': {
        '_default': 'Bot-Änderung: überflüss. Kodierung für [[Und-Zeichen]] ersetzt, vgl. [[Spezial:Diff/233908956|Anfrage]]',
    },
    'replacements': [
        [r' %26 ', ' & '],
    ]
}

Sakura emad (talk) 09:03, 29 May 2023 (UTC)

No, that is not the reason:

1. Inspite I wondered about this change (see next item), I tested it and the error message did not disappear.
2. Though it could be outdated on Manual:Pywikibot/user-fixes.py, the syntax with parentheses is still visible there.
3. Also, I actually had copied older entries and adjusted them to new task. Back in the past all had worked fine. Speravir (talk) 21:53, 29 May 2023 (UTC)

: ) New Answer:

If correcting the replacements field did not resolve the issue, here are a few additional suggestions to investigate further:

Check for any recent changes or updates in the Pywikibot library or related dependencies that could be causing compatibility issues. It's possible that there might be a newer version or a specific configuration that is causing the error.

Double-check the command line arguments and options being used to start Pywikibot. Ensure that all necessary arguments are provided correctly, and there are no missing or incorrect options in the command.

Verify if you have the latest version of Pywikibot installed. While you mentioned version 8.1.2, it's worth checking if there are any newer releases available that might include bug fixes or improvements related to the issue you are facing.

I recommend reaching out to the Pywikibot community or forums for assistance. They have extensive experience with the library and might be able to provide specific guidance or insights based on similar issues they have encountered. Sakura emad (talk) 09:47, 30 May 2023 (UTC)

Could you please add the following lines in front of line 1008 in replace.py:

from pprint import pprint

pprint(fix)

Seems that the replace.py script found an evaluated fixes entry different from that given above. The fix itself is read as tuple instead a dict somehow.  @xqt 04:24, 31 May 2023 (UTC)

With how much indentation? I did it this way:

if isinstance(fix['msg'], str): # line 1007
    from pprint import pprint
    pprint(fix)
    set_summary = i18n.twtranslate(site, str(fix['msg']))

The error message remains.

I also have deleted the whole pywikibot (without my user scripts and config, but unintentionally the logs) and then re-extracted the zip file and then temporarily uninstalled mwparserfromhell, because I noticed the subdirectory in pywikibot. No success at all.

Side question: Can I without any risk add -simulate as generator option into user-fixes.py? Speravir (talk) 00:27, 2 June 2023 (UTC)

> With how much indentation? I did it this way:

Seems you have updated the script in meantime. What you should do is the following: Replace the following code

if not fix['replacements']:

pywikibot.warning(f'No replacements defined for fix {fix_name!r}')

continue

with

from pprint import pprint

print(type(fix))

pprint(fix)

if not fix['replacements']:

pywikibot.warning(f'No replacements defined for fix {fix_name!r}')

continue

The reason is that fix seems not to be a dict but a tuple. Maybe you have added a comma at the end of the ampCode fix in your fixes.py.  @xqt 11:57, 2 June 2023 (UTC)

> “Maybe you have added a comma at the end of the ampCode fix in your fixes.py.“

Oh. My. God. This is sooo embarrassing! Auf Deutsch: Ich möchte im Boden versinken!

Yes, this was my simple, but that extensive mistake.

On one hand I am glad not having opened a Phabricator task, but on the other hand (again) so much stress and effort for such a tiny cause. Nethertheless thank you, Xqt, for having the right idea!

Just for the record and for comparison: With the addition of the print code this will be displayed in the terminal (but not the log) as first lines after the command input:

• in case of defunct user-fix.py with spurious comma (obviously changed code for test case):

<class 'tuple'>                                                                                                      
({'generator': ['-ns:0', '-search:insource:"eszett"'],                                                               
  'msg': {'_default': 'Test Pywikibot, Replace-Skript'},                                                            
  'regex': False,                                                                                                    
  'replacements': [('eszett', 'ß')]},)

• in correct working state without this comma:

<class 'dict'>                                                                                                       
{'generator': ['-ns:0', '-search:insource:"eszett"'],                                                                
 'msg': {'_default': 'Test Pywikibot, Replace-Skript'},                                                             
 'regex': False,                                                                                                     
 'replacements': [('eszett', 'ß')]}

Speravir (talk) 22:53, 2 June 2023 (UTC)

https://gerrit.wikimedia.org/r/c/pywikibot/core/+/926657 :-D  @xqt 10:20, 3 June 2023 (UTC)

Speravir (talk) 22:45, 3 June 2023 (UTC)

> the print code this will be displayed in the terminal (but not the log)

Yes this is intentional. print and pprint does not go throught the ui wrapper but directly to the terminal. I usually use them for debugging. Schönes Wochenende wünsch' ich.  @xqt 09:51, 3 June 2023 (UTC)

> Side question: Can I without any risk add -simulate as generator option into user-fixes.py?

This does not work because generator options are only taken as pagegenerators options. It may contains generators and filters, see generator options and filter options. -simulate is a global option and must be given with the replace.py script or the pwb wrapper script.  @xqt 11:13, 2 June 2023 (UTC)

Thanks, I will have a look. Speravir (talk) 22:22, 2 June 2023 (UTC)

Error handling with Pywikibot

Hi, I upload files on Commons with Pywikibot (see https://commons.wikimedia.org/w/index.php?title=Special:ListFiles/YannBot&ilshowall=1 ). It works but I get a lot of errors, mainly due to time out from server side. Now it would OK if I could just tell the bot to start again if the upload fails. I use a bash script to generate all the needed data, but the output is always true whatever happened. How to do error handling? Thanks,

$PYTHON ~/core_stable/pwb.py upload "$filename" $sim $chunk -noverify -filename:"$uploadname" -keep -abortonwarn -descfile:"$desc" Yann (talk) 16:59, 18 June 2023 (UTC)

As far as I can see, it does exit successfully upon server side errors at least. If you want to change that, I think you can submit a feature request (or even better, a patch to the source code) at phabricator. whym (talk) 02:23, 25 June 2023 (UTC)

I don't know Python enough to propose a patch. See phab:T338969. Yann (talk) 11:53, 25 June 2023 (UTC)

If you already created the task, that's a good start. It doesn't have to be a "feature request" per se, as long as the problem is correctly described. Other users/developers may or may not re-categorize it later. whym (talk) 12:38, 25 June 2023 (UTC)

unable to work on items with pywikibot

I have my family file, user-config.py and user-password.py in the directory that i am running the following script from

```

import os, sys

import pywikibot as pwb

pwb.config.register_families_folder(os.getcwd())

pwb.login.ClientLoginManager().login()

os.environ['PYWIKIBOT_DIR'] = os.getcwd()

site = pwb.Site()

repo = site.data_repository()

pg = pwb.ItemPage(repo, 'Q10')

pg.get()

```

i get the following error `AttributeError: DataSite instance has no attribute 'entity_sources' `.

But when i use `pg=pwb.Page(repo, 'Main Page'); print(pg.text) `, i get an output.

How do i get pywikibot to work with items on my wikidata instance ? Keystester (talk) 13:15, 4 July 2023 (UTC)

Seems your wikibase family file does not define the entity_sources method. Refer wikidata_family.py or commons_family.py as a sample. Maybe we should add a abstract base class for repository families to give a better message.  @xqt 13:29, 4 July 2023 (UTC)

They don't have an entity_sources method either.

Edit: commons_family.py has it but wikidata_family.py's method only has a 'return {}'. I will try modifying my family file Keystester (talk) 13:55, 4 July 2023 (UTC)

adding the following to my family file got it working.

```

def entity_sources(self, code):

return {}

```

Would have been nice if the documentation for third party wiki's had this mentioned Keystester (talk) 14:04, 4 July 2023 (UTC)

See https://gerrit.wikimedia.org/r/c/pywikibot/core/+/935449  @xqt 15:03, 4 July 2023 (UTC)

Get wd item from wikipedia page in bulk

Hello, calling pywikibot.ItemPage.fromPage(page) for each page takes a long time.

I'm using wikidata.preloadpages(list) to get page content in bulk, it works well, is there similar preload function for ItemPage.fromPage()?

Thanks! Framawiki (talk) 20:21, 23 August 2023 (UTC)

There is DataSite.preload_entities. Note that it doesn't maintain the implicit relationship between pages and items. Matěj Suchánek (talk) 20:42, 23 August 2023 (UTC)

if there is preloadpages, that's logical that preload_entities exists too, thanks Matěj! Framawiki (talk) 21:18, 23 August 2023 (UTC)

Edits made by Pywikibot edits not showing up in Recent Changes

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

As the title says, I cannot see the edits (mostly new page creation) in Recent Changes, even with all filters removed. I can only see the edits when I go to the User Contributions page of the user account (which is in the Bots user group).

Here is the setup of my wiki:

Installed software

Product	Version
MediaWiki	1.39.4
PHP	7.4.33 (fpm-fcgi)
MariaDB	10.4.20-MariaDB-1:10.4.20+maria~buster-log
ICU	65.1

One big reason I want my edits to show up is in case I need to delete all the pages created in a run (which is probably 200-300 pages), and Mass Delete only works on edits that show up in Recent Changes--so Mass Delete doesn't see any of these new pages.

Is there a setting somewhere in the Pywikibot configuration to change this behavior? Thanks! Lost Student (talk) 06:53, 12 September 2023 (UTC)

Hi, indeed if you remove all filters from recentchanges, and see No active filters. All contributions are shown (like on this link) you should see all changes made on your wiki, including ones made by pywikibot.

If it's not the case that would be a bug in mediawiki itself rather than pywikibot, as pwb is this an API customer doing it's job (you see the edit on the website on some places, so they were well sent by pwb to mediawiki). You can then open a topic on Project:Support desk. Framawiki (talk) 18:52, 12 September 2023 (UTC)

Thanks for the quick reply. I verified that no filters are applied at the Recent Changes and still nothing shows up, yet I know there are edits that were successfully made (such as here, for example--there are hundreds more made by the same user in the last 7 days).

I will go ahead and open a topic at the Support desk. Lost Student (talk) 19:45, 12 September 2023 (UTC)

indeed I don't see them on your site the way they are usually. Maybe there's a reason why edicts aren't saved in the "recentchanges" table, but only in "revisions" table, while they should be in both as far i know, if you want to look in the database.

Will be interesting to see the other answers on support desk too :) Framawiki (talk) 19:55, 12 September 2023 (UTC)

@Framawiki, after doing more investigation and trials, it looks like no edits are showing up in Recent Changes, even ones made manually by a non-bot account. So this certainly has nothing to do with Pywikibot. Lost Student (talk) 20:05, 12 September 2023 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

How connect to own wiki?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

How can I connect to my local private wiki on 127.0.0.1:8080?

site = pywikibot.Site(url='http://127.0.0.1:8080')

page = pywikibot.Page(site, 'Main_Page')

raises SiteDefinitionError: Invalid AutoFamily('127.0.0.1:8080') Dimka665 (talk) 16:53, 18 September 2023 (UTC)

Hello too. You can take a look at Manual:Pywikibot/Use_on_third-party_wikis. Framawiki (talk) 18:27, 18 September 2023 (UTC)

Thank you. It's hard to find this link. There is no link in main page: Manual:Pywikibot and docs https://doc.wikimedia.org/pywikibot/stable/ Dimka665 (talk) 00:01, 19 September 2023 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.