Jump to content

Manual talk:Pywikibot/2014

Add topic
From mediawiki.org

Cant login to bot to test it...

[edit]

Having a bit of an issue logging into my bot.

c:\inetpub\wwwroot\wikified\pywikipedia>python pwb.py login.py Traceback (most recent call last):

 File "pwb.py", line 131, in <module>
   tryimport_pwb()
 File "pwb.py", line 30, in tryimport_pwb
   import pywikibot
 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\__init__.py", line 418

, in <module>

   from .page import Page, ImagePage, Category, Link, User, ItemPage, PropertyP

age, Claim

 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\page.py", line 17, in

<module>

   import pywikibot.site
 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\site.py", line 32, in

<module>

   from pywikibot import pagegenerators
 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\pagegenerators.py", li

ne 31, in <module>

   from pywikibot.comms import http
 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\comms\http.py", line 6

2, in <module>

   version=pywikibot.version.getversiondict())
 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\version.py", line 52,

in getversiondict

   (tag, rev, date, hsh) = getversion_git(_program_dir)
 File "c:\inetpub\wwwroot\wikified\pywikipedia\pywikibot\version.py", line 150,
in getversion_git
   stdout=subprocess.PIPE).stdout.read()
 File "C:\Python27\Lib\subprocess.py", line 709, in __init__
   errread, errwrite)
 File "C:\Python27\Lib\subprocess.py", line 957, in _execute_child
   startupinfo)

WindowsError: [Error 2] The system cannot find the file specified

User Config: mylang = 'en' family = 'wikifiedlearning' usernames['wikifiedlearning']['en'] =u'MySiteBot' console_encoding = 'utf-8' transliteration_target = None

I'm running on Windows Server 2012 r2, if it's pertinent. If you should need anymore info, let me know.

Thanks 98.122.31.31 19:25, 22 February 2014 (UTC)Reply

I've heard others (Wiki13?) having this problem too on Windows but I didn't manage to figure it out, so I'm interested in the answer too. Nemo 08:22, 24 February 2014 (UTC)Reply
Same as above: it would be the best to file the bug at bugzilla: https://bugzilla.wikimedia.org/enter_bug.cgi?product=Pywikibot  @xqt 15:27, 27 March 2014 (UTC)Reply

WindowsError: [Error 2]

[edit]

Microsoft Windows XP [Versione 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp.

C:\Pywikipedia>login.py WARNING: Running on Windows and transliteration_target is not set. Please see http://www.mediawiki.org/wiki/Manual:Pywikipediabot/Windows Traceback (most recent call last):

 File "C:\Pywikipedia\login.py", line 59, in <module>
   import query
 File "C:\Pywikipedia\query.py", line 30, in <module>
   import wikipedia as pywikibot
 File "C:\Pywikipedia\wikipedia.py", line 6019, in <module>
   versionrev=(version.getversiondict()["rev"])
 File "C:\Pywikipedia\pywikibot\version.py", line 52, in getversiondict
   (tag, rev, date, hsh) = getversion_git(_program_dir)
 File "C:\Pywikipedia\pywikibot\version.py", line 150, in getversion_git
   stdout=subprocess.PIPE).stdout.read()
 File "C:\Python27\lib\subprocess.py", line 679, in __init__
   errread, errwrite)
 File "C:\Python27\lib\subprocess.py", line 893, in _execute_child
   startupinfo)

WindowsError: [Error 2] Can not find the file specified

C:\Pywikipedia>pywikibot

Better to file the bug at bugzilla: -> https://bugzilla.wikimedia.org/enter_bug.cgi?product=Pywikibot  @xqt 15:25, 27 March 2014 (UTC)Reply
I copied your report to bugzilla:63280. Helder.wiki 19:33, 30 March 2014 (UTC)
That has been reported already and I made a patch to fix it but It's not still merged until then you can use this:
http://tools.wmflabs.org/dexbot/compat.zip Ladsgroup (talk) 19:42, 30 March 2014 (UTC)Reply

replace.py not working, just bringing up help script

[edit]
When I use replace.py, it asks me first for text to replace. I fill this in. After that, it asks me for an edit summary. When I fill the summary in, and press enter, it stops. It just brings up a massive long help help list.

When I use replace.py, it asks me first for text to replace. I fill this in. After that, it asks me for an edit summray. When I fill the summary in, and press enter, it stops. It just brings up a massive long help help list. From my terminal;

Global arguments available for all bots:

-dir:PATH         Read the bot's configuration data from directory given by
                  PATH, instead of from the default directory.

-lang:xx          Set the language of the wiki you want to work on, overriding
                  the configuration in user-config.py. xx should be the
                  language code.

-family:xyz       Set the family of the wiki you want to work on, e.g.
                  wikipedia, wiktionary, wikitravel, ...
                  This will override the configuration in user-config.py.

-user:xyz         Log in as user 'xyz' instead of the default username.

-daemonize:xyz    Immediately return control to the terminal and redirect
                  stdout and stderr to xyz (only use for bots that require
                  no input from stdin).

-help             Show this help text.

-loghandler:xyz   Choose a value for 'xyz' from 'TRFH' (TimedRotatingFile-
                  Handler) or 'RFH' (RotatingFileHandler). Has to be defined
                  before '-log' on command line.

-log              Enable the logfile, using the default filename
                  "replace.log"
                  Logs will be stored in the logs subdirectory.

-log:xyz          Enable the logfile, using 'xyz' as the filename.

-nolog            Disable the logfile (if it is enabled by default).

-maxlag           Sets a new maxlag parameter to a number of seconds. Defer bot
                  edits during periods of database server lag. Default is set by
                  config.py

-putthrottle:n    Set the minimum time (in seconds) the bot will wait between
-pt:n             saving pages.

-verbose          Have the bot provide additional output that may be
-v                useful in debugging.

-cosmeticchanges  Toggles the cosmetic_changes setting made in config.py or
-cc               user_config.py to its inverse and overrules it. All other
                  settings and restrictions are untouched.

-simulate         Disables writing to the server. Useful for testing and
(-dry)            debugging of new code (if given, doesn't do any real
                  changes, but only shows what would have been changed).
                  DEPRECATED: please use -simulate instead of -dry

STDOUT: 
This bot will make direct text replacements. It will retrieve information on
which pages might need changes either from an XML dump or a text file, or only
change a single page.

These command line parameters can be used to specify which pages to work on:

-cat              Work on all pages which are in a specific category.
                  Argument can also be given as "-cat:categoryname" or
                  as "-cat:categoryname|fromtitle" (using # instead of |
                  is also allowed in this one and the following)

-catr             Like -cat, but also recursively includes pages in
                  subcategories, sub-subcategories etc. of the
                  given category.
                  Argument can also be given as "-catr:categoryname" or
                  as "-catr:categoryname|fromtitle".

-subcats          Work on all subcategories of a specific category.
                  Argument can also be given as "-subcats:categoryname" or
                  as "-subcats:categoryname|fromtitle".

-subcatsr         Like -subcats, but also includes sub-subcategories etc. of
                  the given category.
                  Argument can also be given as "-subcatsr:categoryname" or
                  as "-subcatsr:categoryname|fromtitle".

-uncat            Work on all pages which are not categorised.

-uncatcat         Work on all categories which are not categorised.

-uncatfiles       Work on all files which are not categorised.

-uncattemplates   Work on all templates which are not categorised.

-file             Read a list of pages to treat from the named text file.
                  Page titles in the file must be enclosed with [[brackets]]
                  or separated by newlines. Argument can also be given as
                  "-file:filename".

-filelinks        Work on all pages that use a certain image/media file.
                  Argument can also be given as "-filelinks:filename".

-search           Work on all pages that are found in a MediaWiki search
                  across all namespaces.

-namespace        Filter the page generator to only yield pages in the
-ns               specified namespaces.  Separate multiple namespace numbers
                  with commas. Example: -ns:"0,2,4" (Take care of quotation
                  marks as comma may qualify as command line separator.)
                  Will ask for namespaces if you write just -namespace or -ns.

-interwiki        Work on the given page and all equivalent pages in other
                  languages. This can, for example, be used to fight
                  multi-site spamming.
                  Attention: this will cause the bot to modify
                  pages on several wiki sites, this is not well tested,
                  so check your edits!

-limit:n          When used with any other argument that specifies a set
                  of pages, work on no more than n pages in total

-links            Work on all pages that are linked from a certain page.
                  Argument can also be given as "-links:linkingpagetitle".

-imagelinks       Work on all images that are linked from a certain page.
                  Argument can also be given as "-imagelinks:linkingpagetitle".

-newimages        Work on the 100 newest images. If given as -newimages:x,
                  will work on the x newest images.

-new              Work on the 60 recent new pages. If given as -new:x,
                  will work on the x newest pages.

-recentchanges    Work on new and edited pages returned by
                  [[Special:Recentchanges]]. Can also be given as
                  "-recentchanges:n" where n is the number of pages to be
                  returned, else 100 pages are returned.

-ref              Work on all pages that link to a certain page.
                  Argument can also be given as "-ref:referredpagetitle".

-start            Specifies that the robot should go alphabetically through
                  all pages on the home wiki, starting at the named page.
                  Argument can also be given as "-start:pagetitle".

                  You can also include a namespace. For example,
                  "-start:Template:!" will make the bot work on all pages
                  in the template namespace.

-prefixindex      Work on pages commencing with a common prefix.

-titleregex       Work on titles that match the given regular expression.

-transcludes      Work on all pages that use a certain template.
                  Argument can also be given as "-transcludes:Title".

-unusedfiles      Work on all description pages of images/media files that are
                  not used anywhere.
                  Argument can be given as "-unusedfiles:n" where
                  n is the maximum number of articles to work on.

-unwatched        Work on all articles that are not watched by anyone.
                  Argument can be given as "-unwatched:n" where
                  n is the maximum number of articles to work on.

-usercontribs     Work on articles that were edited by a certain user.
                  Example: -usercontribs:DumZiBoT
                  Normally up to 250 distinct pages are given. To get an other
                  number of pages, add the number behind the username
                  delimited with ";"
                  Example: -usercontribs:DumZiBoT;500
                  returns 500 distinct pages to work on.

-<mode>log        Work on articles that were on a specified special:log.
                  You have options for every type of logs given by the
                  <mode> parameter which could be one of the following:
                      block, protect, rights, delete, upload, move, import,
                      patrol, merge, suppress, review, stable, gblblock,
                      renameuser, globalauth, gblrights, abusefilter, newusers
                  Examples:
                  -movelog gives 500 pages from move log (should be redirects)
                  -deletelog:10 gives 10 pages from deletion log
                  -protect:Dummy gives 500 pages from protect by user Dummy
                  -patrol:Dummy;20 gives 20 pages patroled by user Dummy
                  In some cases this must be written as -patrol:"Dummy;20"

-weblink          Work on all articles that contain an external link to
                  a given URL; may be given as "-weblink:url"

-withoutinterwiki Work on all pages that don't have interlanguage links.
                  Argument can be given as "-withoutinterwiki:n" where
                  n is some number (??).

-random           Work on random pages returned by [[Special:Random]].
                  Can also be given as "-random:n" where n is the number
                  of pages to be returned, else 10 pages are returned.

-randomredirect   Work on random redirect target pages returned by
                  [[Special:Randomredirect]].  Can also be given as
                  "-randomredirect:n" where n is the number of pages to be
                  returned, else 10 pages are returned.

-gorandom         Specifies that the robot should starting at the random pages
                  returned by [[Special:Random]].

-redirectonly     Work on redirect pages only, not their target pages.
                  The robot goes alphabetically through all redirect pages
                  on the wiki, starting at the named page. The
                  argument can also be given as "-redirectonly:pagetitle".

                  You can also include a namespace. For example,
                  "-redirectonly:Template:!" will make the bot work on
                  all redirect pages in the template namespace.

-google           Work on all pages that are found in a Google search.
                  You need a Google Web API license key. Note that Google
                  doesn't give out license keys anymore. See google_key in
                  config.py for instructions.
                  Argument can also be given as "-google:searchstring".

-yahoo            Work on all pages that are found in a Yahoo search.
                  Depends on python module pYsearch.  See yahoo_appid in
                  config.py for instructions.

-page             Work on a single page. Argument can also be given as
                  "-page:pagetitle".

-xml              Retrieve information from a local XML dump (pages-articles
                  or pages-meta-current, see http://download.wikimedia.org).
                  Argument can also be given as "-xml:filename".

-page             Only edit a specific page.
                  Argument can also be given as "-page:pagetitle". You can
                  give this parameter multiple times to edit multiple pages.

Furthermore, the following command line parameters are supported:

-regex            Make replacements using regular expressions. If this argument
                  isn't given, the bot will make simple text replacements.

-nocase           Use case insensitive regular expressions.

-dotall           Make the dot match any character at all, including a newline.
                  Without this flag, '.' will match anything except a newline.

-multiline        '^' and '$' will now match begin and end of each line.

-xmlstart         (Only works with -xml) Skip all articles in the XML dump
                  before the one specified (may also be given as
                  -xmlstart:Article).

-save             Saves the titles of the articles to a file instead of
                  modifying the articles. This way you may collect titles to
                  work on in automatic mode, and process them later with
                  -file. Opens the file for append, if exists.
                  If you insert the contents of the file into a wikipage, it
                  will appear as a numbered list, and may be used with -links.
                  Argument may also be given as "-save:filename".

-savenew          Just like -save, except that overwrites the existing file.
                  Argument may also be given as "-savenew:filename".

-saveexc          With this parameter a new option will appear in choices:
                  "no+eXcept". If you press x, the text will not be replaced,
                  and the title of page will be saved to the given exception
                  file to exclude this page from future replacements. At the
                  moment you may paste the contents directly into 'title' list
                  of the exceptions dictionary of your fix (use tab to indent).
                  Reading back the list from file will be implemented later.
                  Argument may also be given as "-saveexc:filename".
                  Opens the file for append, if exists.

-saveexcnew       Just like -saveexc, except that overwrites the existing file.
                  Argument may also be given as "-saveexcnew:filename".

-readexc          Reserved for reading saved exceptions from a file.
                  Not implemented yet.

-addcat:cat_name  Adds "cat_name" category to every altered page.

-excepttitle:XYZ  Skip pages with titles that contain XYZ. If the -regex
                  argument is given, XYZ will be regarded as a regular
                  expression. Use multiple times to ignore multiple pages.

-requiretitle:XYZ Only do pages with titles that contain XYZ. If the -regex
                  argument is given, XYZ will be regarded as a regular
                  expression.

-excepttext:XYZ   Skip pages which contain the text XYZ. If the -regex
                  argument is given, XYZ will be regarded as a regular
                  expression.

-exceptinside:XYZ Skip occurences of the to-be-replaced text which lie
                  within XYZ. If the -regex argument is given, XYZ will be
                  regarded as a regular expression.

-exceptinsidetag:XYZ Skip occurences of the to-be-replaced text which lie
                  within an XYZ tag.

-summary:XYZ      Set the summary message text for the edit to XYZ, bypassing
                  the predefined message texts with original and replacements
                  inserted.

-sleep:123        If you use -fix you can check multiple regex at the same time
                  in every page. This can lead to a great waste of CPU because
                  the bot will check every regex without waiting using all the
                  resources. This will slow it down between a regex and another
                  in order not to waste too much CPU.

-query:           The maximum number of pages that the bot will load at once.
                  Default value is 60. Ignored when reading an XML file.

-fix:XYZ          Perform one of the predefined replacements tasks, which are
                  given in the dictionary 'fixes' defined inside the files
                  fixes.py and user-fixes.py.
                  The -regex, -recursive and -nocase argument and given
                  replacements and exceptions will be ignored if you use -fix
                  and they are present in the 'fixes' dictionary.
                  Currently available predefined fixes are:
                  * HTML        - Convert HTML tags to wiki syntax, and
                                  fix XHTML.
                                    **) NOTE below
                  * isbn        - Fix badly formatted ISBNs.
                                    **) NOTE below
                  * syntax      - Try to fix bad wiki markup. Do not run
                                  this in automatic mode, as the bot may
                                  make mistakes.
                  * syntax-safe - Like syntax, but less risky, so you can
                                  run this in automatic mode.
                                    **) NOTE below
                  * case-de     - fix upper/lower case errors in German
                  * grammar-de  - fix grammar and typography in German
                  * vonbis      - Ersetze Binde-/Gedankenstrich durch "bis"
                                  in German
                  * music-de    - Links auf Begriffsklärungen in German
                  * datum-de    - specific date formats in German
                  * correct-ar  - Corrections for Arabic Wikipedia and any
                                  Arabic wiki.
                  * yu-tld      - the yu top-level domain is disabled
                  * fckeditor   - Try to convert FCKeditor HTML tags to wiki
                                  syntax.
                                  http://lists.wikimedia.org/pipermail/wikibots-l/2009-February/000290.html

                                    **) NOTE: these fixes are part of the
                                        cosmetic_changes.py. You may use
                                        that script instead.

-always           Don't prompt you for each replacement

-recursive        Recurse replacement as long as possible. Be careful, this
                  might lead to an infinite loop.

-allowoverlap     When occurences of the pattern overlap, replace all of them.
                  Be careful, this might lead to an infinite loop.

other:            First argument is the old text, second argument is the new
                  text. If the -regex argument is given, the first argument
                  will be regarded as a regular expression, and the second
                  argument might contain expressions like \1 or \g<name>.
                  It is possible to introduce more than one pair of old text
                  and replacement.

-replacementfile  Lines from the given file name(s) will be read as if they
                  were added to the command line at that point. I.e. a file
                  containing lines "a" and "b", used as
                  python replace.py -page:X -replacementfile:file c d
                  will replace 'a' with 'b' and 'c' with 'd'. However, using
                  python replace.py -page:X c -replacementfile:file d will
                  also work, and will replace 'c' with 'a' and 'b' with 'd'.

Examples:

If you want to change templates from the old syntax, e.g. {{msg:Stub}}, to the
new syntax, e.g. {{Stub}}, download an XML dump file (pages-articles) from
http://download.wikimedia.org, then use this command:

    python replace.py -xml -regex "{{msg:(.*?)}}" "{{\1}}"

If you have a dump called foobar.xml and want to fix typos in articles, e.g.
Errror -> Error, use this:

    python replace.py -xml:foobar.xml "Errror" "Error" -namespace:0

If you want to do more than one replacement at a time, use this:
    python replace.py -xml:foobar.xml "Errror" "Error" "Faail" "Fail" -namespace:0

If you have a page called 'John Doe' and want to fix the format of ISBNs, use:

    python replace.py -page:John_Doe -fix:isbn

Let's suppose, you want to change "color" to "colour" manually, but gathering
the articles is too slow, so you want to save the list while you are sleeping.
You have Windows, so "python" is not necessary. Use this:

    replace.py -xml -save:color.txt color colour -always
You may use color.txt later with -file or -links, if you upload it to the wiki.

This command will change 'referer' to 'referrer', but not in pages which
talk about HTTP, where the typo has become part of the standard:

    python replace.py referer referrer -file:typos.txt -excepttext:HTTP

Please type "replace.py -help | more" if you can't read the top of the help.

Why is it doing this? I have installed the latest python version and, I believe, plenty of packages. Technobliterator (talk) 21:52, 21 June 2014 (UTC)Reply

Problem solved, nevermind. Technobliterator (talk) 15:17, 22 June 2014 (UTC)Reply

Removed category is re-added

[edit]

Hello.

When I use my bot to remove certains categories from articles (python category.py remove), in the first case the bot works OK, but when I select other category that the same article has, the bot re-adds the category.

Example:

Thanks. Benfutbol10 (talk) 09:56, 18 July 2014 (UTC)Reply

Worth checking that you're not getting outdated wikitext from the API, though replag looks ok right now. Nemo 11:06, 18 July 2014 (UTC)Reply
bugzilla:55165 could be related. If using the core branch, try replacing all occurrences of
oldtext = self.get(get_redirect=True)
in pywikibot/page.py with
oldtext = self.get(get_redirect=True, force=True)
Ricordisamoa 16:14, 18 July 2014 (UTC)Reply

Making old script work with core

[edit]

Hello! I built a script some months ago using old pywikipedia script:

<pre># -*- coding:utf-8 -*-


import pagegenerators, wikipedia

lloc = wikipedia.getSite("en","wikipedia") x = pagegenerators.AllpagesPageGenerator("Taxonomy/A", 10, True,lloc ) for plantilla in x :

      print plantilla
      titol = plantilla.title()
      if titol[9:18] != "Taxonomy/" : break
      titol = 'Txantiloi:'+ titol[9:] # per treure Template: i posar Plantilla: (canvia-ho al nom en euskara)                                
      try :
              text = plantilla.get()
      except :     
                              continue #Para pasar de los redirects.print text                             
      pagina = wikipedia.Page( wikipedia.getSite('eu',"wikipedia"), titol)
      if pagina.exists() :
              wikipedia.output(titol)
              print "Dagoeneko badago"
              continue 
      pagina.put(text,"Taxonomia txantiloiak esportatzen")

Now it doesn't work with the new core system. It says that there's no "pagegenerators" or "wikipedia" modules to load.

Someone knows how to migrate it? -Theklan (talk) 08:58, 16 September 2014 (UTC)Reply

Theklan, I suggest reading "README-conversion.txt" [1] which is found in the root directory of the pywikibot core package. whym (talk) 11:26, 25 September 2014 (UTC)Reply
Thanks!
The now code is as follows
  1. -*- coding:utf-8 -*-
import pywikibot
from pywikibot import pagegenerators
gunea = pywikibot.getSite("en","wikipedia")
x = pagegenerators.AllpagesPageGenerator("Taxonomy/A", 10, True,gunea )
for plantilla in x :
print plantilla
izenburua = plantilla.title()
if izenburua[9:18] != "Taxonomy/" : break
izenburua = 'Txantiloi:'+ izenburua[9:] # per treure Template: i posar Plantilla: (canvia-ho al nom en euskara)
try :
text = plantilla.get()
except :
continue #Para pasar de los redirects.print text
pagina = pywikibot.Page( pywikibot.getSite('eu',"wikipedia"), izenburua)
if pagina.exists() :
pywikibot.output(izenburua)
print "Dagoeneko badago"
continue
pagina.put(text,"Taxonomia txantiloiak esportatzen")
Theklan (talk) 10:19, 28 September 2014 (UTC)Reply

Unable to log in with core login.py

[edit]

I am using the latest 2.x.x Python, and when I try to login (using login.py), it says there's no module to import (the PWB module, TBP). Its doesn't work out in core, but in compat, it works just fine. Is this by an chance a dir problem, does the user-config need to be in the same dir along with login.py? Ankit Maity (talk) 16:53, 4 November 2014 (UTC)Reply

If you tried to login with python scripts/login.py, try python pwb.py login instead. whym (talk) 02:12, 8 November 2014 (UTC)Reply

Why does pywikipediabot try to access github.com?

[edit]

I'm trying to get the bot to access our wiki, but for some reason my firewall says it is trying to access github.com at port 443. Why is this, and how can I stop it?--89.0.98.186 12:14, 22 November 2014 (UTC)Reply

You should enable https support.  @xqt 10:05, 26 November 2014 (UTC)Reply