Manual:Pywikibot/Workshop Materials/How to run basic scripts

This page is a resource for workshop organizers who want to run a workshop about basic Pywikibot scripts. By following the instructions on this page, you will learn everything you need to know to teach others about this subject. This page features a complete script based on a past workshop, and provides useful tips for people with little experience running such events.

Note that you don't need to follow this guide to the letter. Feel free to adapt it to your needs, and contribute back to it based on your experience.

If you want to learn more about this subject before committing to running a workshop, or need more concise materials, see User:KBach-WMF/Sandbox/SWT Workshop Materials/How to run a basic scripts (self-study).

How to use this guide?
Read the guide in its entirety to decide if the content of this workshop matches your expectations. Change whatever you feel doesn't match your style or isn't interesting for your audience.

Next, read the relevant linked materials to make sure you know everything you need to run this workshop.

Finally, prepare for the workshop by going through the section.

Requirements
To run this workshop you should know how to run scripts included with Pywikibot. This means basic familiarity with Python, Pywikibot, your terminal or command line of choice, and PAWS.

Organizer
To prepare for this workshop, follow the steps below.


 * 1) Prepare a local Pywikibot environment for the  part of the workshop.
 * 2) * Make sure that Pywikibot and its scripts are available on your local environment. The best way to do this is to download the code from the Pywikibot repository as described on Manual:Pywikibot/Installation. You will then be able to run Pywikibot from the core directory by calling . The rest of this page assumes you have used this method of installing Pywikibot.
 * 3) * Run, select the wiki you want to use for your demonstrations. In this workshop we recommend working with https://test.wikimedia.org, which you can select by picking ```wikipedia```, followed by the ```test``` site code. Specify your user credentials if you prefer.
 * 4) * Pick a categories and pages on the test Wikipedia that you will use to present how to manage categories and pages. Read through the instructions on this page first and then select the best categories and pages for your presentation. Create new ones if necessary.
 * 5) Prepare the PAWS environment for the  part of the workshop.
 * 6) * Create or upload a page file to use when presenting how to create a new page from file.
 * 7) * Create a discussion page with threads you want to archive using a bot.
 * 8) Make the following decisions on how you want to run the workshop.
 * 9) * Do you want the participants to follow along and perform the described activities as you talk about them, or wait until a designated time to perform the activities?
 * 10) * Do you want the participants to interrupt you to ask questions, or wait for the designated time instead?
 * 11) * Do you want to record the workshop? If so, ask the participants if they consent to being recorded.
 * 12) Share your decisions during the introduction to the workshop.
 * 13) Consider if there are any prerequisites or preparations required from the participants and share them before the workshop.

Participants
Be sure to inform workshop participants about the following points they need to consider to prepare for this workshop.


 * To run Pywikibot scripts locally, participants must have Pywikibot installed. For information on how to do this, see Manual:Pywikibot, other small wiki toolkits workshops (see small wiki toolkits workshops), and other self-study guides (TODO).
 * To run Pywikibot scripts on PAWS, participants need a Wikipedia/SUL account. They can create it on any Wikipedia page.

Workshop script
Start sharing your screen and recording if applicable.

Introduce yourself and the topic of the workshop. Explain the organizational choices and rules you've decided upon (for example, whether the participants should ask questions immediately or later - see for details).

Present the agenda of the workshop, outlining what scripts you you plan to run and how. Talk about the purpose of the workshop.

Working with categories
Explain that in this section you will focus on Pywikibot scripts that operate on page categories. Go through the scripts you plan to present.

Adding categories to pages


Explain that you can use the  script to add a category to a list of pages.

Show a page you intend to provide to the script.

Run the  script and explain what's happening.

Pywikibot asks you to specify a page with the list, and then a category. It then awaits your confirmation and adds the category to all pages linked from the page you provided.

Show the page again, pointing out the added category.

Mention that Pywikibot doesn't add the category directly to the page you specify, only to linked pages. Also, note that you can pass the  option if you want the script to create the linked pages that don't exist. Otherwise the bot script skips them by default.

Sources:


 * Manual:Pywikibot/category.py
 * category script documentation on doc.wikimedia.org

Removing categories from pages


Explain that you can use the  script to remove the category from all pages with that category. Mention that this script does not allow you to specify pages to remove the category from. Its primary use is for when you want to delete a category entirely.

You can use a page generator to limit the reach of this script. For example, you can choose to only remove a category from pages that only exist in another, specified category. You can read more about this in the documentation.

Demonstrate the use of this script by running  and then picking a category from the test Wikipedia to remove. The best way to do this is to remove the category you have added earlier.

Sources:
 * Manual:Pywikibot/category.py
 * category script documentation on doc.wikimedia.org

Creating a list of pages in a given category


Explain that you can use the  script to generate a list of pages with a specific category.

Demonstrate the use of this script by running  and then providing category name and the name of a list page to create. Show the newly created page on the wiki.

Note that you can use the  option to overwrite a page if it already exists.

Sources:
 * Manual:Pywikibot/category.py
 * category script documentation on doc.wikimedia.org

Moving pages from one category to another


Explain that you can use the  script to move pages from one category to another.

Demonstrate the use of this script by running, and showing the result of this script.

Explain that you can combine this option with other options that can be interesting to participants. You can read about them in documentation.

Sources:
 * Manual:Pywikibot/category.py
 * category script documentation on doc.wikimedia.org

Page maintenance
Scripts covered in this section provide various page maintenance utilities.

Cleaning sandbox


Explain that you can use the  script to clean up your sandbox page. By default, this script sets the sandbox to display the  template.

You can specify the  option to clean up a custom page, and use the   option to set page content to custom text instead of the default.

See the documentation for information about other interesting options, for example  and   that allow for basic scheduling of this bot.

Sources:
 * Manual:Pywikibot/clean_sandbox.py
 * clean_sandbox script documentation on doc.wikimedia.org

Checking for broken links


Explain that you can use the  script to identify broken external links on a page. Note that this script requires a Python package called.

Run the script using. Explain that when you run the  script for the first time, it generates a binary file that's a starting point for evaluating links. The script creates this file in the  directory. If you run the script again on the same page after at least a week, it will generate a list of links that are still broken, in a human-readable format.

This mechanism helps you to avoid removing temporarily broken links.

As with other scripts, this script also provides more options you can read about in the documentation.

Sources:
 * Manual:Pywikibot/weblinkchecker.py
 * weblinkchecker script documentation on doc.wikimedia.org

Deleting pages


Explain that you can use the  script to delete or restore specific pages. You can specify a page directly using the  option, or use page generators to construct a list of pages to delete. For example, you can use the  option to delete pages that link to a certain page, or the   to delete pages in a category.

Run the  script to show how it works. You can follow it up with. Note that you might not have the necessary permissions to directly delete pages on test Wikipedia. In that case, the script offers to mark the page for deletion instead.

Sources:
 * Manual:Pywikibot/delete.py
 * delete script documentation on doc.wikimedia.org

Running Pywikibot scripts using PAWS


PAWS is a Jupyter Notebook instance hosted by the Wikimedia Foundation. Jupyter Notebook is a web application that allows you to run code and present results without needing to install anything on your computer.

PAWS has Pywikibot preinstalled. You can use it to run Pywikibot scripts in many ways. In this part of the workshop, you will see how to use the PAWS terminal to run built-in Pywikibot scripts. Then, you will see how to write a basic custom script in the notebook interface.

Explain that PAWS is a perfect way to run your less CPU-intensive scripts, or scripts that you don't need to schedule. It's also a great way to learn more about Pywikibot and experiment with it without having to set it up locally.

Note that PAWS is not a good solution for collaborative development or longer, scheduled Pywikibot tasks.

Show the PAWS page and explain that it's a good starting point for running PAWS and learning more about it. Then show how to log in to PAWS by clicking the link on that page or accessing https://hub.paws.wmcloud.org directly.

Sources:
 * PAWS
 * PAWS/Getting_started_with_PAWS
 * PAWS/PAWS_and_Pywikibot
 * Jupyter project documentation

In terminal
Demonstrate how to launch a terminal in PAWS. To do that, scroll down the Launcher page to the Other section and select Terminal.

Note that if you closed the Launcher page, you can reopen it using the New Launcher button in the top left corner of the PAWS screen.

Explain that you can use similar syntax to call Pywikibot from PAWS. The only difference is that you omit  in your commands.

Explain that running  in terminal works without any configuration. This command logs you in to test Wikipedia using your own Wikipedia account. This means you can immediately start experimenting with Pywikibot.

Creating pages from a file


To create a page from a file, you need to prepare that page, place it in a file on PAWS, and then use the  script. You can use the code sample below as source of your page content.

Demonstrate how to do it directly in PAWS.

Explain the syntax of the page file. Every page must start with  and end with. The first line of the file serves as a title and a heading of the page. The rest of the page follows regular wikitext syntax rules.

{{Codesample
 * name = page.txt
 * code = {{-start-}}

Pywikibot Workshop In MONTH YEAR

This was held on DATE

List of Participants
}}
 * Participant 1
 * Participant 2

Create the page using. Show the page on test wiki after the process finishes.

Mention that this script has a lot of other options. You can learn more about them in the documentation.

Sources:
 * Manual:Pywikibot/pagefromfile.py
 * pagefromfile script documentation on doc.wikimedia.org

Archiving page discussion


Explain that you can use the  script to archive old discussions. Show a discussion page with threads that you intend to archive.

To automatically archive threads, you need to use a configuration template on the discussion page. One commonly used template is, documented on wikipedia:User:MiszaBot/Archive_HowTo.

To configure your page for automatic archiving, add the template at the top of the page.

Explain the meaning of template parameters (see the documentation on doc.wikimedia.org for details):
 * - name of the archive page. The script moves all archived threads to this page. You can generate its name dynamically using parameters described in the documentation.  is one such parameter.
 * - archiving algorithm - in this case, the script archives every thread that's at least 10 seconds old. Typically, you would set this option to a few weeks or months. You are using a small number here to demonstrate how the script works.
 * - used to dynamically generate the archive page name. You can change it to create a new archive page.
 * - minimum number of discussion threads to keep on the page. If the bot can't fulfill this requirement, it doesn't archive any threads.
 * - minimum number of threads to archive. If the bot can't fulfill this requirement, it does not archive any threads.

Once the template is present on the discussion page, run  and explain the script's parameters. Note that the name of the template is the last option when calling this script and has to match the template you used on the page.

Show the discussion page and point out how the number of threads decreased.

Sources:
 * Manual:Pywikibot/archivebot.py
 * archivebot script documentation on doc.wikimedia.org

Checking images


Explain that you can use the  script to automatically analyze images for problems, for example with description or license.

Run the script by calling. This simulates the check of the newest 10 images. You can also specify a page with images that you want to check using the  option.

Sources:
 * Manual:Pywikibot/checkimages.py
 * checkimages script documentation on doc.wikimedia.org

In a notebook
Demonstrate how to create a new Jupyter notebook by opening the Launcher page and then choosing Python 3 (ipykernel) in the Notebook section.

This opens a new notebook where you can start creating your custom Pywikibot script.

Creating a basic page


Copy and paste the script below into the notebook and run it using the Run the selected cells and advance option in the toolbar.

Explain what the script does based on the comments. Open the Pywikibot Workshop page to show how it looks on test Wikipedia.

Page generators
Mention that you can use page generators with some scripts you showed in this workshop. Page generators allow you to run these scripts on a dynamically generated lists of pages. You don't have to know the names of all pages you want to change using a bot. Instead, the generators automatically create a list of those pages based on specific criteria, for example pages in specific categories, or pages that are not watched by anyone.

To learn more about page generators, read the documentation.

Sources:
 * Manual:Pywikibot/Page Generators
 * Page generators on doc.wikimedia.org

Closing
Thank the participants and other organizers. Leave some time for final questions the participants might have. List links to extra materials on your last slide if applicable. Let the participants know where they can find the slides and any helpful resources.

Frequently asked questions
(questions asked by participants during previous workshops)

Resource reference
(list all links for quick access)