Project:Pywikibot/Documentation RFC

From mediawiki.org

February 2016[edit]

Based on the Lyon Hackathon discussion and some later IRC discussions, I have started to clean up the pywikibot manual. I have classified all current pages into five categories:

  • documentation on getting started and running an existing bot? Keep, but clean up
  • documentation on scripts: Move to Manual:Pywikibot/Scripts/ and update for core
  • compat-only page? Move to Manual:Pywikibot/Compat/ and document on Manual:Pywikibot/Compat (done)
  • discussion page? Move to Project: namespace, (done)
  • documentation on writing a bot? Move to doc.wm.o
  • documentation on contributing to pywikbot? Move to doc.wm.o


PAGE GETTING STARTED SCRIPTS COMPAT PWB: WRITING DEV / to repo
Pywikibot X
Manual:Pywikibot/2.0
Manual:Pywikibot/2.0/Conversion
Manual:Pywikibot/2.0/login.py
Project:Pywikibot/2.0/Porting status x
Manual:Pywikibot/2.0/Porting status/header x
Manual:Pywikibot/2.0/Porting status/row x
Manual:Pywikibot/add text.py x
Manual:Pywikibot/archivebot.py x
Manual:Pywikibot/archivebot.py/setup x
Manual:Pywikibot/articlenos.py x
Manual:Pywikibot/Basic use x
Manual:Pywikibot/basic.py x
Manual:Pywikibot/blockpageschecker.py x
Manual:Pywikibot/casechecker.py x
Manual:Pywikibot/catall.py x
Manual:Pywikibot/category redirect.py x
Manual:Pywikibot/category.py x
Manual:Pywikibot/cfd.py x
Manual:Pywikibot/checkimages.py x
Manual:Pywikibot/claimit.py x
Manual:Pywikibot/clean sandbox.py x
Manual:Pywikibot/Code of conduct RFC x
Manual:Pywikibot/commons category redirect.py
Manual:Pywikibot/commons link.py x
Manual:Pywikibot/commonscat.py x
Manual:Pywikibot/Communication x x x
Manual:Pywikibot/Compat x
Manual:Pywikibot/Compat deprecation x
Manual:Pywikibot/coordinate import.py x
Manual:Pywikibot/copyright.py x
Manual:Pywikibot/cosmetic changes.py x
Manual:Pywikibot/create categories.py x
Manual:Pywikibot/Create your own script x
Manual:Pywikibot/delete.py x
Manual:Pywikibot/delinker.py x
Manual:Pywikibot/Development x
Manual:Pywikibot/Development guidelines x
Manual:Pywikibot/disambredir.py x
Manual:Pywikibot/djvutext.py x
Manual:Pywikibot/Documentation RFC x
Manual:Pywikibot/editarticle.py x
Manual:Pywikibot/featured.py
Manual:Pywikibot/fixes.py x
Manual:Pywikibot/fixing redirects.py x
Manual:Pywikibot/flickrripper.py x
Manual:Pywikibot/Flow x
Manual:Pywikibot/freebasemappingupload.py x
Manual:Pywikibot/Generate user files.py x
Manual:Pywikibot/Gerrit x
Manual:Pywikibot/Global Options x
Manual:Pywikibot/harvest template.py x
Manual:Pywikibot/History x
Manual:Pywikibot/i18n x
Manual:Pywikibot/i18n conversion x
Manual:Pywikibot/i18n progress x
Manual:Pywikibot/illustrate wikidata.py x
Manual:Pywikibot/image.py x
Manual:Pywikibot/imagecopy.py x
Manual:Pywikibot/imageharvest.py x
Manual:Pywikibot/imagerecat.py x
Manual:Pywikibot/imagetransfer.py x
Manual:Pywikibot/imageuncat.py x
Manual:Pywikibot/Installation x
Manual:Pywikibot/Installation/compat x
Manual:Pywikibot/Installation/Labs redirect to TL docs
Manual:Pywikibot/Installation/SVN x
Manual:Pywikibot/interwiki.py x
Manual:Pywikibot/interwiki.py/Phabricator Project x
Manual:Pywikibot/interwiki.py/Wiktionary functionality discussion x
Manual:Pywikibot/interwiki.py/Wiktionary functionality discussion/2007 x
Manual:Pywikibot/isbn.py x
Manual:Pywikibot/listpages.py x
Manual:Pywikibot/login.py x
Manual:Pywikibot/lonelypages.py x
Manual:Pywikibot/maintainer.py x
Manual:Pywikibot/maintcont.py x
Manual:Pywikibot/makecat.py x
Manual:Pywikibot/Migrating to bugzilla x
Manual:Pywikibot/misspelling.py x
Manual:Pywikibot/movepages.py x
Manual:Pywikibot/newitem.py x
Manual:Pywikibot/noreferences.py x
Manual:Pywikibot/nowcommons.py x
Manual:Pywikibot/OAuth x
Manual:Pywikibot/Overview
Manual:Pywikibot/pagefromfile.py x
Manual:Pywikibot/pagegenerators.py x
Manual:Pywikibot/panoramiopicker.py x
Manual:Pywikibot/PAWS x
Manual:Pywikibot/picasacopier.py x
Manual:Pywikibot/protect.py x
Manual:Pywikibot/pwb.py x
Manual:Pywikibot/query.py x
Manual:Pywikibot/rciw.py x
Manual:Pywikibot/Recipes x
Manual:Pywikibot/redirect.py x
Manual:Pywikibot/reflinks.py x
Manual:Pywikibot/replace.py x
Manual:Pywikibot/replicate wiki.py x
Manual:Pywikibot/revertbot.py x
Manual:Pywikibot/Scripts x
Manual:Pywikibot/selflink.py x
Manual:Pywikibot/solve disambiguation.py x
Manual:Pywikibot/spamremove.py x
Manual:Pywikibot/speedy delete.py x
Manual:Pywikibot/standardize interwiki.py x
Manual:Pywikibot/standardize notes.py x
Manual:Pywikibot/Survey2012 x
Manual:Pywikibot/table2wiki.py x
Manual:Pywikibot/template.py x
Manual:Pywikibot/templatecount.py x
Manual:Pywikibot/Test coverage x
Manual:Pywikibot/Third-party Wiki Quick Start x
Manual:Pywikibot/touch.py x
Manual:Pywikibot/transferbot.py x
Manual:Pywikibot/unusedfiles.py x
Manual:Pywikibot/upload.py x
Manual:Pywikibot/User-agent x x
Manual:Pywikibot/user-config.py x
Manual:Pywikibot/user-fixes.py x
Manual:Pywikibot/Version table x
Manual:Pywikibot/version.py x
Manual:Pywikibot/warnfile.py x
Manual:Pywikibot/watchlist.py x
Manual:Pywikibot/weblinkchecker.py x
Manual:Pywikibot/welcome.py x
Manual:Pywikibot/Wikidata
Manual:Pywikibot/Wikidata/compat x
Manual:Pywikibot/Wikidata/Rewrite proposal x
Manual:Pywikibot/wikilogbot.py x
Manual:Pywikibot/wikipedia.py/doc x
Manual:Pywikibot/Windows x

Conclusion from Lyon Hackathon discussion[edit]

See https://etherpad.wikimedia.org/p/Pywikibot_doc_sprint / https://phabricator.wikimedia.org/T100109#1306324



pywikibot documentation[edit]

This is an initial braindump on issues with the current pywikibot documentation and possible solutions, by Valhallasw (talk) 19:49, 20 May 2014 (UTC).[reply]


Current issues[edit]

  1. The documentation is spread out. There is the documentation here on mw.org, but also documentation on basically every Wikipedia (e.g. nl:Help:Gebruik_van_bots), every wikibook and even on wikiversity. Then there's also botwiki: and various guides on wikia. There should be only one authoritative and comprehensive place for users to go for help.
  2. The documentation is outdated. The wikia links above are the worst (Python 2.3 and CVS), but even more recent documentation is only valid for compat, and often documentation does not clarify for which version it was written. The documentation is rarely updated by developers.
  3. The documentation lacks structure. This is, of course, a direct result of the wiki structure. People add a page to write down what they did, instead of trying to merge it into the central documentation, or they add documentation in a section it doesn't really belong.
  4. There is no API documentation

The good things[edit]

There are also some things awesome with the current documentation:

  1. It's really easy to contribute: anyone with an mw.org account can edit/improve a page;
  2. Documentation can be translated relatively easily (which is important)
  3. Documentation is in a sensible location -- the same place as MediaWiki documentation.

What can we do?[edit]

To tackle the 'bad' points mentioned above, I think there are a few things we should get:

  1. Curation: developers need to take an active role in curating content added to the documentation by others (spread out; outdated; structure)
  2. Developers need to take an active role in keeping documentation up to date (spread out; outdated)
  3. Some sort of automated documentation generation (outdated; API docs)

However, we should do that without losing the good points mentioned above.

Options[edit]

A few possible options:

Curation Active updating Auto-docs Easy contribution Translations Location Availability
Status quo

(docs on mw.org)

Oppose Negative

No active review possible (Flaggedrevs has been disabled)

Oppose Negative

Outside the normal developer workflow and therefore hard to enforce.

Neutral Neutral- Positive

Possible with bots

Positive

SUL login; familiar syntax and interface

Positive

Built-in

Positive

mw.org

Positive

status quo

Sphinx

(docs on docs.mw.org or readthedocs)

Positive

Changes pass through Gerrit

Positive

Changes pass through Gerrit, so patches can be held for merge until documentation is added.

Positive

sphinx-autodoc can create documentation from code docstrings

Oppose Negative

Needs git; uses ReStructuredText instead of wikitext; previewing is difficult (needs local installation)

Oppose Negative

Sphinx does support it via .po files, but these also would need updating. Maybe possible via translatewiki?

Neutral Neutral

docs.mw.org is hard to find; readthedocs is used a lot for python apps, so good from the framework perspective

Neutral Neutral

initial attempt in Gerrit

Hybrid (A)

Documentation in git, exported to (locked) pages on wiki

Positive

Changes pass through Gerrit

Positive

Changes pass through Gerrit, so patches can be held for merge until documentation is added.

Neutral Neutral-Oppose Negative

Full doc-gen to mediawiki is difficult: sphinx has no MW output and sphinx.autodoc is hard to use outside of sphinx

Oppose Negative

Needs git; previewing is difficult (probably would need to edit on-wiki, then copy-paste to editor?); wikitext is familiar.

Positive

On mw.org

Positive

mw.org

Neutral Neutral-Oppose Negative

needs a dump of Help:Pywikibot/* pages to text files & a bot/jenkins job to update pages on-wiki

Hybrid (B)

Documentation in git, but with a wiki-to-git bridge.

Positive

Changes pass through Gerrit

Positive

Changes pass through Gerrit, so patches can be held for merge until documentation is added.

Neutral Neutral- Positive

Should be possible through jenkins, maybe with some sphinx-autodoc help?

Positive

No git necessary; previewing is easy (built-in!), wikitext is familiar

Positive

On mw.org

Positive

mw.org

Oppose Negative

needs a dump of Help:Pywikibot/* pages to text files, a bot/jenkins job to update pages on-wiki as well as some way to move wiki edits to git

Suggestions from pywikibot gold-standard evaluation[edit]

As a part of the gold standard evaluation for pywikibot, I identified several documentation-related issues: API:Client code/Evaluations/Pywikibot#Suggested TODOs. --Fhocutt (talk) 19:07, 25 July 2014 (UTC)[reply]

Changes that would improve the documentation include:

  • Add code samples to the documentation to provide easy-to-use examples for common tasks (including queries/lists/searches and edits)
  • Add easy-to-find, complete, and intuitive installation instructions to the documentation. This should include installation via pip (https://bugzilla.wikimedia.org/show_bug.cgi?id=65176) and into virtual environments (https://bugzilla.wikimedia.org/show_bug.cgi?id=61783).
  • Add documentation for people who aren't running bots with existing scripts (particularly researchers who extract data via the API and beginning/intermediate bot writers).
  • Method docstrings should include a link to the corresponding mw:API subpage.