API:Client code/Evaluations/Pywikibot

Pywikibot is a full-featured API client framework geared towards power users who want to edit wikis: wiki maintainers, bot-runners, and others who want to automate work on MediaWiki sites.

Particularly useful or notable features of pywikibot include:
 * makes Wikibase API accessible
 * good and consistent code style, readable code
 * uses API requests efficiently; supports MediaWiki API query generators
 * handles deprecated methods/branches
 * has very good unit test coverage
 * very comprehensive and targeted to its intended usecase, running bots.

=In-depth evaluation=

Easy to install
Pywikibot is now installable through pip. Instructions are in this Phabricator task. The clearest and simplest installation and configuration instructions are in the slides from the 2013 Pywikipediabot/Hackathon 2013. There are also installation instructions at Manual:Pywikibot/Installation, which are extensive and may be confusing to the casual user. It is possible to install through pip using the instructions in this bug report. However, it is not easy to figure out how to set up user_config.py.
 * Installation instructions are correct and easy to find
 * Library is packaged for installation through appropriate package library (PyPI, CPAN, npm, Maven, rubygems, etc.)
 * Platinum standard: library is packaged for and made available through Linux distributions

Easy to understand
The lowest-level functions to access the API can be found in api.py. https://github.com/wikimedia/pywikibot-core/blob/master/pywikibot/data/wikidataquery.py
 * Well designed--makes all intended API calls available with the intended level of abstraction with no redundancies
 * Platinum standard: makes the Wikidata API available
 * Well documented:
 * Code is commented and readable
 * Clear coding with a consistent style. Docstrings are present, classes and functions and are generally clearly written and helpful. Pywikibot's Development guidelines are being followed.


 * Documentation is comprehensive, accurate, and easy to find
 * Many of the issues with Pywikibot's documentation are summarized in Pywikibot/Documentation RFC. Documentation is now located on doc.wikimedia.org, and links some of the more important documentation pages from MediaWiki. It could be linked in the Pywikibot README for better discoverability, and could be featured more prominently on PWB.


 * Deprecated functions are clearly marked as such
 * Pywikibot has been rewritten to use the API, and the rewritten version is the new "core" branch. There is some debate on deprecation of "compat": http://lists.wikimedia.org/pipermail/pywikipedia-l/2014-June/008764.html. Additionally,  has deprecated methods: "deprecated methods still work, but print a warning message in debug mode" (from Manual:Pywikibot/compat-to-core conversion).


 * Platinum standard: Documentation is understandable by a novice programmer
 * Documentation assumes considerable expertise with MediaWiki. There is also little easily available documentation aimed at enabling users to write new scripts.


 * Code uses idioms appropriate to the language the library is written in

Easy to use

 * Has functioning, simple, and well-written code samples for common tasks
 * Demonstrates queries


 * Demonstrates edits
 * There are a number of available scripts. For simpler examples, see Manual:Pywikibot/Create your own script


 * Handles API complications or idiosyncrasies so the user doesn't have to:
 * Login/logout


 * Cookies


 * Tokens


 * Query continuations


 * Requests via https, including certificate validation


 * Courteous API usage is promoted through code samples and smart defaults
 * gzip compression is used by default
 * does this automatically.


 * Examples show how to create and use a meaningful and unique user-agent header (as in https://meta.wikimedia.org/wiki/User-agent_policy)
 * See Manual:Pywikibot/User-agent for more information. The configuration file now contains user-agent settings as well.


 * Platinum standard: generates a unique user-agent string given name/email address/repository location


 * Efficient usage of API calls
 * If scripts use PageGenerator, API generators and batching will be used to make requests more efficient.

Core supports Python 3.
 * Can be used with the most recent stable version of the language it is written in (e.g. Python 3 compatible)

Easy to debug

 * Contains unit tests for the longest and most frequently modified functions in the library


 * Platinum standard: Unit tests for many code paths exist and are maintained


 * Terrible hacks/instances of extreme cleverness are clearly marked as such in comments
 * Documentation links to the relevant section/subpage of the API documentation

Easy to improve
Some interaction takes place on pywikibot, which can at times be contentious and which tends to assume expertise. Interaction also takes place on on IRC.
 * Library maintainers are responsive and courteous, and foster a thoughtful and inclusive community of developers and users
 * Platinum standard: Project sets clear expectations for conduct for spaces where project-related interactions occur (mailing list, IRC, repository, issue tracker). It should:
 * State desired attitudes and behaviors
 * Provide examples of unwelcome and harassing behavior
 * Specify how these expectations will be enforced

A start for setting community expectations for behavior can be found in the IRC channel: "Don't ask to ask, ask. Please be patient for an answer. Do not private message unless asked."


 * Pull requests are either accepted or rejected with reason within 3 weeks (Platinum standard: 3 business days)

Pywikibot uses Phabricator, and most bugs are responded to. However, there is a backlog of old bugs that are not very active.
 * Issues/bugs are responded to in some manner within 3 weeks (Platinum standard: 3 business days) (but not necessarily fixed)


 * The library is updated and a new version is released within 3 weeks (Platinum standard: 3 business days) when breaking changes are made to the API

API maintainers and Pywikibot developers met to talk in the Lyon Hackathon.
 * Platinum standard: library maintainers contact MediaWiki API maintainers with feedback on the API's design and function

MIT License
 * Library specifies the license it is released under

= Suggested TODOs = Pywikibot has been significantly improved since the first evaluation in July 2014, particularly in its technical aspects. It is now Python 3-compatible, it uses API query generators and query batching to improve efficiency, it is easy to generate a unique and Wikimedia-compliant user-agent, installation through pip is much improved, and test coverage remains excellent. Remaining TODOs primarily focus on further improvements to the project's documentation and processes--not trivial tasks, but important for the project's usability and sustainability.


 * Code-related:
 * Make installation through pip more straightforward. Specifically, make it easier to find the correct command (users should not need to check the issue tracker) and make it easier to create.
 * In a fresh virtualenv,  and then   should either work as-is or yield an informative error message that points to the next step of set-up.
 * Watching users who are new to pywikibot install and get started writing code will help identify more specific issues and more user expectations.


 * Process-related:
 * Foster a hospitable attitude on the pywikibot mailing list and IRC channel, especially to new and/or inexperienced users. Consider agreeing on specific community standards for interaction and on responses to behavior that does not meet those standards. The discussion on a proposed code of conduct for Wikimedia technical spaces may be helpful as it develops. The Recurse Center social rules may be another useful starting point.
 * Continue to create more centralized and updated documentation, including the items below. Also see Manual:Pywikibot/Documentation RFC.
 * Easy-to-find, complete, and intuitive installation instructions, including installing via pip and into virtual environments.
 * Code samples for common tasks, including queries. There are samples for edits, but none for queries.
 * Documentation for users who aren't running bots with existing scripts (particularly researchers and beginning/intermediate bot writers).
 * Links in method documentation to the corresponding API subpages or API help submodules for API module wrapper methods. Functional links are best, but if this is not possible in the autogenerated documentation then adding a copy/pasteable link is better than having none.
 * Streamline or add more resources to the patch review process to reduce the backlog of unreviewed patches.
 * Release a new version within three weeks of relevant API breaking changes.
 * Continue discussions with MediaWiki API developers and feedback on the MediaWiki API itself.