Requests for comment/pywikibot 2.0 packaging

Pywikibot 2.0 is intended to be released as a stable package available via https://pypi.python.org/pypi/pywikibot, pip, and other package distribution systems (rpm, etc)

There is a common held understanding that the Pywikibot library should be designed to be separate from the scripts, and Pywikibot 2.0 codebase mostly achieves that, however there are some packaging complexities which have not been hashed out.

Background
The four issues which complicate packaging are:
 * cosmetic changes
 * i18n
 * scripts
 * family classes

cosmetic changes
The Pywikibot library depends on the cosmetic changes script in the Page save logic. This uses i18n, and is import as 'scripts.cosmetic_changes'; see sections 'i18n' and 'scripts' below for problems resulting from each.

i18n
The Pywikibot library includes an i18n module, which uses i18n messages are currently extracted from TranslateWiki and placed in a git repository pywikibot/i18n, which is a loaded by git as a submodule of both core and compat. In core, it is loaded into the directory 'scripts/i18n'. Currently the i18n messages are Python data files. There is a pending change to migrate to using JSON data files so that the TranslateWiki team can terminate their Python support.

The Pywikibot library is almost entirely only using English language in user interaction, logging, etc, except it uses i18n for four messages in pagegenerators.py (See https://gerrit.wikimedia.org/r/#/c/166421/2/pywikibot/pagegenerators.py,cm for each in context):
 * 1) pywikibot-enter-category-name
 * 2) pywikibot-enter-file-links-processing
 * 3) pywikibot-enter-page-processing
 * 4) cosmetic_changes-append

The main usage of the i18n module is by the scripts.

In order to package 'Pywikibot', either the three messages used by pagegenerators.py need to be included, or the dependency on those i18n messages needs to be removed. Patches for both approaches have been submitted; i18n_data and i18n fallbacks. Other options have not yet been explored.

scripts
The Pywikibot scripts currently reside in the same repository as the Pywikibot library, and are collectively placed in a Python package called 'scripts'. There are many instances of a script importing another script, using syntax which isnt supported in Python 3.4. This change 'fixes' that, but continues to call the package 'scripts'.

The name 'scripts' is not an appropriate name for the Pywikibot scripts collection if they were to be packaged on pypi.

There have been numerous scattered discussions about moving the scripts out of the Pywikibot library repository.

family classes
Family classes provide a lot of configuration data about wiki families, needed for pywikibot to operate correctly on those wikis. Most of this information could be dynamically determined at runtime without family classes, and progress is being made on replacing family class functionality with dynamic algorithms. However that effort wont be completed for Pywikibot 2.0. As a result, family classes need to be shipped, and these classes need to change quickly if a wiki site configuration changes.

It would undesirable to issue a new minor release of Pywikibot every time a Wikimedia site configuration change requires a modification to the family file, such as a new language being added to a family.

Proposals
TBD. Please discuss on the talk page.