Jump to content

Manual:Pywikibot/Test coverage

From mediawiki.org

To run coverage, install coverage, and optionally nose.

Library

[edit]

To use coverage as a standalone tool, run the following in the pywikibot clone directory:

$ coverage-2.7 run --source=python -m unittest discover -vv -p "*_tests.py"

(and look at results starting with 'pywikibot')

To use coverage via nose, see their documentation and run the following in the pywikibot clone directory:

$ nosetests --with-coverage --cover-package=pywikibot tests

Pywikibot 'core' test coverage on python 2.7 as at 1 November 2014:

Name                                               Stmts   Miss  Cover
----------------------------------------------------------------------
pywikibot                                            292     73    75%
pywikibot.bot                                        445    306    31%
pywikibot.comms.http                                 137     30    78%
pywikibot.comms.threadedhttp                         202     42    79%
pywikibot.config2                                    313    106    66%
pywikibot.data.api                                   731    219    70%
pywikibot.data.wikidataquery                         243     23    91%
pywikibot.date                                       368     37    90%
pywikibot.diff                                       180    154    14%
pywikibot.echo                                        22     16    27%
pywikibot.exceptions                                 116     10    91%
pywikibot.family                                     232     68    71%
pywikibot.i18n                                       267     97    64%
pywikibot.logentries                                 121     42    65%
pywikibot.login                                       86     58    33%
pywikibot.page                                      1803    782    57%
pywikibot.pagegenerators                             726    599    17%
pywikibot.plural                                       3      0   100%   
pywikibot.site                                      2339    915    61%
pywikibot.textlib                                    630    238    62%
pywikibot.throttle                                   172     45    74%
pywikibot.tools                                      295     92    69%
pywikibot.userinterfaces.terminal_interface            7      1    86%
pywikibot.userinterfaces.terminal_interface_base     202    101    50%
pywikibot.userinterfaces.terminal_interface_unix      18     11    39%
pywikibot.userinterfaces.transliteration            2718     15    99%
pywikibot.version                                    221    165    25%
pywikibot.weblib                                      31      3    90%
pywikibot.xmlreader                                  102     20    80%
----------------------------------------------------------------------
TOTAL                                              13028   4268    67%   
Notes
  • the default output gives lines of code which are not exercised; that has been omitted from the table above.(they are included in the table for scripts)
  • the library contains lines which are Python 2.6 or Python 3+ specific, and cant be included in coverage when run from python 2.7
  • there are a few modules which are not included in this table because they are not loaded by the test suite.
  • PYWIKIBOT2_TEST_WRITE=1 was not used when generating the above statistics.

Scripts

[edit]

To use coverage as a standalone tool, run the following in the pywikibot clone directory:

$ coverage-2.7 run --source=python -m unittest discover -vv -p "*_tests.py"

(and look at results starting with 'scripts')

To use coverage via nose, see their documentation and run the following in the pywikibot clone directory:

$ nosetests --with-coverage --cover-package=scripts tests

(FIXME: both of the above commands is horribly inefficient, as it runs all tests, not just tests for scripts.)

Pywikibot 'core' test coverage as at 1 November 2014:

Name                     Stmts   Miss  Cover   Missing
------------------------------------------------------
scripts.archivebot         332    278    16%   149-157, 169-174, 189-198, 202-204, 214, 217, 220, 223, 240-245, 248, 253-263, 266, 270, 273-286, 298-321, 325-350, 354-362, 365, 369-379, 393-414, 417, 420-422, 425, 429, 435-438, 441-451, 462-473, 476-514, 517-550, 562-653, 657
scripts.data_ingestion      81     24    70%   25-26, 53, 126-129, 132-150, 153, 156-157, 160-165
scripts.upload             215    191    11%   53-54, 76-91, 95, 99-153, 158-227, 231-234, 244-289, 294-313, 325-383, 386
------------------------------------------------------
TOTAL                      628    493    21%   

Notes
  • the number of lines which are only exercised in Python 2.6 or Python 3+ should be so small as to not affect the overall results.
  • the TOTALs in this table are very misleading; there are many scripts which are not listed as they are tested in a way that is not visible to coverage.
  • there were no write tests covering scripts as at 1 November 2014.