Help:Pickle/en

'This is a page describing a proposed solution. It is not finalized, but an on-going work.'''

Sorry about that, to all those that want to do full spec-tests in Lua right away!''

''At some point this extension will probably be renamed, most likely to "pickles". You pick on something, and use gherkin-like languages, so pickles seems like a good name.''

Pickle is a project to integrate automated testing into an environment for continuous integration, like Wikipedia which use Lua-scripts. Within Wikipedia Lua is used to implement advanced templates, and the same solution is used on a lot of sites and projects. Continuous integration is a core element of continuous deployment, which is very important for sites that must be up and running 24×7. Pickle use among other spec-style testing, which can be described as a variant of unit testing, or how to make the thing right. What to build, and how to build it, is not the only important thing. It is also utterly important to reduce the overall risk associated with running live Lua development within an active Mediawiki project.

Background
Testing allows us to say something qualitative about the software, that is the modules in a wiki, such as the number and type of known defects, especially for functional requirements. That helps the developers to identify and fix defects during development, but also builds the community’s confidence in the final module. When someone later on want to use the module as part of a template, then she can check out the results, possibly being assured that this is in fact a good module, before calling the module as part of the template. If something happen later on, possibly because of a defect in some other included module, then the tests will hopefully identify the root cause of the problem.

In this solution anyone can make the necessary tests, but it should be expected that the user developing the code would also develop the tests. That user knows best what to test and how to test it. You don't accept half-done code, you should neither accept half-done tests. Still it is easy to forget some important cases, and when you see such cases do add them! Better test coverage means less defects now and in the future.

Quick tour
How to set up and use pickle tests, run them interactively and understand the reports. Some concepts are described on Help:Pickle/Basic idea, but it is only a very coarse and simple description. There are some specific terms concerning pickle tests in general and its use on wikis in particular. They have been defined in.

Advanced tools
Some advanced tools are available, for specific purposes. They have been regrouped on these subjects:


 * The pickle docpage can be given a special template that not only report the results on the doc page but also categorizes the page if the tests fails.
 * The pickle docpage can be given a special template that not only report the results on the doc page but also categorizes the page if the tests fails.


 * Both test and source pages can run tests in the console. At the test page only a single test set will be run, but at the source page all subpages that is recognized as pickle pages will be run.
 * Both test and source pages can run tests in the console. At the test page only a single test set will be run, but at the source page all subpages that is recognized as pickle pages will be run.

Under construction

 * A subpage is looked up and called as part of a parsed message. This lookup and parsing makes it possible to define a localized fallback chain, or even include and parse new test frameworks.
 * A subpage is looked up and called as part of a parsed message. This lookup and parsing makes it possible to define a localized fallback chain, or even include and parse new test frameworks.


 * The parsed text from the subpage will be processed by a simple extractor to see if it is possible to identify specific terms and phrases. If anyone are found then a specific token will be set before further processing.
 * The parsed text from the subpage will be processed by a simple extractor to see if it is possible to identify specific terms and phrases. If anyone are found then a specific token will be set before further processing.


 * The test report can be the final tokens used by the extension, but it can also be according to the Test Anything Protocol. It will then make an attempt to rewrite the tap-report into the final token.
 * The test report can be the final tokens used by the extension, but it can also be according to the Test Anything Protocol. It will then make an attempt to rewrite the tap-report into the final token.


 * All modules that has specs will have a page indicator for the current result of the test, both as part of the symbol (by coloring) and the message (as text).
 * All modules that has specs will have a page indicator for the current result of the test, both as part of the symbol (by coloring) and the message (as text).


 * All modules that has specs will have a tracking category for the current result of the test. It will follow the result as shown by the page indicator.
 * All modules that has specs will have a tracking category for the current result of the test. It will follow the result as shown by the page indicator.


 * All modules that has specs will add entries to the test log for the current result of the test. It will follow the result as shown by the page indicator.
 * All modules that has specs will add entries to the test log for the current result of the test. It will follow the result as shown by the page indicator.


 * It is possible to calculate software metrics for modules, and especially cyclomatic complexity is a very interesting metric as it gives upper and lower bounds for the number of necessary tests.
 * It is possible to calculate software metrics for modules, and especially cyclomatic complexity is a very interesting metric as it gives upper and lower bounds for the number of necessary tests.