Help:Pickle/Glossary

Glossary for pickle tests is a list of core terms and concepts, both for spec>#spec|spec and step>#step|step style tests. Terms are borrowed from the itil>w:ITIL|ITIL, istqb>w:ISTQB|ISTQB, and rspec>w:RSpec|RSpec framework, and predicate logic>w:predicate logic|predicate logic. The different frameworks are linked as necessary. Try to keep close to the definitions, but adapt to this testing environment.

Terms and concepts

 * After
 * Also known as teardown.
 * Marks functions to be run after describe>#describe|describe, context>#context|context, and it>#it|it. It uses a single queue so all functions will be run in the correct sequence.


 * AfterAll
 * Marks functions similar to after>#after|after, except it is run once. Strictly speaking this function is not necessary as code can be added inline.  It use a single queue so all functions will be run in the correct sequence.  It use the same queue as after so it will be run at the correct time in the first test.  At later invocations it will not be run.


 * Around
 * An alternate to after>#after|after that is only run if the test throws an exception>#exception|exception. If an exception is thrown, and there is an around function registered, then no after function will be run.


 * Assertion
 * Test that focuses on the actual code, and assertions about that code, and their falsy condition>#condition|conditions. In depth article on Assertion.


 * Before
 * An alias for this is setup.
 * Marks functions to be run before describe>#describe|describe, context>#context|context, and it>#it|it. It use a single queue so all functions will be run in the correct sequence.


 * BeforeAll
 * Marks functions similar to before>#before</>|before, except it is run once. Strictly speaking this function is not necessary as code can be added inline.  It use a single queue so all functions will be run in the correct sequence.  It use the same queue as before so it will be run at the correct time in the first test.  At later invocations it will not be run.


 * Carp
 * A spy>#spy</>|spy that adds a message to stack without exiting the test, printing the caller's name and its arguments. Partly emulating the <tvar|carp> </> function from perl.


 * Cluck
 * A spy>#spy</>|spy like carp>#carp</>|carp, but also prints a stack trace starting one level up. Partly emulating the <tvar|cluck> </> function from perl.


 * Condition
 * A logical expression that can be evaluated as True or False, e.g., A>B.


 * Confess
 * A spy>#spy</>|spy like croak>#croak</>|croak, but also prints a stack trace starting one level up. Partly emulating the <tvar|confess> </> function from perl.


 * Context
 * Marks functions used as examples, but also a function <tvar|context> </> itself. See example>#example</>|example for details.


 * Coverage
 * The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.


 * Criteria
 * Decision rules used to determine whether a test item (function) or feature has passed or failed a given test. In specs it is only used simple pass/fail.


 * Croak
 * A spy>#spy</>|spy like carp>#carp</>|carp, but also stops the running test (the user provided anonymous function). Partly emulating the <tvar|croak> </>function from perl.  Because it throws an exception it will always trigger a stack trace.


 * Describe
 * Marks functions used as examples, but also a function <tvar|describe> </> itself. See example>#example</>|example for details.


 * Design
 * Also known as test design
 * The process of transforming general test objectives into tangible test conditions and test cases.
 * The document that describes the implementation details of the test, or results of, the system or in our case the module. The document is a subsection on the   subpage for the spec page.


 * Developer
 * A person that write code. See tester>#tester</>|tester


 * Documentation
 * Also known as test documentation
 * The document that describes plans of, or results of, the system or in our case the module. The document is the   subpage for the spec page.


 * Examples
 * The levels in the describe>#describe</>|describe, context>#context</>|context, and it>#it</>|it ladder. The name and level are a bit arbitrary as they all have nearly the same function.  The context is often left out, and only describe and it are used.


 * Exceptions
 * Error states reported from within the code. They will be caught by the test framework.  In depth article on Exception handling.


 * Expectations
 * Tests that focus on the provided objects and examples, and their truthy states.


 * Fail
 * A test is deemed to fail if its actual result does not match its expected result.


 * Feature
 * An attribute of a component or system specified or implied by requirements documentation (for example reliability, usability or design constraints).


 * Fixture
 * The function defining the test is the test fixture, and a frame without a test fixture it will be skipped.


 * Good
 * See tester>#pass</>|pass for details.


 * Harness
 * A test environment comprised of stubs and drivers needed to execute a test.


 * It
 * Marks functions used as examples, but also a function <tvar|it> </> itself. See example>#example</>|example for details.


 * Mock
 * A simulated object that act as a strictly controlled replacement for a real object. A mock is a type of test double>#test double</>|test double.  In depth article on Mock object.


 * Overfitting
 * Happens when there are too tight couplings between the test and source code. Usually happen when tests use spies to verify internal data structures in source code.


 * Pass
 * A test is deemed to pass if its actual result matches its expected result.


 * Pending
 * A frame marked with skip>#skip</>|skip or todo>#todo</>|todo will be in a pending final state. This state is an override of pass>#pass</>|pass ("ok") and fail>#fail</>|fail ("not ok") and makes it possible to code and test with less noise.


 * Return
 * The outermost zero-level that encapsulates the test results and returns them in a formatted fashion.


 * Skip
 * The system (or optionally the author) can mark a test as skipped, either in the description or in the code. Further processing within the frame will then be terminated and the current state used.


 * Spec
 * A type of testing to make sure we build the thing right, usually by writing some kind of unit tests, but can also be more high level.


 * Spies
 * Functions that can be registered on other functions, or injected into code to report or alter internal states. Spying on public calls made before the module is available to the testing regime will not be possible.


 * Step
 * A type of testing to make sure we build the right thing, usually by writing some kind of acceptance tests, which is at the integration level.


 * Stub
 * Simulated code, usually a function in this context, that acts as a strictly controlled replacement for some real code. A stub is a type of test double>#test double</>|test double.  In depth article on Test stub.


 * Subject
 * The object under test. It can be explicitly set or set as part of the examples>#examples</>|examples.  It is passed on to expect>#expect</>|expect.


 * Test
 * An activity to verify proper operation of a system, given that it is executed under specific conditions, and with observed and recorded results.


 * Test case
 * A set of input values, execution preconditions, expected results and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. Can also include documentation for how to run the test case.


 * Test doubles
 * Code replacing pieces of code that are not tested, and should return only known values. In depth article on Test double.


 * Test item
 * The individual element to be tested. There usually is one test object and many test items.


 * Tester
 * A skilled user who is involved in the testing of a component or system. See developer>#developer</>|developer


 * Throw
 * The test code can run into a statement that either can't be executed and thus "throws" an exception or explicitly "throws" an exception at that point. When the code throws an exception further execution is stopped and a report is initiated.


 * Todo
 * The user (or optionally the system) can mark a test as todo, either in the description or in the code. Further processing within the frame will continue, but the final state set to pending anyhow.