Written by Member
Software Testing Techniques
Here is a description of some variations of the fundamental metrics and some less commonly use metrics.
Function Coverage
This metric reports whether you invoked each function or procedure. It is useful during preliminary testing to assure at least some coverage in all areas of the software. Broad, shallow testing finds gross deficiencies in a test suite quickly.
Call Coverage
This metric reports whether you executed each function call. The hypothesis is that bugs commonly occur in interfaces between modules.
Also known as call pair coverage.
Linear Code Sequence and Jump (LCSAJ) Coverage
This variation of path coverage considers only sub-paths that can easily be represented in the program source code, without requiring a flow graph [Woodward1980]. An LCSAJ is a sequence of source code lines executed in sequence. This "linear" sequence can contain decisions as long as the control flow actually continues from one line to the next at run-time. Sub-paths are constructed by concatenating LCSAJs. Researchers refer to the coverage ratio of paths of length n LCSAJs as the test effectiveness ratio (TER) n+2.
The advantage of this metric is that it is more thorough than decision coverage yet avoids the exponential difficulty of path coverage. The disadvantage is that it does not avoid infeasible paths.
Data Flow Coverage
This variation of path coverage considers only the sub-paths from variable assignments to subsequent references of the variables.
The advantage of this metric is the paths reported have direct relevance to the way the program handles data. One disadvantage is that this metric does not include decision coverage. Another disadvantage is complexity. Researchers have proposed numerous variations, all of which increase the complexity of this metric. For example, variations distinguish between the use of a variable in a computation versus a use in a decision, and between local and global variables. As with data flow analysis for code optimization, pointers also present problems.
Object Code Branch Coverage
This metric reports whether each machine language conditional branch instruction both took the branch and fell through.
This metric gives results that depend on the compiler rather than on the program structure since compiler code generation and optimization techniques can create object code that bears little similarity to the original source code structure.
Since branches disrupt the instruction pipeline, compilers sometimes avoid generating a branch and instead generate an equivalent sequence of non-branching instructions. Compilers often expand the body of a function inline to save the cost of a function call. If such functions contain branches, the number of machine language branches increases dramatically relative to the original source code.
You are better off testing the original source code since it relates to program requirements better than the object code.
Loop Coverage
This metric reports whether you executed each loop body zero times, exactly once, and more than once (consecutively). For do-while loops, loop coverage reports whether you executed the body exactly once, and more than once.
The valuable aspect of this metric is determining whether while-loops and for-loops execute more than once, information not reported by other metrics.
Race Coverage
This metric reports whether multiple threads execute the same code at the same time. It helps detect failure to synchronize access to resources. It is useful for testing multi-threaded programs such as in an operating system.
Relational Operator Coverage
This metric reports whether boundary situations occur with relational operators (<, <=, >, >=). The hypothesis is that boundary test cases find off-by-one mistakes and uses of the wrong relational operators such as < instead of <=. For example, consider the following C/C++ code fragment:
if (a < b)
statement;
Relational operator coverage reports whether the situation a==b occurs. If a==b occurs and the program behaves correctly, you can assume the relational operator is not suppose to be <=.
Weak Mutation Coverage
This metric is similar to relational operator coverage but much more general. It reports whether test cases occur which would expose the use of wrong operators and also wrong operands. It works by reporting coverage of conditions derived by substituting (mutating) the program's expressions with alternate operators, such as "-" substituted for "+", and with alternate variables substituted.
This metric interests the academic world mainly. Caveats are many; programs must meet special requirements to enable measurement.
Table Coverage
This metric indicates whether each entry in a particular array has been referenced. This is useful for programs that are controlled by a finite state machine.
