Ordered t-way Combinations for Testing State-based Systems: NIST releases Cybersecurity White Paper 26

Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine whether code has been exercised sufficiently to establish confidence in its functions.

NIST Cybersecurity White Paper 26, Ordered t-way Combinations for Testing State-based Systems, describes the use of ordered t-way combinations of input values in system testing and how to construct adequate blocks of test inputs for state-based systems. This method can be used to ensure that relevant combinations of input values have been tested with adequate diversity of ordering to ensure correct operation. Examples of application areas include network protocols, transaction processing, and other systems where continuous streams of inputs are processed.