Ordered t-way Combinations for Testing State-based Systems
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if code has been exercised sufficiently to ensure confidence in its functions. Measures are needed to ensure that relevant combinations of input values have been tested with adequate diversity of ordering, to ensure correct operation. Combinatorial testing and combinatorial coverage measures have been applied to many types of applications, but have some deficiencies for verification and testing of state based systems where the response depends on both input values and current system state. In such systems, internal states change as input values are processed. Examples include network protocols, which may be in listening, partial connection, full connection, disconnected, and many other states depending on the values of packet fields and order of packets received. Similarly, merchant account balances in credit card systems change continuously as transactions are processed. This publication introduces a notion of ordered t-way combinations, proves a result regarding the construction of adequate blocks of test inputs, and discusses the application of the results to verification and testing of state based systems.
, Raunak, M.
and Kacker, R.
Ordered t-way Combinations for Testing State-based Systems, OTHER, National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.CSWP.26, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=934829
(Accessed October 2, 2022)