Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Ordered t-way Combinations for Testing State-based Systems

Published

Author(s)

D. Richard Kuhn, M S Raunak, Raghu N. Kacker

Abstract

Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if the code has been exercised sufficiently to ensure confidence in its functions. Measures are needed to ensure that relevant combinations of input values have been tested with adequate diversity of ordering to ensure correct operation. Combinatorial testing and combinatorial coverage measures have been applied to many types of applications but have some deficiencies for verifying and testing state-based systems where the response depends on both input values and the current system state. In such systems, internal states change as input values are processed. Examples include network protocols, which may be in listening, partial connection, full connection, disconnected, and other states depending on the values of packet fields and the order of packets received. Similarly, merchant account balances in credit card systems change continuously as transactions are processed. This publication introduces a notion of ordered t- way combinations, proves a result regarding the construction of adequate blocks of test inputs, and discusses the application of the results to verify and test state-based systems.
Proceedings Title
IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)
Conference Dates
April 16-20, 2023
Conference Location
Dublin, IE
Conference Title
12th International Workshop on Combinatorial Testing

Keywords

combinatorial coverage, combinatorial methods, combinatorial testing, software testing, structural coverage, test coverage

Citation

Kuhn, D. , Raunak, M. and Kacker, R. (2023), Ordered t-way Combinations for Testing State-based Systems, IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), Dublin, IE, [online], https://doi.org/10.1109/ICSTW58534.2023.00050, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=936178 (Accessed December 7, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created May 29, 2023, Updated July 17, 2024