Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Practical Combinatorial Testing: Beyond Pairwise

Published

Author(s)

David R. Kuhn, Yu Lei, Raghu N. Kacker

Abstract

With new algorithms and tools, developers can apply high-strength combinatorial testing to detect elusive failures that occur only when multiple components interact. In pairwise testing, all possible pairs of parameter values are covered by at least one test, and good tools are available to generate arrays with the value pairs. In the past few years, advances in covering-array algorithms, integrated with model checking or other testing approaches, have made it practical to extend combinatorial testing beyond pairwise tests. The US National Institute of Standards and Technology (NIST) and the University of Texas, Arlington, are now distributing freely available methods and tools for constructing large t-way combination test sets (known as covering arrays), converting covering arrays into executable tests, and automatically generating test oracles using model checking (http://csrc.nist.gov/acts). In this review, we focus on real-world problems and empirical results from applying these methods and tools.
Citation
IT Professional (IEEE)
Volume
10
Issue
3

Keywords

all-pairs testing, assurance, combinatorial testing, debugging, faults, pairwise testing

Citation

Kuhn, D. , Lei, Y. and Kacker, R. (2008), Practical Combinatorial Testing: Beyond Pairwise, IT Professional (IEEE), [online], https://doi.org/10.1109/MITP.2008.54 (Accessed December 6, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created June 1, 2008, Updated November 10, 2018