NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.

Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.

U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

Practical Combinatorial Testing

Published

Author(s)

David R. Kuhn, Raghu N. Kacker, Yu Lei

Abstract

Combinatorial testing can help detect problems like this early in the testing life cycle. The key insight underlying t-way combinatorial testing is that not every parameter contributes to every fault and most faults are caused by interactions between a relatively small number of parameters. This publication provides a self-contained tutorial on using combinatorial testing for real-world software, including how to use it effectively for system and software assurance. It introduces the key concepts and methods, explains use of software tools for generating combinatorial tests (freely available on the NIST web site csrc.nist.gov/acts), and discusses advanced topics such as the use of formal models of software to determine the expected results for each set of test inputs. With each topic, a section on costs and practical considerations explains tradeoffs and limitations that may impact resources or funding. The material is accessible to an undergraduate student of computer science or engineering, and includes an extensive set of references to papers that provide more depth on each topic.
Citation
Special Publication (NIST SP) - 800-142
Report Number
800-142
NIST Pub Series
Special Publication (NIST SP)
Pub Type
NIST Pubs

Download Paper

DOI Link

Keywords

Combinatorial methods, computer security, software assurance, software testing
Conformance testing, Cybersecurity and privacy and Software testing

Citation

Kuhn, D. , Kacker, R. and Lei, Y. (2010), Practical Combinatorial Testing, Special Publication (NIST SP), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.SP.800-142 (Accessed October 15, 2025)

Issues

If you have any questions about this publication or are having problems accessing it, please contact [email protected].

Created October 7, 2010, Updated November 10, 2018
Was this page helpful?