NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.

Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.

U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

Combinatorial Testing for Cybersecurity and Reliability

Published

Author(s)

David R. Kuhn, Raghu N. Kacker, Larry Feldman, Gregory A. Witte

Abstract

This bulletin focuses on NIST's combinatorial testing work. Combinatorial testing is a proven method for more effective software testing at lower cost. The key insight underlying combinatorial testing's effectiveness resulted from a series of studies by NIST from 1999 to 2004. NIST research showed that most software bugs and failures are caused by one or two parameters, with progressively fewer by three or more.
Citation
ITL Bulletin - May 2016
Report Number
May 2016
NIST Pub Series
ITL Bulletin
Pub Type
NIST Pubs

Download Paper

Local Download

Keywords

algorithms, combinatorial testing, constraints, covering array, software testing, test suite
Information technology

Citation

Kuhn, D. , Kacker, R. , Feldman, L. and Witte, G. (2016), Combinatorial Testing for Cybersecurity and Reliability, ITL Bulletin, National Institute of Standards and Technology, Gaithersburg, MD, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=920921 (Accessed October 9, 2025)

Issues

If you have any questions about this publication or are having problems accessing it, please contact [email protected].

Created May 12, 2016, Updated February 19, 2017
Was this page helpful?