Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Combinatorial testing for building reliable systems

Published

Author(s)

M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei

Abstract

Combinatorial testing is an approach where test suites are developed by efficiently covering interactions of parameter values and configuration settings. Multiple studies over the years have shown the interesting phenomenon where almost all defects in a system originate from interactions of a few specific parameters or settings. Efficient algorithms compressing these value combinations into a small number of tests have made this method practical for industrial use, providing better testing at lower cost. Through this approach, fault detection close to the level of exhaustive testing can be achieved with a 20x to 700x reduction in the test suite size. Since most defects in systems can be discovered with systematic testing using 2- to 6-way interactions of parameter values, utilizing this approach can help us develop highly reliable systems.
Citation
IEEE Reliability Magazine
Volume
1
Issue
1

Keywords

combinatorial testing, reliability

Citation

Raunak, M. , Kuhn, D. , Kacker, R. and Lei, Y. (2024), Combinatorial testing for building reliable systems, IEEE Reliability Magazine, [online], https://doi.org/10.1109/MRL.2024.3355874, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=957313 (Accessed December 5, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created February 5, 2024, Updated August 22, 2024