Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Measuring and Specifying Combinatorial Coverage of Test Input Configurations

Published

Author(s)

David R. Kuhn, Raghu N. Kacker, Yu Lei

Abstract

A key issue in testing is how many tests are needed for a required level of coverage or fault detection. Estimates are often based on error rates in initial testing, or on code coverage. For example, tests may be run until a desired level of statement or branch coverage is achieved. Combinatorial methods present an opportunity for a different approach to estimating required test set size, using characteristics of the test set. This paper describes methods for estimating the coverage of, and ability to detect, t-way interaction faults of a test set based on a covering array. We also develop a connection between (static) combinatorial coverage and (dynamic) code coverage, such that if a specific condition is satisfied, 100 % branch coverage is assured. Using these results, we propose practical recommendations for using combinatorial coverage in specifying test requirements, and for improving estimates of the fault detection capacity of a test set.
Citation
Innovations in Systems and Software Engineering

Keywords

combinatorial testing, configuration model, covering array, fault coverage, state-space coverage, t-way testing, verification and validation (V&V)

Citation

Kuhn, D. , Kacker, R. and Lei, Y. (2015), Measuring and Specifying Combinatorial Coverage of Test Input Configurations, Innovations in Systems and Software Engineering, [online], https://doi.org/10.1007/s11334-015-0266-2 (Accessed May 24, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created November 14, 2015, Updated November 10, 2018