Measuring and Specifying Combinatorial Coverage of Test Input Configurations

Published

November 14, 2015

Author(s)

David R. Kuhn, Raghu N. Kacker, Yu Lei

Abstract

A key issue in testing is how many tests are needed for a required level of coverage or fault detection. Estimates are often based on error rates in initial testing, or on code coverage. For example, tests may be run until a desired level of statement or branch coverage is achieved. Combinatorial methods present an opportunity for a different approach to estimating required test set size, using characteristics of the test set. This paper describes methods for estimating the coverage of, and ability to detect, t-way interaction faults of a test set based on a covering array. We also develop a connection between (static) combinatorial coverage and (dynamic) code coverage, such that if a specific condition is satisfied, 100 % branch coverage is assured. Using these results, we propose practical recommendations for using combinatorial coverage in specifying test requirements, and for improving estimates of the fault detection capacity of a test set.

Citation

Innovations in Systems and Software Engineering

Pub Type

Journals

Download Paper

https://doi.org/10.1007/s11334-015-0266-2

Keywords

combinatorial testing, configuration model, covering array, fault coverage, state-space coverage, t-way testing, verification and validation (V&V)

Citation

Kuhn, D. , Kacker, R. and Lei, Y. (2015), Measuring and Specifying Combinatorial Coverage of Test Input Configurations, Innovations in Systems and Software Engineering, [online], https://doi.org/10.1007/s11334-015-0266-2 (Accessed March 5, 2025)