NIST logo

Publication Citation: Estimating Fault Detection Effectiveness

NIST Authors in Bold

Author(s): David R. Kuhn; Raghu N. Kacker; Yu Lei;
Title: Estimating Fault Detection Effectiveness
Published: April 01, 2014
Abstract: [Poster] A t-way covering array can detect t-way faults, however they generally include other combinations beyond t-way as well. For example, a particular test set of all 5-way combinations is shown capable of detecting all seeded faults in a test program, despite the fact that it contains up to 9-way faults. This poster gives an overview of methods for estimating fault detection effectiveness of a test set based on combinatorial coverage for a class of software. Detection effectiveness depends on the distribution of t-way faults, which is not known. However based on past experience one could say for example the fraction of 1-way faults is F(sub)1 = 60%, 2-way faults F(sub)2 = 25% F(sub)3 = 10% and F(sub)4 = 5%. Such information could be used in determining the required strength t. It is shown that the fault detection effectiveness of a test set may be affected significantly by the t-way fault distribution, overall, simple coverage at each level of t, number of values per variable, and minimum t-way coverage. Using these results, we develop practical guidance for testers.
Conference: Third International Workshop on Combinatorial Testing
Proceedings: Proceedings of the Seventh IEEE International Conference on Software, Testing, Verification and Validation (ICST 2014)
Pages: pp. 154 - 154
Location: Cleveland, OH
Dates: March 31-April 4, 2014
Keywords: combinatorial testing; software testing; test coverage
Research Areas: Information Technology, Math, Software Testing Metrics
DOI:  (Note: May link to a non-U.S. Government webpage)
PDF version: PDF Document Click here to retrieve PDF version of paper (34KB)