Estimating Fault Detection Effectiveness

Published: April 01, 2014


David R. Kuhn, Raghu N. Kacker, Yu Lei


[Poster] A t-way covering array can detect t-way faults, however they generally include other combinations beyond t-way as well. For example, a particular test set of all 5-way combinations is shown capable of detecting all seeded faults in a test program, despite the fact that it contains up to 9-way faults. This poster gives an overview of methods for estimating fault detection effectiveness of a test set based on combinatorial coverage for a class of software. Detection effectiveness depends on the distribution of t-way faults, which is not known. However based on past experience one could say for example the fraction of 1-way faults is F(sub)1 = 60%, 2-way faults F(sub)2 = 25% F(sub)3 = 10% and F(sub)4 = 5%. Such information could be used in determining the required strength t. It is shown that the fault detection effectiveness of a test set may be affected significantly by the t-way fault distribution, overall, simple coverage at each level of t, number of values per variable, and minimum t-way coverage. Using these results, we develop practical guidance for testers.
Proceedings Title: Proceedings of the Seventh IEEE International Conference on Software, Testing, Verification and Validation (ICST 2014)
Conference Dates: March 31-April 4, 2014
Conference Location: Cleveland, OH
Conference Title: Third International Workshop on Combinatorial Testing
Pub Type: Conferences


combinatorial testing, software testing, test coverage
Created April 01, 2014, Updated February 19, 2017