Take a sneak peek at the new NIST.gov and let us know what you think!
(Please note: some content may not be complete on the beta site.).
NIST Authors in Bold
|Author(s):||Laleh Ghandehari; Jacek Czerwonka; Yu Lei; Soheil Shafiee; Raghu N. Kacker; David R. Kuhn;|
|Title:||An Empirical Comparison of Combinatorial and Random Testing|
|Published:||April 01, 2014|
|Abstract:||Some conflicting results have been reported on the comparison between t-way combinatorial testing and random testing. In this paper, we report a new study that applies t-way and random testing to the Siemens suite. In particular, we investigate the stability of the two techniques. We measure both code coverage and fault detection effectiveness. Each program in the Siemens suite has a number of faulty versions. In addition, mutation faults are used to better evaluate fault detection effectiveness in terms of both number and diversity of faults. The experimental results show that in most cases, t-way testing performed as good as or better than random testing. There are few cases where random testing performed better, but with a very small margin. Overall, the differences between the two techniques are not as significant as one would have probably expected. We discuss the practical implications of the results. We believe that more studies are needed to better understand the comparison of the two techniques.|
|Conference:||Third International Workshop on Combinatorial Testing|
|Proceedings:||Proceedings of the Seventh IEEE International Conference on Software, Testing, Verification and Validation (ICST 2014)|
|Pages:||pp. 68 - 77|
|Dates:||March 31-April 4, 2014|
|Keywords:||combinatorial testing, random testing, software testing|
|Research Areas:||Information Technology, Math, Software Testing Metrics|
|DOI:||http://dx.doi.org/10.1109/ICSTW.2014.8 (Note: May link to a non-U.S. Government webpage)|
|PDF version:||Click here to retrieve PDF version of paper (731KB)|