Take a sneak peek at the new NIST.gov and let us know what you think!
(Please note: some content may not be complete on the beta site.).

View the beta site
NIST logo

Publication Citation: Applying Combinatorial Testing to the Siemens Suite

NIST Authors in Bold

Author(s): Laleh Ghandehari; Mehra N. Borazjany; Yu Lei; Raghu N. Kacker; David R. Kuhn;
Title: Applying Combinatorial Testing to the Siemens Suite
Published: March 22, 2013
Abstract: Combinatorial testing has attracted a lot of attention from both industry and academia. A number of reports suggest that combinatorial testing can be effective for practical applications. However, there still seems to lack systematic, controlled studies on the effectiveness of combinatorial testing. In particular, input parameter modeling is a key step in the combinatorial testing process. But most studies do not report the details of the modeling process. In this paper, we report an experiment that applies combinatorial testing to the Siemens suite. The Siemens suite has been used as a benchmark to evaluate the effectiveness of many testing techniques. Each program in the suite has a number of faulty versions. The effectiveness of combinatorial testing is measured in terms of the number of faulty versions that are killed. The experimental results show that combinatorial testing is effective in terms of killing most of the faulty versions with a small number of tests. In addition, we report the details of our modeling process, which we hope to shed some lights on this critical, yet often ignored step, in the combinatorial testing process.
Conference: Second International Workshop on Combinatorial Testing
Proceedings: Proceedings of Sixth IEEE International Conference on Software Testing, Verification and Validation ICST 2013
Pages: pp. 362 - 371
Location: Luxembourg, -1
Dates: March 18-22, 2013
Keywords: Combinatorial Testing, Input Modeling, Software Testing
Research Areas: Software, Math
DOI: http://dx.doi.org/10.1109/ICSTW.2013.47  (Note: May link to a non-U.S. Government webpage)
PDF version: PDF Document Click here to retrieve PDF version of paper (1MB)