NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.
Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.
An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Applying Combinatorial Testing to the Siemens Suite
Published
Author(s)
Laleh Ghandehari, Mehra N. Borazjany, Yu Lei, Raghu N. Kacker, D. Richard Kuhn
Abstract
Combinatorial testing has attracted a lot of attention from both industry and academia. A number of reports suggest that combinatorial testing can be effective for practical applications. However, there still seems to lack systematic, controlled studies on the effectiveness of combinatorial testing. In particular, input parameter modeling is a key step in the combinatorial testing process. But most studies do not report the details of the modeling process. In this paper, we report an experiment that applies combinatorial testing to the Siemens suite. The Siemens suite has been used as a benchmark to evaluate the effectiveness of many testing techniques. Each program in the suite has a number of faulty versions. The effectiveness of combinatorial testing is measured in terms of the number of faulty versions that are killed. The experimental results show that combinatorial testing is effective in terms of killing most of the faulty versions with a small number of tests. In addition, we report the details of our modeling process, which we hope to shed some lights on this critical, yet often ignored step, in the combinatorial testing process.
Proceedings Title
Proceedings of Sixth IEEE International Conference on Software Testing, Verification and Validation
ICST 2013
Conference Dates
March 18-22, 2013
Conference Location
Luxembourg, LT
Conference Title
Second International Workshop on Combinatorial Testing
Ghandehari, L.
, Borazjany, M.
, Lei, Y.
, Kacker, R.
and Kuhn, D.
(2013),
Applying Combinatorial Testing to the Siemens Suite, Proceedings of Sixth IEEE International Conference on Software Testing, Verification and Validation
ICST 2013, Luxembourg, LT, [online], https://doi.org/10.1109/ICSTW.2013.47, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=913206
(Accessed October 1, 2025)