Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Isolating Failure-Inducing Combinations in Combinatorial Testing using Test Augmentation and Classification

Published

Author(s)

Kiran Shakya, Tao Xie, Nuo Li, Yu Lei, Raghu N. Kacker, D. Richard Kuhn

Abstract

Combinatorial Testing (CT) is a systematic way of sampling input parameters of the software under test (SUT). A t-way combinatorial test set can exercise all behaviors of the SUT caused by interactions between t input parameters or less. Although combinatorial testing can provide fault detection capability, it is often desirable to isolate the input combinations that cause failures. Isolating these failure-inducing combinations aids developers in understanding the causes of failures. Previous work directly uses classification tree analysis on the results of combinatorial testing to model the failure inducing combinations. But in many scenarios, the effectiveness of classification depends upon whether the analyzed test set is sufficient for classification. In addition, generating combinatorial tests for more-than-6-way combination is generally expensive. To address these issues, we propose an approach that uses existing combinatorial testing results to generate additional tests that enhance the effectiveness of classification. In addition, our approach also includes a technique to reduce the complexity of the resulting classification tree so that developers can understand the nature of failure-inducing combinations. We present the preliminary results of our approach applied on the TCAS benchmark.
Proceedings Title
Proceedings of the IEEE Fifth International Conference on Software, Testing, Verification and Validation (ICST 2012)
Conference Dates
April 17-21, 2012
Conference Location
Montreal, CA
Conference Title
First International Workshop on Combinatorial Testing

Keywords

Classification Tree, Combinatorial Testing, Fault Localization

Citation

Shakya, K. , Xie, T. , Li, N. , Lei, Y. , Kacker, R. and Kuhn, D. (2012), Isolating Failure-Inducing Combinations in Combinatorial Testing using Test Augmentation and Classification, Proceedings of the IEEE Fifth International Conference on Software, Testing, Verification and Validation (ICST 2012), Montreal, CA, [online], https://doi.org/10.1109/ICST.2012.149, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=911450 (Accessed March 29, 2024)
Created April 20, 2012, Updated October 12, 2021