NIST logo

Publication Citation: Combinatorial (t-way) testing for software: an adaptation of design of experiments

NIST Authors in Bold

Author(s): Raghu N. Kacker;
Title: Combinatorial (t-way) testing for software: an adaptation of design of experiments
Published: November 01, 2013
Abstract: Most modern instruments of measurements for science, engineering, and commerce have embedded software. Also software is required for data handling and mathematical computations. Therefore verification and validation of software used in metrology is important. Combinatorial testing is a versatile methodology for testing software. It is based on the insight that while the behavior of system may be affected by a large number of factors, only a few factors are involved in a fault. We introduce combinatorial testing as an adaptation of design of experiment methods for testing software. Combinatorial testing began as pairwise testing in which first orthogonal arrays and then covering arrays were used to make sure that all relevant pairs of the test setting of test factors were tested. Subsequent investigations of actual software failures showed that pairwise (2-way) testing may not always be sufficient. Until recently efficient tools for generating test suites for combinatorial (t-way) testing for t > 2 were not widely available. Combinatorial (t-way) testing for t  2 has become practical because efficient and free downloadable tools for t-way testing with support of constraints (to exclude invalid combinations) have become available.
Conference: 20-th IMEKO World Congress
Volume: 46
Issue: 9
Pages: pp. 3745 - 3752
Location: Busan, -1
Dates: September 9-14, 2012
Keywords: Cyber-security, Covering Arrays, Design of Experiments, Metrology, Orthogonal Arrays, Software Engineering, Software Testing
Research Areas: Math, Software
PDF version: PDF Document Click here to retrieve PDF version of paper (226KB)