Combinatorial (t-way) testing for software: an adaptation of design of experiments

Published

November 1, 2013

Author(s)

Raghu N. Kacker

Abstract

Most modern instruments of measurements for science, engineering, and commerce have embedded software. Also software is required for data handling and mathematical computations. Therefore verification and validation of software used in metrology is important. Combinatorial testing is a versatile methodology for testing software. It is based on the insight that while the behavior of system may be affected by a large number of factors, only a few factors are involved in a fault. We introduce combinatorial testing as an adaptation of design of experiment methods for testing software. Combinatorial testing began as pairwise testing in which first orthogonal arrays and then covering arrays were used to make sure that all relevant pairs of the test setting of test factors were tested. Subsequent investigations of actual software failures showed that pairwise (2-way) testing may not always be sufficient. Until recently efficient tools for generating test suites for combinatorial (t-way) testing for t > 2 were not widely available. Combinatorial (t-way) testing for t  2 has become practical because efficient and free downloadable tools for t-way testing with support of constraints (to exclude invalid combinations) have become available.

Volume

46

Issue

9

Conference Dates

September 9-14, 2012

Conference Location

Busan

Conference Title

20-th IMEKO World Congress

Pub Type

Conferences

Download Paper

Local Download

Keywords

Cyber-security, Covering Arrays, Design of Experiments, Metrology, Orthogonal Arrays, Software Engineering, Software Testing

Citation

Kacker, R. (2013), Combinatorial (t-way) testing for software: an adaptation of design of experiments, 20-th IMEKO World Congress, Busan, -1, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=910783 (Accessed October 15, 2025)