NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.

Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.

U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

Automated Combinatorial Test Methods: Beyond Pairwise Testing

Published

Author(s)

David R. Kuhn, Raghu N. Kacker, Yu Lei

Abstract

Pairwise testing has become a popular approach to software quality assurance because it often provides effective error detection at low cost. However, pairwise (2-way) coverage is not sufficient for assurance of mission-critical software. Combinatorial testing beyond pairwise is rarely used because good algorithms have not been available for complex combinations such as 3- way, 4-way, or more. In addition, significantly more tests are required for combinations beyond pairwise testing, and testers must determine expected results for each set of inputs. This article introduces new tools for automating the production of complete test cases covering up to 6-way combinations.
Citation
Crosstalk (Hill AFB): the Journal of Defense Software Engineering
Volume
21
Issue
6
Pub Type
Journals

Download Paper

Local Download

Keywords

combinatorial testing, formal methods, model checking, pairwise testing, software assurance, software testing
Software testing

Citation

Kuhn, D. , Kacker, R. and Lei, Y. (2008), Automated Combinatorial Test Methods: Beyond Pairwise Testing, Crosstalk (Hill AFB): the Journal of Defense Software Engineering, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=152162 (Accessed October 25, 2025)

Issues

If you have any questions about this publication or are having problems accessing it, please contact [email protected].

Created June 2, 2008, Updated February 19, 2017
Was this page helpful?