Ensuring Reliability through Combinatorial Sequence Coverage

Published

June 9, 2025

Author(s)

M S Raunak, David Kuhn, Raghu Kacker

Abstract

Errors in software and hardware systems do not always result from only a single set of input values. Instead a series of input values or events often lead to failures. These sequence of events may or may not need to be contiguous. In either scenario, structural coverage metrics prevalent in traditional test selection approaches often fall short in exercising the code sufficiently to ensure exercising of different sequence of inputs or events thoroughly to build confidence in its functions. We need to have metrics and measures to gauge how much of relevant combinations of input values and input sequences have been tested and verified for correct operation. Going beyond simple coverage of sequences in a test, value combinations over multiple tests in a sequence are also important for building highly reliable systems. Combinatorial sequence coverage as well as non-adjacent and adjacent ordered combination coverage (OCC and OCCa) measures provide efficient means of achieving this type of verification. This paper discusses combinatorial sequence coverage, OCC, OCCa, and their application and usefulness in runtime verification, vulnerability and fault detection, and reliability assurance.

Citation

IEEE Reliability Magazine

Pub Type

Journals

Download Paper

Local Download

Keywords

Combinatorial Testing, Sequence Coverage, Ordered Sequence Coverage

Citation

Raunak, M. , Kuhn, D. and Kacker, R. (2025), Ensuring Reliability through Combinatorial Sequence Coverage, IEEE Reliability Magazine, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=960007 (Accessed March 31, 2026)