Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Combinatorial Coverage Measurement of Test Vectors used in Cryptographic Algorithm Validation (Presentation)



Dimitris Simos, Stavros Mekesis, D. Richard Kuhn, Raghu N. Kacker


We measured the combinatorial coverage of test vectors provided by the NIST Cryptographic Algorithm Validation Program (CAVP). Input models were defined and test vectors measured and analyzed for 2-way, 3-way, and 4-way combinatorial coverage. The results of our measurement show that some test vectors do not achieve a full 2-way to 4-way combinatorial coverage, so we generated the missing combinations for these vectors and extended the test suites to achieve a full 2-way coverage. We also conducted differential testing on popular AES implementations, such as OpenSSL, LibreSSL, Crypto++, PyCrypto, using the extended test suites. Our differential testing of AES implementations on these test suites showed no discrepancies between the implementations. Finally, we use the NIST Cryptographic Algorithm Validation System (CAVS) as a golden system against which the AES implementations are tested.
Conference Dates
September 25-28, 2017
Conference Location
Gaithersburg, MD, US
Conference Title
IEEE Software Technology Conference


AES, combinatorial testing, combinatorics, computer security, cryptography


Simos, D. , Mekesis, S. , Kuhn, D. and Kacker, R. (2017), Combinatorial Coverage Measurement of Test Vectors used in Cryptographic Algorithm Validation (Presentation), IEEE Software Technology Conference, Gaithersburg, MD, US, [online], (Accessed May 23, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created September 27, 2017, Updated October 12, 2021