Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Search Publications by: D. Richard Kuhn (Fed)

Search Title, Abstract, Conference, Citation, Keyword or Author
Displaying 1 - 25 of 189

Leveraging Combinatorial Coverage in the Machine Learning Product Lifecycle

June 27, 2024
Jaganmohan Chandrasekaran, erin lanus, tyler cody, laura freeman, Raghu N. Kacker, M S Raunak, D. Richard Kuhn
The data-intensive nature of machine learning (ML)-enabled systems introduces unique challenges in test and evaluation. We present an overview of combinatorial coverage, exploring its applications across the ML-enabled system lifecycle and its potential to

On Combinatorial Security Testing for the Tor Anonymity Network Client

April 7, 2024
Dimitris Simos, Bernhard Garn, Dominik-Philip Schreiber, Manuel Leithner, D. Richard Kuhn, Raghu N. Kacker
In this paper, we present an application of combinatorial security testing to the well-known anonymity network Tor. Rigorous testing of the Tor network is important to evaluate not only its functionality, but also the security it provides to its users

Predicting ABM Results with Covering Arrays and Random Forests

June 26, 2023
Megan Olsen, M S Raunak, D. Richard Kuhn
Simulation is a useful and effective way to analyze and study complex, real-world systems. It allows researchers, practitioners, and de- cision makers to make sense of the inner working of a system that involves many factors often resulting in some sort of

Ordered t-way Combinations for Testing State-based Systems

May 29, 2023
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if the code has been exercised sufficiently to

Synthetic Data Generation Using Combinatorial Testing and Variational Autoencoder

May 29, 2023
Krishna Khadka, Jaganmohan Chandrasekaran, Yu Lei, Raghu N. Kacker, D. Richard Kuhn
Data is a crucial component in machine learning. However, many datasets contain sensitive information such as personally identifiable health and financial data. Access to these datasets must be restricted to avoid potential security concerns. Synthetic

AI Assurance for the Public -- Trust but Verify, Continuously

October 3, 2022
Phillip Laplante, D. Richard Kuhn
Artificial intelligence (AI) systems are increasingly seen in many public facing applications such as self-driving land vehicles, autonomous aircraft, medical systems and financial systems. AI systems should equal or surpass human performance, but given

Ordered t-way Combinations for Testing State-based Systems

June 13, 2022
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if code has been exercised sufficiently to ensure

Developing multithreaded techniques and improved constraint handling for the tool CAgen

June 8, 2022
Michael Wagner, Manuel Leithner, Dimitris Simos, D. Richard Kuhn, Raghu N. Kacker
CAgen is a state-of-the-art combinatorial test generation tool that is known for its execution speed. In addition, it supports an extensive list of features such as constraint handling, higher-index arrays, and import and export of models/test sets in

A Data Structure for Integrity Protection with Erasure Capability

May 20, 2022
D. Richard Kuhn
This document describes a data structure, referred to as a data block matrix, that supports the ongoing addition of hash-linked records while also allowing for the deletion of arbitrary records, thereby preserving hash-based integrity assurance that other

The Path to Consensus on Artificial Intelligence Assurance

March 15, 2022
Laura Freeman, Feras Batarseh, D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Widescale adoption of intelligent algorithms requires that Artificial Intelligence (AI) engineers provide assurances that an algorithm will perform as intended. Providing such guarantees involves quantifying capabilities and the associated risks across

A Pseudo Exhaustive Software Testing Framework for Embedded Digital Devices in Nuclear Power

June 14, 2021
Athira Jayakumar, D. Richard Kuhn, Brandon Simons, Aidan Collins, Smitha Gautham, Richard Hite, Raghu N. Kacker, Abhi Rajagopala, Carl Elks
The major challenge faced by the nuclear industry related to software testing of digital embedded devices is the identification of practical software (SW) testing solutions that provide a strong technical basis and is at the same time effective in

Combinatorially XSSing Web Application Firewalls

April 12, 2021
Bernhard Garn, Daniel S. Lang, Manuel Leithner, D. Richard Kuhn, Raghu N. Kacker, Dimitris Simos
Cross-Site scripting (XSS) is a common class of vulnerabilities in the domain of web applications. As it remains prevalent despite continued efforts by practitioners and researchers, site operators often seek to protect their assets using web application

Combinatorial Testing Metrics for Machine Learning

April 12, 2021
Erin Lanus, Laura Freeman, D. Richard Kuhn, Raghu N. Kacker
This short paper defines a combinatorial coverage metric for comparing machine learning (ML) data sets and proposes the differences between data sets as a function of combinatorial coverage. The paper illustrates its utility for evaluating and predicting

Combinatorial Test Generation for Multiple Input Models with Shared Parameters

March 17, 2021
Chang Rao, Nan Li, Yu Lei, Jin Guo, YaDong Zhang, Raghu N. Kacker, D. Richard Kuhn
Combinatorial testing typically considers a single input model and creates a single test set that achieves t-way coverage. This paper addresses the problem of combinatorial test generation for multiple input models with shared parameters. We formally

A Trusted Federated System to Share Granular Data Among Disparate Database Resources

March 15, 2021
Joanna DeFranco, David F. Ferraiolo, D. Richard Kuhn, Joshua D. Roberts
Sharing data between different organizations is a challenge primarily due to database management systems (DBMSs) being different types that impose different schemas to represent and retrieve data. In addition, maintaining security and privacy is a concern

Combinatorial Methods for Explainable AI

October 24, 2020
David R. Kuhn, Raghu N. Kacker, Yu Lei, Dimitris Simos
This paper introduces an approach to producing explanations or justifications of decisions made by artificial intelligence and machine learning (AI/ML) systems, using methods derived from fault location in combinatorial testing. We use a conceptually

Vulnerability Trends in Web Servers and Browsers

September 11, 2020
M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Richard Kogut
In previous work we have looked at trends in vulnerabilities due to ordinary programming errors [2, 3]. This analysis focuses on two of the most widely used types of software in today's internet, web browsers and web servers. In addition to reports of

Effectiveness of dataset reduction in testing machine learning algorithms

August 25, 2020
Raghu N. Kacker, David R. Kuhn
Abstract— Many machine learning algorithms examine large amounts of data to discover insights from hidden patterns. Testing these algorithms can be expensive and time-consuming. There is a need to speed up the testing process, especially in an agile