U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Search Publications by: D. Richard Kuhn (Fed)

Search Title, Abstract, Conference, Citation, Keyword or Author
Displaying 1 - 25 of 195

Proxy Validation and Verification for Critical AI Systems

September 26, 2024
Author(s)
Phillip Laplante, Joanna DeFranco, D. Richard Kuhn, Jeff Voas
This white paper offers a suggestion that prior testing artifacts from similar AI systems can be reused for new AI software. Testing AI and Machine learning software is difficult, and if prior testing results from similar systems could be applied as a

Leveraging Combinatorial Coverage in the Machine Learning Product Lifecycle

June 27, 2024
Author(s)
Jaganmohan Chandrasekaran, erin lanus, tyler cody, laura freeman, Raghu N. Kacker, M S Raunak, D. Richard Kuhn
The data-intensive nature of machine learning (ML)-enabled systems introduces unique challenges in test and evaluation. We present an overview of combinatorial coverage, exploring its applications across the ML-enabled system lifecycle and its potential to

Ensuring Reliability Through Combinatorial Coverage Measures

May 14, 2024
Author(s)
M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei
A key question in software assurance is, "How much testing is enough?" Coverage criteria such as statement or branch coverage were developed to help answer this question, but we also need to measure whether tests are sufficiently representative of inputs

On Combinatorial Security Testing for the Tor Anonymity Network Client

April 7, 2024
Author(s)
Dimitris Simos, Bernhard Garn, Dominik-Philip Schreiber, Manuel Leithner, D. Richard Kuhn, Raghu N. Kacker
In this paper, we present an application of combinatorial security testing to the well-known anonymity network Tor. Rigorous testing of the Tor network is important to evaluate not only its functionality, but also the security it provides to its users

Challenges of Assured Autonomy

February 28, 2024
Author(s)
D. Richard Kuhn
This article summarizes some recent novel approaches to the problem of verification, testing, and assurance of autonomous systems. These include proxy verification and combinatorial methods for input space coverage measurement, which also has applications

Combinatorial testing for building reliable systems

February 5, 2024
Author(s)
M S Raunak, D. Richard Kuhn, Raghu N. Kacker, Yu Lei
Combinatorial testing is an approach where test suites are developed by efficiently covering interactions of parameter values and configuration settings. Multiple studies over the years have shown the interesting phenomenon where almost all defects in a

Use Cases for Secure and Trusted Granular Data Sharing Among Disparate Databases

January 30, 2024
Author(s)
Joanna DeFranco, David Ferraiolo, Joshua Roberts, D. Richard Kuhn
Sharing data among disparate organizations can be extremely difficult, when the data comes from different database management systems (DBMS). Most problematic is that data stored at another organization most likely uses different DBMS schemas and

Predicting ABM Results with Covering Arrays and Random Forests

June 26, 2023
Author(s)
Megan Olsen, M S Raunak, D. Richard Kuhn
Simulation is a useful and effective way to analyze and study complex, real-world systems. It allows researchers, practitioners, and de- cision makers to make sense of the inner working of a system that involves many factors often resulting in some sort of

Ordered t-way Combinations for Testing State-based Systems

May 29, 2023
Author(s)
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if the code has been exercised sufficiently to

Synthetic Data Generation Using Combinatorial Testing and Variational Autoencoder

May 29, 2023
Author(s)
Krishna Khadka, Jaganmohan Chandrasekaran, Yu Lei, Raghu N. Kacker, D. Richard Kuhn
Data is a crucial component in machine learning. However, many datasets contain sensitive information such as personally identifiable health and financial data. Access to these datasets must be restricted to avoid potential security concerns. Synthetic

AI Assurance for the Public -- Trust but Verify, Continuously

October 3, 2022
Author(s)
Phillip Laplante, D. Richard Kuhn
Artificial intelligence (AI) systems are increasingly seen in many public facing applications such as self-driving land vehicles, autonomous aircraft, medical systems and financial systems. AI systems should equal or surpass human performance, but given

Ordered t-way Combinations for Testing State-based Systems

June 13, 2022
Author(s)
D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Fault detection often depends on the specific order of inputs that establish states which eventually lead to a failure. However, beyond basic structural coverage metrics, it is often difficult to determine if code has been exercised sufficiently to ensure

Developing multithreaded techniques and improved constraint handling for the tool CAgen

June 8, 2022
Author(s)
Michael Wagner, Manuel Leithner, Dimitris Simos, D. Richard Kuhn, Raghu N. Kacker
CAgen is a state-of-the-art combinatorial test generation tool that is known for its execution speed. In addition, it supports an extensive list of features such as constraint handling, higher-index arrays, and import and export of models/test sets in

A Data Structure for Integrity Protection with Erasure Capability

May 20, 2022
Author(s)
D. Richard Kuhn
This document describes a data structure, referred to as a data block matrix, that supports the ongoing addition of hash-linked records while also allowing for the deletion of arbitrary records, thereby preserving hash-based integrity assurance that other

The Path to Consensus on Artificial Intelligence Assurance

March 15, 2022
Author(s)
Laura Freeman, Feras Batarseh, D. Richard Kuhn, M S Raunak, Raghu N. Kacker
Widescale adoption of intelligent algorithms requires that Artificial Intelligence (AI) engineers provide assurances that an algorithm will perform as intended. Providing such guarantees involves quantifying capabilities and the associated risks across

A Pseudo Exhaustive Software Testing Framework for Embedded Digital Devices in Nuclear Power

June 14, 2021
Author(s)
Athira Jayakumar, D. Richard Kuhn, Brandon Simons, Aidan Collins, Smitha Gautham, Richard Hite, Raghu N. Kacker, Abhi Rajagopala, Carl Elks
The major challenge faced by the nuclear industry related to software testing of digital embedded devices is the identification of practical software (SW) testing solutions that provide a strong technical basis and is at the same time effective in

Combinatorially XSSing Web Application Firewalls

April 12, 2021
Author(s)
Bernhard Garn, Daniel S. Lang, Manuel Leithner, D. Richard Kuhn, Raghu N. Kacker, Dimitris Simos
Cross-Site scripting (XSS) is a common class of vulnerabilities in the domain of web applications. As it remains prevalent despite continued efforts by practitioners and researchers, site operators often seek to protect their assets using web application

Combinatorial Testing Metrics for Machine Learning

April 12, 2021
Author(s)
Erin Lanus, Laura Freeman, D. Richard Kuhn, Raghu N. Kacker
This short paper defines a combinatorial coverage metric for comparing machine learning (ML) data sets and proposes the differences between data sets as a function of combinatorial coverage. The paper illustrates its utility for evaluating and predicting

Combinatorial Test Generation for Multiple Input Models with Shared Parameters

March 17, 2021
Author(s)
Chang Rao, Nan Li, Yu Lei, Jin Guo, YaDong Zhang, Raghu N. Kacker, D. Richard Kuhn
Combinatorial testing typically considers a single input model and creates a single test set that achieves t-way coverage. This paper addresses the problem of combinatorial test generation for multiple input models with shared parameters. We formally

A Trusted Federated System to Share Granular Data Among Disparate Database Resources

March 15, 2021
Author(s)
Joanna DeFranco, David F. Ferraiolo, D. Richard Kuhn, Joshua D. Roberts
Sharing data between different organizations is a challenge primarily due to database management systems (DBMSs) being different types that impose different schemas to represent and retrieve data. In addition, maintaining security and privacy is a concern