U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Search Publications by: Paul E. Black (Fed)

Search Title, Abstract, Conference, Citation, Keyword or Author
Displaying 1 - 25 of 90

Vulnerability Test Suite Generator (VTSG) Version 3

October 13, 2023
Author(s)
Paul E. Black, William Mentzer, Elizabeth Fong, Bertrand Stivalet
The Vulnerability Test Suite Generator (VTSG) Version 3 can create vast numbers of synthetic programs with and without specific flaws or vulnerabilities. Such programs are useful for measuring static analysis tools. VTSG was designed by the Software

SATE VI Report: Bug Injection and Collection

June 14, 2023
Author(s)
Aurelien Delaitre, Paul E. Black, Damien Cupif, Guillaume Haben, Loembe Alex-Kevin, Vadim Okun, Yann Prono, Aurelien Delaitre
The SATE VI report presents the results of a security-focused bug finding evaluation exercise carried out from 2018 to 2023 on various code bases using static analysis tools. Existing bugs were extracted from bug tracker reports and the CVE/NVD database

Impact of Code Complexity On Software Analysis

February 23, 2023
Author(s)
Charles D. De Oliveira, Elizabeth Fong, Paul E. Black
The Software Assurance Metrics and Tool Evaluation (SAMATE) team studied thousands of warnings from static analyzers. Tools have difficulty distinguishing between the absence of a weakness and the presence of a weakness that is buried in otherwise

Guidelines on Minimum Standards for Developer Verification of Software

October 6, 2021
Author(s)
Paul E. Black, Vadim Okun, Barbara Guttman
Executive Order (EO) 14028, Improving the Nation's Cybersecurity, 12 May 2021, directs the National Institute of Standards and Technology (NIST) to recommend minimum standards for software testing within 60 days. This document describes eleven

Algorithms and Data Structures for New Models of Computation

February 1, 2021
Author(s)
Paul E. Black, David W. Flater, Irena Bojanova
In the early days of computer science, the community settled on a simple standard model of computing and a basic canon of general purpose algorithms and data structures suited to that model. With isochronous computing, heterogeneous multiprocessors, flash

SATE VI Ockham Sound Analysis Criteria

May 19, 2020
Author(s)
Paul E. Black, Kanwardeep S. Walia
Static analyzers examine the source or executable code of programs to find problems. Many static analyzers use heuristics or approximations to examine programs with millions of lines of code for hundreds of classes of problems. The Ockham Sound Analysis

Static Analyzers: Seat Belts for Your Code

April 17, 2020
Author(s)
Paul E. Black
Just as seat belt use is wide spread, we argue that the use static analysis should be part of ethical software development. We explain some of the procedures of the four Static Analysis Tool Expositions (SATE), and some of the lessons we learned

Opaque Wrappers and Patching: Negative Results

November 21, 2019
Author(s)
Paul E. Black, Monika Singh
When a patch is released for buggy software, bad actors may be able to analyze the patch and create an attack on unpatched machines. A wrapper could block attacking inputs, but it, too, gives attackers critical information. An opaque wrapper hides such

Formal Methods for Statistical Software

October 4, 2019
Author(s)
Paul E. Black
"Statistical software" encompasses several distinct classes of software. This report explains what formal methods, tools, and approaches may be able to increase assurance of results of using statistical software and implementing differential privacy. To

Information Exposure (IEX): A New Class in the Bugs Framework (BF)

July 9, 2019
Author(s)
Irena Bojanova, Yaacov Yesha, Paul E. Black, Yan Wu
Exposure of sensitive information can be harmful on its own and in addition could enable further attacks. A rigorous and unambiguous definition of information exposure faults can help researchers and practitioners identify them, thus avoiding security

SATE V Report: Ten Years of Static Analysis Tool Expositions

October 23, 2018
Author(s)
Aurelien M. Delaitre, Bertrand C. Stivalet, Paul E. Black, Vadim Okun, Terry S. Cohen, Athos Ribeiro
Software assurance has been the focus of the National Institute of Standards and Technology (NIST) Software Assurance Metrics and Tool Evaluation (SAMATE) team for many years.The Static Analysis Tool Exposition (SATE) is one of the team’s prominent

Juliet 1.3 Test Suite: Changes From 1.2

June 14, 2018
Author(s)
Paul E. Black
The Juliet test suite is a systematic set of thousands of small test programs in C/C++ and Java exhibiting over 100 classes of errors, such as buffer overflow, OS injection, hardcoded password, absolute path traversal, NULL pointer dereference, uncaught

SARD: Thousands of Reference Programs for Software Assurance

October 31, 2017
Author(s)
Paul E. Black
A corpus of computer programs with known bugs is useful in determining the ability of tools to find bugs. This article describes the content of NIST's Software Assurance Reference Dataset (SARD), which is a publicly available collection of thousands of

SATE VI Ockham Sound Analysis Criteria

July 11, 2017
Author(s)
Paul E. Black
In preparation for SATE VI, we present our current thoughts on the Ockham Sound Analysis Criteria track. First, we explain the purpose of the Ockham track and define some terms, such as "sound", "finding", and "site". Then we present the general flow for

Impact of Code Complexity On Software Analysis

February 9, 2017
Author(s)
Charles Daniel De Oliveira, Elizabeth N. Fong, Paul E. Black
The Software Assurance Metrics and Tool Evaluation (SAMATE) team evaluated approximately 800 000 warnings from static analyzers.We learned that elements that we call “code complexities” make the detection of warnings more difficult. Most tools cannot not

Dramatically Reducing Software Vulnerabilities

January 18, 2017
Author(s)
Paul E. Black, Larry Feldman, Gregory A. Witte
This bulletin summarized the information presented in NISTIR 8151: Dramatically Reducing Software Vulnerabilities: Report to the White House Office of Science and Technology Policy. The publication starts by describing well known security risks and

Defeating Buffer Overflow: One of the Most Trivial and Dangerous Bugs of All!

October 31, 2016
Author(s)
Paul E. Black, Irena Bojanova
The C programming language was invented over 40 years ago. It is infamous for buffer overflows. We have learned a lot about computer science, language design, and software engineering since then. As it is unlikely that we will stop using C any time soon

The Bugs Framework (BF): A Structured Approach to Express Bugs

October 13, 2016
Author(s)
Irena Bojanova, Paul E. Black, Yaacov Yesha, Yan Wu
To achieve higher levels of assurance for digital systems, we need to answer questions such as, does this software have bugs of these critical classes? Do these two tools generally find the same set of bugs, or different, complimentary sets? Can we