Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Software Assurance with SAMATE Reference Dataset, Tool Standards, and Studies



Paul E. Black


Today's avionics systems depend more and more on software from many sources: vendors, subcontractors, in-house, and open source. System interactions are exposed to external agents in contexts from air-to-ground links to OS patches downloaded via the Internet. This is a huge amount of software with the risk of attack from distant global sites. Yet users need assurance that the software will work and not create security problems. We focus on NIST's Software Assurance Metrics And Tool Evaluation (SAMATE) project and its contribution. SAMATE is developing specifications, metrics, and automated test suites for software assurance tools. For instance, source code security analyzers can help developers produce software with fewer security flaws. They can also help identify malicious code and poor coding practices that lead to vulnerabilities. The project's publicly available reference dataset, the SRD, contains more than 1800 flawed (and fixed!) program examples to help evaluate software assurance tools and algorithms. These metrics and reference datasets help purchasers confirm tool vendors' claims. We also study the assurance impact of tool use, methods, and techniques.
Conference Title
26th Digital Avionics Systems Conference


Homeland security, samate reference dataset, software assurance, static analysis


Black, P. (2007), Software Assurance with SAMATE Reference Dataset, Tool Standards, and Studies, 26th Digital Avionics Systems Conference, [online], (Accessed May 24, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created October 1, 2007, Updated January 27, 2020