The NIST SAMATE (Software Assurance Metrics And Tool Evaluation) project is dedicated to improving software assurance by developing methods to enable software tool evaluations, measuring the effectiveness of tools and techniques, and identifying gaps in tools and methods. This project supports the Department of Homeland Security's Software Assurance Tools and R&D Requirements Identification Program - in particular, Part 3, Technology (Tools and Requirements), the identification, enhancement and development of software assurance tools. The scope of the SAMATE project is broad: ranging from operating systems to firewalls, SCADA to web applications, source code security analyzers to correct-by-construction methods.
The SAMATE project is an inter-agency project between the U.S. Department of Homeland Security and NIST, and consists of two parts:
The first part classifies Software Security Assurance tools and develops metrics and tests for each tool class. Source/object code vulnerability scanners are an example of one possible class. A series of workshops is used to develop recommendations for:
Metrics and tests for these functionalities are developed. Classification and testing activities proceed simultaneously. As a result, a draft specification and test methodology for the highest priority tool class is developed. These activities develop the infrastructure needed for testing efforts in following years.
The SSA tool testing effort supports the second part of the project: identifying deficiencies in SSA methodologies and tools. Like the activities in the first part of the project, this part of the project develops an infrastructure for assessment and periodic reporting.