Abstract
This document provides a set of metrics, including test suites and methods, to determine how well a particular source code security analysis tool conforms to the requirements specified in Source Code Security Analysis Tool Functional Specification Version 1.0 [SCA]. Each relevant programming language in [SCA] has a corresponding set of test suites. The test suites are intended to be used by tool developers and tool users alike to increase their level of confidence in product quality. Each test suite consists of test cases that are designed to evaluate against various requirements of [SCA], including mandatory features and optional features. Each test case contains test description, weakness contained in the test case, expected result and test code. The detailed information of the test case, such as start parameters, procedures for executing a test file and test file itself can be retrieved from the SAMATE Reference Dataset (SRD)
http://samate.nist.gov/SRD/. As this document evolves, new versions will be posted to the web site at
http://samate.nist.gov/index.php/Source_Code_Security_Analysis.html.