Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

SATE V Ockham Sound Analysis Criteria



Paul E. Black, Athos Ribeiro


Static analyzers examine the source or executable code of programs to find problems. Many static analyzers use some heuristics or approximations to handle programs up to millions of lines of codes. We established the Ockham Sound Analysis Criteria to recognize static analyzers whose findings are always correct. In brief the criteria are (1) the analyzer's findings are claimed to always be correct, (2) it produces findings for most of a program, and (3) even one incorrect finding disqualifies an analyzer. This document begins by explaining the background and requirements of the Ockham Criteria in more detail. In Static Analysis Tool Exposition (SATE) V, one tool, Frama-C, examined pertinent parts of the Juliet 1.2 test suite to participate. We reviewed eight classes of warnings, including improper buffer access, NULL pointer dereference, integer overflow, and others. This document details the many technical and theoretical challenges we addressed to classify and review the warnings against the Criteria. It also reports anomalies, our observations, and interpretations. Frama-C reports led to the discovery of three unintentional, systematic flaws in the Juliet test suite involving 416 test cases. Our conclusion is that Frama-C satisfied the SATE V Ockham Sound Analysis Criteria.
NIST Interagency/Internal Report (NISTIR) - 8113
Report Number


software assurance, sound analysis, static analysis, Ockham Criteria


Black, P. and Ribeiro, A. (2016), SATE V Ockham Sound Analysis Criteria, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], (Accessed April 21, 2024)
Created March 22, 2016, Updated May 6, 2019