New Report on Improving Software Tools that Improve Software

If the shiny new software on your computer or mobile phone runs without crashing, you may have another computer program to thank--a static analyzer. Static analyzers try to find weaknesses in other programs that could be triggered accidentally or exploited by hackers. A new report by the National Institute of Standards and Technology (NIST) documents the Static Analysis Tool Exposition (SATE), an exercise by NIST and static analyzer vendors to improve the performance of these tools.

The report is the culmination of a lengthy effort to host and then digest the results of SATE, begun in February 2008 to help toolmakers assess their products' ability to find security defects in other software. Eight tool developers, along with a ninth team of professional human reviewers, participated in SATE, which provided a non-competitive environment for the vendors to compare their program analysis techniques for the benefit of the entire group.

Software assurance tools may be obscure outside the world of professional software development. However, their importance has increased as programs grow longer, more sophisticated, and increasingly are required to interact with other programs over computer networks. The number and subtlety of attacks from hackers has also increased. Because it is impossible to anticipate every combination of inputs a given piece of software might receive, static analyzers attempt to use mathematical and logical tools to rigorously predict the behavior of the program and examine it for weaknesses based on its code or set of instructions.

NIST software assurance expert Vadim Okun says SATE was a long-overdue idea.

"Most modern software is too lengthy and complex to analyze by hand," says Okun. "Additionally, programs that would have been considered secure ten years ago may now be vulnerable to hackers. We're trying to focus on identifying what in a program's code might be exploitable."

The participating vendors brought a range of tools that possessed different features and analyzed programs written in two different languages. According to Okun, the depth of the field made SATE as much a learning experience for the NIST team as it was for the participants.

"We intend to hold more expositions in the future and will use this experience to help shape their focus," Okun says.

And according to the organizers and several participants, a good deal of research remains to be done. The effort was not only highly demanding, but it also showed some goals may be out of reach. While users want static analyzers to find all the problems in a piece of software, but also raise no false alarms, "that's not achievable," Okun says. "We want to show people that this isn't a trivial process, but the tools are improving and it makes good sense to use them."

The SATE report is available online at http://samate.nist.gov/docs/NIST_Special_Publication_500-279.pdf. NIST is planning the next SATE and invites tool makers interested in participating to get in touch with Okun at vadim.okun [at] nist.gov (vadim[dot]okun[at]nist[dot]gov) or (301) 975-3268.

V. Okun, R. Gaucher and P.E. Black (editors). Static Analysis Tool Exposition (SATE). NIST Special Publication 500-279. June 30, 2009.