Workshop on Defining the State of the Art in Software Security Tools

[SAMATE Home | IntrO TO SAMATE | SARD | SATE | Bugs Framework | Publications | Tool Survey | Resources]

10 and 11 August 2005

LOCATION: U.S. National Institute of Standards and Technology

NIST North Building (off the main campus)
820 West Diamond Ave
Gaithersburg, MD

 

 

Contents

• 1 PURPOSE
• 2 DISCUSSION MATTER
• 3 ATTENDANCE and REGISTRATION
• 4 AGENDA
• 5 SCHEDULE
• 6 PROGRAM COMMITTEE

PURPOSE 

Software assurance (SA) tools can help software developers produce software with fewer known security flaws or vulnerabilities. They can also help identify malicious code and poor coding practices that lead to vulnerabilities. There are more than a dozen source code scanners alone, in addition to dozens of other software security tools and services. Reference datasets of clean code and code with security flaws, along with metrics, can help advance the state of the art in software security tools. These metrics and reference datasets can also help purchasers confirm tool vendors' claims. To help develop metrics and reference datasets, the Information Technology Laboratory of the U.S. National Institute of Standards and Technology (NIST) is planning a workshop. One goal of the workshop is to understand the state of the art of SA tools in detecting security flaws and vulnerabilities.

Participants will also discuss

• possible metrics to evaluate the effectiveness of SA security tool
• finding, collecting, or developing a set of flawed and "clean" software to be reference code for such evaluation.

As a result of the workshop, we will publish a report on classes of known software security vulnerabilities and the state of the art of security SA tools.

DISCUSSION MATTER 

We have published references to, rough drafts, preliminary versions, or sketches of the following to help generate discussion and comment:

• classes of software security flaws and vulnerabilities,
• a survey of software assurance security tools and companies,
• the state of the art in software assurance security tools,
• possible metrics to evaluate software assurance security tools (includes features of code scanning tools), and
• properties of a reference dataset of "flawed" and "clean" software.

ATTENDANCE and REGISTRATION 

To help us plan the workshop, please send a brief position statement and professional background information. The position statement should address one or more issues in the workshop purpose. The background information should describe your experience this area and your interest, for instance whether you are a vendor, a user, or a researcher of SA security tools. So that we can get you a NIST visitor pass, please include your full name and country of citizenship. If you are not a U.S. citizen, also include your title (e.g., CEO, Program Mgr.), employer/sponsor, and address.

We invite those who develop, use, purchase, or review software security evaluation tools. Academicians who are working in the area of semi- or completely automated tools to review or assess the security properties of software are especially welcome. We are looking for participation from researchers, students, developers, and users in industry, government, and universities.

Send plain text or PDF submissions to Liz Fong <efong [at] nist.gov (efong[at]nist[dot]gov)>. Your submission constitutes permission for us to publish your position statement and identifying information in workshop proceedings.

AGENDA 

August 10, 2005 

8:45 am Registration 

9:00 am Welcoming Remarks Shashi Phoha, Director, NIST ITL

9:10 am Round Robin Introductions and Workshop Goals Paul Black 

9:30 am Tools Survey and Categorization Facilitator: Elizabeth Fong 

10:15 am Break 

10:25 am Taxonomy of Software Assurance Functions Facilitator: Mike Kass 

11:30 am Lunch (order in) 

1:00 pm Recommended Best Practices, or, State of the Art in SA Tools Facilitator: Brad Martin 

2:00 pm Software Assurance Vulnerability List and Taxonomy Facilitator: Mike Koo 

3:30 pm Break 

3:45 pm Software Assurance Tool Metrics Facilitator: Paul Black 

5:00 pm End of Day 1 

 

August 11, 2005

9:00 am Recap of Previous Day Paul Black 

9:15 am Reference Dataset Facilitator: Mike Sindelar 

10:45 am Break 

11:00 am Next Step Facilitator: Paul Black 

11:30 am Develop Consensus on Workshop Report Facilitator: Paul Black 

12:30 pm End of Workshop 

Workshop Chair: Paul Black

SCHEDULE 

18 June 2005 - Deadline for submission of position statements.
11 July 2005 - Agenda and references, drafts, sketches, etc. published.
10-11 August 2005 - Workshop.
23 September 2005 - Report and proceedings published.

PROGRAM COMMITTEE 

• Paul E. Black - NIST
• Michael Kass - NIST
• Carl E. Landwehr - NSF
• W. Bradley Martin - DOD