Technical Guidelines Development Committee (TGDC)
Security and Transparency Subcommittee (STS) Teleconference *
June 26, 2007, 10:30 a.m.
Draft Minutes


1) Administrative Updates
2) Review for TGDC meeting
3) Discussion of open ended vulnerability testing
4) Other items

Attendees: Alexia Scott-Morrison, Alicia Clay, Allan Eustis, Andrew Regenscheid, Barbara Guttman, Bill Burr, David Wagner, Elle Colver (EAC), Helen Purcell, John Wack, Mat Masterson (EAC), Nelson Hastings, Quynh Dang, Sharon Laskowski

Administrative Updates:

  • Allan: The meeting material for the July 3rd plenary has been posted on the web, sent via email, and fedexed on CDs.
  • Allan: The draft VVSG has been posted on the web in final format, and CDs containing the VVSG have been sent to TGDC members.
  • Allan: We will be using a "hand raising" internet software tool (see: ) for questions and comments during the plenary meeting. Instructions have been provided to TGDC members. If internet access is unavailable to any member during the meeting, please speak up.
  • Allan: Sections of the VVSG will be presented by subcommittee at the July 3rd meeting.
  • Nelson: The plan for the STS subcommittee presentation at the plenary meeting is to go through each section and point out major changes. There will also be a discussion on OEVT and Software Independence.
  • Nelson will be presenting STS material at the June 29th HFP teleconference - HFP is interested specifically the sections on VVPR, electronic records, and auditing. Nelson will forward HFP's agenda to STS when it is available.

Open Ended Vulnerability Testing (OEVT) (Alicia Clay):

Alicia went over requirements 1.1.2.F and 1.1.2.G regarding failure criteria, and noted that Santosh Chokani had comments that still needed to be taken into consideration. It was clarified and verified that the OEVT team does not have to exploit a threat; they just need to analyze and prove the vulnerability to fail the system. Two ways to fail a system were discussed. A system would fail if it does not meet all the requirements laid out in the VVSG. A system is also fail-able if there are vulnerabilities discovered (even if not covered directly in the standard) that could be exploited to change the outcome of an election. Requirements 1.1.2.F and 1.1.2.G will be reworded to clarify specifics. It was also decided to change the current draft requirements so that the OEVT team did not "define failure criteria" to "use failure criteria". There will be informative discussion text added to clarify what is meant by serious vulnerabilities. It was noted that the testing labs are the ones that make the final call on pass/fail of a system based on the OEVT team's report.

Alicia discussed questions and feedback received on the OEVT requirements:

  • Why three experts? Requirements were written based on review of other teams set up to do similar work. One to two people are not enough, large numbers cause communication problems.
  • Can we provide more guidance on the expertise of team members requirement? The requirements, as written, call for a minimal level of experience. It was decided to reference requirements done for other "penetration teams". Team members will be chosen based on experience, professional reputation, and previous "red team" experience if any.
  • Cost - how do we justify? The cost was derived by the level of effort needed to do credible OEVT testing - level of effort, number of weeks needed. David Wagner described the level of effort in a high level overview - there are three steps: 1) documentation review and understanding system architecture, 2) architectural level analysis to understand where critical security risks are, and 3) narrowly targeted investigations of specific parts of the architecture as guided by analysis and documentation review.
  • Has this been done in any states already?

The question arose as to how the OEVT requirements would be presented and the group was informed that all requirements were presented to the EAC as draft recommendations.

There was discussion regarding whether a system failed if during the testing process the OEVT team was able to break into the system but evidence was left that this had occurred. It was discussed that there is no uniform rule about evidence tracking on the system and that each requirement being tested must be looked at individually. It was decided that based on conversation there would not be a change to the requirements but that a "Rules of Engagement" paper should be written.


  • There is a pre-draft document circulating around on Independent Voter Verifiable Records. This will be discussed in more detail after the June 29th HFP meeting.
  • VVPR and SI requirements were added to the Conformance Clause for those wishing to review.
  • REMINDER: Anyone wishing to attend the June 29th HFP meeting on Friday to go over STS material is welcome.

[* Pursuant to the Help America Vote Act of 2002, the TGDC is charged with directing NIST in performing voting systems research so that the TGDC can fulfill its role of recommending technical standards for voting equipment to the EAC. This teleconference served the purposes of the STS subcommittee of the TGDC to direct NIST staff and coordinate voting-related research relevant to the VVSG 2007. Discussions on this telecon are preliminary and do not necessarily reflect the views of NIST or the TGDC.]

Teleconferences from 2004, 2005, 2006 and upcoming in 2006.


Link to NIST HAVA Page

Last updated: July 25, 2007
Point of Contact

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department