Technical Guidelines Development Committee (TGDC)
Human Factors and Privacy Subcommittee (HFP)*
August 10, 2007
Draft Minutes


1. Administrative updates (Eustis)
2. Overview of status of benchmarks: renamed, reworded in Chapter 3. Revised white paper will be forthcoming.
3. Discussion of how to set benchmarks.
4. Summary of other HFP materials in preparation for TGDC meeting.
5. Other issues

Attendees: Alice Miller, Allan Eustis, David Baquis (U.S. Access Board), John Cugini, John Wack, Nelson Hastings, Philip Pearce, Sharon Laskowski, Whitney Quesenbery

Administrative Updates (Allan Eustis):

  • IEEE has nominated Dr. Cem Kaner (Florida Institute of Technology) to be its representative on the TGDC. He has a PhD in experimental psychology, Juris Doctor, and an extensive background in computer science.
  • Last week Dr. Jeffrey announced his NIST departure for September 3, 2007.
  • The next scheduled STS teleconference (Tuesday, Aug 14) will be dedicated for reviewing the VVSG. All TGDC members invited to attend.
  • NIST has been working on a number of logistical items including an executive summary and companion documents for the VVSG. The web will have a set of matrices posted that map all requirements to their source
  • The TGDC plenary is still scheduled to begin at 11:30 ET. We will be using the TRACE hand raising software. We hope to have all power point presentations by next Wednesday.

Benchmarks (Sharon L):

  • Sharon has distributed a detailed technical report outlining the benchmarks work. She has also produced a 7 page overview summarizing the reporting and providing the benchmarks.
  • We have three benchmarks: voter inclusion index (a function of the average error rate, used to be called the accuracy index), total completion rate, and perfect ballot index (now a ratio).
  • The question we're currently dealing with is how to write them into the current standard. Currently the standard is written with the requirements stating the systems have to meet these 3 benchmarks, (which we are including current numbers for) with a statement saying that we are continuing to test and final numbers are forthcoming. Discussion was held about how high and how stringent those numbers should be. This question will be posed to the TGDC as a whole at the plenary meeting.
  • HFP would like to make recommendation to EAC that these benchmark numbers are reviewed periodically (this does not involve changing the standard or the requirements, just the benchmark numbers).

List of Notable Changes (John Cugini):

  • Introduction, section 2.2 and 2.3 (What's New) - nice summary of what's new from VVSG 2005
  • Introduction, section 3.5 (Relationship of VVSG and HAVA) - this applies to the whole standard but originated from HFP
  • Part 1, section 6.4.5 (Maintainability of voting system) - no longer in
  • Part 2, sections 3.6.2-A (vendor reporting of their own summative usability tests), 4.4.6-A (usability of poll worker documentation), 4.4.6-B (vendor must document the system's accessibility procedures) and 6.1-R (test lab must report the metrics for usability of the system) Moved back to Chapter 3 in Part 1 from the document requirements section.
  • Part 3, section 3.3 (Performance Testing (Benchmarking)) - talks about performance testing benchmarks in general
  • Part 3, section 4.5 (Accessibility Test Methods) - J. Cugini feels this could should be dropped


  • Sharon will be working on her presentation slides over the weekend to present benchmarking materials and recommendations, as well as changes to HFP's sections. Any comments should be emailed to her about any other material needing to be covered at meeting.
  • Allan discussed the approval process for the VVSG. Each subcommittee will present and then a resolution will be offered to approve that section. At the end of the meeting a resolution will be offered to approve the document in whole. Over the next 2 to 3 weeks edits will be made.
  • Everyone is invited to the STS teleconference on Tuesday, August 14, 10:30 am to go over the VVSG.
  • TGDC plenary on Friday, August 17, 2007, 11:30 a.m. ET.


[* Pursuant to the Help America Vote Act of 2002, the TGDC is charged with directing NIST in performing voting systems research so that the TGDC can fulfill its role of recommending technical standards for voting equipment to the EAC. This teleconference discussion serves the purposes of the HFP subcommittee of the TGDC to direct NIST staff and coordinate its voting-related research relevant to the VVSG 2007. Discussions on this telecon are preliminary and do not necessarily reflect the views of NIST or the TGDC.]



Link to NIST HAVA Page

Last updated: July 25, 2007
Point of Contact

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department