Technical Guidelines Development Committee (TGDC)
Human Factors and Privacy Subcommittee (HFP)*
June 8, 2007
Draft Minutes


1. Administrative Updates (Allan Eustis)
2. Discussion of Edits for Cast Vote, and Other Small Edits
3. Benchmark Status
4. Other Issues

Attendees: Alexis Scott-Morrison, Allan Eustis, Commissioner Davidson (EAC), David Baquis (U.S. Access Board), John Cugini, Mat Masterson (EAC), Sharon Laskowski, Wendy Havens, Whitney Quesenbery (late)

Administrative Updates (Allan Eustis):

  • We are planning logistics for the July 3rd plenary teleconference. We will be using internet based "Hand Raising" software to cue up questions for the Chair at the meeting. Authors are busily working on chapters for the VVSG, we will be giving them as much time as possible. CD of the VVSG will be sent out a week before the plenary; Hard copies will be available on the 28th to those that request them.

Modifications to VVSG Chapter (John Cugini):

  • Under 3.1.2 (Special Terminology): Added the phrase "summative usability testing" to the "common industry format" definition. Added the definition for "summative usability testing" following ISO/IEC 25062:2006, with regards to voting. This answers a request by STS.
  • Under 3.2.2.D (Notification for Ballot Casting): Added requirements to include notifications for both successful ballot casting and for unsuccessful casting. It was broken into 3 requirements. Requirement pertains to DREs and PCOS systems. John Cugini feels it would not be good to discuss the testing strategy in the discussion field. Sharon will check with Whitney.
  • Under (Privacy): Added discussion to clarify requirement. No system shall provide mechanism for anyone to be able to tell how another person voted.
  • 3.2.5.G (Legibility): These are new requirements. [In the usability section, we use the term poor vision as opposed to partial or low vision.]
  • NOTE: The U.S. Access Board recommends changing <partial vision> to <low vision>. This is OK with subcommittee, John Cugini to make changes.
  • 3.2.7 (Alternative Languages): There were questions from the last HFP meeting about using English vs. alternative language vs. primary language. There's nothing changed here from last week, we're waiting on feedback from EAC.
  • 3.3.2 (Partial Vision): Per earlier conversation this will be changed to Low Vision. Added a reference to point to other visual requirements.

Usability Performance Benchmarks (Sharon Laskowski):

There were four different types of voting systems tested, with about 50 participants for each system. We'd like to have about 100 people each to tighten the confidence intervals. Using the statistics collected and analyzed, we will have benchmarks for completion rate, perfectly cast ballots, and accuracy. The voting systems used had been used before for federal elections and they were only certified with VVS 2002 standards. The fixes that need to occur on the voting systems appear to be easily accomplishable; we do not feel we are setting impossible benchmarks. Other data was also collected and will be reported for time and subjective satisfaction (there will not be benchmarks for these). The TGDC agreed that accuracy measures could be a pass/fail benchmark and that time/efficiency/satisfaction would be informational data points.

Sharon will pick benchmarks over the weekend and fill them into the standard. The data used to set these benchmarks will be available for anyone wishing to see how numbers were reached. The test protocol will be made public.


David Baquis brought up the issue of two font sizes and the requirements having to do with magnification. It was decided that the requirements would be left as written.

Our next telecon is scheduled for: Friday, June 15, 2007 at 11 AM ET

[* Pursuant to the Help America Vote Act of 2002, the TGDC is charged with directing NIST in performing voting systems research so that the TGDC can fulfill its role of recommending technical standards for voting equipment to the EAC. This teleconference discussion serves the purposes of the HFP subcommittee of the TGDC to direct NIST staff and coordinate its voting-related research relevant to the VVSG 2007. Discussions on this telecon are preliminary and do not necessarily reflect the views of NIST or the TGDC.]



Link to NIST HAVA Page

Last updated: July 25, 2007
Point of Contact

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department