CRT Teleconference
Thursday, August 31, 2006

Participants: Alan Goldfine, Allan Eustis, Dan Schuster, David Flater, John Wack, Max Etschmaier, Sharon Laskowski, Thelma Allen, Wendy Haven


1) Administrative updates (Allan E. and John W.)

2) Critical Issues for Formulating Reliability Requirements (Alan G. and John W.)

Max Etschmaier presentation:

Max Etschmaier paper:

3) Issues List (David F.) See:

4) Any other items.

Meeting began with introductions at 10:05 a.m.

Administrative Updates:

  • Allan Eustis - Just returned from the Wyoming primaries where he was reviewing post election activities. Met with Laramie County IT people - they would very much appreciate a standard XML for voting result outputs. They also thought other states would find a uniform XML valuable.

  • John Wack - Attended EAC meeting. Discussion about expanded scope and issues. Concerns about poll workers, including electronic books. Brian Hancock says these systems will have to be certified. Walked away with thoughts on how requirements must be satisfied. Confusion arose about how to include it.

Critical Issues for Formulating Reliability Requirements - Max Etschmaier

(Note paper and presentation URL above)

  • Alan Goldfine introduced Max and informed the group that he would be looking at old reliability issues in existing specifications. He is not developing conclusions or requirements at this time.

  • Max began with a quick background introduction of his experience, most from aviation (both civil and military). Aviation was quite different then, but still better than our voting systems of today. Reliability was looked at because of their concerns with costs.

  • Although voting machines are different from airlines, the idea of looking at system reliability still applies. Hopefully the same successful outcome will happen.

  • Many of the barriers are institutional barriers.

  • The definition of reliability emphasizes that reliability analysis cannot be focused entirely on obtaining measures, but needs to look at the purpose of the system and the environment, to define measures. Reliability defines the frequency with which "failures" occur.

  • Reliability depends on equipment, maintenance process, and logistics support.

  • Our analysis will require 3 steps: 1) examine definition of "voting system", 2) examine other requirements to see what functions are required of the voting system, and 3) operating environment.

  • Current guidelines require reliability requirements at the machine as well as precinct.

  • Voting machine examination must include software examination.

  • Definition of voting system is consistent with HAVA definition.

  • Functions of a voting system were discussed. Other functions were suggested: verify eligibility of voter, make sure voter has not voted previously, exception handling (disputes problems).

  • Threats to the integrity of these functions come from malfunctions, damage, and tampering.

  • There are two types of failures: critical (essential for completion of system mission or may lead to unacceptable consequences) and non-critical (may disturb operation or have economic consequences).

  • The language of VVSG2005 says critical failures have to be avoided - the requirement is unconditional. If a machine is does not exclude critical failures, it cannot be certified.

  • Next step is to determine cost of a "critical" failure.

  • Max's recommendation is to stay with the current requirement unconditional requirement of no critical failures.

  • Usage pattern - machines are idle most of the time. Use is predictable.

  • Critical failures in the form of interference can only occur during active phase; however, a skilled person could access machine and manipulate it to cause a critical failure outside the active phase - the failure would be that the protection against tampering failed.

  • Two strategies to look at - neither permits critical failures during active stage.

  • Strategy 1: No failure of any kind will occur during active phase. Easy to manage. Works well for simple machines. Certification: Demonstrate machines under conditions similar to actual operations would not fail at a rate that exceeds the limit. Analysis performed by vendor and audited by certifying agency.

  • Strategy 2: Only critical failure are excluded during active phase, non-critical failures are permitted and corrected with maintenance. Requires management and maintenance in place during active phase. Also spare machines. Certification: A disciplined process for managing the logistics system is included in certification of reliability.

  • Discussion of MTBF under strategies 1 and 2 (no failures during critical phase vs. failures by activity)
  • Discussion of narrowing scope
  • Need to approach accepted reliability of today's computers
  • Reliability does not stand alone
  • Need to consider failure in terms of testing context
  • Max will continue one on one dialogue with TGDC members to refine approach.

Meeting adjourned at 11:35 a.m.


Link to NIST HAVA Page

Last updated: July 25, 2007
Point of Contact

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department