Thursday, August 31, 2006
Alan Goldfine, Allan Eustis, Dan Schuster, David Flater, John Wack, Max
Etschmaier, Sharon Laskowski, Thelma Allen, Wendy Haven
updates (Allan E. and John W.)
2) Critical Issues
for Formulating Reliability Requirements (Alan G. and John W.)
3) Issues List
(David F.) See: http://vote.nist.gov/TGDC/crt/CRT-WorkingDraft-20060823/Issues.html
4) Any other items.
Meeting began with
introductions at 10:05 a.m.
- Allan Eustis -
Just returned from the Wyoming primaries where he was reviewing post
election activities. Met with Laramie County IT people - they would
very much appreciate a standard XML for voting result outputs. They
also thought other states would find a uniform XML valuable.
- John Wack - Attended
EAC meeting. Discussion about expanded scope and issues. Concerns about
poll workers, including electronic books. Brian Hancock says these systems
will have to be certified. Walked away with thoughts on how requirements
must be satisfied. Confusion arose about how to include it.
for Formulating Reliability Requirements - Max Etschmaier
(Note paper and
presentation URL above)
- Alan Goldfine
introduced Max and informed the group that he would be looking at old
reliability issues in existing specifications. He is not developing
conclusions or requirements at this time.
- Max began with
a quick background introduction of his experience, most from aviation
(both civil and military). Aviation was quite different then, but still
better than our voting systems of today. Reliability was looked at because
of their concerns with costs.
- Although voting
machines are different from airlines, the idea of looking at system
reliability still applies. Hopefully the same successful outcome will
- Many of the barriers
are institutional barriers.
- The definition
of reliability emphasizes that reliability analysis cannot be focused
entirely on obtaining measures, but needs to look at the purpose of
the system and the environment, to define measures. Reliability defines
the frequency with which "failures" occur.
- Reliability depends
on equipment, maintenance process, and logistics support.
- Our analysis will
require 3 steps: 1) examine definition of "voting system",
2) examine other requirements to see what functions are required of
the voting system, and 3) operating environment.
- Current guidelines
require reliability requirements at the machine as well as precinct.
- Voting machine
examination must include software examination.
- Definition of
voting system is consistent with HAVA definition.
- Functions of a
voting system were discussed. Other functions were suggested: verify
eligibility of voter, make sure voter has not voted previously, exception
handling (disputes problems).
- Threats to the
integrity of these functions come from malfunctions, damage, and tampering.
- There are two
types of failures: critical (essential for completion of system mission
or may lead to unacceptable consequences) and non-critical (may disturb
operation or have economic consequences).
- The language of
VVSG2005 says critical failures have to be avoided - the requirement
is unconditional. If a machine is does not exclude critical failures,
it cannot be certified.
- Next step is to
determine cost of a "critical" failure.
- Max's recommendation
is to stay with the current requirement unconditional requirement of
no critical failures.
- Usage pattern
- machines are idle most of the time. Use is predictable.
- Critical failures
in the form of interference can only occur during active phase; however,
a skilled person could access machine and manipulate it to cause a critical
failure outside the active phase - the failure would be that the protection
against tampering failed.
- Two strategies
to look at - neither permits critical failures during active stage.
- Strategy 1: No
failure of any kind will occur during active phase. Easy to manage.
Works well for simple machines. Certification: Demonstrate machines
under conditions similar to actual operations would not fail at a rate
that exceeds the limit. Analysis performed by vendor and audited by
- Strategy 2: Only
critical failure are excluded during active phase, non-critical failures
are permitted and corrected with maintenance. Requires management and
maintenance in place during active phase. Also spare machines. Certification:
A disciplined process for managing the logistics system is included
in certification of reliability.
- Discussion of
MTBF under strategies 1 and 2 (no failures during critical phase vs.
failures by activity)
- Discussion of
- Need to approach
accepted reliability of today's computers
- Reliability does
not stand alone
- Need to consider
failure in terms of testing context
- Max will continue
one on one dialogue with TGDC members to refine approach.
at 11:35 a.m.
to NIST HAVA Page
Last updated: July 25, 2007
Point of Contact
policy / security notice / accessibility statement
NIST is an agency of the U.S. Commerce Department