2010-05-21 14:00 EDT Auditability Working Group telecon Minutes rev. 2010-05-27 - * - Pursuant to the Help America Vote Act of 2002, the Technical Guidelines Development Committee (TGDC) is charged with directing NIST to conduct voting systems research so that the TGDC can fulfill its role of recommending technical standards for voting equipment to the Election Assistance Commission (EAC). This teleconference discussion is for the purposes of the Auditability Working Group of the TGDC to direct NIST staff and coordinate voting related research relevant to the development of Voluntary Voting Systems Guidelines (VVSG). Discussions on this teleconference including referenced discussion papers are preliminary and pre-decisional, and do not necessarily reflect the views of the National Institute of Standards and Technology, or the TGDC. - * - Agenda: * Finalize minutes of previous meeting. * Review available records of the VVPAT accessibility decision(s). * Reach a consensus regarding whether paperless approaches are in scope. * Start narrowing the scope for auditability. Pending items: * Awaiting more information on EBM-scanner hybrid devices. Attendees: David Flater Diane Golden Nelson Hastings Marty Herman Ann McGeehan Russ Ragsdale Andrew Regenscheid David Wagner Karen Yavetz -- Finalize minutes of previous meeting Forgot to do this--deferred to next meeting. -- Review available records of the VVPAT accessibility decision(s) In 2003, the Department of Justice published an opinion that the presence of a contemporaneous paper record on a DRE does not violate HAVA or the Americans with Disabilities Act, but they did not address the specific question of whether that remains true if the VVPAT is the ballot of record and/or is part of a process in which the voter is expected to verify recorded-as-cast. [Where HAVA Sec. 301 (a)(1)(A)(i) said "permit the voter to verify," it was in the context of reviewing one's selections for voting errors--the summary screen on a paperless DRE being sufficient for that purpose.] On the "shall" requirement in VVSG 1.0 Req. I.7.9.7.b, EAC counsel's office did not issue a decision as was previously believed. The Commissioners requested the change based on comments received during the public comment period for the VVSG. Thus, we have no explicit and incontrovertible legal opinion on the question of how HAVA and/or the Americans with Disabilities Act would apply. A Discussion Draft of the "Holt Bill" (111th Congress, 1st Session) dated March 13, 2009, that was provided by Diane Golden, clearly takes the position that accessible readback of a VVPAT (as well as a solution to paper handling) should be required. Diane explained the softer, socio-political side of the issue. To the extent that voters in general are expected to participate in the process through some substantive activity (in this case, directly verifying their ballots), that activity must be implemented in a way that does not block voters with disabilities from participating in it. Though we may agree that the voter-verification act is not really for the individual doing it but is rather for the integrity of the election as a whole (for which purpose it is not necessary that every voter do it), that act is still a form of substantive participation in the process, and to implement it in a way that excludes voters with disabilities specifically "smacks of discrimination" and therefore "won't fly." Diane confirmed that providing a readback from the paper record would satisfy the objection at a general level [assuming that the paper handling issue was also addressed somehow], but the acceptability of shortcuts such as pulling the text from memory instead of OCRing it from the ballot or relying on a bar code that is not what gets tabulated remains controversial. -- Path forward The working group discussed a strawman manifesto proposed by David Flater. Following is a version amended to reflect the consensus of the working group, to be validated at the next meeting. 1. It is not our job to write election law, resolve conflicting laws, or translate election law which varies from one jurisdiction to the next into VVSG requirements. 2. It is not our job to design a voting system. 3. Our job is To define auditability; To require auditability to the level needed for defensible elections; To ensure that drafted requirements are consistent with federal law such as HAVA and the Americans with Disabilities Act; To leave jurisdictions with choices of what systems to deploy; And to be candid about the compromises and risks inherent in each one. As a specific case where this matters: The auditability properties of VVPAT have nothing to do with whether it is the legal ballot of record. The auditor gets multiple records and checks them for consistency. As previously agreed, it suffices to detect evidence of material error. We can do that much without regard to the legal precedence of one record over the other. However, there was not consensus that the legal term "ballot of record" could be thrown entirely out of scope since it could still affect the application of federal law with which we are required to remain consistent. -- Scope of auditability and paper vs. no-paper David Flater asserted that parallel testing is an audit method for recorded-as-cast, which is the same scope addressed by VVPAT. Setting aside the question of whether we consider parallel testing to be good enough in the final analysis, does this not give us a way to talk about auditability without excluding paperless systems a priori? David Wagner agreed in principle that you could call it that, but the practical concerns are substantial. - Parallel testing tends to be "de-adopted" because of cost. - Difficult to do it right. - Effectiveness hard to demonstrate. Russ expressed concern that parallel testing was an election procedure and therefore out of scope. David Flater: Granted, the VVSG cannot require parallel testing, but the availability of parallel testing as an audit method means that we can include paperless systems in the scope of what we talk about in the context of auditability. Russ / David Wagner: Are you suggesting then that every voting system is auditable? So we will have no meaningful requirements? David Flater: Not like that. Propose that we stop thinking about auditability as yes or no and start thinking about it in relative terms, and in terms of getting different assurances from different systems that maybe aren't even directly comparable. First we do the analysis of the entire space; then we write the requirements. The requirements will be yes-or-no, but we draw the line somewhere that leaves jurisdictions with more than one voting system to choose from, and we don't draw it the same place as SI, which answer was already rejected. Diane Golden and David Wagner each expressed fears that this process would lead to the same end as before, but were prepared to give it diligence. -- Next steps Everyone begin analyzing the various kinds of auditability found in all of the voting systems that we have heard about and prepare to narrow down a definition of auditability. Next meeting scheduled for June 4 at 2 PM EDT. Adjourned at 3:20 PM EDT.