Participants: Allan Eustis, David Flater, Donetta Davidson, John Cugini, John Gale, John Wack, Nelson Hastings, Sharon Laskowski, Wendy Havens, Whitney Quesenbery
Agenda for December TGDC Meeting
Sharon Laskowski reviewed the agenda for the meeting. Commissioner Davidson has made it clear that we are committed, because of congressional testimony, to deliver the VVSG 2007 in July of 07. The VVSG should be a document that is useful to the election community (vendors and election officials) for the next four years - We do not want to redo in two years. The VVSG 2007 will not necessarily be implemented immediately.
[Noted that some TGDC members felt we should be fixing things in the VVSG 2005 iteration. Also concern has been expressed about hardware changes due to new requirements in 07.]
In VVSG 07, HFP has been filling gaps, reducing ambiguity, and pushing items forward that were not pushed in the previous version because we didn't have research backup. The big item left for HFP is getting the usability conformance test done, which is currently ongoing. Two stages: 1) getting it firm enough to write into 07 standards and 2) being sure the detailed test protocol can be written.
The recent elections have shown STS and CRT that there are issues with security reliability and quality and to fix those, they are recommending requirements that would change the hardware in some respects in big ways.
Compared to CRT and STS, the HFP requirements in 2005 were done more completely. In those (CRT,STS) areas, there is a lot more to be revised.
The question arose as to whether we needed all allotted time at the TGDC meeting - the other subcommittees may need more time. [John Cugini pointed out that we may be satisfied with certain issues but they may cause controversy outside HFP group.] HFP seems to be on track, other committees may be looking for direction.
ACTION: Sharon and Allan to work on agenda, realizing we can use less time.
Subcommittee chairs met with EAC. Donetta Davidson will be attending as many meetings as possible until someone is hired to represent them at meetings.
John Gale: In regards to these new standards (that will stand for four years), it seems we're scrambling to deal with current technology, and in some areas there are huge advancements, what happens to the standards in the next four years for that next generation or do these hold the industry to the status quo? Donetta will speak at TGDC meeting about what are current timeframes are, and how the new VVSG applies to it. Everything that has been purchased so far is only to 2002 requirements. At the December 2007 date, we'll no longer certify anything to the 2002 standards; they will have to meet 05 requirements. In the past, manufacturers have been reactionary instead of futuristic. We have to allow time for the design, build, test, and NVLAP certification. NIST test scripts have to be written to the new ones. It's very important that we talk about the time factor - We are not going to have this done in two years.
STS has been looking at new innovation classes of voting equipment, for when manufacturers come up with new technology ideas.
[NOTE: This is the difference between performance and design standards. Performance standards are technology neutral; we say it has to reach a certain effectiveness and efficiency. This is a good reason to get the usability conformance test done. Maybe STS should do the same thing for the security tests that HFP did for this.]
Donetta: TGDC's goal is to make the new future elements there possible so someone can design a new piece of equipment. We're talking about future equipment -We don't want to tell states to get rid of their equipment, and we also don't want to stifle new innovation.
With the assistance of NIST, EAC is planning a workshop on the cost related to testing voting equipment. We hope to gain Congressional awareness that cost has to be considered.
Summary of STS activities (wireless and independent verification) - John Wack
Wireless presentation that STS is going to make is not going to be as controversial as earlier thought. STS making the point that NIST didn't explain wireless well in 2005 - we did not mean to ban transmission of results. Radio frequency (RF) is a type of wireless that is difficult to secure and easy to disrupt. If used, you would need backup. Hardly used, expect wireless modems. Only one vendor uses wireless LAN - they'll need significant changes to meet 2005 requirements. NIST will present the argument that the key management protocols currently out there that are used to distribute encryption keys to authenticate and secure the transmissions are still immature and hard to manage so it would be hard to manage an election. It would be better and simpler not to put modems directly on voting machines. Proposal: No RF built in the actual voting station. A white paper will be circulated and on the TGDC web page on Monday.
STS will also be discussing software independent systems. From an engineering point of view, NIST and STS are asserting that future voting systems need an audit trail - current DREs do not have one. The presentation will say that these sorts of systems will be required in VVSG 2007. They are called software independent (SI) because you can take the audit trail and verify that the electronic records are correct, therefore you are not relying on the accuracy of the voting system software. This may cause issues at meeting. People may think all we'll have is paper machines. In the future systems may also use cryptography. [Whitney owes Ron Rivest a revised section on usability to the draft SI paper - it doesn't completely meet this committee's approval.]
The question was raised about the number of different ballot types out there. You have to look at each machine individually to look at it's vulnerabilities - not all machines need the same fix. New designs have to show that they meet basic requirements, but also that they are usable. It doesn't appear that much effort has been put into improving paper-based ballots.
Industry has to be spurred into coming up with secured paperless approaches - so far, not much work in this area. We might want to propose a requirement that says when you propose a cryptographic solution, you have to consider all the humans in the system that make it work. To come up with these requirements and do them right, there is not enough time for 2007.
It comes back to performance standards, and if things are developed in the future, we have a way to address them. From a usability scope, we need some wording about capturing the notion of having a vendor specify for his solution the end-to-end usability and accessibility, the interface with people.
STS is saying that after looking at sound practices, the DRE route is not good for future, paper works for now, but we have to work on making that a usable solution.
Any CRT issues? Pretty much on track. Some concern over the presentation that says we want to see the way changes are developed and monitored after deployment to achieve reliability and accuracy to levels that can't be verified through operational testing alone. CRT has a long list of items to discuss. There is a collection of discussion papers on CRT's website that the committee may want to review. CRT's work needs to be simplified and more understandable.
Tweaks or more to HFP Section? For anything we've talked about but no new material. JC will work over next week to complete minor changes.
No meetings until after the December TGDC meeting.
Participants: Alice Miller, Allan Eustis, Sharon Laskowski, Tricia Mason, Wendy Havens, Whitney Quesenbery
TGDC Presentation Summary:
Next meeting is Friday, November 17, 2006, at 11:00 a.m. ET.
Next telecon is Thursday, November 9, 11:30AM ET.
Participants: Allan Eustis, David Baquis, John Cugini, John Wack, Nelson Hastings, Sharon Laskowski, Tricia Mason, Wendy Havens
Discussion of updated Usability and Accessibility Sections
John C. has marked the items where he feels need to be discussed with question marks. Anyone who has other issues is welcome to introduce them.
3.1.3. New section called "the Relationship between HAVA and VVSG". There's confusion on the interaction. This material was added to clarify the distinction between HAVA legal requirements and "requirements" within the VVSG. Maybe this belongs in a general VVSG section, not just an HFP section. [Allan agrees it should be in the overview.] Will stay as is unless there are objections. [David: Questions about enforcement. Under VVSG, do we really want the test labs to provide enforcement or should it be EAC? JC and SL agreed to change from test labs to EAC (they are rule making body).]
220.127.116.11 B. Overall Efficiency. Clarification. The overall requirements apply to all voting stations. The accessibility stuff is extra. The usability requirements need to be applied in a different way at the accessible stations, in particular with the differences between video and audio. We do not have a benchmark for this yet. We want a requirement for efficiency - how long it takes voters to vote; note different benchmarks for audio for visual. No objections.
18.104.22.168. Performance Requirements for Specific Tasks. Thinking may be going against this because it's too detailed and too hard to measure, so this may be going away. We don't want testing by labs to be too detailed because of costs.
22.214.171.124.C. Handling of Marginal Marks. Handed off from CRT. This is for paper ballots. Will stay as is unless there are objections. [David: Clarification. Will they be filled out by hand and inserted into scanning machine? Yes, this applies to opscan systems.]
3.2.4.G. Icons and Language. Also from CRT. This is because we can't rely on color alone. Must require icon and text. No objections.
3.2.5.G. Visual Access to VVPAT. Most accessibility issues for VVPAT covered by general requirements. Deals with comparing paper to screen. If you want comparable formats between two, you need to pull that out specifically. [What does same posture mean? Paper visible at same height as machine.] "Posture" the best way to write this? Sharon/John to re-word. [Whatis meant by by same format? Summary capabilities must be the same.] Default position: doing nothing for now on this requirement, John W to come up with some language. [David to take posture issues to ADA specialist for recommendations.]
Can TGDC recommend areas to EAC that they may want to consider research in? Hopefully so. STS wants to do this as well.Note: May want to create list of things to consider.
126.96.36.199 Timing Issues. New section, we should look at carefully. You have to do different things for audio versus video. The requirements have changed. [David: Clarification on section C, is this about initializing? This is any kind of response from machine. No good way to put an end time on audio since it is content dependent. We should have input from disability community.]
3.2.9. Usability for Poll Workers. This is one we gave away to the Core requirements sub committee . Originally this was a section about usability of documentation. It seemed more appropriate to give to CRT since they were responsible for other documentation. It should be practical and usable for average poll workers. There should be style guides on how to put documentation together. [David to provide documents on accessibility of documentations.] Sharon not sure how far to go into accessibility for this topic.
3.3.2. (C) and (D) High Contrasts for Displays and Adjustable Saturation for Color Displays. Thinks the purpose is that people with vision problems have choice of high & low contrast and high & low saturation, not actual color change. John reformulated and simplified this section. We should put out guidance about universal colors, put out best practices. [David: Good place to harmonize between this and ADA 508 standards.]
3.3.3.D Ballot Activation. Clause about "normal procedure" a little funny. Who knows what the normal procedure is. John would like to reword. Must be based on equipment, not procedures in the polling place.
3.3.5.B Allowance for Assistance. New requirement. Suggested in comment period. No objections. [David will pass to ADA specialist - "adequate room" is arbitrary and not measurable.]
John C. intends to have this to the formatting contractor by next weekend. Any comments should get to John a.s.a.p.
Adjustable Controls: A lot of these things are adjustable by the voters, except contrast, which should be adjustable either by the voter or the poll worker. Not sure why contrast is singled out. John proposes making all visual aspects adjustable by the voter. Will discuss further with Whitney.
Update on benchmark experiments and ballot instruction experiments - Sharon
We're trying to get benchmarks from the usability testing. Don't have formal report, still looking at preliminary statistics to figure out how to write benchmarks. Initial "rough" Results (which collected paper optical scan and DRE with VVPAT) show that timing wasn't much different between the two. All voters were confident and had average satisfaction with voting experience. This is the first batch. Out of 23 DRE users, 15 made mistakes. With the paper optic scans, 7 people made mistakes. Outcome: We are able to measure error rates.
Discussion of next steps to talk about at the TGDC meeting - Sharon
Any pressing issues to discuss with Bill Jeffrey?
Next telecon is Thursday, November 9, 11:30AM ET.
Participants: Alexis Scott-Morrison, Alice Miller, Allan Eustis, David Baquis, John Cugini, John Wack, Nelson Hastings, Sharon Laskowski, Sharon Turner-Buie, Tricia Mason, Whitney Quesenbery
Discussion of VVSG Version 2 Draft
3.2.7 Alternative Languages
This section got muddled in VVSG 05. Some states needs this and some don't. We don't want to mandate to every vendor that they must have this capability - but if they have it this is what must be done. Add a statement about "if supporting language, then …" No objections to comments
3.2.8 on privacy already covered so this section was skipped.
3.2.9 Usability for Poll Workers
This section is in the roughest shape of any of our sections. It was in a different place in VVSG 05, we're trying to pull it together.
188.8.131.52 Operation - System must be easy to operate, easy to set up and break down, and have adequate documentation.
This is new material. Sharon L is talking to experts to help cover these concerns. There is also concern about testing this in a test lab. Experts may not be realistic. We need a "typical" poll worker. Statement added about documentation usefulness - "must be suitable for use at polling place."
184.108.40.206 Maintenance - Old material, clean up, but same as before. No issues.
Bullet "B" about quality control is not written very well. If we don't understand it, how can we test it. Maybe it should be removed. We should ask former committee members about it's intent, if not understandable, then remove it. Bullet "C" is good, maybe we should start with it.
NOTE: Reminder that this section is above and beyond what every system must do. Implication that all the usability requirements must apply to the accessibility systems.
3.3 Accessibility Requirements
Hasn't changed much since VVSG 05. It was really hashed out then.
Same as VVSG 05 with a little rewording.
3.3.2 Partial Vision [John C will be rewriting this section. NOTE: May want a new section on "Initiating use of voting system - mode selection aspect"]
Bullet "B" - Why is this here - "how will millimeters be calculated"? Because of comments from 05.
Comment received for VVSG 05 - Guarantee room for human assistant. Do we want to address this. David - Please respond to this issue via email
NOTE: For the next meeting we want to post that we invite access board members to participate.
Moved a piece about hearing aids to this section
New. No specific design issues. Discussion section includes specific features.
3.3.8 English Proficiency
Treated as a disability.
Next meeting November 3, 2006, 2:00 p.m. EST.
Participants: Alexis Scott-Morrison, Alice Miller, Allan Eustis, David Flater, John Cugini, John Wack, Nelson Hastings, Philip Pearce, Sharon Laskowski, Wendy Havens,
Meeting commenced at 11:00 a.m.
Draft HFP Section of VVSG
John Cugini walked the subcommittee through the following sections of the draft HFP sections for VVSG 2007:
3.2.3 Functional Capabilities - Basic things machines can do to make life (VOTING) easier
220.127.116.11 Editable Interfaces
18.104.22.168 Non-Editable Interfaces (such as manually marked paper ballots and optical scans
3.2.4 Cognitive Issues
3.2.5 Perceptual Issues
3.2.6 Interaction Issues
22.214.171.124 Timing Issues
NOTE: Whitney - Before we get into the accessibility section, including alternative languages , we should have the members appointed to the Access Board participating . Wait until next telcon.
3.2.7 Alternative Languages (this section was touched on only briefly)
NOTE: Whitney - We need to identify "big issues" that need to be discussed before the December meeting.
Next teleconference is scheduled for October 20, 2006, at 2:00 p.m. ET.
Participants: Allan Eustis, John Cugini, Philip Pearce, David Baquis, Alexis Scott-Morrison, Whitney Quesenbery, John Gale
Review of Report (John Cugini):
Next meeting September 29, 2006, 11:00 a.m.
Taxonomy of Voting System Records Production Approaches
This is a brief, high-level paper on voting system approaches for the purposes of ballot records auditing. It presents an approach to categorizing these approaches in the VVSG 2007 using the class structure. It is meant for the purposes of discussion only.
We group different approaches to voting system design into two broad categories: software-independent and software-dependent approaches. Software-dependent approaches are best exemplified by today's DRE systems: the accuracy of the captured votes depends to a large extent on the accuracy of the software used to record the votes. DREs do not produce other records that can be used to positively verify the accuracy of the captured votes.
Software-independent approaches, on the other hand, produce voting records in such a way that their accuracy can be verified even if the voting system software contains errors or deliberate fraud. Such approaches should be, in theory, less expensive to test than software-dependent approaches. While VVPAT is one example of this approach, some end-end cryptographic approaches are also software-independent. The category Independent Dual Verification (IDV) consists of a variety of different voting system approaches, including current VVPAT and Op Scan (combined with Electronic Ballot Marking devices), and the more theoretical Witness approaches.
While some of these designs, e.g., VVPAT, are purely software-independent, other designs such as Witness are somewhat software-dependent. This bears more explanation, as follows:
In VVPAT, for example, the voter's indirect verification of the DRE's electronic record is backed up by the voter's direct verification of the paper record. Furthermore, the paper record cannot be changed by the voting system after the voter has verified it, thus it can be used in useful comparisons with the electronic record(s). Of course, some software is still involved and paper can be mishandled at later stages, so further security measures are still required. But, the two records can be compared for accuracy and errors/fraud in
In the Witness design, (Witness is a theoretical approach that no vendors admit to pursuing but it is useful for illustrative purposes) a camera takes a picture of the DRE's summary screen immediately after a voter finalizes his or her ballot and the voter does
Thus, one indirect verification takes place - if the camera displays the photo it has taken, two indirect verifications are possible. But, the camera-related software involved is hopefully relatively small and thus more easily verified for correctness than, say, the DRE itself. Two or more records are produced, and the DRE's electronic records can be compared against the digital photos and verified.
This approach would be preferred over the pure DRE approach. Consequently, some software-dependent approaches are preferred over others. More testing of these approaches is warranted, with some sort of a sliding scale going from IDV approaches (less testing) to DRE approaches (more testing).
The high-level taxonomy of software-independent and -dependent approaches, then, would be as follows:
1. Software-Independent Approaches
2. Software-Dependent Approaches
Participants: Adam Ambrogi (EAC), Alexis Scott-Morrison, Alice Miller, Allan Eustis, David Baquis, David Flater, John Cugini, John Wack, Nelson Hastings, Philip Pearce, Sharon Laskowski, Thelma Allen, Wendy Havens, Whitney Quesenbery
Allen Eustis called the meeting to order at 11:03 a.m.
Administrative Updates - John Wack
Sharon Laskowski Update
VVSG Usability and Accessibility Section
Next Telecon on September 8, 2006, at 2:00 p.m.
Meeting adjourned at 11:37 a.m.