MEETING OF THE TECHNICAL GUIDELINES
DEVELOPMENT COMMITTEE
NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY
Wednesday, March 9, 2005
(START OF AUDIOTAPE)
MALE SPEAKER 1:	-- the habit now of having these meetings on the coldest day of the month so we decided not to break that and we’ll try to keep you warm inside with the topics that we cover today. 
A couple of administrative things. First of all, welcome to the public and not only here but on the Webcast.  The Webcast will be archived and can be accessed after the meeting as well.  We would request that the public here wear their name tags at all times for security reasons and also so that we know who you are.  If you would turn off or turn down your cell phones that would be most appreciated as well.  We have a tight schedule so we will move right along during the morning and we have had weather delays and actually postponements of capabilities for people to be here because of the weather so, that can’t be helped but we have a number of members joining us on Tele-conference as well.  So, we are happy that that capability is available as well.

Does anyone, or if anyone, needs interpretation for the deaf our interpreter is over here and you might want to move over to that part of the Green Auditorium.  Last time we had a little practice so you need to really watch this.  We are in the Green Auditorium and if you go to the back doors, either one, you take a left and then a right and you get right outside and you will know there’s a fire drill here because the Klieg lights start to go on and off and you will hear the sirens just like in school so we are concerned for your safety but there are a number of ways out of here.  We have lunch at 12:30 today.  That will be in the main cafeteria which is just down the hall from where we are here.  There are several bathrooms around.  There are several bathrooms right here.  In other words once you go out, take a left for the handicapped and, as I said, if there is any other emergencies you will hear the, the P.A. system will go on and we’ll be advised on how to go from there.
Briefly, pretty much what I have covered here on the fire, severe weather and security emergencies and with that I think I’ll say that I’m here to assist you in any way during the conference, the public that’s here.  There are handouts in the back for the public.  All of that information is on the web site as well.  They are big green folders.  Feel free to take one with you.  Power Point presentations will be on the web as well early tomorrow morning.  With that I would like to hand it over to Dr. Smergian, the Chair of the Technical Guidelines Development Committee.
DR. SMERGIAN:	Thank you, Alan.  Good morning everybody.  I am Rach Smergian.  I am the acting director of NIST and I would like to welcome all of you to NIST, to this meeting of the Technical Guidelines Development Committee.  I hereby call to order the third meeting of this committee today, Wednesday, March 9, 2005.  Let us now stand and pledge allegiance.
GROUP:		I pledge allegiance to the flag of the United States of American and to the Republic for which it stands, one Nation, under God, indivisible with liberty and justice for all.
DR. SMERGIAN:	We have currently nine members here.  We also have Dr. Revest and Ms. Turner-Bowie on the phone with us.  We are still expecting a couple more people to join us.  Given the weather conditions I am sure some of the flights are delayed and two other members, Mr. Harding and Mr. (undecipherable) will join us in the afternoon.  They are actually attending another meeting this morning so they will be joining us for the afternoon session.

At this time, oh, I would also like to acknowledge Commissioner Martinez who is here with us and sitting back there.  He wants to make the point that he is an observer.  I believe that Commissioner DeGregorio just came in.  So, we are very pleased to have them with us at this meeting.		
With that I now recognize Mr. Craig Burkhart as the TDGC Parliamentarian and request that he determine if a quorum of the committee is present.  Mr. Burkhart.
MR. BURKHART:	I call the roll.  Williams.  Williams is here.  Burger.  Burger.  Not here.  Caldess.  Caldess is here.  Kraft.  Kraft is here.  Davidson.  Davidson is here.  Eleckies.  Not here.  Gannon.  Gannon is here.  Harding.  Not here.  Miller.  Miller is here.  Crystell.  Crystell is here.  Quisenberry.  Quisenberry is here.  Revest.  Revest is here.  Schutzer.  Schutzer is here.  Turner-Bowie.  Turner-Bowie is here.  Smergian.  Smergian is here.
Mr. Chair a quorum is present.  You may declare a quorum is present.
DR. SMERGIAN:	Thank you Mr. Burkhart.  I also thank the members of this committee for arranging your busy schedules to allow your participation at this meeting.  As I have said in the past and it certainly bears repeating, your willingness to volunteer significant time to the work of this committee is a mark of the highest ideals of citizenship and civic responsibility.  I am sure every American voter will benefit from your commitment.  At this time I will entertain a motion to adopt the March 9, 2005 meeting agenda for the Technical Guidelines Development Committee which is in your binder with the tab “Agenda”.  Do I hear a motion?
FEMALE SPEAKER 1:	(undecipherable).
DR. SMERGIAN:	Do we have a second?
FEMALE SPEAKER 2:	Second.
DR. SMERGIAN:	Any questions or discussion?  Hearing none, I would move for a vote.
MR. BURKHART:	Take a voice vote.

DR. SMERGIAN:	I guess we’ll take a voice vote on that.  All in favor say “aye”.  Any opposed?  The agenda has been adopted.
At this time I will entertain a motion to accept the minutes of the January 18 and 19, 2005 meeting of the Technical Guidelines Development Committee.  Do I hear a motion to accept the minutes of the last meeting?
MALE SPEAKER 2:	I so move.
DR. SMERGIAN:	We have a motion.  Do I have a second?  Any questions or discussion?  Having none, all those in favor say “aye”.  Any opposed?  Thank you.  The minutes, as presented, have been accepted.
Having seen the material that was sent to you earlier I hope you agree with me that these scientists have made significant progress.  Mr. Kraft.
MR. KRAFT:		Yes sir.  We seem to have a recurring problem in seeing the materials for the meeting.  I received a blank C.D. from Mr. Ustus and actually got my hands on the materials Monday night and spent the time that I had blocked on my schedule to review the materials in a very frustrating exercise downloading materials off the NIST web site, some of which were in a PDF format that made it difficult to save them.  I’m beginning to feel like I’m trying to teach the cat to fetch.  I’m not going to object to the materials in this particular meeting in the interest of getting the work done but we absolutely must have these materials in hand five days before the meeting so we can review them.  We passed a Resolution on that last time and NIST needs to work on its delivering.
DR. SMERGIAN:	I believe the material was sent on Tuesday or Wednesday?  It was sent by Federal Express on Wednesday so that you all should have gotten it Thursday.  Is that correct?

MR. KRAFT:		The material that I got was a blank C.D.  I received an e-mail Monday from you, Allan, after –
ALLAN:		There was another one (undecipherable).
MR. KRAFT:		I didn’t get that one.  My administrative assistant did a search for any e-mails from you this morning.
DR. SMERGIAN:	I believe we tried three different drafts by e-mail, by hard copy through FedEx and by, which also included a C.D.  Certainly our intention was and is to fully comply with the Resolution.  We appreciate the fact that there is voluminous material and that people need to have sufficient time to review them.  On the other hand as you all know, since you are all involved there is work being done until the very last minute so its sort of a balance but we certainly have tried to meet the spirit of the Resolution that was passed last time and to avoid any of these communication problems we tried to do multiple, you know, parallel processing, try to use as many of the channels available to us for communicating the material to you.  I assure you that we will try to do better.
MR. KRAFT:		Okay.  I appreciate it.
DR. SMERGIAN:	Thank you, Mr. Kraft.
As I was saying the scientists have made significant progress on technical tasks defined in the thirty-one Resolutions adopted by this committee at the January plenary meeting.  The TDGC members will have an opportunity today to provide further guidance to NIST on these tasks.  In addition, two new Resolutions have been submitted for consideration.

Proposed Resolutions number 36-05 and number 37-05 as well as preliminary task reports were sent to the committee on March 2 in accordance with the advanced notice required in Resolution number 1-05.  Obviously we did have some problems but certainly we tried to meet the spirit of that Resolution.  In addition this material has been posted on the public web site “NIST.gov”.  As a brief review for the public in attendance and hearing the web site, Webcast, Public Law 107-252, the Help America Vote Act also know as HAVA, establishes the Technical Guidelines Development Committee.  HAVA charters the members of this committee to assist the Election Assistance Commission with the development of voluntary voting system guidelines.  This committee’s initial set of recommendations for these guidelines are due to the Executive Director of the Election Assistance Commission in accordance with HAVA’s nine month deadline.  In the interim the 2002 voting system standards adopted by the Federal Election Commission serve as the first set of voluntary voting system guidelines under HAVA.  
At this time I note that the latest revised version of the Roberts Rules of Order was adopted on July 9, 2004 to govern Technical Guidelines Development Committee and subcommittee proceedings.  I call on Mr. Burkhart to review the logistics of this third meeting of the TGDC.  Mr. Burkhart.
MR. BURKHART:	Good morning.  I’m Craig Burkhart and I’m the Chief Legal Counsel for Technology at the Department of Commerce.  I welcome you on behalf of the Secretary of Commerce Gutierrez as well as President Bush who again are so thankful to the TGDC members for being willing to spend you very valuable volunteer time to serve on this committee.
I’ll very briefly review the strategy that we will be following today to see that we make as much progress as possible according to the TGDC’s agreement.  We are doing three things today.  There will be a brief review of the prioritization of the work projects in response to one of your Resolutions last meeting.  The second thing is we will review the work product which will be given to you, which was given to you.  And then finally there will be a couple of new Resolutions toward the end of the meeting.  

The goal, the most important goal is to review this work product which has been given to you so that the NIST staff can continue to fine tune and develop that into a finalized work product which you will be capable of adopting during your April meeting.  Probably one key concept to keep in mind is that during this meeting we are not adopting or approving any of this work product as part of your initial recommendations to the Election Assistance Commission.  We are just simply making a check on whether or not the work product to date is following the spirit and letter of the Resolutions which you have previously adopted.  You will, in any case, be able to take a look at the finalized work product which you will get during your April meeting and that will be the critical time when you will actually take vote to approve work product to recommend to the Election Assistance Commission.
MALE SPEAKER 3:	So, if I understood that the point of today’s meeting is primarily to give the NIST staff the committee’s feedback on the work product they’ve presented.

MR. BURKHART:	That’s correct.  Here is the explanation for what we’ll be doing for the large bulk of this meeting.  You, of course, have received written work product and then you will be receiving brief presentations by the senior NIST staff person who is charged with developing that particular segment of the work product.  The combination of that oral presentation and the written work product we’re calling that the preliminary report to you on that particular subject matter.  After the live presentation there will be a question and answer session during which you will be able to dialogue with the NIST staff person.  After that, among yourselves as a TGDC you can have a discussion as to whether or not this suits you well, the way the project is being developed or whether there are supplemental instructions that you would like to give or even corrections if there is something about the development of the work project which you disagree with, that you don’t think meets the letter of the Resolutions that you have previously passed, you want to bring that up at that time.  
The intent is that if the group is in agreement as to some supplemental instructions or corrections that if it’s agreed there can be unanimous consent for some type of supplemental directions being given to the NIST staff person.  In fact, at the end of each presentation the chair will read a brief statement saying we believe this is in response to the certain Resolutions you have passed.  Are there any corrections and that will begin the discussion.  If there are no corrections or supplemental instructions, no vote whatsoever is taken and the NIST staff will continue along the line that they have developed.  If, however, there are supplemental instructions, if it’s by consensus that will be what’s called unanimous consent.  It will have the same effect as though you had taken a contested vote.
Finally, there may be some contested issues where there is not unanimity of opinion on something and then a vote can actually be taken in the normal manner where somebody moves a particular correction or a particular supplemental instruction and it can be debated in the normal manner.  
So, that’s what we will try to do today.  You might say this is your speak now or hold your peace.  Make sure that you believe that the NIST staff is following the directions that you have previously given.  
Finally, of course, what you will get in April will be the finalized draft work products.  And you will remember that you will get two.  One will be, if you will the addendum to the 2002 standards and the second product will be those things which are not currently embraced in the 2002 standards and are more the long term type projects.  So, you will actually get two work products which will be based upon the projects that you are going to hear about today.

And then, finally, there will be a couple of new Resolutions at the end which will be handled in the same manner that we have worked the last meeting.  That would be all Mr. Chair.
DR. SMERGIAN:	Thank you Mr. Burkhart.  The scientists and I look forward to the committee’s comments and questions on the preliminary reports presented today and we have an ambitious agenda today as we did at your January meeting so we will try to move as expeditiously as possible.  Last time we went through the introductions of the committee.  We’ll save some time and not repeat that however we are very pleased that Secretary Davidson joined us who was not with us at the last meeting.  I’ll ask her if she would introduce herself and make a few comments.  Secretary Davidson.
SECRETARY DAVIDSON:	Thank you very much.  Well, as stated my name is Donnetta Davidson and I’m the Secretary of State in the State of Colorado.  I’m very honored to be on this committee and I think that we do have a lot of work.  A lot of work has been done but there’s a lot of things that need to take place.  I won’t take anymore of your time so we can move on with the important agenda today.  Thank you.
DR. SMERGIAN:	Thank you very much.  So, as Mr. Burkhart pointed out today as a committee we will review and, where appropriate, provide supplemental direction to NIST scientists.  This guidance is critical to the completion of a draft of voluntary voting system guidelines that you will receive for review in our April meeting.
The time required to accomplish the agenda items means that the committee cannot take public comments at this meeting however; there will continue to be opportunities for the public to comment on relevant issues.  Comments and position statements should be sent to; you want to send an e-mail to voting@NIST.gov where they will be posted on the NIST voting website VOTE.NIST.gov.  The comments we have received to date have been posted and reviewed by NIST staff and TGDC members.  

As I mentioned in my introduction, this is the third plenary meeting of the Technical Guidelines Development Committee.  At the first and second plenary sessions of the TGDC in July of 2004, in January of 2005, Resolutions were adopted that have guided the committees and NIST work up to this phase of the guideline development process mandated by HAVA.  Welcome, Mr. Burger.
From July 2004 until today three subcommittees engaged in information gathering, research and analysis including the taking of public testimony at a two day hearing in September 2004.  In addition, public testimony has been requested, accepted and posted electronically at the NIST voting web site, VOTE.NIST.gov.  
The work product of the subcommittees’ work was presented at the January 2005 meeting as a series of Resolutions.  The Resolutions were debated and, in many cases, amended.  Adopted Resolutions form the basis for the document drafting phase of the guidelines development process.  The Resolutions assigned NIST specific technical tasks.  NIST scientists are here today to report back to the committee on their progress. 
At this time I would like to call on Mark Scolo of the NIST Information Technology Laboratory to review a prioritization of the committee’s adopted Resolutions and a proposed strategy for implementing them.
MARK SCOLO:	Thank you Dr. Smergian.  Welcome.  Nice seeing you all again.  I would like to welcome the public and the EAC Commissioners.
DR. SMERGIAN:	Mark, if I could interrupt you just for a minute.
MARK SCOLO:	Sure.
DR. SMERGIAN:	Just for the record I would like to welcome Dr. Burger to the committee meeting.
DR. BURGER:	(Undecipherable).
DR. SMERGIAN:	We quite understand.

MARK SCOLO:	So this morning I would like to talk to you about the priorities we have developed for the Resolutions that you have passed and the strategy for implementing those priorities.
As background as you well know this committee did yeoman work in passing thirty-one –
DR. SMERGIAN:	By the way, you have copies of these presentations I know.  You may get a pain in the neck trying to look back.  So, if you do have copies in front of you –
MARK SCOLO:	That’s right.   They are in the small workbook.  Thank you.  So this committee has done tremendous work in passing thirty-one Resolutions at the January plenary and those Resolutions were very substantive and there is an awful, awful lot of work to do.  We have a lot on our plate and we decided that we would prioritize the Resolutions so that NIST can focus on the important ones early on.  
So, before I go into the strategy for determining the priority I just want to tell you a little bit about the NIST thinking and the TDGC thinking at least as much as we have been able to communicate with respect to how we are going to, or at least or plan or our approach, to packaging the work products.  We have a couple of goals that really drive us.  Goal number one is we want to produce the best standard possible.  This means, at least in our opinion, building on the strengths of the 2002 VSS but making changes when, in fact, those changes are appropriate and the Resolutions refer to some of the things that need to be changed or at least looked at to determine whether they need to be changed.  The coding standards, for instance, the exemption for COT software, quality management standards, rewriting some requirements to make them more precise and testable.  

So, the new futuristic standard will probably not look a whole lot like the 2002 VSS but it will clearly build upon the strengths.  It will be reorganized to reflect our thinking and how to better present this.  At the same time it is clear to us that the goal, and we’ve heard this from the states, we’ve hear this from the commissioners; one of the goals is to help the states get through the 2006 election.  They have an election to conduct and we’re very sensitive to that.  They need as much help as possible.  In order to do that, one can argue we should change the 2002 VSS as little as possible so that, in fact, they work they have already done in getting systems qualified should stay but also give them guidance on the gaps in the VSS 2002 such as voter verified paper audit trail, such as wireless, such as human factors.  
What we’ve done is look at both of these goals and developed a two pronged approach to produce essentially two standards.  A shorter term, what we call augmented VSS, which we used to call an addendum but its probably not going to be in the form of an addendum.  So, an augmented VSS which will in fact fill in the gaps that exist now in the VSS 2002 and correct errors that are in there while at the same time developing the first draft of the longer term redesign standards.  That is our approach.
With that in mind the strategy for determining the priorities centers on creating the highest priority items for the guidance needed for the states to conduct the 2006 election cycle.  The highest priority Resolutions are addressed in the April work products that impact the 2006 election cycle and focus on, again, improving the 2002 VSS by filling in the gaps, correcting errors and also addressing the issues facing the states such as helping to ensure that installed voting software is the same as the software that’s been tested.
So, how with the strategy be implemented?  This is what I just really explained but to summarize, the highest priority items will be packaged into an addendum or perhaps incorporated into a document that augments and corrects the 2002 VSS while at the same time a draft of the new redesign voting system standard will be developed in parallel.  

So, looking at grouping the priorities, Group 1, the work targets the highest priority Resolutions, it augments and corrects the 2002 VSS and the guidance and requirements will be complete by April.
Group 2, that work targets the second highest priority Resolutions and this is work in developing a new redesign standard.  Again, it builds upon the 2002 VSS but enhances many areas including security, human factors, precision and testability of requirements.  The initial work product for this document will be produced in April with planned completion in November.
Group 3 targets the remainder of the Resolutions.  They will be addressed after April.  Plans for completion, again, in November.
So, getting into a little bit of detail, Group 1, targets voter verified paper audit trail which we know is a burning issue for the states since many states have enacted legislation to produce such an audit trail and I try to tie in the Resolutions to the actual priorities.  This is not only a Resolution but specifically the AC has mentioned this to us as has the TGDC off line.  The second part of Group 1 is the accessibility and usability requirements.  The third, software distribution and set of validation procedures.  Use of wireless technology.  Next the interim conformance clause that’s clearly needed for the existing 2002 VSS and we are going to revise the glossary and update it.
Group 2 includes an analysis of the 2002 VSS to determine what things need to be changed, an analysis of voter verifiability, additional requirements and test methods for human factors and accessibility, test strategies and analysis addressing COT software documentation requirements, the issue of non-conforming voting systems, publicly available test suits and test reports is part of this group of priorities, strategy for addressing multiple representation of ballots.  We are going to analyze Federal standards that we have developed and see whether, how much of those apply to voting systems and essentially draft the next version of the VSS.

Group 3, the lowest priority, the usability of the standards, common ballot format specifications, quality management standards, maintenance of the VSS, sharing information and disqualification of voting systems and the last assessment papers on recommendations for future work.
Now this is a document you have in your big folder, I think the sleeve.  Its, its in my handout as well.  It’s a separate document.  This is sort of an index of all the documents that you have obtained with references to what committee the fall under, which of the subcommittees.  They are identified by tabs in the book, A through W.  Its on the next line.  Thank you.  The Resolution number the priority grouping and the disposition whether in fact they are in the augmented VSS or in the new revised version.  For those of you who haven’t memorized it yet this the second half.
My last slide just introduces a couple of issue I’d like to bring up and get some feedback on if at all possible.  The first has to do with packaging of the April work products that augment and correct the VSS 2002.  It seemed to us at the beginning there were two alternatives.  Essentially do an addendum or do a new version of the 2002 VSS with changes in line and perhaps highlight the changes.  It seems to us there are going to be a lot of changes and they will be in different places in the 2002 VSS, interspersed throughout, not just in one place.  So, it seemed to us an addendum would be unwieldy.  It would be referring to many, many changes, page 3, line 6, and probably difficult to do.  So, our recommendation is to basically do the changes in line and highlight the ones that are different.

A more, perhaps, minor question is what do we call the augmented VSS 2002 in the new redesigned standard?  Some thoughts were there have been two versions of the VSS so far, as you could think of the augmented version as Version 2.1.  The new redesign standards as 3.0.  Other suggestions are call the augmented version VSS 2002 revision 1.  Its not really important to us.  If you have strong feelings we would like to know.
The third is perhaps a little more substantive.  Up until now all the versions of the VSS, the first and second, have had requirements for voting systems and, by implication, manufacturers and vendors of voting systems as well as for testing labs.  Clearly we are in little bit of a different environment.  Now, we are still going to have those requirements but we are going to have additional requirements for voting officials and if you u look at Resolution for wireless, for voter verified paper audit trail, many of those Resolutions talk about what has to be present in the technology but as well, certainly how that technology is implemented in systems.  I’m sorry, Paul, did you have a question?
PAUL:	Yes.  I thought we had been around this issue of keeping election administrative requirements.  Requirements for voting officials out of the voting system standards.  That is a separate body of work that vests mainly in state elections codes unless Congress chooses to pass something that overrides.  There was a best practices work product for election administrations run by the EAC but election administrative issues, in my belief, and I thought it was a consensus of the committee have no place in the voting system standards and we continue to hit that issue.
MARK SCOLO:	Whitney, yes.

WHITNEY QUISENBERRY:	Whitney Quisenberry speaking.  I would like to disagree.  I think we went around this issue the last time and we talked about the difference between the administration of an election and the requirements for the deployment of a machine to effectively meet the standards.  Those are quite different things in my mind.  I think that, for instance, you could make a perfectly accessible machine but if you place it a foot from the wall, backed up against the wall, you have essentially rendered those entire accessibility requirements moot.  So, it is my understanding in talking at least with human factors and privacy staff that what we are talking about is not how to run an election but how to deploy a machine in the service of an election.
PAUL:	Okay.  I guess to respond to that Whitney, the concept that I thought we had consensus on was, yes, that’s true.  How you use the machine is a very important of it being able to do its job but how the machine is properly used is one of the things that you put within the system boundary of the requirements in the operator’s manual and, of course, if you are using a machine and you are not following the manufacturer’s procedures that were included in the certification, then you are not using the certified system.

MARK SCOLO:	Can I just respond to that?  I guess I would concur with Whitney’s interpretation of what we had discussed.  We had discussed this further in the CRT group that I think TGDC did not have full access to.  I guess my question is, in my mind its not just the operator’s manual.  For instance, how do we give guidance on voter verified paper audit trail and wireless, for instance, if we don’t give guidance?  These don’t have to be in the form of requirements, they could be best practices, we are not arguing how they should be said.  Somehow these Resolutions talk about implementing things well beyond the capabilities provided by the voting system and the operator’s manual.  You need to know as a voting official how to set up voter verified paper audit trail, to ensure privacy, to ensure all the things we are talking about.  These are instructions for how to set them up.  Wireless is the same thing.  You can have the capability of that wireless but you have to know when to turn it on, when to turn it off, when it use it.  So, my question is how do we follow the Resolutions if we don’t provide guidance and how that guidance is provided is, in my mind, completely up to the TGDC.  It could be not mandatory, it could be best practices, but somehow we need to document that, is my feeling.  So, I’d love to hear –
PAUL:	What needs to flow into the documentation requirements for the system and that needs to be part of the evaluation of the system.
WHITNEY QUISENBERRY:	I’m sorry but I believe that may be how it is promulgated and how it is communicated and I would argue that it should not be a separate system if you want the answer to the actual question you posed Mark.  I do think that what we’re saying is that; for instance, let’s just stay with my mobility example.  The system documentation shall provide proper layout and shall meet that requirement.  I’m not arguing that we are saying that here is the measurements that a voting official might use.  If you would like to say it comes out of the documentation that’s fine but I believe that we, we set up a lot of requirements for the design and manufacture of voting systems and this would be one of them.
MARK SCOLO:	Yes, Brit.
BRIT:		We’ve got precedence for this sort of thing now.  Voting system vendors take the optical scan.  They may specify in their documentation that you use a certain pen to mark the ballots; usually it’s a pen that they sell.  Fifty percent of the people that use that system use a number two pencil.  I don’t know, I definitely think that in the set up for the ITA’s for the laboratories, we should, they should specify, the vendor should specify how the system is to be used.  It should be tested to that standard.  Now, if a jurisdiction chooses not to use it that way, that’s their choice but, they have to do that realizing that they are not then running a certified system.  

Now, here’s what I would like to recommend.  One of the real problems we have right now is that the voting system standards and I don’t think this one’s going to be any better are so convoluted and difficult to read that your typical local election official can’t read it.  They don’t know what they are supposed to be doing.  So, your suggestion down here of putting that in a separate document is, I think, an excellent suggestion.  That’s a heck of a lot of extra work but if you could do that I think that’s a great idea.
WHITNEY QUISENBERRY:	I forget the number of the Resolution but I would point us back to the human factors and privacy Resolution on the usability of the standards and that was exactly what we were thinking about when we wrote that Resolution.
MARK SCOLO:	Yes.  I think you will hear discussions, I think, in some of the presentations following mine that will perhaps allude to some of this and I think we need closure by the end of the day on this issue.  Perhaps we should take it up at the end of the plenary after you have heard all of the discussions.  I mean, I have a lot of questions about just documentation because people producing printers.  They are not concerned with the privacy issue, for instance, so the documentation wouldn’t talk about how to use it to ensure privacy.  That’s clearance guidance for voting officials that can only come from this group, I believe.  So, yes, I’m sorry.
FEMALE SPEAKER 3:	I think we also need to keep in mind our limitations on getting the polling places ready and even having polling places that change at the last minute.  We certainly need to keep those kind of practical things in mind of what the election official deals with on Election Day.
MARK SCOLO:	Yes, absolutely.  Thank you.  I think Ron –
DR. SMERGIAN:	Ah, yes.  Dr. Revest?
DR. REVEST:		(Undecipherable - too far from microphone).
DR. SMERGIAN:	Thank you.
MARK SCOLO:	Mr. Chairman I would suggest that we revisit this issue at the end of all the presentations.
DR. SMERGIAN:	Any other questions or comments?

MARK SCOLO:	Thank you.
DR. SMERGIAN:	Thank you, Mark.  This leads the proceeding plenary report on the prioritization of adopting Resolutions from the January 2005 meeting and the proposed prioritization of work product; technical support response to all currently adopted Resolutions by the committee.  So, unless there are supplemental directions or corrections the technical support and related work product will continue to be developed consistent with this preliminary report.  Are there any other questions, further directions or corrections?
MALE SPEAKER 4:	You need a motion to be adopted.
DR. SMERGIAN:	I think we agreed that we don’t need to have a motion unless you are recommending a change in the direction.  If you think we are going, you know, in concert with the spirit of the Resolution, we don’t feel that we need a vote at this time.
MALE SPEAKER 4:	Let me propose a compromise to my point that I think will address Whitney’s issue.  I guess my problem with the Power Point was the wording of it.  Both standards will have requirements for voting officials.  I think perhaps requirements for proper use of the system as certified or as developed would be a little clearer to the intent.
DR. SMERGIAN:	I think I’ll, just to move things along, I will suggest that you perhaps work off line with Mr. Scolo and suggest the wording that would be acceptable to you and we’ll then agree to modify this presentation so that it is part of the record, the language is part.  Is that acceptable?

MARK SCOLO:	I just want to mention one thing.  Certainly we will talk and come to the correct wording.  One of the things we do in the conformance clause and that’s the reason I put voting officials.  I had no idea if that’s a proper term.  We want to identify the entities that conform and maybe requirement is too strong a word as well, recommendations, I don’t know.  So in conformance we are very interested in making sure we know who it is that’s responsible for these things.  That’s why I have a party, so let’s talk off line and see if we can –
DR. SMERGIAN:	Thank you.  At this time I call on Dr. Sharon Lescosio of the information technology laboratory to present a preliminary report on this approach to usability and accessibility requirements for the augmented 2002 VSS.
DR. LISKOWSKI:	Thank you.  Good morning commissioners and good morning committee members.  I’m going to be reporting on our usability and accessibility progress thus far.
This serves as an outline of what I’m going to be talking about.  I’ll briefly go over the definitions, talk about the gap analysis we did, then the draft of the augmented use accessibility requirements, then the outline of the augmented usability requirements for planning.  I’m going to talk very briefly as an intro to the next talk about accessibility and privacy requirements that relate to voter verified paper audit trail.  Go through our road map and then I’m not going to talk about this, I’m just going to mention it at this point because after April we are going to be looking at usability test protocols for the system.  We need test ballots.  I have been collecting, with the help of many state election directors who kindly have sent me samples of actual election ballots, a collection that we can analyze so that we get a good test set of varying complexity, suitable for qualification testing.

First I would like to talk about a couple of issues that have arisen as we’ve been doing the work and that is one is the difference between the augmented versus the redesigned VSS with respect to usability and accessibility?  The current draft that you will be seeing finalized in April, the augmented are small additions and corrections and for the most part usability and accessibility will be just additions to the VSS.
Longer term the redesigned VSS due in November will be more reorganization and also performance requirements and as those performance requirements in the long run evolve, we see also that the need for very specific design requirements will be lessen, hence this reorganization will look somewhat different.  As it evolves over the next few years, we hope, will have a more of a performance based tack.  But as I say, that is beyond the April deadline.
We’ve already talked about the equipment standards versus what I call the process standards.  I’m not going to go into the detail there.  Our April deliverable basically focused on the equipment standards but there are these other things, especially critical and accessibility which we will talk about in the April deliverable about setting things up properly.  The deployment of the equipment.  The design of the ballot.  We do see those within the scope of what we are doing.
The accessibility requirements that you see are based on a fairly well established body of literature that has been evolving for accessibility of IT products.  Because it’s a bit more mature than usability requirements, specifically for voting systems, you’ll see more design requirements for accessibility so that we can firm action and coverage across a range of disabilities.  As always, my proviso is design requirements do not capture everything.  You still, even in this case, need some usability testing.
The think I want to point out about the glossary definitions that we’ve put in is first, accessibility –   voting systems because of the large audience that’s trying to understand what’s going but a more conceptual definition was important to get us started with accessibility so, hence, why we diverted from that.  

The other thing that you see with the focus of a lot of accessibility recommendations is that we’ve talked about legally defined disabilities.  However, HAVA under the accessible umbrella does talk about accessibility for Native American, Alaskan native citizens and voters with limited English proficiency.  So we put that under the scope of accessibility although typically from other standards you don’t see that kind of coverage.  I’ll let you read these off line because that’s a pretty standard kind of definition.  
The privacy definition or really, just what it is with respect to the voting system is a little bit different than what we normally think about privacy in everyday situations because here privacy in the voting system means it makes it impossible for others to find out the voter voted.  Usually privacy is just concerned about keeping information you don’t want released private.  Here, the additional thing that you can’t sell your vote so it has to be virtually impossible for others to find out how you voted a subtle distinction specific to voting.
Okay, on to the large deliverable, our assessment of the current status of human factors and privacy for voting and we relied on lots of literature here.  Certainly what is in the VSS with respect to accessibility?  The ADAG, section 508, the web contact accessibility guidelines, HAVA, things like not allowing other voting.  These are human factors constraints.
MALE SPEAKER 5:	(undecipherable).
DR. LISKOWSKI:	Yes.
MALE SPEAKER 5:	I had a need to web cast accessibility here.  I’m just getting a blank screen nor any PDF.  Are you using slide (undecipherable)?
DR. LISKOWSKI:	Yes, I am.
MALE SPEAKER 4:	Is there a file name or something?  I don’t have any – I can’t see what you’re showing.  Some files are shown but yours aren’t included.  Maybe (undecipherable).  The web cast is going but I have nothing.		

DR. LISKOWSKI:	Alan?  Alan will send you the slides.
MALE SPEAKER 4:	Sorry to interrupt.
FEMALE SPEAKER 3:	What I would say Ron, that what she’s doing is doing a high level summary of the documents of the, in this case, of her gap analysis documents.
MALE SPEAKER 5:	Thank you.
DR. LISKOWSKI:	So, there is quite a large body of literature for accessibility.  For usability there is the Appendix C of the VSS.  What we’ve outlined in the human factors report.  Some things in the I Triple E draft.  Some things on testing that came out of a UPA workshop.  The usable guidebooks that the FEC did in conjunction with AIR.  There’s not a whole lot of specific things with respect to usability hence our gap analysis, the next slide, recommends a better organization for the VSS to make the usability and accessibility clearer, improving the standards and test for accessibility, putting in some performance standards for usability, creating a test methodology for usability testing to be used by the ITAs and one thing that we haven’t done in a formal way is to examine voter privacy issues of privacy concerning the voter’s choices.  We will do a principled analysis of that in the next month, I hope.

Now, let me talk about some of the issues of the draft of the augmented accessibility that you have in your handouts.  As I said its composed of accessibility requirements that have been gathered up from all the resources identified in the assessment and gap analysis with the exception of section 255 of the Telecommunications Act.  There’s some hearing aid kind of things that we haven’t looked at yet but its on our list of things to do.  As you know, this is just a preliminary draft so its rough around the edges.  Speaking of rough around the edges we do need to go through the draft and make sure that we use consistent, generally accepted, terminology both in the voting community and the community of the disabled.  
We are having some discussion of do we organize accessibility by disability type or not.  Its organized b y disability type at the moment but we are going, for the final draft, decide whether that’s the way we want to go.  As I mentioned before we do include the limited English proficiency and Native American and Alaskan native in our umbrella for accessibility.  We need a major pass through because it is a little rough around the edges and we have to talk more about alternate formats, usability tests, absentee ballots because there’s a question we had.  Also, accessibility of the smart card machine. 
You also have an outline of the augmented usability VSS and here our major issue is how much do we do with respect to putting in design guidelines for usability which may be – a little technical glitch here on the screen.  So, how hard do we work between now and April to put in a whole lot of design guidelines that well go away when we get the performance requirements?  So, it’s a delicate balance.  I just lost my view graphs, Alan.  I’ll do a little tap dance while we’re waiting.
MALE SPEAKER 6	While you’re waiting on that can I make a couple of comments?
DR. LISKOWSKI:	Yes.
MALE SPEAKER 6:	On this privacy issue, selling votes in minuscule.  The main reason I want privacy is I don’t want my wife to know how I voted.
DR. LISKOWSKI:	Very good point.
MALE SPEAKER 6:	Vote selling itself is pretty minuscule.  The other is, if you are collecting ballots, you may be collecting the wrong thing.  You might want to collect state laws that define ballots.  These ballots are developed in compliance with some very explicit laws.

DR. LISKOWSKI:	That’s a good point.  We’ll definitely look into that.  I do want to see actual ballots because we also want to develop the actual test ballots so I want them to see how they are rendered across the board, but absolutely, you’re right.
So, right now we are discussing how much time do we put into pulling together all of these design requirements versus getting to the performance based usability testing and, so, my feeling, you know, given the tight deadlines is that we sort of gather up what’s out there now in terms of usability and put that into the April version and then start the new work.
I promised John Wack who is going to be talking about easy pack that I would spend a couple of minutes talking about accessibility, usability and privacy with respect to it and I view it as a three legged stool.  If you break one of the legs the stool falls over. 	With respect to accessibility of a paper audit trail that is voter verifiable you need to provide, its generally accepted that there should be an accessible way to do that for people with disabilities.  So, just to give you kind of a generic example one might imagine an accessible VRE that prints out an optic scan ballot.  That ballot is fed into a separate reader that has audio and reads it for those that are blind but also might have a screen that lets you adjust the contrast and the larger print, etc. for other kinds of disabilities or combination of disabilities.  From the various information I’ve gathered having this kind of secondary reader as long as its separate from the voting system seems to be an acceptable way of doing the verifiability.  For example, the blind community typically makes heavy use of readers in all their IT but suppose the audit trail is under glass.  How do you then feed this paper into something that’s accessible?  One idea might be to have it cabled to the reader the same feed that‘s going into the printer.  Nothing like that is implemented at the current time.  

You might say: why don’t we just do some alternate format paper audit trial like larger print, etc. Then you get into privacy issues because to preserve privacy you want the ballot, all the audit trails to be indistinguishable from one another.  You could print on the ballot output the different choices and all the alternate formats of different font sizes and languages and you get something very long.  That doesn’t sound very usable.  You have got to have a paper audit trail that’s readable by both the voter and the election officials if they are going to use it for a recount and there’s also kind of load if you are looking at your voting work station, say a VRE.  If you are looking over to the paper, it’s a different medium.  It looks very different.  How many errors do you get if people who think when they try to audit that its different and it really isn’t, especially if you have something long.  
There is a bunch of usability issues as well and that’s the usability of the set up, the printer, the paper, expert readers, etc. and then you are back to pulling a whole bunch of things together and is the whole thing accessible as well.  There’s a lot of issues fighting in there that we have to think about very carefully and John’s going to talk a little bit more about that in a broader context.
I do think that there are definitely solutions out there but paper being inherently inaccessible means we have to think about it because it’s a complex issue.

Briefly, our road map for April, the two augmented versions of additions to the VSS for accessibility and usability focus on capturing the indication of the voter’s choice.  Just a draft gathering up existing guidelines for polling place, defining some usability performance measures to set the stage for the next work, a plan for an assessment of what we are going to use with respect to universal design.  I forget the Resolution number that talked about that but there is a Resolution, and just kind of looking at sort of current, what’s out there for ballot design, instruction, error message collection, coming up with a plan to then address that.  There’s a couple of requirements on testing spreadsheet that you’ll hear about later.  Today, we need to look at the human factors and privacy issues there and address them in that spreadsheet and revise our VV path analysis as well.  
For November we expect the redesign usability and accessibility requirements.  Some initial performance benchmarks for usability and a test protocol and then some revised polling place guidelines, universal design guidelines, again, using our verbiage today that are requirements for the deployment of the voting equipment as well as ballot design, instruction error message guidelines and just looking over the whole of this work product and looking at any other general human factors and privacy issues that may have arisen in what we’ve done thus far and some usability evaluation, the redesigned VSS in general.
I guess I’ll leave the road map up.  That concludes my summary of where we are thus far and I’d like any feedback or questions or clarifications.
MALE SPEAKER 6:	It looks like a good plan.
DR. LISKOWSKI:	Thank you.
FEMALE SPEAKER 5:	I just have a question for you Sharon.
DR. LISKOWSKI:	Yes.
FEMALE SPEAKER 5:	I know you have been getting some feedback and input especially on the accessibility requirements in the community.  Would you speak a little bit about that?
DR. LISKOWSKI:	Yes, thank you for reminding me.  We are in close consultation with the Access Board and, in fact, they have been very kind to provide, I think we got it Thursday or Friday a lengthy commentary on what we’ve done so far and so we are going to go back and address all those comments and be in both e-mail and verbal discussions with the Access Board.
FEMALE SPEAKER 5:	I know I have seen some comments come in from accessibility and disabilities commissions around the country.

DR. LISKOWSKI:	Yes, we have read all the comments that have come in and we are going to be addressing each one, one by one to make sure we incorporate those into our accessibility documents.
DR. SMERGIAN:	Thank you, Sharon.  	
DR. LISKOWSKI:	You’re welcome.
DR. SMERGIAN:	This leaves the preceding preliminary report of technical support titled “NIST Approach to Usability and Accessibility Requirements for the Augmented 2002 VSS” and “Response to TGDC Resolutions Number 2, 3, 4, 5, 6, 8, 9, 10, and 11".  They had their work cut out for them.  So, unless there are supplemental directions or corrections the technical support and related work product will continue to be developed consistent with this preliminary report.  Are there any questions, further directions or corrections?
MALE SPEAKER 7:	No, I was very pleased with that.
DR. SMERGIAN:	Looks like everybody is happy with what they’ve seen and the direction they are following.  Thank you.
At this time I call on John Wack of NIST Information Technology Laboratory to present the preliminary report on NIST approach to voter verified paper audit trails requirements for the augmented 2004 VSS.  John.

JOHN WACK:	Thank you.  Can you all hear me fairly well?  Okay, well, hello, all.  I’m John Wack.  I’ve worked in the computer security division at NIST for a number of years and Mark Scull said this is a burning issue and I can assure you that I am burning.  I feel a little bit at a loss discussing this with you because this issue has been discussed so much that what can I really tell you that you don’t already know and I was hoping to plan a couple of things, you know, earth shaking, like it reduces cholesterol or makes people feel better, you know, reduces impotency, things like that.  What I generally found out is that it really being in security doesn’t necessarily make me the best person to address this issue.  Usability is critical and there are a number of small issues that need to be addressed that aren’t always brought out to light and that some of the state legislation is very good, some of it is at a higher level and more than ever I think we really need your help.  We’ve come up with some requirements.  We may have set the bar high, maybe we need to go higher, maybe you want to take it down lower, but in general, any comments, any suggestions, any criticisms, are very much appreciated.
The other thing is I just want to briefly thank the members of the NIST voting team.  I’ve got a lot of assistance and the STS, especially Dr. Ron Revest.  Its been a pleasure to work with him.   The I Triple E very graciously allowed us to use their draft P1583 standard.  I wanted to also thank Leslie Reynolds.  Ms. Leslie Reynolds of the NASS who helped in some analysis of state legislation.
Lets see if this thing works.  I’ve got about ten slides and I am really just going to go through the first eight very quickly and just basically to give you an idea that we did try to do some due diligence in figuring out what the issues are.  You know the genesis for this.  We really are basically working on a addendum to the existing 2002 VSS.  We are really addressing DRE systems that basically produce a printed summary of a voter’s choices.  The voter can then compare that with the electronic record.  If the voter is satisfied with that, he can then do something that causes the electronic record to be recorded and that is the record that is counted.  The paper record is used for audits and recounts.  That’s the system we are talking about.

Let me go on to the next one.  Basically, in addressing this issue we, number one, there is plenty of literature out there, a number of strong opinions, a number of good analyses that are out there.  A number of those are available on our web site.  We have been discussing things with some election officials.  That’s been what I’ve found critical.  We can sit and think about requirements and think about the most secure ways of doing things and then we have to talk to the people that actually have to pick up the paper, turn the machines on, do statistical analysis at the end of the day in a hurried environment and so really getting back to the security issue, it has to be secure, it has to do what its supposed to do but if its not usable its really not practical and workable.  Then we looked at what material we had out there that we could derive requirements from and found that in general some of the state legislation was very good and from there we went on addressing the major issues and came up with some requirements.
From looking at things, looking at the literatures out there, discussions, dealing with people, its no surprise that the issues I have up here are really the important issues, usability, accessibility, robustness of the technology.  The fact that the electronic and paper records need to correspond with each other because they are two distinct records of votes and they need to correspond carefully and be handled appropriately.  Then there are various issues that arise in resolving complications and disputes.  So, we looked at those issues.  One other thing that was brought out to me is that usability, we tend to think of that as for the voter but its critical that it also be for the poll worker, for the election official.  Usability of the paper record has to be looked at by the voter but somebody else has to audit that.  That’s a very important issue.  
The other thing is that DRE’s with VV pack capability may be larger. It may take up more space.  If a poll site generally has a line of voters it may increase the line of voters because it takes more time.  I don’t know how much more time but it takes more time to verify your vote and cast it that way.  So, there are just some practical issues associated with that as well.

Looking at the state legislation I have to admit I looked at it at first thinking I’m not going to find anything and actually I found a wealth of very good material.  A number of states have done a very good job, I think, in addressing some of the requirements and figuring out some of the issues.  Some states looked at it at a higher level and again I think this is where we come in where I think we can do some good at least specifying the core fundamental issues and requirements of best practices or what have you that the states really need to take a look at to make sure that they do this correctly.
A couple of things I’ll point out.  I found a couple of surprising things that I didn’t expect to find but some of the states actually wanted a strict correspondence between the electronic and the paper records.  They wanted to be able to look at the paper record and go to the specific electronic records.  So, they wanted identifiers on the two to map them back and forth.  So wanted machine readability for the paper records.  Accessibility was covered; I thought, fairly well, at least mandates for accessibility.  
A couple of other things.  I didn’t delve too much into the state legislation in other areas besides VV pack capability.  I did find that five states overall had enacted legislation with a lot of VV pack requirements.  At the most recent count I think twenty-three other states were dealing with legislation.   The legislation had been introduced and was in various stages.  We pretty much looked at what had been enacted thus far.  I don’t know how the other legislation will turn out.
Again, no surprises.  What did we find out based on our initial analysis of issues and state legislation, again, usability, usability, usability.  Basically procedures needed to handle the paper.  VV pack capability should not simply just reintroduce paper problems. As much as possible we should address that issue so that paper is easier to handle and so that auditing is more accurate.  It should not be a surprise.

I’m going to skip over the major features and the major benefits because it sounds like I’m trying to sell you something here.  I’ll just go directly to the requirements and what I would like to do is just kind of go over them a little bit and, see if I can find my notes here, give you a brief summary and some of the issues involved in that and, Alan, I’ve got about five minutes left?  I don’t want to talk too much.  Oh, I can keep going.  Okay.
What we did in our document was we came up with eight core requirements and then we have a number of derived requirements for each of those eight.  Other documents I looked at came up with slightly different requirements, slightly different language, so we are not wedded to this but this is our way of looking at it.  
The first one is a fundamental requirement of DRE capability,  DRE VV pack capability and it basically just says that a paper record has to be produced and at the end of the day you end up with two distinct, independently handled records of votes.  You have two separate things there. You don’t necessarily have to use the paper record for auditing and recount.  I think HAVA says that the paper record is the record to be used as the last resort but you do end up with two records that have to be handled carefully.
The second one, permitting the voter to compare the paper record and the electronic record with maximum uses.  As you can figure out a usability related requirement.  Basically if the verification procedure is not used, or if it draws complaints, obviously that’s a bad thing.  Let’s say one in every thirty voters actually performs the verification procedure.  That’s statistically may be a significant figure depending on the size of the election.  Its still an indication that the usability is probably pretty poor.  I don’t think if the usability is very good we are going to have everybody verifying their vote but we should strive for more than one in thirty.

Number three, the method for vote verification shall be accessible.  I’m going to point back to Dr. Liskowski there a little bit more.  What I know about this subject is just enough to make me dangerous.  Basically I do know that handling accessibility requirements is fundamentally a little bit different from doing it otherwise.  It requires special features.  A couple of things I find interesting, for example, if you print out a paper record, let’s say in Spanish, then is that going to make it harder for somebody auditing the paper records to deal with that.  We have to design the paper records in certain ways so that they are still accessible to auditors as well as voters.  A number of issues associated with that, privacy has to be safeguarded.  If there is only one Spanish speaking voter at that site then its going to be pretty easy to figure out what their paper record is.  We have to be careful about that.
Number four.  The DRE VV pack shall permit the voter to accept or reject the paper record and reenter ballot choices.  Basically, we need to keep a strict correspondence of records even if they are spoiled.  A high rate of spoilage may indicate that there are some usability problems with that system.  In a sense with that particular requirement we need to come up with a number of procedures that deal with spoiling limits, how that’s going to be handled.  What we’ll do in basically marking ballots in the presence of voters that they are spoiled or accepted.  A number of issues are coming up there.

Number five.  Voter’s privacy and anonymity shall be preserved.  Well, one of the obvious things is when you have two records of the votes, especially a paper records that can be handled and read, then you have to address the privacy related issues with that.  I’ll talk a little bit more about that when I deal with the next requirement.  One of the interesting things that we are struggling with is that the manner in which the paper record is produced and then stored ought to safeguard voter privacy.  Ideally it would come off the printer and it would fall in some sort of random fashion.  Somebody could pick it up.  They wouldn’t know what system it came from, what voter.  That may be difficult to do.  It may be very difficult for every single DRE to do that.  I don’t know but it seems to me that that could cause some problems and it may involve some compromises such as people may need to handle the paper ballot.  They may need to walk it to a central ballot box.  It may be that, you know, one possible compromise is that it may be that the papers all come out in the order in which people voted and then poll workers need to safeguard the security of that or randomize it some way.  There are a number of issues there.  Ideally, though, we would like the technology to address the security issue automatically and not rely so much on people but we may not be able to do that.
Number six.  The DRE VV pack shall permit robust auditing, forensic analysis, full recount capability.  That’s really the more interesting one.  That deals kind of with the situation of when a voter goes to verify that the paper record and the electronic record match and if they don’t match, and then we have some interesting issues there.  I’m not sure what exactly to do whether we call in the armed guards, whether we pull the plug on that voting system or what.  Basically we need to be able to provide a paper record that points back specifically to its corresponding electronic record to improve auditing capability.  We need to do the same for spoiled ballots as well.  We need to have a one to one correspondence.  It would be good if the paper ballots contained more information such as an indication of the serial number perhaps of the DRE.  The name of the election.  The precinct.  A number of other things that can be looked at after the fact to help identify problems, forensics analysis, things of that sort.  Let’s see if I’ve left anything out about that.  Also got a couple of requirements in there for how electronic records ought to be exported from voting systems.  It would be good if voting systems exported the electronic records in a format that could then be used and analyzed on a different computing platform and if you are going to export them it would be good to export them with some sort of a hash, ideally a digital signature but provide some authenticity.

MALE SPEAKER 8:	I just wanted to make sure I understand because I’m a little bit confused about what the readers really provide.  So, let me make sure I understand the process.  I’m a voter.  I see on my screen the votes.  I look down and I see on a piece of paper, under glass, let’s say, perhaps, a vote.  I see they match.  I’m happy.  I walk away.  Isn’t it likely that the meddling would be after it leaves the screen?  It get manipulated through the data base.  What I see is not what necessarily going to match.  So to me I don’t, it’s a lot of effort and it doesn’t really solve the problem.  Is that the way people are talking about it right now? It would seem to be more likely to, let’s say, have it in a data base which is used for counting, put some random number, and I have a random number here, and I can go through the process of accessing it from the data base.  Even then there’s some risk but a little less risk.  At least it has physically left the screen and gone to some data base for counting.  Something like that.
JOHN WACK:	Well, I guess as far as opinions about whether, you know, this is a good thing to do or not, I’ll probably refrain from that.  I think, though, that establishing some traceability between individual records certainly allows someone to determine whether paper ballots have been inserted or whether they have been removed.  I think, I don’t really know, based on my research if that aspect of fraud is the biggest issue.  Its probably not.  I do think making the records correspond to each other strictly does help in that regard.  Did I answer your question?
MALE SPEAKER 8:	Yes.
JOHN WACK:	Okay.  Another question.

MALE SPEAKER 9:	I think within the scope of what I understand we are to work on and what the commission has an interest in is providing standards for the states who now have mandated this technology.  So, the issue of how you spoil ballots, what you do with the spoiled ballot, what you do when you reach a conflict, how you are going to run the forensic analysis, you are going to have to alternately look to state law for that and those states that have required this and haven’t addressed that will find themselves addressing it either in statute or in the election contest activity.  
I was interested in the use of the word auditing and I just checked the prepared glossary and I agree with that definition there and I just wanted to make sure in your analysis we are being consistent with that definition which in the glossary it is the systematic, independent, documented process of obtaining records, statements, fact or other relevant information and assessing them objectively to determine the extent to which specified requirements are being filled.  That is very different from the use of the word auditing or reference to recount type proceedings.
JOHN WACK:	Yeah, yeah and I think we are talking about the auditing, you know, the aspect – there’s a possibility that definition does cover that in a very general sense.  The auditing I’m talking about more is the correctness of the electronic record via auditing it with a paper records.  I’m talking about the usual one percent manual recount or auditing that we generally see in state law.
MALE SPEAKER 9:	So you are talking about what I think in some states is called a discovery recount.
JOHN WACK:	Right.
MALE SPEAKER 9:	Once again, that’s something that some states have and some don’t.  Once again, you are going to have to look, even though I know a lot of its high level and there’s not much there, in some states, you are going to have to look back to the state laws because that’s, those are the people who this particular standard is going to serve.

JOHN WACK:	You raise a good point.  I meant to bring it out.  I don’t think we are in a position to say what auditing you ought to do.  That’s clearly not our area.  What I try to do is at least make the records more accurate and more traceable and in a sense give people tools so that, if they choose to use them, they can and we assert that that will result in better accuracy especially in resolving disputes.  As far as the rate of auditing I’ve read a lot of material about that and opinions vary.  I don’t want to get into that so much, its just basically here are the tools so that you can do auditing.
MALE SPEAKER 9:	Okay.
DR. SMERGIAN:	If I can, John, we’ve talked about this.  I think this is a very important point.  We are not advocating any of the particular uses technologies, etc.  The point here is, if a state decides to do certain things we want to make sure that the standards and therefore the tools exist or are present to be able to do those.  This is strictly if all the statements have to start with the “If” statement, you know.  If a state decides to do this or if they have a legislation to do such and such how can that process be done in the best possible way.  I think that’s a very important issue here.  We don’t want to give any impression that we are in any way advocating, you know, this or that technology or to do a particular procedure.  I just wanted to emphasize that.
MALE SPEAKER 9:	But we do want to have the tools available.
JOHN WACK:	Yes, thank you.
MALE SPEAKER 9:	I’m just a little sensitive about the misuse of the word audit because, I mean, its something that election officials have been beat up on by activists and interested parties who have complained that they have no audit process when they have very good internal controls, they have very good documentation of the process, they have separation of functions and they have a whole bunch of stuff that you can audit but those are election administration controls more so than audit.  I think the definition here in the proposed glossary speaks to that.  I just wasn’t sure if we were being consistent.

JOHN WACK:	Well, what I would like to do is talk with you off line.  I like the glossary.  I just like the fact that we are trying to focus on common language that we understand.  That’s another thing that I kind of learned in state legislation is that people use terms differently depending on what state and that must be confusing.  It would be better if we all stuck with the same definitions.  Yes.
FEMALE SPEAKER 6:	I just wanted to make sure and I think the question was answered but I just want to make sure that when we are talking about whether its in a roll, our receipt is in a roll and that the elector has the ability to read that or if it is cut and given to or made available where the elector can put it in a box.  I don’t want us to take a stand on that because many of us really have concerns about the elector because if they walk out with that receipt then it doesn’t match.  Whether you call it auditing or recount or whatever the case might be, we are in trouble.  I want to make sure that we are not indicating one way or another and leave that up to the states for their state law.
JOHN WACK:	Okay, well, that’s a good comment.  I would say what we have thus far is leaning towards the direction of basically saying that it is easier when you have individual pieces of paper to make then random.  Its easier to feed them perhaps through a machine and automatically scan them.  If you keep them on a paper roll obviously you preserve the order in which voters cast their votes.  As to which way you want to go that’s really not, you know, our role to say.  They both have advantages and disadvantages.

FEMALE SPEAKER 6:	I agree.  I totally agree with you.  I do think even if its on a roll it can be handled by procedures and by rules that its not marked because, I mean, if you only have one piece of equipment to meet the disability requirement of the law in that polling place and even if you have more than one, you are not going to know how the voter voted even if you tried to track back unless you marked that, what machine they voted on and obviously that’s the thing we should be concerned about, that there’s never, ever an indication whether the individual was voting a scanned ballot or on a DRE or which piece of equipment that they were voting on.  I think that’s a real concern there because we have to protect the privacy of the voter.  That’s my main concern when you get into an issue like that.  I agree with you totally but I think there is ways to count those, whether its underneath a glass or a bigger ballot, I mean, they are both coming up with ways of being able to count that other than just by hand.
JOHN WACK:	Could I respond real quickly.  We, no, you are more important.  Go ahead.
FEMALE SPEAKER 7:	Well, I guess, I actually wanted to start by congratulating you on a really coherent and dispassionate presentation of this issue.  One that has caused raised voices in hallowed halls and many meetings.  So, I think that, in itself, is quite an achievement and I congratulate NIST.  This is exactly the sort of careful analysis I had hoped we would be getting from the NIST staff.  So, thank you very much.
I would also just like to pick up on what Secretary Davidson was saying which is I was looking at the structure of the document as you have been proposing.  It was kind of hierarchal requirements and I think that there is a lot of value in being able to say, if you do it this way, then here are some requirements.  If you have chosen a paper roll then you must have surrounding administrative procedures or if you have, whatever, so that we are not providing a single solution but are providing the requirements in all of the appropriate permutations of solutions and I think that one of the things that will do indirectly is provide guidance on how difficult to administer or how difficult to design or prepare or qualify or certify actually, both of those two things, a particular design of system might be.  I think that, in itself, would be a helpful kind of indirect analysis that would help state officials make good decisions.  
JOHN WACK:	Thank you.
FEMALE SPEAKER 7:	Sure.

MALE SPEAKER 10:	John, just two thoughts, you know, I didn’t give it much thought.  With respect to the concern about the order in which ballots are and seeing what people have designed, now this may not work for all districts, it may not work for any but, if you had a large enough number of people you could perhaps say have then sign their books, have bunches of lets say five or six, use a randomization of the individual before they go into the voting booth.  Therefore what you are seeing on the books, you know, isn’t necessary the order in which they voted.
FEMALE SPEAKER 9:	That won’t work in my precinct because we have a single machine per precinct.
MALE SPEAKER 9:	Okay.
FEMALE SPEAKER 9:	I live in a very small town.
MALE SPEAKER 10:	No, even if a single machine but you have to have at least enough people so you couldn’t, wouldn’t make – thirty minutes until another three or four come along.
MALE SPEAKER 11:	Another issue that you have is poll watcher who basically are allowed to sit in a precinct and would be capable of documenting the order in which people voted.
MALE SPEAKER 12:	It is possible in the disability issue that even though it’s a standard paper, if you had feeding after that machine into another special machine that could render it in an particular media or so forth, you might be able to get around it that way.  It’s the same paper that’s now being scanned and read and it gives an audio voice or a different print or whatever it is that you want from the machines in which you have counted is a possible solution for the disability one.

MALE SPEAKER 13:	John, could you address on number six to clarify.  What it seems to read is that both the electronic and paper records would be used for a full recount?  I thought there should be a precedence as to which is used for recount as opposed to saying both.
FEMALE SPEAKER 10:	I believe –
JOHN WACK:	Well, that precedence –
FEMALE SPEAKER 10:	Isn’t that a matter of statute?
MALE SPEAKER 14:	Yes, it would come from state law.
MALE SPEAKER 13:	But that’s my point.  I think this recommendation seems to imply that both, as opposed to saying that there is a precedence here.
JOHN WACK:	Okay, I understand.  Yes.
MALE SPEAKER 13:	That’s all I’m saying is a clarification of how –
DR. SMERGIAN:	John, do you mean both or do you mean either?
JOHN WACK:	I guess really either.  
DR. SMERGIAN:	I think what you mean is that depending on the situation either one may be used.  Is that what you really mean?

JOHN WACK:	Right.  We should do our best to end up with a situation where either record could be used.  As to which one must be used or shall be used that’s not our decision but it is best if we design the records and produce them in a way that if they both have to be used, the can.  Really when it comes to resolving disputes, if the two records disagree, I think the more information we can draw about how the records were created the better off we are and I wanted to come back to an issue you addressed and it seems that from what I can understand there’s this tension between usability of the paper records, let’s say, versus auditing and information on the paper records to make auditing usable and more accurate.  If you don’t include any identifiers on the ballots or on the paper records, I should say, if you don’t include the serial number of the DRE or a unique identifier, things of that sort then you can take those spools of paper, you can put them in a box, they can be stored for a long time, nobody can go back and figure out who voted first on a particular machine.  At the same time it does make it a little more difficult to audit the paper records against the electronic records and the better the correspondence, the better you are able to do a number of things and also just basically come up with post election statistics and analyses that would be good for usability of systems.
One of the things that we proposed in the requirements is the possibility of, you could have a paper ballot, let’s say like this, and its got a human readable part and then an encoded part.  The encoded part could be basically an encoded representation of what’s on the human readable content.  That way you could hide some information such as unique identifiers and things of that sort.  It would have to be an open format basically, the major shippers use similar sorts of encodings and formats, easily scanned.  It would require some auditing to make sure the two representations worked.
FEMALE SPEAKER 11:	Too far from microphone to understand what was being said.
JOHN WACK:	The major shippers.  I was waiting for a package and a shipper showed up with basically a box that had some dirt and scratches but they were still able to use a hand scanner to go through and read off information on the package.  Its fairly robust technology.  That would, that may help in this situation.  It may hide some information.  It would also making scanning the paper records more accurate.  There is this tension here, you know, I think the more of these sorts of things we can do to make auditing perhaps more accurate we still have this tension with usability of the records and perhaps reduce overall accuracy in auditing.  Maybe I’m misusing it a little bit but in recounts that may be acceptable.  I really don’t know.
DR. SMERGIAN:	Secretary Davidson and then Dr. Williams and then Mr. Gant.

SECRETARY DAVIDSON:	 And I agree with you.  I think that, you know, there is so much that you can do with a bar code.  You have to be very careful not to put numbers on there because the public will view that that we can identify how they voted.  Its very, very contentious.  A bar code that could not ever identify how somebody votes.  We don’t want to ever do that, be able to with a bar code or with numbers.  We have to make sure that they understand that it is only for auditing to make sure the votes were there, not tracking how they voted.  It is very important.  When we are wording it, evidently it has to cover that because we don’t want the public to take it the wrong way.  It’s got to be very common wordage (sic) so that they understand that, I think
JOHN WACK:	That’s a good point, thank you.
DR. WILLIAMS:	I have several points.  One is on this issue of putting information in bar codes.  I have to point out that the voter can’t verify that bar code so you don’t have a voter verified ballot anymore if you use that bar code to count those paper ballots.  You are not   (END OF TAPE 1) were introduced in this country voters have handled their ballots.  One of the criticisms I’ve heard of these VAT systems is the fact that that things under glass and the voter has trouble reading it and can’t hold it in their hand and look at it.  I’m glad to see you are considering that.
The third thing that’s kind of a nagging problem for me is that I haven’t heard any discussion of the impact that all of these innovations are going to have on the voter.  We know a little bit about voter behavior and we know that most people who vote are in a hurry.  If you start putting impediments in the voting place, regardless of what that impediment is, whether I’ve got to verify this or take it over to another station, or what, you are going to have a drop off in your participation so we don’t want to create an unintended consequence here where we come up with an enormously secure system that people stay away from in droves.

JOHN WACK:	Well, you raise a question and that is what if a voter doesn’t want to verify their vote.  Should there be a way of saying, I’m happy, press a button I’m out of here.  Yeah.  The issue about the voter, in a sense not being able to verify the encoded representations that is a good point and I’ve thought about that.  That does require some manual auditing.  How much, one percent, I think it would probably be likely a lot less than that but you do have to do that.
MALE SPEAKER 15:	We could miss the whole underlying concept here is to establish voter confidence.  When we are in the back room verifying that this bar code corresponds to that printed record the voter is not participating in that process and it doesn’t do anything for his confidence.
JOHN WACK:	I anticipated a question about this and all I can say is, I agree at the same time I think its, if you handle a piece of paper you can wrinkle it, the paper has got to be more robust, a number of things.  If there is a robust encoded bit of information on there that can be machine scanned it makes it easier to handle the paper and you more accurately scans so there is this, you’ve identified a conflict here.
Was there one other question?
MALE SPEAKER 16:	John, I just wanted to kind of close up my question.  So, in the section 3.3.6 which is the detailed set of requirements that you will add some text or another item to kind of clarify the recount as being a different process than the audit as described there.  In other words there is nothing in there that really talks about the full recount being based upon one or the other, based upon state law, etc.  I just want to be sure that that section gets clarified.  That you are not expecting that a recount is done with both.

JOHN WACK:	Okay.  All right.  I understand and again, maybe afterwards I can grab you for a minute or two and make sure I get the language down.  I think I may be running over and I just have one last thing to say.  I didn’t address just two of the last requirements and I should do them in maybe one sentence.  They are fairly obvious.  One is printer technology must be very robust and even reloading a printer with paper or ink may be too much maintenance to do on a printer during the day.  So we are addressing those issues and we, I’m a little sensitive to prescribing procedures or requirements for procedures but we are identifying the need for definition of procedures to handle all of the complication that can occur because, you know, what can go wrong will go wrong.  There are a lot of complexities to providing this.  So, people who have the proper training, who can anticipate and answer questions, so on and so forth.  That’s another important issue that can’t be neglected.  Yes.
SECRETARY DAVIDSON:	I have one additional question.  Have you thought about and I apologize, but have you thought about or looked about retrofitting old equipment.  States are saying you’ve got to have the paper verification on DREs that have already been bought and retrofitting that.  Are you moving into that area whatsoever in the suggestions because that is coming up in states.
JOHN WACK:	Well, that’s a good point and we haven’t addressed that specifically.  I think its possible we could not address it specifically but still issue requirements.  That may get back to what Whitney’s mentioned that we may have to do something like if you are going to do it this way, if you are going to have separate sheets of paper do it this way.  If you are retrofitting do it this way.  That will be a little more work, I think, but I agree its an important issue.
Okay.  Thank you very much.

DR. SMERGIAN:	Thank you, John.  Did we take care of all the questions, comments?  Well, I do believe the preceding preliminary report of technical support titled “NIST Approach to VV Pat Requirements for the Augmented 2002 VSS Response to TDGC Resolution 1205".  Unless there are supplemental directions or corrections, certainly the comments made will be taken into account.  The technical support and related work product will continue to be developed consistent with this preliminary report.  Let me ask again; are there any questions, further directions or corrections?  If not – yes.
SECRETARY DAVIDSON:	I do believe that we need to do something about the retrofitting and that would be an additional requirement, I’m sure.  I think that states looking at your vendors right now are looking into it.  They could use some guidance I do believe because they are just beginning to assess that, what it would take.  Obviously, it would have to go through certification and we feel like the equipment would have to go back through certification if you are retrofitting an additional piece on there.  So, I think that should be something that we should add to it.
DR. SMERGIAN:	Do we need a motion for that?  I think that’s a minor addition to the instructions.  Do we have unanimous consent, basically, to add retrofitting issue to the instructions?
MALE SPEAKER 16:	That’s governed in your present standards anyway because the present standards say that any change to the voting system requires re-qualification, re-certification.
DR. SMERGIAN:	All right.  So, hearing – did you have another question?
SECRETARY DAVIDSON:	No.  I just, I agree with Britt on the certification but on the, giving them guidelines, you know, what we think that retrofitting should be, I think should be there.
DR. SMERGIAN:	Okay.  Well, not hearing any further –
MALE SPEAKER 17:	Well, why don’t we go ahead and make a motion, make it a Resolution and then that kind of ends the question of whether we need to or not.

DR. SMERGIAN:	Well, we just did.  I mean, by unanimous consent we will add retrofitting to the instructions.  Is that acceptable to everybody?  Okay.
Well, amazingly we are ahead of schedule.  Instead of starting the next presentation I suggest that we take a little earlier break.  We basically had allowed for a twenty minute break so I suggest we come back at 10:40 this time and we will start again.  So, please try to get back a few minutes before that so we will be able to get started at 10:40.  Thank you.
DR. SMERGIAN:	Okay.  If we could resume our meeting.  At this time I call on David Siefert of NIST Information Technology Laboratory to present a preliminary report on NIST approach to wireless requirements for the augmented 2002 VSS.  David.
DAVID SIEFERT:	Thank you.  
DR. SMERGIAN:	Can we sit down please?
DAVID SIEFERT:	Based on the little discrepancy with whether you’ve had time to review the document or not, my main concern is have you at least had time to read the Executive Summary which is just a cover page?  If not, I’m going to read that so you have some idea of where my presentation goes because the hope was, I wasn’t going to rehash what I have in the presentation or what I have in the document but rather highlight some of the issues that I think need to be addressed.
I’m going to read from the Executive Summary from the document: “Based on Resolution 35, number five, titled ‘Wireless’ NIST is directed to research and draft standard documents for the use of wireless communications in voting systems.  Since a blanket statement about wireless communication devices in voting systems is neither prudent nor appropriate given the wide variety of wireless communication devices and possible usage in the numerous and diverse voting systems the approach to considering wireless communication devices in voting systems will be on a case by case basis.  

To this end this will create a guide showing where wireless communication may potentially be placed in a particular voting system and some of the associated security risks.  This guide will contrast the hype for wireless technology usage versus the real needs and/or requirements for the wireless technology to improve the performance or operation of a voting system.  The placements described are not to be construed as suggesting that wireless technology should be used in these locations nor is the list exhaustive of all the current or future usage of wireless technology placements, nor does it replace those pre-existing wireless requirements currently stated in the VSS 202 or the I Triple E draft.”  I’m going to skip the next paragraph and read the last, final paragraph.
“An exhaustive investigation of all possible wireless technologies or, more importantly, all implementations of wireless technologies will never be practical.  Therefore specific wireless technologies will only be used as examples.”
With that said,
SECRETARY DAVIDSON:	Could you move the mike closer to your –
DAVID SIEFERT:	Can’t hear me?  All right.  Better.  I’ll talk louder.  Okay.  So, the draft as it stands is to respond to 3505.  First we need to reach a common understanding of the subject of wireless communication devices and I’m trying to do that by providing background information.  Wireless, as a term, is broad.  I also try to identify any technical requirements for the use of wireless communication devices for the voting system by review VSS 202.  The end goal is what you want to provide guidelines on the use of wireless communication devices.

The progress so far is the draft before you.  I’ve tried to do this in three, trying to address three separate potential people who are concerned with wireless usage.  As was mentioned in the opening statements by Mark Scolo, we have to prioritize these based on the voting system itself, the users of the system and also the technologists who have to try to define features that will be available to those systems.  So, if you look at the draft, its laid out in these areas.  The first section, Section 3, which actually goes through the VSS and notes any possible location of where someone might interpret as a wireless communication device being needed or being used.  
Section 4 addresses the issue of the Election Day or preparation for election.  The equipment itself, how the election is run, wireless need, where and when wireless may be used because there has been discussions that, well, you can use wireless except for the Election Day.  Well, not necessarily, on Election Day there may be times when a system to use wireless may be appropriate or may be convenient.  Its not a time of day that you can turn wireless on and off. 
Finally, the major issue that I had reviewing VSS 202 and wireless, whether its use, was in the realm of telecommunications.  Telecommunications has a section unto itself, Section 5, and it lists some wireless technologies or wireless as an example of usage cases but it doesn’t go into any more detail.  If I look at that strictly from a requirements point of view, wireless has no more requirements than a telecommunication network does.  I can have that perspective but I want to call that out that that’s an assumption.  Also, there is a discrepancy between the telecommunications network being public or private.  A statement is made in VSS 202 that private networks have to be the same or have to follow the same requirements as public networks.  Now, I know that public networks have a lot more regulation than private networks and it may be that regulation that makes a public network more trusted than a private network.  So, I wish to call that out as well.  They have to be separated.  The difference between telecommunications and wireless and the difference between a public network and a private network.  Okay.

Now, this is the most interesting picture and its hard to see but, I’ve tried to place wireless anywhere that appears to be a place in the VSS that wireless may be used.  It doesn’t say it has to be used.  So, this figure shows many different arrows and the arrows are to be representatives of wireless communications.  Along with each arrow is a piece of information that may be transported by that wireless communication.  As part of wireless usage, and securing wireless usage you need to know what is being transmitted and what do you need to secure.  
So, in the simplest case, we have the keyboard and the mouse going to a DRE or a vendor system to program either the ballots or the program itself.  How secure or insecure are those links?  When you think of mouse click, who needs anything from a mouse click?  What information could someone obtain by having a mouse click?  The same is true for a keyboard.  Well, guess what, passwords are put on keyboards.  How many wireless keyboards are encrypted so that no one can intercept the password?  A vital piece of information.  Also we talked this morning about whether the system has a capability and whether we need to state that a person makes sure that that capability is implemented and followed correctly.  A password is a perfect example.  I can make a requirement that a system has to have a password but how do I force the user to make sure that that password is used?  We all have the default enter and that’s a password.  Its not very efficient for security.  Wireless is no different.  We can have the capability but unless we have policy and procedure as well, its not going to do it.
MALE SPEAKER 17:	You can get software that you have to have (undecipherable).
MALE SPEAKER 18:	If we do that you put in software that the person puts in two characters requiring eights, it won’t accept it as a valid password.
DAVID SIEFERT:	Okay and now, --

MALE SPEAKER 18:	You can set up a password that’s not long enough and doesn’t have the requirements that you want to have.
DAVID SIEFERT:	Okay.
MALE SPEAKER 18:	If software – someone couldn’t do a space bar and set that up as a password, they just couldn’t on those kinds of systems.  You couldn’t do it on our system.
DAVID SIEFERT:	Okay.  Now, does your system also require that that password has to be changed frequently?
MALE SPEAKER 18:	Yes.
DAVID SIEFERT:	And how frequently is that?
MALE SPEAKER 18:	Well, that’s a matter of our policy.  I think a voting system might have a different kind of policy because they don’t run it every day.  We do require every month, every two months.  It depends upon the policy of the particular business and the sensitivity of the systems that can be accessed and so forth.  But that also can be enforced by software.  That’s what it is.  I go in there and I go to enter my password and I automatically get a new screen.  I have to reenter, select a new password.  When I go to select that password, if it’s an old password it won’t accept.  If its less than eight characters, it won’t accept it, and on and on and on.  I could do trailing zeros.  Its whatever they put in that software control to sort of force me to put it in.  That certainly can be done.  It is done.
DAVID SIEFERT:	Okay.
MALE SPEAKER 18:	I mean, we don’t get certified unless we do it.
DAVID SIEFERT:	All right.  Well.  Again, that is, I guess the question there becomes on password, eight characters is not very sufficient –
MALE SPEAKER 18:	I was just giving that as an example.
DAVID SIEFERT:	prep.  

MALE SPEAKER 18:	Actually there is work going on at NIST right now in this area in support of GSA and we’re doing a study with them and they are looking at what we would have to be to be certified as a certifying authority for our customers to access their sites if we were to authenticate it.  They have a whole set of tests, that part of NIST as to what we would have to do in order to be qualified as a certifying authority.  Part of it deals with this whole issue of passwords.
DAVID SIEFERT:	Okay.  What happens if you forget your password and you are locked out?  But anyhow.  Okay.  So, again, back with the examples present.  We just heard, prior to the break, about the verified paper audit trail.  If the printer needs to be added and there’s no printer port to the device it may be easier to implement that connection by a wireless connection.  I’m not saying do it but its another place where its available.  The only place that I can find that has any kind of a requirement and its an if requirement, its conditional, is for the access for handicapped.  Its marked as a wireless coupling if you are using a telephone handset and that is required in the VSS and that is a wireless communication.  So, there is no way to make a blanket statement, you can’t have wireless because there’s a perfect example of where wireless exists and would have to be supported if that device was done.
The public, back on the public telecommunication network there may be wireless inside the public telecommunication network through microwave communication or satellite communication that is impossible for us to control.  To add security features there, we have no authority to do so.  So, wireless, in some cases, and that’s why I said it’s a case by case basis, we are not going to be able to provide you with any help in those areas because its beyond our control as well.

Okay.  I guess I’m doing okay on time.  Now here are the numbers trying to address the issue of all the possible wireless communications.  I’ve been participating at I Triple E land man working group and there are five different wireless standards that they have published.  There are seven different physical layers that use wireless technology, infra red or radio frequencies and they are building more.  That’s asking a lot just from the I Triple E, if I just look at the I Triple E standards.  
Go to cellular technology and I can count more than eight standards available for cellular technology and as cellular technology grows and the shrinkage in electronics and the advancement in the capacity of those devices more and more things will be able to be done on a cell phone.  We’ll have to call them P.C.’s too. 
Now, the last statement, I have a visual graphic, I didn’t put it in my slide.
MALE SPEAKER 19:	Does it – you’ve included Blue Tooth and everything under that top five?
DAVID SIEFERT:	Blue Tooth is the lower layers of the Blue Tooth specification were standardized by 80215.1 but the Blue Tooth technology itself is an entire seven layer stacked protocol.  It does not follow the I Triple E structure for putting layers above it.  It’s a completely defined self-contained package.
MALE SPEAKER 19:	Right, but isn’t that becoming more common for these detached keyboards and headsets and many of these wireless connections?
DAVID SIEFERT:	Right.  Blue Tooth is getting more and more places.  Correct.  There are Blue Tooth mice.  There are Blue Tooth keyboards.  Where Blue Tooth is the initial penetration point was for the cell phone because it was pushed by a cell phone manufacturer.  They were doing wireless headsets for that purpose.  Wireless is being put in top of the line cars now too.  I don’t know why but most of the time just to connect your phone.  Okay?

So, this is a frequency allocation spectrum and its very hard to read.  This is October 2003 from NTIA, our sister organization.  This is just a radio frequency spectrum and if you count all the little colors you have well over five hundred possibilities.  If only one technology was present in those and I know they are not because I Triple E 802 sticks a lot of technologies in one frequency band.  This is not a doable project.  Yes.
MALE SPEAKER 20:	A couple of things.  One is – they also – IR is used a lot.
DAVID SIEFERT:	Sorry?
MALE SPEAKER 20:	IR.
DAVID SIEFERT:	Yes.  That’s not in this radio frequency band.
MALE SPEAKER 20:	Right and IRE is very directional.  Its very hard for someone to intercept.  It would seem to me that if you specify something like IR, with some encryption, if you are really paranoid, then you have solved the problem of the wireless doesn’t provide any security problems.  As far as the wireless in the public telecommunications system, I bet that every single precinct is going to go to the telecommunication system does eventually hit some wireless links, I agree.  I would doubt that there is a single precinct that doesn’t have that situation if they are communicating externally they don’t have wireless somewhere along those lines.  So, the bottom line is if you are sending sensitive information out of the physical voting precinct by telecommunications, public telecommunications to some other location and you think its sensitive its got to be encrypted.
DAVID SIEFERT:	Right.  Encrypted is the requirement but the level of –
MALE SPEAKER 20:	By except for the issues of a spectrum such as IR being much more directional and therefore won’t radiate away and therefore be harder for someone to intercept, connection keyboards and everything else, the frequency is sort of irrelevant, isn’t it?

DAVID SIEFERT:	Not necessarily.  My example, the example for encryption is the I Triple E 802.15.4 which is a very, very low rate wireless network and its designed to send very infrequent and very small pieces of information and its to fit within 128 or 256 K of memory.  You cannot put encryption into that system.  However, without knowing what is being transmitted on these voting systems, if I would go back a slide, you would see that there was a place, an ability to just send a signal of open the poll or close the poll.  That would not require very much data information and technology would fit in that case.  If it had to be encrypted because you didn’t want somebody to be able to open the polls or close the polls out of when they should be, you’ve got to have encryption to protect that and you can’t.
MALE SPEAKER 20:	Well, I think this is very good.  If you can follow along these lines, you know, consulting with the experts here, I think you could probably begin to determine which ones need protection.  For example, uploading a ballot and a program, maybe that doesn’t need protection if I have some procedures and check points in the system afterwards to ensure that, you know, hashes, checks and something there that hasn’t been modified in transit.  There’s no vote, there’s no secrets going on, so, the question would just be, what is happens, what’s the threat to each one of these and what could you do to eliminate that threat and does it require encryption and then you could go through it and you could determine where, if they wanted to use wireless, the would have to have encryption and that.
DAVID SIEFERT:	Correct.
MALE SPEAKER 20:	That’s your intent?
DAVID SIEFERT:	That’s my intent.  The main missing piece is wireless gives you access to the systems.  So, the protection of the voting information that we think of absolutely needs to be protected but we also have to protect the machines themselves from having anybody interfere with them while the wireless link is open.  Yes.

DR. SMERGIAN:	I thought there might be somebody –
MALE SPEAKER 21:	Can I take a turn?
DR. SMERGIAN:	Yes, go ahead Dr. Revest.
DR. REVEST:	I just want to support (undecipherable) approach to that and David’s response.  I think that sets sort of wireless is definitely just a threat that is encompassed there and if you divide them into transmitting and receiving the – any transmitting from a voting system the concern is the sensitive information that might be disclosed typically and the reception of information is a possibility of modifying behavior of the machine in a way that’s not authorized.  (Undecipherable), you know individual signatures and so on if the information is otherwise public.  That may be appropriately handled.  I think the threat base now is the use of wireless (undecipherable) and I support the momentum in that direction.
DAVID SIEFERT:	Ms. Quisenberry.
MS. QUISENBERRY:	I just want to make sure I understand what’s in the actual draft standards.  So you actually have an appendix draft standard which covers private networks.  Could you just briefly go over where thinking on those requirements?
DAVID SIEFERT:	Okay.  Section B is a direct copy of Section 6, the security section with wireless placed separately because I do believe wireless needs to be addressed separately as opposed to being hidden in the current requirements.  As was stated previously, we didn’t know whether we were going to rewrite the draft or make suggested changes so annex B at the moment is a suggested addendum to be put in the security section for wireless.
MS. QUISENBERRY:	Could you just run down what you are saying in that section?  I can read it but I don’t necessarily understand the implications of the decisions you are making at this point.

DAVID SIEFERT:	Okay.   I guess I don’t know what – the main thing is I’m making a distinction between private and public which is one of the items I’m calling out.  I’m also calling out that we are definitely using wireless communication as opposed to telecommunications.  Those are the two main reasons for this.  There are no specific additions.  There was, I think, a subtraction because it said something about a physical, you needed physical security which doesn’t exist in wireless communications.  If I had the other text I could do a line by line comparison and tell you but those are the three items I can remember at the moment of what I’m trying to do.  The real requirements, specific wireless requirements, are in Section 5 of the main document.  Again, trying to do three different views of wireless in systems both from a VSS, an election process and then from a technology point of view needs to be coordinated but at the moment three separate – I need to address the specific issues but all three sections must be coordinated and list the same information.  Annex B may or may not be present depending on which direction we go with the document.  I was trying to cover all points of view and say, okay, here’s what it would look like if we did it this way.  Here’s what it would look like if we did it another way.  Yes, there is information in Section 3 that’s not in 5 or its in 4, its not in 5.  They all have to be coordinated.  They all contain the same information.
DR. SMERGIAN:	David, are you finished with your presentation or did you have –
DAVID SIEFERT:	I was finished with my presentation, yeah.  My last slide was the numbers.
DR. SMERGIAN:	Then, Mr. Kraft, then Mr. Berger and them Dr. Williams.

MR. KRAFT:		Okay.  David, I think I heard you say that number 1 we are using wireless.  You can’t go down the road of prohibiting the use of wireless and two, its almost impossible to conceptualize and bring into a measurable standard that will address every case and time is short so, I guess, what do you recommend to the committee that we do to pursue this issue.  I guess one of the things that we have done in the past in dealing with telecommunication and networking issues is actually address which data can be transmitted.  For example, there was a prohibition in the 1990 FEC standards against unstipulated ballot data being telecommunicated before it was, you know, tabulated and before there was some controls.  Where are you going with that?
DAVID SIEFERT:	Well, I guess, from what I understood you to say, you want me to tell you what pieces of information would be distributed using wireless technology.
MR. KRAFT:		Well, no, I was kind of giving that as a suggestion as one alternative to trying to get our arms around standards for encryption and protocols and all these different wireless technologies.  One of many options is actually evaluating what pieces of information or what level of information risk you could take within the systems.  I guess what I’m asking is are you looking at other avenues to try to get some sort of a measurable standard built around this?  Are we kind of at the point where we are adrift?
DAVID SIEFERT:	Well, I guess, again, this is why I left this picture here.  This is where my view is.  This is my understanding of the information that may be distributed and therefore that’s what I need to examine for threat.  Again, back to the questions of having everything encrypted, that is not a possible statement either because, again, back to the handicapped situation.  That wireless technology cannot be encrypted.  It is at the voice level and therefore if you encrypt it I can’t hear it.  So, having encryption required on every wireless link doesn’t work either.  I guess, yes, I’m at the point where I don’t know where to go either.  I can continue to do these analyses.  

The other issue that was spoken of earlier, the upload of the program.  There is a requirement that that program has to be on right one media which means it could never be – that program could never be transmitted over wireless.  So, one may go away simply because of an existing requirement of the information cannot be distributed in that manner.  That one may be restricted because of another requirement in the spec.  It really is going to be on a case by case basis and these are the links that I’m going to have to try to address.
The printer point of view – going to the printer I don’t have to worry about information necessarily coming back to me.  Its an output device although I don’t want anybody to intercept the ballot.  If they transmit from voting precinct to the main voting location that information is a tally vote and I need to encrypt it but if somebody knows the numbers who cares as long as they don’t modify it.  The modification of those numbers is really what’s important in an election.
DR. KRAFT:		Do you, or have you had good reliable information from the industry in this process on what data they are actually moving because some of the things in this model, while its very good, are very actually, you know, somewhat hypothetical but I was wondering in keeping with EACH’s interest and I think, mandating the best of current technology nationwide, perhaps we could look and get some cooperation from the industry in disclosing exactly what data they are transmitting.
DR. SMERGIAN:	That’s a good suggestion.  It could minimize your work if you know what they are physically doing.  One point in the handset area, I’m not saying you should go to that expense but there is an older technology that we used in the analog communications world which did voice scrambling, frequency scrambling and them B scrambling can be used to make it very difficult for someone to intercept and understand what’s being said.  You can do that.
DAVID SIEFERT:	But you are correct but in the application its designed for that cannot be used.  Its going to the hearing aid and unless I put the encryption in the hearing aid as well, it doesn’t work.

DR. SMERGIAN:	Right.  That’s what you would have to do.  You would have to have a specially designed – I’m not saying its worth it but it can be done.
DAVID SIEFERT:	Well, that’s also an example of where what’s the risk?  How feasible is it that someone is going to get between the loop and the hearing aid.
MALE SPEAKER 22:	I agree with you.  For efficiency purposes I think you ought to just scan this stuff and say this link’s not a risk for these reasons.  You don’t have to worry about it.  This link isn’t really being used or contemplated to be used for wireless.  Don’t worry about that one.  You can get them down to a manageable number to work with.
DAVID SIEFERT:	But I’m hearing immediate help from us are the industry.
MALE SPEAKER 23:	In terms of the task here, David I saw you (undecipherable) a big job here and to get started on.  I think that the committee is giving you good advice here on how to proceed.  We have a Resolution that says that wireless poses a severe risk and should be approached with extreme caution.  I think that the task is, (undecipherable) the thoughts of no wireless and any wireless and wireless where it is encrypted and they are trying to work on an exception. But I think that the approach that’s being suggested are reasonable points for possible exception if you can identify and try to see if you can (undecipherable).  Its not a question of trying to cover the whole base but just the pole position.

DAVID SIEFERT:	Inherently very risky and a manufacturer that wants to use wireless for this better be prepared to justify it very carefully and show many of the risks that go along with it and to the extent, excuse me, you can identify closed applications where wireless really was justified we can standardize on those ahead of time.  That would be the right thing to do.  A hand full of those where wireless was justified either because there is already mitigating technology in place or because there is such (undecipherable) it is really necessary or for some other reason where the data is public and protection is probably an issue. That’s when we can standardize on those.
DR. REVEST:	We understand that you might be able to make a case for exception to handle the risk but I think we are trying to stop it.  The original wording of the Resolution ways that wireless does introduce threats and must be approached with extreme caution.  The thought here is whether the (undecipherable) are exceptional.  We’ve got the general (undecipherable).
DR. SMERGIAN:	Thank you Dr. Revest, Mr. Berger.
MR. BERGER:	A couple of comments.  Let me start with a little history on the hearing aid compatibility issue because I think it points to something important in this issue.  
In the old Western Electric telephone days, receivers put out a magnetic flux leakage.  It was jut a leakage that happened naturally and someone discovered that to be useful in coupling into hearing aids and help people with hearing aids get better reception.  As technology changed in 1988 the Congress mandated that that be continued and that’s an FCC requirement to this day and that’s what you are talking about.  
I think the important point is that wireless is always present over every connection if there are electrons flowing, currents flowing there is a wireless emanation.  It is just a matter of degree at that point and from a security standpoint and I know there is a lot of people in this who know a lot more about this than I do, if all these wireless connections are wired, all you’ve done is change the degree to which someone can monitor what’s going on.  To that degree, I think, the comments that each link needs to be analyzed as to what’s the risk and are those risks appropriately mitigated is real important.  

I think there is some general technology trends that will help circle this because to a large degree the wireless technology is irrelevant.  Let me go to your cloud there.  Its not generally understood for those who don’t spend a lot of time in it but since the mid-90's there’s been a huge trend away from circuit switch traditional telephone system to packet switch, to an IP based data transmission, particularly data transmission.  Its almost impossible to send data, wireless or wired, now through the public network.  Its not IP based.  In some ways I think the question is not wireless but is it possible to set up a packet switched IP based data exchange that’s sufficiently secure for our purposes.  The transport at that point is almost irrelevant.  
Actually I had an interesting discussion with the chair of 802 and I think his advice was germane to this work.  He said, you ought to assume wire to wireless that someone can tap into that wire and if you are satisfied that you have appropriate security, you are okay.  
MALE SPEAKER 25:	I’m missing your point.  Number one, I don’t understand the different between whether I’m tapping into an IP stream or any other kind of stream.  I still can have equipment that can read it.  Your other comment about the emissions, the biggest threat there if you don’t address it is you have emissions coming out from the screen which I can pick up and replicate copies of every screen shot and I can demonstrate the equipment that can do that.  Its relatively inexpensive.  So, if you are really worried about capturing votes and so forth like that, and I agree with you, it doesn’t matter whether its wired or wireless, then you would have to talk about either some design of the equipment that its got emission controls or the cubicle in which you put that has some kind of a screen in it that would prevent the radiation outside of that screen.  That wouldn’t match some of the pictures I’ve seen.  In some of the precincts we have these PCs in open areas like that and it would almost require you to have some kind of a —

MR. BERGER:	No.  I think that’s a great illustration.  The only one I’d add to it is, if all those wireless links are wired, the quality of the shielding of the wire and all those could be very easy to pick up in a similar manner.
I think the point is and I think you made it well is, what’s the risk and what degree of mitigation is appropriate, you know, to the risk.
DR. SMERGIAN:	Dr. Williams.
DR. WILLIAMS:	Yeah, this is a little too theoretical to be useful in the short term.  We need to look at how voting systems are using wireless technologies and right now what happens is that the primary use of wireless technology in a voting system right now is to program voting stations, to load ballot definitions into voting stations.   If I’m Fulton and I’ve got three thousand voting stations and I’m having to load those with PC cards I’ve got to sit down --
(START OF TAPE 2).  I’m in the precinct and I want to, its time to open the polls and I’ve got anywhere from ten to thirty voting stations sitting there.   Its very convenient to be able to sit down at a console and wirelessly open all thirty of those voting stations.  The same thing with closing the poll at the end of the day.  Now, during the day that wireless capability is not a threat because you turn it off.  So, under what circumstances can we use that and then if the close of the polls, if I want to use that wireless facility to do my roll up in the precinct.  These are the issues; these are the questions that local voting officials are asking us to address for them.  They have systems out there that are capable of doing that.  They want to know what are the risks if I use that capability and if I do choose to use it what safeguards can I reasonably put in place.  Whether my mouse communicates wireless, I could care less or my keyboard or my printer.  These are not the issues.  These are voting systems and the issues are transmitting ballot images and opening and closing polls and transmitting election results at the end of the day.

MALE SPEAKER 26:	Let me give some thoughts on that for a second then.  I think then you do, let’s do a top level kind of threat analysis.  Let’s say one of the primary uses is in the administration and loading and so forth.  I think you might want to spend some time to address how we can do that wirelessly and how to have the checks and balances so that initial set up and close down.  That’s important to do.  I think I would tend to agree that those things that are just output types of devices and those wireless (undecipherable) aren’t really that much the issue because, frankly, as we had that discussion, your biggest threat there is privacy and knowing who voted what and there you would really have to be most concerned about my running around with some box in my pocket that could pick up the emissions from the CRT which is a very liable type of thing to do.  So the only other thing I think you would want to protect on those links where you’d think people are actually using it is on the input or transfer side because it is possible then.  Let’s say that I could manipulate that DRE, you know, going to the counting system, modify and insert, you know, what I saw and voted, what actually was done.  So, if you were to use wireless there, but on the other hand if it turns out, so you might show that threat and if no one is currently using it in that area then we might just say, it is recommended not to use wireless in that area.  If someone ever wants to do that it would have to go through a whole threat analysis.  Just focus on the areas where they find it administratively efficient and what are the procedures and the checks and safeguards to insure that.

DR. WILLIAMS:	These systems, you have to have the capability of allowing the poll manager, the poll workers to wirelessly activate voting stations for voters.  Nobody that I know of uses that capability because you don’t want to sit there eight hours or twelve hours with your polls open with that wireless capability hot so they all use the physical cards to activate voting stations.  So, the people that use this and we have some of them sitting in the audience here today, are concerned about this wireless capability and the way they are using it and would like guidance on the problems that they are dealing with, not the theoretical problems that might happen in the next generation of systems.
DR. SMERGIAN:	What I’m hearing is that I think the guidance you are getting is, you don’t need to address all the possible permutations, the possibilities but rather the first order lets look at the processes or applications that are either being used now or where election officials would like to use but they are concerned about the potential risks.  So, focus on those specific applications either current or potentially imminent, you know applications and not address the broader very large domain of possible applications.  I think that a very constructive guidance.  It certainly limits the domain that we have to worry about and make the job, hopefully, more manageable.  Do we, I’m not the specialist here, so I’m trying to sort of capture the general feeling here.
DR. WILLIAMS:	We’d be happy to put you in contact with some of the vendors that have these wireless systems.  That way you can go look at them and get comfortable with how they are capable of operating and how they should be operated.
SECRETARY DAVIDSON:	Or we could get on with getting the systems in house for NIST to work with.
FEMALE SPEAKER 12:	Sorry, we are on this nine month battle to get access to systems for the experts at NIST so, –
DR. SMERGIAN:	They would be available after nine months but – I think the concern here is clearly there is a huge, you know, domain out there that you could look at and you could spend the next ten years looking at them and that’s not what the committee has in mind.  

DR. WILLIAMS:	We’ve got a group from Virginia sitting right here that has one of those wireless systems.  I’m sure they would be happy to have you come down and go through how they use that system and look at it and make recommendations and suggestions to them.
DR. SMERGIAN:	Mr. Berger.
MR. BERGER:	Well, I’m in complete agreement with what’s being said here and I would just like to highlight one point that I thinks been said but very important.  That is simply what constitutes acceptable turning off of a wireless system.  Is a software switch enough?  Do you have to remove the whole module?  A simple question but I think its very important.
DR. WILLIAMS:	These are the issues I think we can deal with in the time frame that we are talking about.
MR. BERGER:	Also, at the mention of the unavailable of equipment to NIST I’ve made eye contact with some vendor representatives here who looked a little confused.  I think perhaps you could get with some of the vendor representatives who are in the audience and go ahead and start making some arrangements today.  I’m see some nods.
DR. SMERGIAN:	Any other questions, especially from the folks on the phone?
DR. WILLIAMS:	Let me comment on this a little further.  Some of us have got some leverage with the vendors.  If you will let us know which systems you would like to see we maybe can help you with that.
DR. SMERGIAN:	I’m sure we all appreciate the help.  Dr. Revest did you have any other comments?
DR. REVEST:	No.  I think that direction here is a good one to focus in on those near term things to the extent that we can have a vendor input as to which application area they have seen the most pressing to them.  That would be helpful.

DR. SMERGIAN:	Well, I consider these comments certainly very constructive but also clarifying in nature.  I don’t really consider these as a change in our direction.  I think I consider these as focusing on the more limited set of work products and outputs and I don’t really see the need for a Resolution or any kind of vote.  I think that there is, the committee seems to be in general agreement that this is an important topic but we don’t need to solve the world’s problems.  We need to address the immediate issues of concern that are for systems that are being used now and perhaps some of the things that may be coming down the pike soon.  I certainly consider that as a very constructive comment and unless we have other questions or comments I think we’ll sort of take those comments to heart and proceed from there.
Thank you David.
DAVID SIEFERT:	Thank you.
DR. SMERGIAN:	So, let me go through my official process here.  Certainly NIST believes the preceding preliminary report of the technical support titled “NIST Approach to Wireless Requirements for the Augmented 2002 VSS, Response to TGDC Resolution 35-05".  Unless there are supplemental directions or corrections the technical support and related work product will continue to be developed consistent with this preliminary report and I should add with the comments made right now.  Are there any further questions, directions or corrections?  I think this was a very productive discussion.  Are there any other questions or comments?  If not, we’ll proceed to the next presentation and at this time I call on Nelson Hastings of the NIST ITL to present a preliminary report on NIST Approach to Software Distribution and Set up Validation for the Augmented 2002 VSS.  Nelson.

NELSON HASTINGS:	Thank you.  My presentation is going to be broken up into two parts specifically addressing the Resolution on software distribution and set up validation.  Due to the time constraints we have today this presentation will give you just highlights of the goals and methodologies that we are thinking of using as well as the scope that we’ve come to on those Resolutions as well as some sample draft recommendations.  At the end of the presentation I will open up the floor, obviously, for discussion.
The goal for software distribution is to ensure that the software has been distributed without modifications.  The approach that we are going to take to that is to develop reference information to that can be used to check that the software hasn’t been modified.  That can take the form of binary images, just to a binary to binary comparison of that software as well as you could use hash values to do a comparison of the hash that generated of the system of the software being inspected and that type of technology is being used in the National Software Reference Library here at NIST in their work with law enforcement.  The last technology is digital signatures which is used in the commercial space when they do signed codes such as signed JAVA applets.
There are different places or different sources of this reference information.  Vendors and independent testing authorities supply reference information in regard to the parts of the voting system software that don’t change based on where its installed or the election that’s being conducted whereas jurisdictions may be -- would be able to provide the reference information on the software that changes on election day, that kind of thing.  I think Georgia does that in the program that they use.  The National Software Reference Library could generate whatever reference information for whatever software they receive from the vendors, the ITAs or the jurisdictions themselves.  

The scope is to look at all executable code and the associated configuration files irregardless of whether its at the polling place, the central counting system, or election management system.  Included in this is third-party software such as operating systems and drivers that are critical to the proper operation of the voting system.  However, determining whether the software is operating correctly is beyond the scope of this Resolution.  It actually will be addressed by another Resolution.  
Some sample recommendations that we’ve put together.  The first one is related to a requirement found in the current VSS where it says that ITAs shall witness the build of the executable that would be used to go under test.  This extends that and just says that once that system has been qualified the ITAs will witness the final build of that system.
The second requirement there is directly from the Resolution itself that basically says that right once media will be used to distribute the software, the binary software.
The third requirement is basically just talking about labeling of the write once media so that you can have traceability back to the authoritative source that created it.  There is a requirement in the VSS something about product marking, marking products for identification.
The next three recommendations are actually related to the I Triple E voting standard, specifically the section on software and firmware installation.  However, these requirements make, refine the requirements found in the I Triple E standard and give it a crisper requirement.
The first one basically just says that Phipps approved functions can be used to generate reference information.
The second one basically says a Phipps approved digital signature skink will be used to generate reference information for digital signatures.

The final requirement calls for a Phipps 140 level one validated cryptographic module be used to general hash values and digital signatures that’s used for reference information.  By imposing this requirement you bring in the testing that’s done to get a cryptographic module validated which insured that the cryptography is operating properly as well as some other security features such as key management.
The goal of set up validation is to insure that unauthorized software is not on the system.  Authorized software is on the system and that the system is in an initial proper state before it is being used.  The approach that we are going to take here is to develop items that need to be inspected to determine that the voting machine is in the proper state.  These items not only include software but registers, the accumulation registers that need to be checked that they are zeroed before being used.
The method will be to leverage off the requirements of the software distribution.  It will be more process oriented or guidelines or best practices for election officials and will probably manifest itself mostly in the vendor documentation that’s provide.  However, there may be some additional technical requirements.  Because the software distribution and set up validation are so heavily related to each other, those two documents that you received previously will probably be merged into one document for the final deliverable.
The set up validation is going to focus on the software itself.  Once again, regardless of whether it’s a polling place system, a central counting system or election management system, general hardware, voting system hardware set up will not be included in this scope so you won’t see things such as one needs to verify that the voting system is plugged into a proper back up power supply or that the voting system’s screen is set to the appropriate contrast level.  So, those types of things are beyond the scope of what this, what we are producing here.
Also beyond the scope is how the authorized software is installed on the system.  How unauthorized software is prevented from being installed on the system and how the voting system reaches its initial state.  All those are very important in the assurance of the voting system.  However, its beyond the scope of this specific Resolution.

Here is some sample draft requirements that we have been working on.  The first one is a documentation requirement that basically says that vendors need to identify all the software required for the system to run properly including the software that needs to be modified by jurisdictions.  The second requirement talks about obtaining reference information for that software and the final one says that jurisdictions will use that reference information to verify that the system software hasn’t been modified.  
These last three requirements are embedded are related to a section of the VSS on, I think its called validation at the polling station and validation at the central counting facility.  Basically there is a requirement in there that says that you need to check registers, accumulation registers are zeroed and things like this.  These requirements just extend that concept to all registers that are used and all variables that may be used in the voting system so that you can make sure that those values are correct.
The first one talks about vendors documenting the variables that they control.  The second requirement says that jurisdictions shall document the variables that they set for given elections and the final requirement says that the jurisdictions will use the documented values and check them on the system.  So, this kind of gives you a flavor of the types of requirements that we are looking at to put towards software distribution and validation set up.  At this point I am opening up for discussion and feedback.
DR. SMERGIAN:	Thank you, Nelson.  Any questions or comments?  That sounds like time to sit down.
NELSON HASTINGS:	Okay.  I will take that.  Thank you.
DR. SMERGIAN:	Seriously, are there any questions.  Dr. Revest, did you have any comments?
DR. REVEST:	(Too far from microphone to hear what he was saying).

DR. SMERGIAN:	Well, not hearing any other questions I believe that, we believe the preceding preliminary report of technical support titled “NIST Approach to Software Distribution and Set Up Validation for the Augmented 2002 VSS Response to TGDC Resolutions 15-05 and 16-05".  This is your last chance.  Unless there are supplemental directions or corrections, the technical support and related work product will continue to be developed consistent with this preliminary report.  Any other questions or comments?  Hearing none.  Thank you.  We will proceed and at this time I call on Lynn Rosenthal of our information technology laboratory to present a preliminary report on Interim Conformance Clause and Glossary for the Augmented 2002 VSS.  Lynn.
LYNN ROSENTHAL:	Thank you.  Moving right along I thought I would be up against lunch and have to do this very quickly.  Now you’ve given me way too much time.  So, we’ll finish early.  I’m just going to give a very highlight view of the conformance clause and then talk a little bit about our glossary and for your pleasure Mark Spoce (sic) is going to be my driver and we are actually going to show you a very early prototype of the glossary.  So, lets get started.
The conformance clause is Resolution number 24 where we were instructed to draft a conformance clause for the VSS.  The way we are approaching this actually to have two conformance clauses.  One for this April deliverable of the Augmented 2002 VSS.  That’s being called the interim conformance clause and then after April we will start working on the more permanent performance clause that would go in this redesigned standard.

Let me talk a little bit more about the interim conformance clause which you have a copy of in your document folder.  The idea there is that there currently is no conformance clause in the VSS and just to remind you what a conformance clause is, it’s a very high level statement or paragraphs or section that talks about who needs to conform, what needs to conform, what defines conformance, how is it defined, how is it specified. It defines any of the conditions, constraints, anything that would be applicable to the voting systems, the tech labs, the vendor with respect to conformance.  Basically, what do you have to do and how do you go about doing it.  The conformance clause talks about the minimum requirements.  These are functional requirements, performance requirements, document test evaluation and procedural requirements.  Now these procedural requirements, as we well know, is a different animal.  They may be requirements that have to do with the set up or the operation of the equipment that would be in the documentation from the vendor or it may be types of requirements that actually end up as a best practice that the states would be able to adopt as they saw fit.  It would relate to aspects of the equipment, the setting up, the maintenance, etc.  In fact, the requirements would be and in fact the conformance clause talks about requirements for those who design, manufacture and sell voting systems, those test entities who do the qualification testing or the certification at the Federal level of the VSS.
DR. SMERGIAN:	I think some people are trying to find your presentation.  They are actually at the end, the right side of your book.  Sorry.
MALE SPEAKER 27:	So, if I understand correctly what you are going to do is just.  We are going to be at the functional requirements level as opposed to the how, the detailed design level that we are going to test.

LYNN ROSENTHAL:	We are not going to get into – the interim conformance clause is going to take what the current VSS already says plus all the new additions and changes.  Its going to talk about, you know, how do you put all that together and which applies to me, you know, what do I have to concern myself with depending on which role I’m in.  Some of those requirements that are currently in the VSS are functional, some are design and some are requirements on the testing people.  So, it’s a mixed bag.
MALE SPEAKER 27:	That (undecipherable) that it had some design ones but I didn’t see design but you really threw design in those cases where the current spec actually goes into design.
LYNN ROSENTHAL:	Correct.  Thank you for the clarification.  So, the other people who may want to be using the document and that may be addressed by the conformance clause as well would be other testing entities.  For instance, at the state level.
Then, finally as Mark indicated this morning, voting officials, we are not quite sure what the correct term would be to incorporate all, whether its poll workers, voting officials, election judges, those who are in the warehouses that are technicians so we came up with just this general name to include all of those people.
I’m going to move on to the glossary unless somebody wants to talk a little bit more about conformance clause.
MALE SPEAKER 28:	Lynn, I have a question.  When you are talking about coming up with definitions for conformance and you are primarily referring to the voting system standards which my understanding will, in some cases, reference other standards that come from other organizations, whether its I Triple E or anyone else, is there a level of criteria that – so if you are referencing a specification from another organization that they have achieved a certain level of specificity in how they define conformance.  Is that part of this, you know, for any reference standard?

LYNN ROSENTHAL:	To the best of my knowledge the VSS doesn’t do that right now.  That is a good point in that within the performance clause we do need to address those normative documents, those normative standards and what role they would play when it comes to implementing the requirements that are being imposed on a system let’s say.  That will be something that we will need to consider.  That’s a good point.  At this, for the interim performance clause, I believe its not going to be a question unless I’m missing something from my reading of the current VSS.  In a sense the conformance clause right now is fluid because we can’t put anything in concrete until we know what all the pieces are going to be and how we are going to want to provide, if you will, guidance on which piece is applied to which role and how they apply whether they are mandatory requirements or whether they are optional requirements or whether it’s a good practice.  Those are the things that the performance clause will take care of and will be done for the April document.
Moving to the glossary this is Resolution – Yes, Ron?
RON:	I just want to make sure I understand with respect to security requirements and so on.  We talked about what kind of (undecipherable) where a vulnerability may be discovered after a system has been qualified and so on.  Does the conformance clause address issues like that of situations where a security vulnerability is discovered?
LYNN ROSENTHAL:	After the fact?
RON:	Yes, I mean is it, I’m trying to figure out how to use the terminology here.  I suppose for conforming the system is on that, you know, meets the requirements then its discovered not to meet after.  Security is funny that way because you could discover that, my goodness, its broadcasting everybody’s vote on radio if we didn’t know that or something.

LYNN ROSENTHAL:	Typically the conformance clause does not address what happens after a system has been tested or a new problem has arisen.  Its used to just identify what are the requirements that apply and at what level do they apply, whether they are mandatory or an optional requirement and if its optional and you chose to pick that option you must do it correctly.  What I think you are talking about has to do with the testing and that aspect of once something is tested and determined to be certified or qualified, then how do you go about decertifying or disqualifying that system now that a vulnerability has been found.  That is not part of what goes on in the conformance clause.  That is a very important aspect that will be addressed in the, I would believe, probably by the testing labs and the requirements for doing the testing.
DR. SMERGIAN:	Yes and part of it could probably be addressed in the maintenance and updating, how do you do that?  That’s really what happens where you have a system, vulnerabilities get discovered, patches eventually get addressed and from what I gather that’s a particular sensitive thing because you don’t necessarily want to do an update of a patch right as you get started to do an election but so there are some thoughts behind it.
MALE SPEAKER 29:	Yeah and I think in the larger scope that is part of an intuitive process.  There are risks that are discovered.  There are anomalies in systems that are not discovered in testing.  On the other side of the coin there are also new functionalities that election officials and state legislators demand of systems.  So, all of these systems evolve as they go through time. 
LYNN ROSENTHAL:	Barbara just pointed out there are two Resolutions, number 31 which is the maintenance of the VSS and 32 which is sharing information and disqualification of voting systems.  I think what we are talking about now would be addressed under those Resolutions.  I think, Paul, you bring up an excellent point is that is with conformance, with standards, with testing, it is a cycle in that which one feeds the other and as things are found out you have to feed it back and maybe change something or add something to the standard.  Maybe change your testing so that it does so in a sort of intuitive feed back loop here.

MALE SPEAKER 30:	So to summarize I do agree that that’s where it belongs in those two sections and you know, just some thought on how we handle that sort of thing which I think might be something you might consider how you do because we also have to be pragmatic about when we do updates and when we don’t do updates.  What we normally do is we --.  Normally we absolutely want to know when a vulnerability gets discovered.  We leave it up to that business or in this case, the election district, to determine when they are going to install the patch and what they are going to do in terms of mitigation and remediation considering the fact that they know they might be going into a system that isn’t necessarily entirely patched with all the vulnerabilities.  So, the vulnerability exists and there is no available patch or its not an appropriate time to do a patch then they have to acknowledge that there is a risk and what they are doing to compensate for that risk, operation procedurally and then we also have procedures on how you do patches and what the order of priority is and, you know, all those kinds of things.  So, I’m sure you have guidance like that so you might want to put some of that in those appropriate sections subject to review, of course.
DR. SMERGIAN:	Mr. Berger and then Secretary Davidson.
MR. BERGER:	I would just like to highlight.  We may want to consider as a policy issue whether conformity and certification should be time limited.  That’s being considered in some other arenas and it offers some unique advantages if you know that a system is certified for some period of time and then it has to be renewed and then that gives you an assured checkpoint where you can make sure security patches are in, that sort of thing.  It’s a larger policy issue but I think its worth considering at some point.

MALE SPEAKER 30:	The only problem with that it sometimes is not related strictly to time.  What happens is a vendor comes in.  They put in an application or some software.  They continue to upgrade in different versions and at some point they no longer support it.  Its not longer to be maintained.  That’s probably when you really want to consider retiring that system.  Exactly when that point is reached is quite variable depending upon the vendor and the system and so forth.
MR. BERGER:	In some of the radio work that’s going on, one of the advantages of an assured time limit is that you can take certain system risks if you know that in two years this piece of equipment has to get a re-authorization to continue to operate.  So if any vulnerabilities get discovered, any problems, you know, basically an upgrade can be required at that point in time.
LYNN ROSENTHAL:	Those are all things that are typically done in the testing procedures and they are often done in both those ways in the sense of a time in the sense of not necessarily that you have to get retested but it has to come up for review to determine whether or not to renew the certification and also when equipment is changed and I believe its already in place, any time the system is changed it has to be retested.
MARK:	I just want to make it clear because it sounds like there is some confusion.  What the conformance clause does is define, excuse me, instructions for what it takes for implementations to conform.  This is related to but separate than the way of proving conformance.  One way of proving conformance is through testing.  There are other ways, self declaration.  So what Lynn is talking about are instructions.  Much of the discussion, a really good discussion about testing and things like that.  They are really outside the scope of the conformance clause.
MR. BERGER:	I just wanted to make sure everyone understands it.
MALE SPEAKER 31:	We recognize that.  We were just digressing into those. 
LYNN ROSENTHAL:	Moving on to the other work products that we –
DR. SMERGIAN:	Secretary Davidson.

MALE SPEAKER 32:	May I ask a quick question?  I’m still confused a little bit about how things change.  So if we talk a little bit about what if the system changes say in the perception of security but when the standards are changing and this is very relevant to our work now.  If we introduce new requirements that all systems must have a Future X then existing systems that have been qualified already and don’t have Future X, does the conformance clause direct the issue of transition, grandfathering (sic) and things like that?
LYNN ROSENTHAL:	No.  How’s that for shorthand?  No but the testing procedures or the exceptions procedures that either the test labs or the states would have would probably address that of when it needs to be retested and how do you handle the phasing in and the grandfathering in of systems.  
The conformance clause is very specific and as Mark said it just has to do with what are those requirements that are in this standard and who do they apply to.
DR. SMERGIAN:	Secretary Davidson.
SECRETARY DAVIDSON:	You know, the only thing I want to say and it goes to back to understanding the whole, you know, testing part of the equipment and testing it before an election.  As Paul said there is many things that take place at the very end of a process that changes and you have to change your whole program.  For instance, a deceased candidate and you can’t count the votes.  So you have to change your programming and the testing and that type of thing at the very last minute or if a candidate withdraws or a judge says you can’t count something.  So, I think that’s the other thing.  I think we need to make sure that they do proper testing at those times that something like that happens at the last minute.
LYNN ROSENTHAL:	I think these are all excellent point.  I’ll move on to the glossary.

This is Resolution number 33 and this was to develop a glossary of terms for the VSS and, in fact, its broader in scope that just the terms that are in the VSS.  It includes additional terms that are needed to understand what voting’s all about, what security is about, human factors and testing.  Right now what we are doing is we are just collecting terms from various sources from the VSS, for the I Triple E standards, from HAVA, from ISO, from various NIST publications, even looking at what some of the states have defined in their standards and legislation and using that.  We are also, if you will allow me, inventing a few terms as we need them as we are doing our research and discovering that we have to define some of these terms and how does it most make sense for the way we are using that term.  
Once we get these terms our next step is to look at them, revise some if need be, harmonize the various terms when you get more or less a similar one from two different sources and they are close but they are not the same, which one do you use and go from there.
Some highlights with the glossary.  We’ve added some new terms related to security, to human factors, terms related to the voter verified paper audit trail.  As Sharon point out, definitions for privacy, accessibility, usability.  The term conformity assessment which comes from our NAV lab area.
Additionally, each term is more than just the term and a definition.  We’ve tried to indicate the source either that the term was extracted directly from or is based on.  We’ve added something called an association.  This is what is this term relevant to.  Is it a voting term?  Is it a testing term?  Is it an accessibility term?  We also have added the See also when there is related terms or other terms that are used in either correctly or incorrectly in place of this term.  

We are discovering lots of questions, lots of areas that we need to look further into.  For instance, what is a ballot?  We are finding lots of different ways of describing a ballot and needs some information from Florida.  Even there they have different definitions for what determines a ballot.  So we need to come up with an appropriate general definition followed by any of the specifics.  
Another area is clarification on terms like accreditation, qualification, certification.  These are not even used consistently between HAVA, between what’s currently in the VSS, between what the NAV lab people here at NIST typically use.  So, we need to come up with either better definitions, maybe deprecate (sic) some of the current definitions and replace them with new ones as we go into accredited laboratories that are no longer being called ITAs but are now called DSTLs, so here we have a transition.  We are looking at all of those and that will be in the glossary as well.
So, next step obviously review and revise the definitions.  We keep adding new terms.  I’m a little bit behind in the sense that I have a stack of terms that need to be added but they are not in there.  Just to give you a preview, we are developing a web version of this glossary and Mark, you can just go there.  The web address that you have on the slide doesn’t work yet.  Its an alias that I’m going to try and see if they can get it working by next week at the latest.  But I did this morning for those on the phone; I sent the real URL this morning to the TGDC list so you should be able to get it.  

This is a very early rough prototype.  Not everything works and its not fully populated but just to give you a view of the what it looks like you can get the full glossary.  You can jump to specific terms.  Lets say you want to see, you know, all the terms starting with B.  Rather than paging down, you can see there is an area that says “also see” and lists the various other related terms.  It has the source.  It has association of what its to.  Now, lets say you don’t want to look at the full glossary, you want a different view.  You can go over to the list of sources on the side and say, give me all the terms that were extracted or based on the I Triple E standard and you will get those, or you say, give me all the terms that have to do with human factors accessibility.  You can go and you can pull out all those terms.  So you can grab it from different ways.  As I said this is a very early prototype to just give you a view of what we would like to have eventually.  We will make sure that its in XML, so its very easy to make sure that this is accessible, has all the right ways of being presented and right now we are just trying to work out the infrastructure behind it so that we can grab the pieces we need to grab.
MALE SPEAKER 32:	Two things.
LYNN ROSENTHAL:	You’re not allowed to talk.
MALE SPEAKER 32:	Accessibility is not spelled right.
LYNN ROSENTHAL:	Thank you.  As I say, very rough, but thank you.  We will correct that.  And thank you to my wonderful driver over here.  Any questions on the glossary?
DR. SMERGIAN:	I have to go through my formal procedure here.  So, NIST believes –
MALE SPEAKER 33:	Question.
DR. SMERGIAN:	Yeah, I’ll get there in a minute.  NIST believes the preceding preliminary report of technical support titled “NIST Interim Conformance Clause and Glossary for the Augments 2002 VSS Response to TGDC Resolutions 24.05 and 33.05".  So, at this time I open the question.  One thing I wanted to point out, all of the morning presentations were related to the augmented 2002 VSS.  So the ones in the afternoon will be associated with the redesign 2002.  So, the fact that there is general agreement and I hope that will be the case for this one also, I think puts us into a very good position to complete these tasks by April.  I didn’t hearing any major change in direction.  I think that’s very good.  We are now open for questions.
LYNN ROSENTHAL:	Thank you.

MALE SPEAKER 34:	Lynn, on the glossary, this is excellent.  I think it will be very helpful.  Question kind of on the scope of the terms.  Most of these appear to be more procedural terms.  Will you be getting into actually listing in there data elements?  For instance, if there is a record transfer between systems, the actual data elements, definitions of those data elements.
LYNN ROSENTHAL:	No.  I don’t believe so.  If I understand the question correctly.  I’m open to put whatever term in the glossary that people need to see in the glossary.  So, the answer is “no” but, you know, its fuzzy.  If there is a terms that needs to be defined in order to relay an understanding related to voting or related to the voting standards, related to the testing of these according to the standards, etc., then we will put the term in or if its just a general term that needs to be understood in this community then I don’t have a problem putting it in.
MALE SPEAKER 34:	I guess, maybe an example is, if you are, if there is a definition or a standard for electronic ballot, an electronic record for a ballot and that record is made up of specific data elements, each of those data elements is defined.
LYNN ROSENTHAL:	Right now that wasn’t envisioned but if that is a term that and those data elements related to, you know, what the record is need to be defined and there is a context for it then it makes sense to put it in but right now its not.
MALE SPEAKER 35:	I think what you are suggesting though is going down the tricky path of setting standards by definitions and I think the approach they have taken as a glossary is to list terms that you need understand the standards.
(START TAPE 3)

DR. SMERGIAN:	If we could come to order, we’ll resume this meeting of the TGDC.  I understand two of our members had other meetings to go to this afternoon so they will try to check in by phone, Ms. Miller and Mr. Berger.  Do we still have Dr. Revest and Ms. Turner Bowie on the phone?  Thank you.  Ms. Turner Bowie are you?  Currently we have ten members, either here, present here or on the phone.  So, we are a quorum so we will proceed as planned.  
At this time I call on John Kelsey of NIST Information Technology Laboratory to present a preliminary report on NIST Approach to Direct and Indirect Verification for the Redesigned VSS.
JOHN KELSEY:	Okay.  Can you hear me?  Okay.  So, let’s see if I can make this work.  Good.  Okay.  I’m John Kelsey which I guess everybody heard.  Now, I’m going to try to address this in a little different way than some of the other talks.  
First I want to talk about this broad security framework we’ve been trying to work out for how we write standards that will lead to secure voting systems.  Systems that can be, you know, that can be tested and implemented and used in a secure way.  I’m going to take about the DVID Resolution and the (undecipherable) Resolution in light of that in a broader security approach.
DR. SMERGIAN:	John, just a minute.  I thought I heard a click on the phone.  Do we have Ms. Turner Bowie on the phone?  
MS. TURNER BOWIE:	Yes, you do.
DR. SMERGIAN:	Okay.  Thank you.  Please proceed John.

JOHN KELSEY:	Okay.  So kind of a high level view of this, of our security framework.  I’m going to start with a picture of a model voting.  The high level, okay, a summary of this is as we looked at the security requirements for voting the first conclusion we came to is you don’t actually need paper to get a secure, to get a good voting system.  You do need some way of checking results.  You need what I’ve been calling audit ability (sic) and I gather now that I have been using that term incorrectly for the voting world so I will have to find another term.  I’m probably going to say audit because otherwise I’m going to be misreading my slides for the entire talk but to substitute in, what I’m trying to get at with the notion of auditing is cross checking records, cross checking independent records.
So, this is a picture of kind of the general voting, kind of a very simple voting model.  This part in the dotted box, this is the interaction between the voter and voting system.  The problem that you can have here is whatever, the only chance we get to get the information about what the voter wants is in this box.  We are going to do this process of capturing and verifying the vote.   Verifying in this sense does not mean in the voter verified paper trail sense.  It just means in the sense of you get a list of what you chose and you get a chance to say yes or no.  It’s a usability issue.  Then we produce a vote record.  Probably there’s a better term for that as well but we’ll produce some final record of this interaction and at the end of the election, after we have done this many times for many voters, this vote record is put into the process of counting and kind of cross checking and auditing and we get a preliminary count and eventually we can also get a recount from that.   So it’s a very broad overview.

There are a couple of interesting properties of this, right?  Its actually really hard to catch errors or fraud in this interaction between the voting system and the voter because I think pretty much all the other steps in the voting process you can have observers watch things.  You can often rerun a process like if you think that the machine has incorrectly read the paper ballot, you can do a recount.  You can’t really do that with this interaction with the voter short of rerunning an election which is a disaster, right?  So, and the other thing is that the privacy requirements for the voter really impose some limits on how much auditing you can do.  You couldn’t put a video, a security camera in here and watch everybody vote at least if you could observe the screen because if you did, you would violate their privacy.  You would get in trouble.  Right now how does, as I understand, at least, requires a step where the voter gets a chance to verify his vote.  That doesn’t mean in the security sense.  That means in the sense that, for example, he gets to look at the piece of paper he’s marked or if this is a touch screen machine or a derry (sic) of some kind, the derry (sic) presents his vote back to him and gives him a chance to say that’s what I meant or that’s not what I meant.  It actually resolves a lot of usability problems and actually even some security problems get caught with that if you are doing something or mis-calibrating the screen to cause some votes to not get counted that would actually help a lot.  Its just having the verification steps.  
So in our framework what we want to do is use this step to provide independent evidence of the voter’s choices, the voter’s interaction with the voting system.  That makes that step auditable or checkable, right, in the broad sense.
MALE SPEAKER 36:	What do you mean by independent?
JOHN KELSEY:		I’ll get to that about two slides from now.  Okay?
MALE SPEAKER 36:	Okay.

JOHN KELSEY:		So, I’ve come up with this name, pretty much out of thin air of a dual verification voting system.  There are a gazillion of these out there.  This is not an invention of ours.  This is just a term that I’m using because I want to get at this idea that we produce two records of independent validity.  Typically one during the capturing, the process of capturing the vote and one during the process of verifying the vote.  One of the requirements here is both of those go into that first step where you are going to cross check and do a count.  Its important that you do that.  Some voter verified paper audit trail ideas; the idea seems to be that you keep the paper around just for recounts.  That really doesn’t work well in security terms because it means that an attacker who can tamper with the election in a way that doesn’t quite cross the threshold to cause a recount gets a way with it and maybe gets caught later and maybe he doesn’t.  There is a reasonable chance of success there.  You can kind of see that there’s a couple of interesting points and I’ll get a little more in this before when you were talking about an independent record.  This process and this process may be independent enough that an attacker who could plausibly compromise this process doesn’t, we don’t just assume that he must also be able to compromise this one and vice versa.  The whole system has to be designed in that sense.  For example, if you had paper audit trail plus electronic records on a memory card you couldn’t have the same people carry put those together, put the memory card in the ballot box with no cryptographic protection or anything and have them carry that to the central facility to count because if you did they wouldn’t be independent anymore.  The same one person would have had a chance to tamper with both records.  You might address that other ways like cryptographically but that’s kind of the broad idea.  
Just kind of covering this a bit the idea is that each record is going to give us some independent evidence of the voter’s choices and here’s where I am going to try to address that a little bit, your question.  By independent evidence and I also say independent validity what I’m trying to get at is two things.  Number one its meaningful to audit one against the other, to check one against the other.  So its not just a matter of saying – an example of some place this wouldn’t make any sense would be a paper ballot where you scan it in electronically and you have an electronic record derived from it.  In that case there is not really a cross check where the electronic record could ever tell you anything about the incorrectness of the paper record.  The errors can only go one way.  In this case the two records can be used to cross check each other.  So an attacker who wants to compromise an election has to compromise both records.

MALE SPEAKER 37:	Lets step back to your paper example because I think its very similar.  You have a paper marked ballot and you have a cumulated totals from scanning repeatedly those ballots and the DRE device you have an electronic image of ballot images and you have registers that accumulate totals for each candidate off those registers so, with the exception of electronic versus paper, I don’t really see any difference in the two processes.
JOHN KELSEY:		Well, if you had just hand marked paper ballots then the processes are rather similar in this view in the sense that in both cases you are looking at a system that looks more like this, where there’s one process by which you are capturing and verifying the votes.  The process is you are using the DRE, you are selecting the votes and its presenting you a verification screen and you say yes or no and there’s a single record.  There’s no way to cross check that record against itself.  The DRE could be producing multiple like independent logs or something.
MALE SPEAKER 37:	It (undecipherable) possible memories.
JOHN KELSEY:		Right.  But those independent memories are still being produced fundamentally by the same process on the same machine.  Maybe there’s some way of partitioning that out.
MALE SPEAKER 37:	But they are not always exact.
JOHN KELSEY:		I’m sorry.
MALE SPEAKER 37:	They are not always exact.
JOHN KELSEY:		They are not always exact copies of each other?
MALE SPEAKER 37:	Right because sometimes machines malfunction and then we actually go into the memories and discover what happened and find which one has the correct totals.  So the multiple memory actually exists for a reason and they actually add value and independence to the process.

JOHN KELSEY: 	I think in that case you are talking about a fairly narrow range of things that can cause the error that involves physical failure of the memory.  If there were an error in the DRE itself or if somebody compromised the DRE itself, then I don’t believe that these multiple memories would add a lot of value.  There’s not any reason to think if I had taken over the DRE that I couldn’t write the same incorrect data, you know, show you that you voted for John Smith and then write the vote for Mary Jones.  There’s no way to cross check that particular interaction with a DRE and similarly with a paper ballot.  With a paper ballot there is sort of a similar issue.  There you’ve got to verify it but then there’s a question –
MALE SPEAKER 37:	But there are ways to validate that software before the election and then make sure that you don’t gain access to it during the election.
JOHN KELSEY:		Right.  So, there are ways to try to prevent the attack.  I’m sorry, go ahead.
SECRETARY DAVIDSON:	But I think the point is that in one case you are checking the operational, you are checking the operational failures of the machine and in another you are actually producing duplicate, sorry, you are producing two different records of the same event as opposed to simply writing the same thing twice to make sure there is no electrical glitches.
JOHN KELSEY:		Let me address that just a little bit, the basic idea because I think its important to get.  One way that you can secure any system, basically there are a couple of different directions you can go in trying to secure any system.  One way is before the fact trying to prevent attacks.  Right.  You lock the system down.  I’m going to talk about that actually in the next talk that I do on system evaluation.  What we do ahead of time to make sure that the box can’t be broken into.  Oh, sorry.

MALE SPEAKER 38:	Here’s the concern and why I like this approach over the other approach.  Now, I can construct an attack that does the following.  You know, I built code that during the test performs properly but at the time of the election what it does is it shows you on the screen, the same code, the same machine.  It shows you on the screen one thing and it prints it the same as it shows on the screen but what it enters in the electronic ballot is something else.  Its an attack that I can do.  
Now, this approach says I have two different machines built by two different vendors.
MALE SPEAKER 37:	You said you can do.  Have you done it?
MALE SPEAKER 38:	Can do what?  Could I do such a thing?
MALE SPEAKER 37:	Can you modify a DRE voting system?
MALE SPEAKER 38:	I believe you can.
MALE SPEAKER 37:	Without being detected where it will take votes from one – have you done it?
MALE SPEAKER 38:	Some of the virus software does that.  They look good for the scanners.  They say there’s no virus and then they hide themselves and they come back later.
MALE SPEAKER 37:	Once again, have you done it?  Its very difficult to start manipulating frame ware –
MALE SPEAKER 38:	I haven’t attempted to do this for this application.
MALE SPEAKER 37:	That’s not –
MALE SPEAKER 38:	I have seen it done in the internet for other kinds of attacks.
MALE SPEAKER 37:	For other kinds of attacks but everybody who keeps saying this is possible.  No one comes and says we’ve done it.  There seems to be enough bright people and resources trying to prove that its possible that I would like to see someone who can say they’ve done it.

JOHN KELSEY:		May I suggest, I think – in my threat analysis I wrote this and I write this in some of the other stuff I’ve done here.  Trying to secure a system assumes that there are attackers who are a threat to your system, right.  One would try to address this a preventative, to lock down the system so well that it can’t be tampered with.  I don’t know of any experimental data where we, where people have taken a working voting system with full procedures and attacked it.  I think we’ve seen attacks on pieces or things that were leaked and I think that there you have this problem that you are seeing one piece of the system in isolation.  Its another thing I will talk about in the testing stuff.  When you look at one piece in isolation you actually, you think you see attacks that may or may not really be there when you see the whole system as its used.
I think, first of all I would say, that in the computer security world there’s not much, there’s a lot of history of people having systems they were convinced couldn’t be attacked, that had been done very well by very competent people who were successfully attacked.  The kind of common wisdom you see in, like the Belvin and Cheswick book on fire walls and internet security is something – I wish I could remember the exact quote but its something like, you know, if you haven’t noticed traces of an attack on your system and your system’s been on the network for a long time it probably means you are not checking your logs.  So, its, you know, I’m probably misquoting that but that’s approximately right.  You can’t assume that a system can’t be attacked just because you haven’t seen evidence yet.  Also we have a problem that we have to worry about attackers in the future, not just the ones today.  I think there’s some argument for that.  I think maybe we can move to the next couple of slides if that’s okay.

The highlight of the idea here is we don’t just want to try to prevent the attacks from happening.  That’s very important for a lot of reasons.  We would also like to detect them if they happen.  We can do that in a lot of ways.  This offers a chance to detect attacks in one area that I don’t believe existing procedures could reliably catch.  You can’t have observers from different parties’ watch the person cast their ballot and then make sure that the ballot is recorded correctly.  If you did that then the person would, the observers would know your vote.  I want to make it clear we are not inventing something new here.  There’s a bunch of things in the literature, a bunch of ideas that have been floated in the world to do this.  The MIT/Cal Tech report has a lot of nice information in it.  It talks about a thing called the Frog Protocol which amounts to doing the same kind of protections you are trying to do with a voter verified paper trail but without having to handle a bunch of paper, doing it all electronically.  There are a lot of good reasons to like that.  There are a bunch of other similar things.  DREs with a camera, DREs with an external screen, (undecipherable).  He pointed out that you could do that to get the same verifications and, of course, voter verified paper another way of getting the same assurances.
Having given you kind of a broad overview of how we want to approach the security and I think I can address these Resolutions.  I’m way behind to what I thought I’d be but I hope its okay.  I’m glad you guys were fast earlier.
Resolution 1205 introduces this DV/IV distinction.  It basically says a DV system is a system where the voter verifies some representation of his vote that recorded with his own senses.  In practice it seems like that’s always paper.  In principle it could be other media but it seems like its always actually paper.  Then with IV its just everything that’s not DV basically.  You verify your vote with the help of a computer.  So, in this kind of broad category, these broad categories something like the Frog Protocol would be IV.  You are verifying your vote with a computer.  DV would be like voter verified paper.

Resolution 2105 talks about multiple representation.  This is an area where I suspect we may run into problems between what’s required for security, what makes sense for security and what works for state election law.  I’m hoping to get some feedback on this.  In multiple representations we have voting systems, voter verified paper, the Frog Protocol or any of these that produce, these voting systems produce two different records or three different records of the vote and there’s a question of what you do if they disagree and is there a clean way to resolve those disagreements.  We’d really like a simple rule to resolve the disagreements and if you look at it in terms of security you can’t have a simple rule to resolve the disagreements.  This is potentially a problem and I’ll get into that in a bit.
So, I’ll talk a little bit about DV and IV.  In our framework these are kind of the definition issues for DV and IV, for a DV system, right.  DV means directly verifiable.  The idea is without any intermediary hardware or software or anything you are verifying some representation of your vote.  Its then actually stored and recorded.  So the idea is the voter verifies the record with his own senses and generally its paper and then you have both paper and electronic somehow used in this first count.  That’s a little confusing.  What I’m trying to get at there is, you cross check them before you use the electronic count.  You have to have an auditing procedure to make sure that the paper agrees with the electronic records.  You don’t just lock those paper records in a box until it comes time for a recount.  If you do that, again, there is an easy attack where you just tamper with the electronic, and assuming you can tamper with the electronic records is an easy attack where you just bypass the paper and there’s no point in having a printer there in the first place.

Now, some interesting problems that arise here. First of all there’s an issue with verifying that the human readable thing equals the scanned in electronic thing.  I think Bert Williams talked about that a minute ago or he talked about that earlier today.  It came up in the voter verified paper audit trail talk.  The thing is, its not just an issue with bar codes and we were talking about bar codes, putting a bar code or some mechanical thing on the ballot to make it easier for the computer to read or more robust.  The natural thing is to think that bar codes are the only place where this issue comes up.  Actually even with optical scan ballots there are known problems where if the thing, if its printed in an offset way you can misaligned it, you can cause differential rates of error because votes for John Smith get counted as errors a little more often than votes for Mary Jones.  There has to be some process by which we verify that the thing that the voter thinks he verified and saw is the same thing as the electronic record that was actually counted.  That’s an independent kind of step that is needed in these systems and is not generally needed in the IV systems.  
There are some interesting issues of error and reliability rates of printers and I think John really covered that pretty well.  One of the things that comes up in the security context is if you have something where the printer goes out and a bunch of your records are blank, you know, you have a bunch of ballot records that are blank on the paper and they are there in the electronic records, that could mask an attack.  That happens very often, it will mask an attack if somebody does one.  Like if you were pulling out real ballots and putting in blank pieces of paper.  Its hard to distinguish between those two cases which is a good --.  Go ahead, oh, sorry.
SECRETARY DAVIDSON:	I’m just trying to understand what you were saying.  Could you repeat that last portion again?
JOHN KELSEY:		Okay.  So, suppose you have a voter verified paper audit trail.  We have something where the printer malfunctions and starts producing just blank pieces of paper and we get through – if we assume that often voters don’t actually look at these papers, at this paper –
SECRETARY DAVIDSON:	Oh, now I understand.

JOHN KELSEY:		we might get ten or fifteen of these ballots records or whatever they are into the ballot box before anybody notices the problem.  We’ve now got fifteen ballots that will never match up with the paper records and from an auditing perspective.  Its very hard to determine whether that’s really what happened or what happened was somebody pulled out fifteen real ballots and put in fifteen blank pieces of paper.  That’s the thing we worry about and that really pushes us towards needing very high reliability rates on these printers.  
There are also privacy problems with unshuffled paper, of course if you have the rolls of papers there are pretty dire privacy problems.  I know there was some discussion, swapping the rolls around to keep people confused about which roll goes to which machine.  I don’t think that will work very well with any kind of auditing procedures because you usually would like to be able to recount.  
A lot of times you have a requirement to recount a certain number of machines.  You say choose one machine from each precinct and recount it.  You have to know which ballots went with which machine or you are going to get really strange answers out of that.  You can’t actually do a verified; you can’t actually check your work that way.  
I think that there are a lot of issues there that need to be resolved and then, of course, the whole accessibility and usability problems with paper.   If you are blind or if you just need a magnifying glass to see small print, the voter verified paper can raise some problems with that.  Those can be addressed but they are problems that need to be addressed.

So, then IV systems in our framework, enormously wide range of systems.  It ranges from the Frog Protocol from the MIT/Cal Tech report to just a camera with a DRE or a DRE with a separate screen for verification, all the way to the cryptographic schemes and I’ll show my bias here.  I’m a cryptographer.  Its what I do.  I think these schemes are just beautiful.  I would like to see these widespread.  I don’t think we are ready.  I think there are reasons why we aren’t ready to standardize on them.  They are really cool schemes.  They do fit in this and they actually fit kind of poorly in this category.  They provide a different set of assurance than any of the other schemes.  Long term I hope we can move toward those but I think that’s a direction that’s, you know, provide some really unique security but, some of the concerns that come up here and this is something else that comes to your question of independence, is when you’ve got independent processes there’s a question of how much independence you need.  
In particular I’ve thought a lot about attacks on these systems.  I’ve spent a huge amount of time.  I tend to attack things as a way of designing.  One of the problems you can run into is, for example, if you have, the Frog Protocol you have a voting station and a verification station and you move a token between them, like a memory card between them to do you vote.  The interesting issue that can come up.  Suppose that both of them have been resourced from independent vendors but they are both running the same version and patch level of Win XP.  Okay.  If I have a new exploit on Win XP, a new attack on it, these machines are vulnerable to.  If they are using wireless networking, it would be a network attack perhaps.  If they were using some other, you know, mechanisms it might be some other thing.  If I can find a way to attack these systems, they are kind of in a mono-culture, if they come from the same operating system or the same or similar designs or something there may be a way to leverage that attack on both those machines at once and defeat the independence.  I’m sorry.
SECRETARY DAVIDSON:	But don’t you also have the problem if you get somebody that’s –
DR. SMERGIAN:	Would you turn your mike on, please.

SECRETARY DAVIDSON:	Excuse me, I’m sorry.  If you have, the individual that is doing the programming, and we do not let any of the vendors do our programming in our state.  If you’ve got an individual doing the programming and they are trying to, you know, obviously throw the election let’s say, they are going to have the ability to program both systems.
JOHN KELSEY:		Okay.  When you say programming, do you mean actually writing code to run on the voting systems or do you mean doing ballots or --?
SECRETARY DAVIDSON:	To do the ballots, I mean, you know, each time before an election is when you are really concerned about this correct?
JOHN KELSEY:		Right.
SECRETARY DAVIDSON:	So, I mean, they may be also able to, if anybody else can hack into it why can’t they figure out how to do it?  So they are actually changing the code to where they could do it on both systems.
JOHN KELSEY:		Right.  I think in the design of this kind of system as we fill out the requirements of the design that’s something we have to address.  First of all, making sure, and in my document I don’t think I got this in the slides but in the full document that I wrote on DV/IV I talk about the fact that one of the requirements of this kind of system is that there is no one election official or small group of election officials who have the authority to do and no vendor, no one person or small group of people have the authority to alter both machines, you know, to alter the function of both machines.  Now, that means you load in a ballot design, we should be able to design a system that doesn’t, or somebody should be able to design a system that doesn’t allow a loaded in ballot design to compromise the security of the machine.  If you are actually talking about getting in and rewriting the programming on the Diebold VRE or something, you know, I don’t --.  Maybe I’m not understanding, is that something your state would do is have somebody rewrite some of the codes for the DRE?

SECRETARY DAVIDSON:	Not necessarily but I mean, we are talking about people hacking in and that could be, I mean, I think one of the main concerns about somebody that’s a hacker they’ve got more capability to the equipment and everything else as an employee than somebody else.  So, that’s where my concern came in is that, you know, we can try to protect the whole world but we also have to remember we have very small counties that only has maybe one person that even can write the program for the election.  So, say they can’t write the program on each system if you require two different systems.  It would be virtually – I don’t know how we would accomplish things like that in a lot of state and a lot of counties.
JOHN KELSEY:		Okay.  I think maybe we should talk about that more off line.  I think clearly we need to make sure that whatever standards are written here require that the machines be usable by the election officials in ways that don’t compromise the security.  That’s true, just as its true with Nelson’s things with software distribution and set up; I think its true here too.  I think that’s a requirement.  It will be interesting to get to that.
DR. SMERGIAN:	I might say that if you have a separate machine and if its functions are really very minimal, you know, that is to say it can receive a representation of a ballot and display it in a variety of ways and that’s all it does, you could design that where its really hard to attack and you could also say that’s not the area where I will tolerate any changes that’s in a read only, non-modifiable type of memory.  You could go to (undecipherable) with that device.  It would not be very expensive.  It would be so stripped down that you could verifiably prove it and you wouldn’t allow anybody to modify it.  It would be the hard memory, the hard ground, it wouldn’t be changeable.  You could play a game like that if you want to.  It’s really just a medium to render it in different views for different disabilities and for someone to see and perhaps have verified.  The less you do the easier it is to prove.

JOHN KELSEY:		I think you are ballooning the number of lines of code once you start talking about alternate –
DR. SMERGIAN:	Yeah, yeah, I know.  I should have kept quiet.
JOHN KELSEY:	I’m told I have very little time left to cover the rest of this so.  Let me get through the rest of these real quickly.
So, these are just a couple of the points here.  Kind of the round up, these are the things I really wanted to make as a point here.  They are somewhat contentious.  I have given some thought to the idea that I was going to have at least two large groups of people in the voting community mad at me at the end of this talk.
The first thing to understand is the distinction between paper, you know DV and IV.  Its actually a little less clear than you think of at first because even paper systems aren’t really pure paper you almost always scan them electronically and you have to deal with that electronic record.  Some of the cryptographic schemes, they look nothing like the paper schemes but they actually provide, in some sense, stronger assurances at the end than the paper ones do.  Neither DV or IV systems are inherently more secure.  That’s an important point to get because there is a lot of idea that, you know, paper is this kind of magic thing you are going to sprinkle on an election system and make it secure.  
DV systems are probably harder to do a wide spread attack where you have a small conspiracy of two or three people that actually tamper with machines all over a state or something.  There are operations reason that are really hard in practice if you think about.  The two things I want to get at – do I have time to cover them (undecipherable) really quickly.  Okay.  I’ll take it from some of my testing time.

So, the kind of the big thing with multiple record and this is something we really need feedback.  Okay.  In the 1205, Resolution 1205, there is this idea, the fundamental representations, the thing that gets counted initially.  There is also this idea of like a ballot of record.  The thing that’s the final, the record that has the final say.  The problem with this sort of system its true with voter verified paper or with any of the dual verification architectures, is both records have independent validity.  If you lost one and you knew it was an accident, you could reasonably reconstruct the results from the other.  You always have to count them against each other in normal process.  There’s not one of these records that can always be seen to dominate in a disagreement.  I gather that that may not work well with election law and I don’t know exactly how to address that because I think that’s state by state.  From a security point of view, assuming the paper records are always right just means that people who can stuff ballot boxes always win and assuming the electronic record is always right just means that people who can tamper with DREs always win. Neither of those are the thing you wanted.  One goal that we can do in the standards is, I’ll make this phrasing, disagreements between these records should be very rare and they should be meaningful.  So, if we see a disagreement between these records we need to be able to figure out what the cause is and it needs to be one of a fairly small, short list of possible causes, like the printer failed or somebody dropped the DRE in a river or something.  I don’t what you would have to do to destroy all the memory copies you keep on a DRE.  You may shoot it out a canon or something.  

The other part of this is, we want to get down to zero misreads which means we use error correcting codes, we make sure error detecting codes, to make sure that the probability that you will ever read a vote for John Smith as a vote for Mary Jones is negligible.   That’s existing technology.  Its off the shelf.  Its well understood.  You can make that less probable than getting struck by lightning twice in your life or something.  Once we have the kind of zero misreads we can remove a lot of the ambiguity about damage or error versus fraud.  That’s the real goal here is to get down to a very, if you see a disagreement, to be able to pretty quickly resolve whether its evidence of fraud or a major error or whether its just a routine thing like the printer failure.
I think that’s basically it.   All the rest of the stuff is covered in my document.  Kind of the big thing is you need to keep the records under separate control for this to work.  I talked about this in the document a lot more.  I think that’s everything.  Our broad approach to security is going to be to look at dual verification systems as the things that we want to see in the standard in the future.  I would like to move toward having those be the things that we allow for electronic voting systems, not having a system where’s there’s a single point of compromise. Exactly how to specify that independence is kind of an interesting question.  We need to talk about that.   Then we’ve addressed DV/IV and multiple representations within that framework.  That’s everything.
MALE SPEAKER 38:	Could you put your slide back up that show a diagram of the system.
JOHN KELSEY:		Yeah, this one?  Okay.
MALE SPEAKER 38:	You’re saying that those two systems are independent.  That those two are independent and should be produced by separate vendors.
JOHN KELSEY:		Yes, that would be the ideal, yeah.
MALE SPEAKER 38:	Okay.  A person’s going to vote on this one on the left.
JOHN KELSEY:		Right.
MALE SPEAKER 38:	And then he’s going to go over and verify it on the right.
JOHN KELSEY:		Right.
MALE SPEAKER 38:	Is going to vote again?
JOHN KELSEY:		No.  It’s a –

MALE SPEAKER 38:	All right.  So how’s the records going to get from here to there?
JOHN KELSEY:		Each of the different architectures does this differently.  I’ll point out; it doesn’t actually have to be that he interacts directly with this process.  One of the designs we’ve talked about is a camera.  People have talked about it.  I don’t know whether anybody’s implemented it.  Is a camera over the DRE screen?  The idea is there is some point at which the voter has the verification screen or several verification screens and the camera takes an independent record of that, of what’s being shown to the voter and then that can be reproduced if there is some questions of whether or not the results are correct.
MALE SPEAKER 38:	A camera over a voting system doesn’t sound to bad but we’ve got 25,000 voting systems.  So we would have 25,000 cameras.
JOHN KELSEY:		That’s right.
MALE SPEAKER 38:	which would operate with about the same reliability as 25,000 printers.
JOHN KELSEY:		Right.  I didn’t say that the camera was the optimal way to do this.  I suspect that something like the Frog Protocol is better but –
MALE SPEAKER 38:	This is my concern.  I’m not sure this is well thought out.  I’m not sure its as independent as you think it is because one set is going to be generated from the other set.  The voter’s not going to vote twice and if they do they are going to get different results because, not many voters will vote a 38 or 40 race ballot and then walk over to another voting station and vote it the same identical way.
JOHN KELSEY:		That’s not even on the table.  Nobody’s talking about having him vote twice.

MALE SPEAKER 38:	So, they are not independent in that sense.  The other question that I have is a lot of people we know from what little experience we’ve had with these systems, that a lot of people do not verify their ballot.  They just don’t.  They are in a hurry.  They want to go in.  They want to vote.  They want to get out.  They are on the way to dinner.  They are on the way home.  They have to pick up the kids.  Most people that vote are in a hurry.  So, you are going to have a lot of unverified material in that verified bucket.
JOHN KELSEY:		Right.
MALE SPEAKER 38:	I appreciate your concept but to require this at the stage we are at right now, I think would be a bad, bad mistake.  This needs a lot of careful thought.  Something that I haven’t seen much talk about in any of these is the voter themselves.  What I the voter’s reaction going to be to this?
JOHN KELSEY:		Okay.
MALE SPEAKER 38:	I think I said this morning, if we are not careful we can have an unintended consequence here of designing a system that the voters just despise and result in a drop off in participation that’s unacceptable.  So, these are the variables we are juggling here.  I like your concept but I think it’s a long, long way from any reasonable implementation.

MALE SPEAKER 39:	Well, I’ve spent even less time thinking about it than he has but if you can envision some kind of a picture where, let’s say, in this voting booth or space is three pizza platters or boxes, all driving the same display, let us say for the sake of argument.  So, from a user’s point of view, he votes on the DRE machine on that screen and the next thing the control shifts through a verification processor which shows him that that vote which he confirms, and then there’s a comparison processor that compares them.  So, he never really has to leave or get in another line.  Now, what happens if he chooses to not really verify, just thick or thin.  Well, if the verification processor’s representation didn’t match the DRE representation then that vote is not – he tried to verify.  We know that they didn’t match.  Like his blank example so you have at least got a situation where you say, when that voter leaves there’s a red light or something.  This guy can’t leave yet because he obviously didn’t verify it because they don’t match.
JOHN KELSEY:		Well, that sounds like a bad (undecipherable).
MALE SPEAKER 39:	So, now if you did this right you could conceive of how we could overcome a lot of those kinds of objections.
MALE SPEAKER 38:	I have trouble seeing the difference between that a lot of what happens now where you are given a verification screen at the end to look at and say yes this is the way I intended to vote.  All of your DREs have that now, every single one.  I don’t see a whole lot of difference in that and what’s up here because the original voting device has got to create that verification image.
JOHN KELSEY:		So, if it creates the verification image and that image then goes to a separate processor, a separate machine, say, then displays that image up and give the voter a chance to verify it on the other machine.  Under normal circumstances what should happen is, if the voter doesn’t want to verify if he just pushes yes, yes, yes, yes and walks away.  You actually don’t need a lot of voters to verify this to catch large scale fraud as long as we can produce the record.  The other side of this is that record that’s been created on one machine is displayed on a different machine. The idea is if this machine, if the first machine that I voted on is sending a different record to the second machine then I’ll see it if I check and that will get recorded.  If its getting the same thing that it showed me but recording a different thing then I have a chance of catching it and if enough people complain, that should raise a red flag.

MALE SPEAKER 38:	And then the last thing and I’ll shut up, is this idea of massive fraud.  In reliability and security work you don’t spend enormous amounts of resource to protect against events that occur with a minuscule probability.  So far, in the forty years we have been using electronic devices to count votes we have not had a single massive electronic fraud.  Lots of massive blunders but not any massive electronic frauds.  It looks to me like we are spending enormous resources to protect ourselves against something that has a rather minuscule probability of occurring.
JOHN KELSEY:		I understand the point there.  I think, maybe this is just my bias, but I think that its hard to estimate the probability that there will be fraud in the future and I think that I worry – well, we are doing the standards so we have, we’re doing standards so we are saying something about the future.  We are saying we think we know how to secure votes so that next year somebody won’t come out, or two years from now there will be a governor’s race where there’s evidence that somebody’s fixed the election and we don’t know what to do.  That’s where we are trying to go so I think that’s a –
MALE SPEAKER 38:	Well, don’t get me wrong.  I’m not saying that we shouldn’t pursue this but what I am saying is watch the expenditure of resources and see if we can’t pursue this with, well two things, expenditure to resources and the demand on the voter because either one of those can destroy this whole concept.
JOHN KELSEY:		I think Sharon would beat me up if I foul usability requirements.  Ron wanted to talk?
DR. SMERGIAN:	We have a question and comment on the phone.  Dr. Revest.
DR. REVEST:	Thank you.  
DR. SMERGIAN:	Can you speak up a little more?

DR. REVEST:	(undecipherable) massive fraud or fraud attempted by software modification.  We are fortunate but he seems to be correct that there haven’t been any major tampering that we’ve seen today on a large scale.  However, one of the things this committee is hearing on a consistent basis from the efforts in the field is that the complexity of the system is making it much easier to mount such an attack and do so undetectably.  (Undecipherable) could sneak a Trojan horse through the back door or something under the rug as well and in such a way that it would be much harder to detect.  We are raising the ease of doing this by computerizing our voting system.  (Undecipherable) making it much, much easier to (undecipherable) (This gentleman was speaking over the telephone and I could not really make any sense of what he was trying to say).
DR. SMERGIAN:	Secretary Davidson.
SECRETARY DAVIDSON:	I only want to make a statement.  I think that we are dealing with voluntary standards and if we make then where they are so cost prohibiting our states will not accept them.  I definitely believe that the states should be accepting them so I guess I’m putting up a red flag, be careful of what we ask for or we won’t get anybody that will do them.  I think there’s so much good in our standards that we have to be very careful of how we proceed.
JOHN KELSEY:		I think cost is clearly a big issue.
DR. SMERGIAN:	Mr. Kraft.

MR. KRAFT:		Yes, I applaud the (undecipherable).  I think, yes, that’s something that we need to be working on.  With the current state of the art there is a verification process in the works.  As I said at the last meeting and I mentioned it to some individuals who I had seen in the audience, there are a number of people working on other ways to get more independence into the process and do more verification.  At the same time we have to remember that we are in the business of prescribing standards to be used in the real world and that means taking what we know about the current state of the art, the current best practice, and putting those into standards.  I guess my caution here is while you are doing new research and new ideas, you shouldn’t be bringing those ideas into standards until they research is complete and the science is mature.
JOHN KELSEY:		Okay.  That’s helpful.  Again, particularly that would apply, especially the cryptographic schemes which are very new and clever but –
MR. KRAFT:	The other thing that I really have problems with and I continue to have problems with this is within your assumptions that you have stated, there are a lot of things that, frankly, I know you have no evidence for.  Its possible and easy to manipulate firmware on an installed system and escape detection.  That’s a theory that some people have advanced.  It’s a theory that if you are going to pursue, somebody needs to go out and actually do some experiments and see what’s really involved in that.  
You made a statement that optical scan based on ballots being misaligned can create more read errors on one candidate than the other.  My experience is no.  Basically when those machines get misalignment problems they reject the ballot but they don’t do read errors and that’s based on thousands of hours of testing those devices.  You need to do real research in those areas before relying on those assumptions.
JOHN KELSEY:		Okay.  I wasn’t assuming that the attack as I understood it was, you caused some of those ballots to be misread as incorrect, as under votes, not as – the misalignment problem is that they don’t get read to the proper vote, not that they –
MR. KRAFT:	No.  The misalignment problem and, as I say, we test for this, the misalignment will cause the ballot to be rejected as being the wrong ballot for the election and when you stick it back in it reads it.
JOHN KELSEY:		Okay.

DR. SMERGIAN:	Any other questions or comments?  We believe that the preceding preliminary report of the technical support titled “NIST Approach to Direct and Indirect Verification for the Redesigned VSS Response to the TGDC Resolution 1205 and 2105".  On this there are supplemental directions or corrections.  The technical support and related work product will continue to be developed consistent with this preliminary report taking into account the comments made here.  Are there any other questions or further directions or corrections?  Do we need substantial change in directions?  
I think my interpretation was so the level of emphasis and some additional parameters such as cost, etc. that we need to keep in mind as we develop these standards.
MR. KRAFT:	Well, I think that’s practice and this dictates and analysis of cost, a real analysis of risk rather than relying on antidotal theories of risk and then moving to look at real world solutions that can be implemented and tested that address those risks that are real.   I think that’s good science and I think that’s what you all are supposed to do here.
JOHN KELSEY:	Does that need to change.
DR. SMERGIAN:	No.  I think that is good advice.   You are back on for a presentation on the second preliminary report on NIST Approach to Security Testing for the Redesigned VSS.
JOHN KELSEY:	Okay.  So, I’m me again and now I’m going to talk about testing and evaluation.

We are trying to address the specific Resolution which is Resolution 1705.  I’m going to kind of show you and overview and I’m going to talk about the Resolution and then talk about kind of this idea of open-ended evaluation of a system.  That’s really the innovated part of that or the new part of that Resolution as opposed to what everybody’s already done which is saying we’ve got to test the system.  We have to include open-ended kind of search for vulnerabilities and we do this a lot in cryptography and computer security.  That’s kind of the thing that people think about a lot.  I think we can, maybe, bring some nice tools to the table here.  We are going to talk about evaluating a design, evaluating the pieces and then some issues that I think are going to come up. 
One of the big issues related to the previous conversation is cost.  I think a good evaluation, where there’s a lot of search for open-ended vulnerabilities is going to cost a lot more than simply going down a checklist.  Fortunately that does actually get amortized over the entire set of voting systems so, hopefully, that’s not prohibitive to the states.
Resolution 1705 and I didn’t try to quote the whole thing because if I tried to include it all in a slide, you couldn’t read any of it.  The italicized part is the interesting part here.  We are supposed to come up with standards to deal with testing of voting systems that include a lot of open-ended research for vulnerabilities.  So, let’s kind of think about this.  Open-ended evaluation, open-ended research for vulnerabilities.
MR. KRAFT:		I think it said significant amount.
JOHN KELSEY:	A significant amount, yeah, did I –
MR. KRAFT:		Not a lot of.

JOHN KELSEY:	I quoted it right, I just said it wrong.  A significant amount.  So, the idea here is a natural way to try to address security.  One that you actually use in any sensible way of addressing security, testing something, is you go down a checklist.  You say, okay, did you lock all the doors, did you lock all the windows, did you turn the alarm on, that sort of thing.  You can apply it to computer systems.  There actually are some nice, I think its 870, that’s the checklist for securing computer systems.  We have some really nice work that’s been done there.  The point of this is a checklist approach gets you so far but you still need somebody to just look at the design at a high level and try to find weaknesses, try to find possible flaws.  One of the kind of core ideas here is that this evaluation needs to be adversarial.  I’ve done some of this and it’s a strange thing if you are working as a consultant because your customer is basically paying you to beat up on them, to tell them what they did wrong.  Maybe people in other fields have had a different experience.  The idea is your goal as an evaluator is to find weaknesses before the system gets out the door.  Its more like you are looking for a reason to fail the system than a reason to pass the system.  Of course, the full process there is after you find the flaws then they go back and fix them and you check and make sure that they’ve got them fixed.  This mind set has to be a little bit adversarial.  You are not looking for an excuse to let them through.  Your goal is really to find weaknesses and find design flaws at a high level sometimes there will be a design, an idea for doing something that just doesn’t, you know, there’s some fundamental reasons why the thing doesn’t work or where you can’t secure it properly.
The other thing is you can’t just trust vendor assertions about verification.  I think that’s already in the VSS, the testing that’s done, right?  You don’t just say, did you do a good job with security and they say yes.  You say, okay, now let’s look at that and check the code and make sure everything works out.  I think Brit used the term before, making them open the kimono or something.  I think that was a good, that’s the approach we want.  They show us everything and we try to attack the system.
FEMALE SPEAKER 15:	In other words, always rely on saying we are doing an acceptance testing, like we are accepting the equipment from the very beginning because we want to prove that the vendors, if he has got any flaws, we want to turn them back.  So, every step of the way you should be approaching it as if you are doing an acceptance testing of the equipment.

JOHN KELSEY:	Okay.  Yeah, that makes sense.  So, kind of the adversarial model of the world that bring to this process is you are assuming the existence of serious attackers.  I talked about this in my risk document a little bit and the idea here, and I know there is some question about whether or not there are going to be attackers that really try to mount a serious attack on the system and what’s plausible but I think it’s a very big mistake in any system design to start with the assumption that your attackers aren’t as smart as you are or don’t have full access to the system internals.  Now, of course, that doesn’t mean you publish the system’s internals, it just means that when you are doing the security analysis you don’t assume that they don’t know it.  You have to assume that they might be in on, they might be working for, they might have some access to it, they might be working for one of the test labs or something.  

The big sin in designing a system and I think, also evaluating it, is underestimating the attackers up front.  So, an open-ended search for weaknesses, just kind of a high level view of what we are going to be expecting.  The evaluation is going to get top level system design and architecture; they are going to get a lot of documentation.  It is actually going to add some documentation requirements to the vendors.  That documentation is going to have to include some discussion of the risks to the voting system and how those risks are addressed and give an argument for why the voting system is secure.  That’s going to be where the evaluation starts.  What these documentations are, they are going to document the procedures that they recommend for the election officials.  All that has to be evaluated.  Somebody has to sit down and look at those and say, here is how you tell us to set up the system.  Does that actually lead to a secure system?  Here’s how you tell us to handle a recount.  Does that actually get a meaningful recount done?  Somebody has to do that.  The evaluator has to check all that and after you have looked at the top level system, you are going to look at the pieces.  You are going to look at software, operating system configuration, COT software, hardware, communications protocols, all this stuff.  Again, you are making these assumptions that the attacker really has a lot of knowledge of the system, insider access and you can’t just assume that they are kind of small scale vandals.
When we evaluate the design we are looking at the whole system.  One thing that, this actually comes up a lot, a lot of the high level discussions of the area systems.  When you are looking at the top level of the design you often find attacks that don’t work when you get down to the details.  As I’ve said, I’ve done this as a consultant and its very common to come up with something, I think I’ve broken your system.  Well, when you actually dig down a little you actually find out that there’s some reason why that doesn’t really work.  They really did think about that.  What you come up with is not so much attacks as lists of possible attacks.  Lists of ways you might be able to compromise the system and that leads to some idea of the things you’ve got to look for in the later parts of the analysis.  Sometimes you are going to find fundamental flaws in the assumptions of the design.  I think the real core of this is this first part where you are looking for attacks, looking for places where the security might be vulnerable and where you find out what you are really relying on for security.  

This last part here about, the second part here about checking to see if the procedures and documentation really address the problems.  I’ve heard from a couple of people in the election community that sometimes they don’t feel like they get procedure that tell them exactly how to address some of the stuff that they are supposed to do in a real clear way.  I think that’s an important thing.  There needs to be clear guidance on how to do each of these pieces.  That has to be evaluated per design because different designs will have somewhat different nature of, you know, what does the recount mean if you are doing it on DREs?  There’s a way to do that in some meaningful way by looking at audit records but it might be a little different than what you do for paper.  It might be a little different than what you do Frog Protocol.  
Once we get down to the pieces, once we get down past that pile of evaluations then basically what we are going is looking at all the pieces.   Some of this is already covered in existing testing rules.  I think we are going to want to expand on that a lot but kind of the idea here is that there is, at each stage of this, there’s this mix of kind of checklist.  You know the sort of broad problems the system can have.  You know DRE voting system.  You some things that you want to make sure doesn’t happen.  You want to make sure somebody doesn’t, you know, load new code on the DRE and take it over.  You are going to have some sort of checklist for things that you want to make sure are being secured.  
You are going to use some automated tools.  For network vulnerabilities they are like port scanners and things.  There are automated tools to look at problems at almost any level of this.  You are also going to just do an open-ended search.  You are just going to look for places where you might be able to get through the defenses and then check to see if you can actually get through.
I want to talk a little bit about these.  I will try and keep this a little short.  In communications, there are two sides of communication, of securing communications.  We talked about this a bit with wireless.  One side of it is figuring out how to secure those communications, you know, encryption, authentication, origin identification.  All those things are kind of standard cryptographic mechanisms.  If it’s a closed network maybe you don’t need that but if its wireless or your if you are transmitting over an untrusted network you, of course, got to have encryption for things that are secret and authentication for things that have got to be, you know, say ballot summaries or summary totals or something from different precincts.  That’s got to go over an encrypted and authenticated link.

The other side of that though, that’s securing the communications.  Kind of think of the risks of somebody tampering with the communications or reading them ahead of time.  The other side of this though is you have got to protect the systems from communications.  Any time there is a communications channel into a computer that’s a potential avenue of attack.  That’s one reason why I think people worry a little about wireless systems in voting machines is because, okay, you’re on a network and there’s no way for the election officials to see if somebody’s connected to you.  Who knows what’s going in there and that’s an avenue for an attack.  There are a lot of attacks on general purpose operating systems that start with network access.  I can send you packets, I can start, you know, talking to you on some port and see if I can turn on a service and do something bad to you.  That’s kind of a broad way to do this.  The tools for this are going to be testing and watching the communications.  Doing kind of the same thing you do with a network where you actually look at the packets going by and see if they are really what they say they are and checking the software and the configuration and everything.

The other layer to this is software.  There are at least three places here that I think are interesting to talk about.  All this involves open-ended search for vulnerabilities here.  One part of this that I’ve seen discussed a little bit is the development environment.  If I were doing a review of a system that was supposed to be secure, I would be very interested in the development environment.  I would want to know, for example, did you actually secure your development network?  Is that open to the world?  If it is, the attacker doesn’t have to be a voting system programmer.  He could be somebody who came into the network.  If you do good version control, that actually makes a lot of attacks harder because it means you’ve got to be responsible for whatever changes you introduce,  what tools you are using, what kind of quality assurance and testing you do internally.  Then, having seen it, and that’s all vendor assertion effectively, but you would want to look at that.  Then having seen that, you then want to talk about testing and vulnerability scans.  So there are some automated tools for looking for problems.  Generally, the program is language specific.  There are things where you load the program up and you run it and you look for memory leaks and things like that.  All of that is stuff you would want to do as part of the review.  Of course, you are going to end up with a code review and that’s where you are really doing an open-ended search for problems.  You are looking at the code and saying well does this really actually count the ballots?  Or does this really capture the ballots correctly?  Is there something weird going on?
The other two sides of this are operating system and COTS software security.  They are actually some interesting issues with the COTS software as far as getting it approved but assuming that we have to, that you have off the shelf operating systems and off the shelf software in your voting system.  There is typically going to be some checklist you are going to go through to lock it down, to make sure that you’re not leaving any known vulnerabilities.  You are going to check against kind of databases of known vulnerabilities in different products and say, is this the most recent patched version?  Is this known to be vulnerable to any attacks?  Also, you could say, even if it is, maybe that’s mitigated because it’s a network based attack and there is no networking access on this machine.  That needs to be checked.  That’s all stuff that has to be checked.  
One of the things that is more open-ended and this is a very old procedure for locking down a box, is everything that is not actually necessary for the functioning of the election system has to be taken off the box.  You don’t leave anything running that’s, you know, if you do network security, that’s a big deal.  

The physical configuration security and I’ve just gone over this very briefly because that’s really way far outside of my level of expertise.  Some things that you would want to do would be making sure that, you know, any hardware that’s supposed to not be there, that’s not supposed to be there, is not there.  Making sure that the ports are blocked, if they are not supposed to be used.  Making sure that the locks and seals work and everything.  That’s more, somebody, we’ll have to find other people to look at the physical security issues.  That’s not an area I feel real qualified to address.
I think that leads us to the place where we get potentially ugly although, hopefully, this is not as bad because its spread out over a lot of systems.  An open-ended evaluation could cost a lot of money.  We are not talking, this is not a matter of running a couple of vulnerability scans and going down a checklist and saying did you use a AES?  Yeah.  Did you use Shaw 256?  Yeah.  Did you do the right things?  This is actually going through and evaluating the thing and looking for vulnerabilities.  
The $64,000.00 number is probably way, way low.  This is the smallest amount that I could imagine.  You are talking about a team of four people.  That’s probably too small but I was thinking if you tried to cover all those levels of expertise.  If you are trying to evaluate cryptographic protocols you are not going to want to bring in a general. You are going to need somebody who knows something cryptographic protocols.  If you want to evaluate software security you are going to need somebody who is good at that.  You are talking about pretty highly paid people who cost some money and you are talking about a fair amount of to do this.  I’m concerned that the cost issue could be big although the good news is, its probably a one time cost, spread out over an awful lot of machines.  

There are also some interesting kind of issues.  A good voting system architecture can reduce how many things have to be checked.  The idea here is, if you had, for example, the voter verified paper trail.  One of the things you are checking, you are checking the DRE, you still have to make sure that the DRE can’t be compromised but you might have fewer concerns about the integrity of that electronic record because there is some way to check it.  If you had the Frog Protocol, you would have a different kind of evaluation.  On the cryptographic voting schemes if they ever get kind of wide spread would be a good example of something where the architecture could really reduce some of the costs.  
There is some interesting incentive in financing issues that basically amount to the relationship between the labs and the voting vendors.  I don’t know how easy it would be, I know in the Phipps 140 program, it seems to work pretty well that the validation labs are willing to reject things that come from different vendors and I don’t have the piece of paper but apparently some really large fraction of the first submissions of these things get rejected.  This is your cryptographic algorithms that are well-defined, anybody can go out and look at the standard and the still get them wrong fairly often.  So, its kind of nice that somebody’s checking.  If they weren’t checking then these products would just not work with each other.  	There is other issues revolving around like the availability of the reports when you are done.  There is a concern that on one side you probably don’t want to give, to produce a record that says everything that the vendor did wrong and how it was corrected and publish it on the web.  That might be a little embarrassing for the vendor.  It might also; you might not want to guide other attackers who are trying to attack other systems.  You also might have some real issues for the vendor because they might say well, look, we went through this $100,000.00 process to get a really secure system and now somebody else is going read the report, never make any of our mistakes and walk away with the same thing.  I don’t really know how to address that because you would like the information on how to secure the system out there but I think there are some incentive and financial issues there.  That’s it.

DR. SMERGIAN:	Thank you John.  NIST believes the preceding preliminary report of technical support entitled “NIST Approach to Security Testing for the Redesigned VSS, Response to TGDC Resolution 1705".  Unless there are supplemental directions or corrections, the technical support and related work product will continue to be developed consistent with this preliminary report.  Are there questions, further directions or corrections?  Mr. Kraft.
MR. KRAFT:		Two things.  One, well first off; I like a lot what you did on that.  That is very similar to the type analysis my program does and I think Brit’s program does in many areas.  There was some intermingling there between concepts that are applicable in certifying a voting system for deployment and actually certifying at installed site.  The voting system standards really have to concentrate on those measurable criteria for evaluating standards or evaluating systems.  I think you need to sort out a little bit of that.
Additionally, I don’t know where, oh, there’s Jeff back there.  I wasn’t watching him during your presentation but I think he must have cringed a little bit every time you said open-ended.  We are going to have to for purposes of accrediting labs, we are going to have to put a definition around open-ended and we are going to have to define what due diligence in doing a search for extraordinary and anomalies that you haven’t previously encountered is going to entail.  We cannot really leave that undefined in the standards.
JOHN KELSEY:	I agree. That’s something I didn’t cover and I should.  We have to think in terms of their being a limit as to how long you keep evaluating this.  So, you don’t say, spend a year on it and see if you can find anything.  I think that’s a major issue.

MR. KRAFT:	And you touched on it a little bit with the concept there of one vendor being able to read the report of another vendor and benefit from that person’s $100,000.00.  There are best practices that are know and for many of these areas once you verify that that best practice is in place, you can move on and not spend a lot of resources.
JOHN KELSEY:	Actually, I think that’s the point of the open-ended evaluation is to say, even when you’ve done the best practices, somebody needs to look at this at a high level and make sure that there hasn’t, you can always combine strong pieces together in a weak way.  That’s just an unpleasant fact about security.  Maybe I misunderstand you, I think.  You would want to see best practices being used and once you saw that, that would make your evaluation a little shorter but you would still want to make sure that it was being done in a way that worked.  There is a concern that you use all the right tools and still end up with a bad product.
MR. KRAFT:	Right.  I think we still have to put bounds around what an open-ended piece is going to be.
JOHN KELSEY:	Right.  I agree.
DR. SMERGIAN:	Any other comments, questions?  Well, hearing none, I think the work is in the right direction of what the appropriate comments will be taken into account.  With that, we are almost on schedule for our break so, I suggest, let me see, why don’t we plan on taking a fifteen minute break.  Come back at 3:00 and then we will hear our final report from Alan Goldstein and then we will go to the Resolutions.  Thank you.  See you in fifteen minutes.
BACK ON RECORD:
DR. SMERGIAN:	Can we get other back please so that we can start quickly and finish up the session today on time?  Can we take our seats please so that we can get started?  
Ron, when you make comments, are you on a speaker phone?  We can barely hear you.  When you want to make a comment you might want to get on the regular phone, perhaps.

Okay.  At this time I call on Alan Goldstein of NIST to present a preliminary report and discussion of NIST analysis of the 2002 VSS for the redesign VSS.  Alan.
ALAN GOLDSTEIN:	Thanks.  Am I on yet?
DR. SMERGIAN:	No.
ALAN GOLDSTEIN:	Okay.  Thanks.  What I’m going to be talking about are some documents, some products that are being developed with in the core requirements and testing group.  These tasks have to do with the analysis and the revision of the 2002 VSS.  Admittedly, this task is, perhaps, a little bit less glamorous than some of the earlier presentations but, nevertheless, is completely necessary and vital to the, you know, to the successful completion of our charge.
Another slight different is that, unlike some of the previous presentations, there are no overriding global well-defined, well-scoped comments or decision points.  What I am going to be talking about specifically are documents, most of which are in the package, in the CDs which some of you haven’t been able to read, that are on the web and so on.  What my goal is is to invite comments as to our overall direction and, most specifically, comments on the actual text, the actual work of the documents that you have in front of you or that you or that you can look at.
The first document is the notorious disposition of VSS Requirements Table, referred to earlier as the spreadsheet, you know, whatever.  Its really just a Word document table.  What this is is a, right now, a complete first draft of a table that extracts requirements from the 2002 VSS, slightly oversimplified.  We went through and took out every bit of text within the 2002 VSS pretty much that had the word shall in it.  That was pretty much what we viewed as a requirement and put this in a table and there are some additional things but primarily we proposed, or have proposed, the disposition of each of these requirements.

MALE SPEAKER 42:	(Too far from microphone to understand what was being said).
ALAN GOLDSTEIN:	I think its what’s up there.  Disposition of VSS Requirements, Volume 1.  That’s what it is on the web.  That’s what I was using.  Pardon me?
MALE SPEAKER 43:	Tab “O”.
ALAN GOLDSTEIN:	Tab “O”.  I never even got an unreadable CD so, I really can’t say.  So are we all on page one.  All on the same page.  Basically, I have fifteen minutes, I’m not going to, you know, go into detail.  I’m going to talk more about the structure and the intent of these documents.  
Basically we went through and for each identified requirements in the VSS categorized them as “E” which is a requirement that can be extracted as is.  No need to rewrite it.  “R” which is a requirement that was not precise, clear or testable.  It needs to be rewritten.  There are quite a few of these.  Some of these refer to minor changes, others we anticipate much larger, more major changes to these requirements.
MALE SPEAKER 44:	(Too far from microphone to understand what was being said).
DR. SMERGIAN:	Welcome aboard, J.R.
ALAN GOLDSTEIN:	Another category which a few fell into was that a requirement that appeared in Volume 1 which this complete table is referring to really was a testing requirement and should be moved to Volume 2.
The fourth category is, for one reason or another, the requirement was extraneous, irrelevant, obsolete, totally misleading, incorrect, whatever and we’re suggesting, proposing that this requirement be deleted.  There aren’t that many of them. 

Most importantly, in addition, I guess, there’s a comments column which currently has initial observations and questions.  Questions for the community, the TGDC, for anyone who reads this.  For the most part these are not rhetorical questions.  These are real questions, you know, what is the answer to this question?  We need comments to be able, to enable us to actually do the rewriting and so on.
Our next step is to incorporate the comments that we have received and will receive and answers to questions, into this document to provide a clear path to revising the VSS requirement by requirement.  The precise quantification of what will be done in April versus what will be done in November, you know, I guess, remains to be seen.  Certainly we have encountered quite a few typos, misprints, ambiguities, bad references and so on.  Those can certainly be corrected in the April time frame.
A second document which is not on the CD, which doesn’t cause many of you any problem at all, didn’t quite make it.  We did put it up on the web last Friday.  Its up there.  Its available for your perusal.  I don’t think you have a hard copy representing it either, is a first draft a similar table for the disposition of requirements for Volume 2, the testing volume of the 2002 VSS.  Its not quite complete, sections 2 through six I think there are one or two additional sections that need to be processed.  This was what we got at the deadline and please take a look at this as well.  I think the format is identical except that “M” now refers to things that are in this volume that should have been in the first volume, obviously.  

The next document in the package is the latest updated version of the process model.  I think there are two files comprising this.  One the model diagram itself and another a text document describing the model.  Its our attempt, this was presented in at the previous plenary so, I don’t think there is anything startling or new about it.  There are some revisions.  Take a look at it. Give us your comments.  It’s the, our attempt at the election process modeled with activities, objects and flows.  We are using it as a reference for organizing and cross-referencing the compliance points that we are developing to see where they fit into the overall process. 
Another document, some of these are a little bit awkward because even if you have the hard copy and look at it there aren’t any titles on some of these.  So, its still hard to figure out which document we are talking about.  There is one called the CRT rationale pieces.  This, at least, is what its called on the web which is a discussion and explanation of issues, sort of fragmentary discussion impacting the CRT subgroup, you know, various issues that we have dealt with.  We’ve tried to summarize, you know, what the discussion was, what the thinking is on a number of these things.  They have to do with the architecture of the revised VSS, some testing issues, coding conventions, quality assurance and configuration management, you know, things that fall under the domain of the CRT group.  
There is also a discussion of what’s called here procedural requirements.  That begins to get into the area of election procedures which, in turn, gets into the area of instructions for voting officials and that particular can of worms.  It’s a bit of our thought on it and why its necessary to, for us to talk about this in some form, the form, I guess, to be determined.  

The document CRT products standard pieces, what this is, is a set of detailed compliance points relating to one of the eight organizing principles that we discussed at the previous plenary.  In particular, the one dealing with accurately accumulate, count and report votes.  It’s a work, its by no means finished.  Attempts to extract, identify and identify precise testable compliance points, clarify, remove redundancy, organize coherently.  If you view the table mentioned a few minutes ago, as sort of like a horizontal analysis of the VSS, this is a vertical drilled down deeply for those requirements, those compliance points that deal with one particular topic, namely, accumulate, you know, accurately accumulate, count and report votes.  It deals specifically with those issues rather than a broad, you know, one liner analysis of every single requirement found in the document.
The document called CRT testing pieces presents a formal mathematical model of voting systems developed by David Slater with terms and definitions.  It applies logic verification to or shows how it can be applied to source code, to voting systems source code and also provides examples of abstract test cases for functional tests, capacity tests, some others, I think, perhaps are in there which would begin to give you a flavor of what we are envisioning for the new testing document.  What types of tests, what type of test specification is at the right level is appropriate for a VSS as opposed to, you know, what, perhaps needs to be developed, needs to be customized by the testing labs.  So, I urge you to take a look at that.

Finally, there is a couple of documents which propose a rewrite of the sections in the 2002 VSS dealing with software standards.  We looked at the VSS and it seemed that the chapters on software were probably the ones that were in the most need of rewrite of all of these.  There’s a lot of material in there that is, well, lets put it generously, obsolete.  It needs to be, it needs to have a major revision.  This is a proposed revision, now more red and blue ink than anything else.  It recommends changes to the coding conventions on the source code.  It does retain and expand conventions, addressing software integrity.  It does retain some size limits on the modules, purges outdated, optional stylistic inventions, you know, irrelevant naming conventions and things like that.  Rather than trying to tell the reader what are the precise practices to use it defines criteria for a published credible coding convention.  You sort of say use what is considered to be the best practices in the world out there and hear are some criteria to help you decide whether or not these are, in fact, the credible best practices.  It required some structured exception handling.
Those are the documents.  None of them are finished although the process model is probably, I think, pretty much complete unless we get some more comments in.  We are eager to get comments, to get answers to our questions.  I’m a particular advocate for the table in there.  I have tried to highlight the questions.  We really do need people, qualified people to go through and say, you know, we missed a point here, or this is the answer, this is the definition of the term, this is the meaning of the term that was used and this is what was intended and so on so we can do the appropriate rewrite.
Okay.  Any questions.
DR. SMERGIAN:		Thank you Alan.  Questions, comments.  Don’t go away.
MALE SPEAKER 45:	Yes.  On the left side is there a hand out document that corresponds to this?
ALAN GOLDSTEIN:	Yeah, its in the package.  I think the whole draft –
MALE SPEAKER 45:	I don’t see anything that’s titled Q1-S4.
ALAN GOLDSTEIN:	I forget what the title is but its there.  As a matter of fact its more than an outline, it’s the entire draft.
MALE SPEAKER 45:	Well, okay, they were jumping back and forth.  I’m just trying to follow your order.
ALAN GOLDSTEIN:	The order is slightly different from what turned out to be the --.  Its all there.  Everything is there except for the Table for Volume 2 of the VSS.
DR. SMERGIAN:		I assume Volume 1, Section 4, is under tab “P”.
MALE SPEAKER 45:	The letters correspond to the front.
DR. SMERGIAN:		And then, Volume 2, Section 5, is under tab “Q”.

ALAN GOLDSTEIN:	These are both dealing with the software one in the requirements part of the VSS and the second one is the equivalent section in Volume 2 of the testing part.
DR. SMERGIAN:		Any other questions, comments?  Anybody on the phone, any questions or comments?
MALE SPEAKER 46:	You’ve done a good job.  Thank you.
DR. SMERGIAN:		Okay, well based on these comments NIST believes the preceding preliminary report of the technical support titled “NIST Analysis of the 2002 VSS for the Redesigned VSS Response to TGDC Resolutions 25-05, 27-05 and 29-05".  Unless there are supplemental directions or corrections which I have not heard, the technical support and related work product will continue to be developed consistent with this preliminary report.  Obviously we continue to welcome, as Alan pointed out, your comments and your additions and corrections and, hopefully, some answers to the questions that he has put in the tables.  So, unless I hear any other comments, we are –
DR. FITZER:	I just want to commend NIST for some very good work, a lot of hard work.  We appreciate it.
DR. SMERGIAN:		Well, we appreciate your comments, Dr. Fitzer.  So we are basically finished with this section of the meeting and will proceed to consider the two Resolutions, two new Resolutions which are in your notebook.  Its tabbed “Proposed New Resolutions”.  It’s the last tab in your binder.
So, we have received two new Resolution from TGDC members that were circulated to the entire committee last week and I will call on Ms. Ann Caldess to introduce Resolution number 36-05.

ANN CALDESS:		Thank you.  Before I do so I would like to clarify that the Resolution is simply the first two paragraphs of these several pages.  The additional background is simply that background and perhaps not appropriate to be part of the, to have been included as part of the documents that reflect the Resolution.  So, that was really background information.  I will also say as I read it that I’m going to edit a phrase out of it.  So, I would like to read the proposed Resolution.
DR. SMERGIAN:		Please go ahead.
ANN CALDESS:		The TGDC recognizes the time frames established in HAVA as necessary to insure a prompt response to the Nation’s voting system issues which it was written to address.  The TGDC further recognizes that a robust, voluntary consensus standards system exists in this country and should be relied on to the greatest extent possible to facilitate the long term development and regular maintenance of voluntary voting system guidelines and standards.  Moreover the TGDC encourages the EAC to rely not only on this recognized expertise but also on the U.S. voluntary consensus and conformity assessment system as exemplified by programs and standards that comply with the requirements set forth on OMB circular A119 as part of the EAC’s long term systemic approach to addressing the Nation’s continuing needs for up-to-date voluntary voting systems guidelines and standards.
DR. SMERGIAN:		Could you point out the changes because –
ANN CALDESS:		I deleted the, because I understand people misinterpreted its inclusion, reference to accredited by the American National Standards Institute and others.
DR. SMERGIAN:		So, and you added “and standards” I believe after programs.
ANN CALDESS:		Yes, programs and standards.  Right.
DR. SMERGIAN:		And then simply continues as which comply with the requirements.

ANN CALDESS:		Right.  That’s correct.  
DR. SMERGIAN:		Was that the only change?
ANN CALDESS:		Yes.
DR. SMERGIAN:		Did everybody catch those changes?	Yes.  Go ahead J.R.
J.R.:	I am just confirming that I heard the changes and I’m comfortable with them.
DR. SMERGIAN:		You are not comfortable with the changes?
ANN CALDESS:		No, they are.
DR. SMERGIAN:		He is comfortable with them.  Okay.
MR. BERGER:		So the Resolution ends with the word standard, voting system guidelines and standards.
DR. SMERGIAN:		We are only talking about the first two paragraphs of that Resolution.  The rest of it is background information.  Is that correct?
ANN CALDESS:		Yes.
DR. SMERGIAN:		And basically have taken out the third line of the second paragraph starting at accredited.
ANN CALDESS:		Right.
DR. SMERGIAN:		And added after programs added and standards.
ANN CALDESS:		That’s right.
(START TAPE 4)								
DR. SMERGIAN:		So I have a second?
MALE SPEAKER 47:	Yes.
DR. SMERGIAN:		Questions, comments.

FEMALE SPEAKER 19:	I understand this is a kind of general statement because it seems to me that we’ve been doing this all along.  I’m a little curious about why this exists but I’m also curious about things like relying on existing Federal regulations.  For instance in the human factors area we are relying rather heavily on Federal regulations known as Section 508.  Given this statement it might suggest that we should be relying more heavily on say, the web accessibility initiative which is a voluntary consensus standard over that Federal regulation.
ANN CALDESS:		I think, perhaps, it’s a, you know, a little bit about OMBA 119 should also be clear that it lists the criteria for standards that are in the public interest as open, as balance, due process and consensus but also recognizes that where its not practical to rely on them, that you would rely on whatever works essentially.  Whether that’s a standard written by one person or a standard written, you know, by hundreds.  Again, it was really for the long term process.
FEMALE SPEAKER 19:	I guess the other point that I would like to raise is that one of the issues with some of the, I don’t know how to say the word, proprietary consensus standards is that they can be expensive.  They are often not available and they are often, in fact, not in the public domain.  So, there are issues with using them at all.
ANN CALDESS:		This does not in any way mandate it.  It encourages the reliance on them, where its appropriate, because such standards are in the public interest because they involve all those who are materially affected stakeholders have an opportunity to contribute in an environment that is characterized by openness, due process and consideration of all views.  It is not; it does not exclude the use of any, but for the long term, you know, recognizes the importance of the existing system in this country.
I would also note that NIST participates on many of those standards development activities.  They are not separate and it was not meant to imply that they were.

DR. SMERGIAN:		If I could ask a question.  Do you feel that this is meant to apply to this immediate period, you know, for the next year or so or are you talking about beyond that?
ANN CALDESS:		I think it’s the long term.  HAVA reads as it does and obviously has to be implemented as such.  I think its important to raise, you know, awareness, you know, that for the long term there is a process that exists that is in the public interest.
DR. SMERGIAN:		Mr. Berger.
MR. BERGER:		I’ll speak in favor of the Resolution.  I think at its heart it calls for a two stage system,   one in the voluntary consensus arena where the door is open, people can come in and join committees as fully participating members.  I think that’s important especially in this arena and that work product can be taken and considered in a process such as this to see its appropriateness and, obviously, particularly in situations where something that’s appropriate isn’t available or perhaps where that kind of open consensus couldn’t find a consensus then a process like this can be used to make decisions where they are necessary.
DR. SMERGIAN:		Well, if I may, that was sort of part of the confusion I had in my own thinking because I thought that this committee and the team, the NIST team, certainly included and utilized I Triple E and other, you know, standards or draft standards or any other informational tool that was available.  Is that the focus of this Resolution or is it more than that.  Basically saying that, well, a voluntary standards process should take over this process in the long run because my impression wasn’t that, I was thinking about more like you pointed out a two step process which feeds into, you know, this kind of a process or are we basically talking about going to an entirely voluntary standards based process?  Could you –

MR. BERGER:		In my view what’s happened recently would be an example of implementing this Resolution so that the first step was to see what was out and available and to the degree its applicable and its sufficient mature to be usable.  Its been incorporated and then where there are gaps, that’s where the staff time is applied.  So, I would see this Resolution as already being exemplified in the current process.
DR. SMERGIAN:		Is that your thinking?
ANN CALDESS:		Yeah, yeah, this doesn’t intend to imply that it would be completely under the voluntary consensus process.
DR. SMERGIAN:		Any other questions from the folks on the phone?
J.R. (Speaking from phone):	Yes, this is J.R.
DR. SMERGIAN:		Go ahead.
J.R. (Speaking from phone):	If I understand it right, we are moving forward on type of being able to screw the ultimate (undecipherable) and then, if it (undecipherable) its going to be much higher in terms of the expect ability to accomplish here and that it would be (undecipherable) baseline for all national elections.  Is that correct?
DR. SMERGIAN:		I don’t think we could hear your last few words.  Could you repeat yourself?
J.R. (Speaking from phone):	The two step meaning (undecipherable) and then indicating to the public that this is the threshold or the standard will be on a higher level of expectation and that we are, in fact, on a (undecipherable) national standard for inclusion of all people in the voting system.
DR. SMERGIAN:		Thank you.  Mr. Kraft.

MR. KRAFT:			Yeah, this is Paul Kraft and J.R. in partial response to you, I am supporting the Resolution.  I think what this Resolution does more so than what you described is to point out that this is a long term, ongoing process that won’t end in ‘06 or ‘08 or, hopefully, ever.  As technology continues to evolve, as we learn more in the areas of accessibility, usability and security, this work needs to be updated at least every other year.
FEMALE SPEAKER 20:	I’m sorry, while I completely agree with you; I don’t see how this Resolution addresses that.
MR. KRAFT:			I’m locking in on that fourth sentence there in the first paragraph, to facilitate the long term development and regular maintenance which maybe we need to say that more clear, or you need to say that more clear.
FEMALE SPEAKER 20:	I mean, its not entirely clear to me that a voluntary consensus standard is the best way to maintain this in the long term.  It might be that an open, fair and transparent process would be the best way to maintain it in the long run but, I haven’t, for instance, seen the access board decide to turn back to voluntary industry consensus standards for designing accessible web sites.
FEMALE SPEAKER 21:	I feel somewhat conflicted because I agree with every word in here but I don’t really quite understand why we have this and so I think that’s some of the hesitation around here.  If this is simply a statement of good principles and I have to say that the human factors and privacy group put out a certain number of statements of good principles in our Resolution so I suppose the pot can’t call the kettle black, then I think I’m fine with it.  I just want to make sure that we are not, in fact, creating Resolution that says more or less than we mean to say.
DR. SMERGIAN:		Yes.

FEMALE SPEAKER 22:	Yeah, I just, I don’t think its inconsistent with 3105 which talks about the need to develop a process for the maintenance and the interpretations and for due process and those are old provisions that are, at least within Nancy’s world, but in other worlds as well are a part of the required aspects of standards development, a voluntary consensus standards development.
DR. SMERGIAN:		Do we have any other questions or comments?
MALE SPEAKER 42:	I call (undecipherable) 
DR. SMERGIAN:		Do I have a second?
MALE SPEAKER 43:	I second it.
DR. SMERGIAN:		Okay, Sagreen would you please take the roll call?
SAGREEN:			This is 3605.  Williams.  
WILLIAMS:			For.
SAGREEN:			Williams votes yes.  Berger?
BERGER:			For.
SAGREEN:			Berger votes for.  Caldess?
CALDESS:			Yes.
SAGREEN:			Caldess votes for.  Kraft?
KRAFT:			For.
SAGREEN:			Kraft votes for.  Davidson?
DAVIDSON:			For.
SAGREEN:			Davidson votes for.  Aleckies?
ALECKIES:			For.
SAGREEN:			Aleckies votes for.  Gannon?
GANNON:			Yes.
SAGREEN:			Gannon votes yes.  Harding?
HARDING:			For.
SAGREEN:			Harding votes for.  Miller?  Miller?
DR. SMERGIAN:		She’s not present.
SAGREEN:			Purcell?

PURCELL:			Yes.
SAGREEN:			Purcell votes yes.  Quisenberry?
QUISENBERRY:		Yes.
SAGREEN:			Quisenberry votes yes.  Revest?
REVEST:			For.
SAGREEN:			Revest votes yes.  Shutzer?
SHUTZER:			Yes.
SAGREEN:			Shutzer votes yes.  Turner-Bowie?
TURNER-BOWIE:		Yes.
SAGREEN:			Turner-Bowie votes yes.  Smergian?
SMERGIAN:			Yes.
SAGREEN:			Smergian votes yes.  That’s fourteen votes in favor. One voter absent.
DR. SMERGIAN:		The motion, Resolution has been adopted, number 36-05.  The next Resolution is offered by Dr. Revest, Resolution number 37-05.  Dr. Revest would you like to make a few comments.
DR. REVEST:		Yes.  Can you hear me okay like this?
DR. SMERGIAN:		Yes.
DR. REVEST (speaking on phone):	Okay.  I would like to read the Resolution and describe it briefly.  It’s a simplification of an earlier Resolution that we had talked at our last meeting regarding the availability of source codes for review.  The proposed Resolution reads as follows:

The TGDC has reviewed the issue of the availability of voting system source codes for review.  The TGDC directs this draft requirement stating that (a) voting system vendors shall supply source codes with any voting system submitted for evaluation; (b) the source code shall be retained and (c) the source code shall be made available for review upon request to local, state or Federal election officials and/or their designated representatives.  (Undecipherable) handle third party software for which source code may be unavailable other than its authorized access to the source code, availability of required tools such as the (undecipherable) for source code review and possible restrictions on reports resulting from source code reviews.  That is the proposed Resolution.  This is in line with previous discussions about transparency and building voter confidence in the systems.  The availability of the source codes and specifics what the voting machine does in detail and having that available to state election officials and representatives I think do a lot to increase public confidence that these voting systems are in fact doing the right thing and (undecipherable).  I’m open for discussion on this.
DR. SMERGIAN:		Do I have a second?
FEMALE SPEAKER 24:	Second.
DR. SMERGIAN:		Discussion, comments?  Go ahead, Dr Berger. (Undecipherable).
MALE SPEAKER 44:	This issue keeps coming up and we don’t seem to be able to drive a wooden stake in its heart.	There are some of that have concerns about open availability of the source code.  We are concerned that when you put this code in the hands of every teenage hacker in the country good things aren’t going to happen.  In my experience which spans over eighteen years of dealing with voting systems, I have never had a vendor deny me access to the source code.  Its been my experience that everyone in this industry who has a reasonable need to know can gain access to this source code.  

So, my first recommendation would be to vote this down entirely but since its on the table I would like to propose an amendment to it.  I’d like to amend section, propose that we amend paragraph (a) to read: Voting system vendors shall provide source code with any voting system submitted to an ITA for evaluation for compliance with the voting system standards.  
I’d like to amend (b) to say: The source code shall be retained by the ITA and I would like to delete paragraph (c) entirely.
DR. SMERGIAN:		Do I have a second?
MALE SPEAKER 45:	I’ll second.
FEMALE SPEAKER 25:	Is it ITA or is there a different –
MALE SPEAKER 44:		Whatever.
MALE SPEAKER 46:	I think its now Voting System Test Lab –
MALE SPEAKER 44:		I think we’ve agreed we’re going to use those interchangeably.
MALE SPEAKER 46:	Yeah.
FEMALE SPEAKER 25:	Okay, just checking.
DR. SMERGIAN:		That was actually the question I was going to ask of Dr. Revest because this didn’t say who is supposed to be given to or who is supposed to keep.  What did you have in mind?
DR. REVEST (via telephone):	This was intended to be left open for NIST to make a resolution of.  I’m happy to view those modifications to (a) and (b) the present amendment.  The proposal to drop (c), I’d view as a very unfriendly.
DR. SMERGIAN:		I didn’t quite hear the comment about (c).
DR. REVEST (via telephone):  	I said I viewed the proposal to drop the language in part (c) as an unfriendly proposal.

MALE SPEAKER 44:		Well, then let me explain why I recommended that.  It talks about making available on request to local state or Federal election representations and/or their designated representatives.  That’s entirely too vague.  Who –
FEMALE SPEAKER 26:	Perhaps this could be worded as to direct NIST to write criteria under which it can be made available to appropriate officials.
DR. REVEST (via telephone):	I think the question of who should have access and how is a little bit vague here.  It needs to be clarified by NIST.  If somebody wanted to hire you, Brit, to look at the source code, this would authorize that.  That’s the kind of thing that we had in mind.
MALE SPEAKER 44:		I don’t need it.  Ron, I have never had a vendor in eighteen years deny me access to their source code.
DR. REVEST (via telephone):	There are some states that (undecipherable) that I wanted to come in look at the source code.  I think we ought to allow that.
MALE SPEAKER 44:		Its allowed by the marketplace.  All the Secretary of State has to do is say if I don’t look at your source code, you aren’t going to do business with us.
DR. REVEST (via telephone):	(undecipherable).
DR. SMERGIAN:		Mr. Kraft.
MR. KRAFT:			Yes.  The issue as to the availability of source code, the fair use of the source code, the protection, the intellectual property rights, that’s primarily, its state law.  In our state we have a law that requires the source code to be provided to our office and there is also a law that makes it a felony for me to disclose it for anyone’s use outside of its intended use.  I think most states have that and also the marketplace control.  First off our whole concept of voting system testing labs here, those labs cannot do their job if they don’t have the source code.  Beyond that any state and any local jurisdiction who is controlling the check to the vendor can see source code any time they want to.

DR. SMERGIAN:		I might add there is another reason beside all this why you might want to consider doing it and, of course, I don’t see the difficulty if you have ever experienced a problem, what’s the problem putting in.  Another reason why you might want to do it, which we write in a lot of our contracts, and I think there’s a distinct possibility with some of the vendors, is that we like to hold the source code in escrow in the event that something happens to that vendor, so if it was not available we would like to at least maintain it and continue on.
MALE SPEAKER 44:		Right.  But that is a contractual.  That’s something that’s usually handled contractually at the time you buy the system because what you want in escrow is the system you are using.  This is talking about asking everybody for everything.  You are talking about specifically escrowing the software you are using and I think everybody addresses that at contract time.
FEMALE SPEAKER 27:	I guess I want to agree with Mr. Schutzer which is that if what we are talking about is embodying in the standards something that is already common practice, I don’t understand what the problem is.  If what we are talking about is creating new requirements that people feel are unacceptable we’ll see that when the requirements come out from NIST.
MALE SPEAKER 44:	Well, this expands what’s already in the standards and what’s in practice.  This basically requires the source code to made available, upon request, to local, state or Federal election officials and/or their designated representatives.  It doesn’t put any bounds around what those jurisdictions may do with that code, who those representatives might be.

FEMALE SPEAKER 28:	I’m sorry but I think we have already proposed, although we didn’t get final language on it, that we bound this to say not at this level of resolution that it doesn’t just say to any official in their thing but that part of the requirement shall specify who and how and, you know, how that’s determined.
MALE SPEAKER 44:	What’s a local official?  If the dog catcher wants it can he get it?
FEMALE SPEAKER 28:	He’s not an election official.
MALE SPEAKER 44:		It doesn’t say election official.
FEMALE SPEAKER 28;	It does.  It does.
MALE SPEAKER 44:		Well, yeah, okay or their designated representative, yeah.
FEMALE SPEAKER 28:	I mean, I would take your point about the open-ended of this but I would say that a part of what the requirements would have to do is say who might be eligible.
MALE SPEAKER 44:		Well, Dr. Revest has accepted my changes to (a) and (b) and, so I will withdraw my recommendation to omit (c) if someone would like to reword that.
DR. SMERGIAN:		May I ask a question?  If this Resolution passed exactly what it is that you are asking NIST to do?  I mean, it seems to me this is already regular practice and it seems to me people are comfortable with the definition of local, state and Federal election officials and/or their designated representatives, so, exactly what it is that you expect NIST to do?
MALE SPEAKER 44:		Well, I guess what I would like to see is bounding this closer to what is actually desired.  The reason we get source code from a vendor is number one, so we can review it and we can make sure that it is properly prepared, decent code and we can do all the analysis and including me that we don’t do now that we will do in the future for the purpose of determining whether or not a system can be qualified.

The second purpose for source code and the reason that we escrow it is not so that we can go compile it and build our own voting system when and if the vendor goes broke, but so that when he brings his next revision into us, and he gives us technical release notes on what he has changed in that code, we can do a code compare, the new code versus the old code and determine if, in fact, he made changes outside of that and we can then lay in test lands that are specific to those changes.  That is really the only utility in this program that source code has.  That’s what the vendors make it available for.  That’s how its used and for those people who are engaged in that particular work, the vendors have no problem making it available.  So to make it available outside that scope, I think its something we shouldn’t do.
FEMALE SPEAKER 28:	Okay.  So, I can propose some wording changes that, if I do it slowly, I think are quite simple and don’t require a lot of redrafting.  I’ll need your help, Paul, (undecipherable).
PAUL:		Okay.
FEMALE SPEAKER 28:	So, back up at the first sentence, TGDC directs NIST to draft requirements stating.  You need to put the colon after stating and drop the that.
PAUL:		Okay.
FEMALE SPEAKER 28:	The word that then needs to appear at the beginning of (a), so stating that voting system vendor shall supply.  Beginning of (b) that the source code shall be retained, and (c) the conditions under which the source code shall be made available for review, remove upon request, and Paul, I’ll let you take the rest of that paragraph.  Hang on for a second.  Go down to the paragraph below it.  These requirements shall address a number of issues.
PAUL:		Okay and NIST shall clarify these requirements.

FEMALE SPEAKER 28:	That these requirements shall address. So the question I have for you is do we need to change this for review, the phrase after for review to local, state or Federal election officials to clarify the point that you are making.  I don’t think that anybody is implying that we should give this away to teenager hackers or even people who just happen to be curious.
PAUL:		How about if we change (c) to the source code shall be made available for review upon request to local, state or Federal election officials for their use in approval, purchase or certification assistance.
DR. SMERGIAN:	I have a fundamental question.  What is this going to allow you to do that you cannot do now?
PAUL:		Nothing.
 MALE SPEAKER 44:	This does not contribute anything to the existing standards.  That’s why I’m 
DR. SMERGIAN:	So, I guess then my question is –
FEMALE SPEAKER 28:	Maybe that question should be addressed to Dr. Revest about why he wanted –
DR. SMERGIAN:	Dr. Revest would you like to comment on that?
DR. REVEST (via telephone):	I think there is a perception that these voting systems are black boxes that, you know, no one except the vendors maybe know what’s going inside.  I understand that common practice has the source code available to state election officials.  I think part of the point of having this as a requirement, even though if they codified what is common practice today, is to increase public confidence that the public’s representatives do have the right type practice but the right standards to have access to the source code and to allow critical of the source code for their purposes.

MALE SPEAKER 44:		I’m not sure they should be making the standards styled in such a way to address things that everybody sitting at the table know really are not an issue.  These are not black box systems and they are not reviewed and black box tested.
DR. SMERGIAN:		Mr. Berger.
MALE SPEAKER 48:		(Undecipherable) software system and a black box standard except at the market forces you are talking about.  I think the standard is simple and the market forces to set expectations (undecipherable) transparency to have in the marketplace.  Its just a matter of protecting the principle, I think.  We have a statement of principles here.  Some of us are happy with principles and I’m sure (undecipherable).
DR. SMERGIAN:		Mr. Berger.
MR. BERGER:		It seems to me the key point on (c) is if the vendor protests the distribution of the software.  Clearly if the vendor agrees to a particular official or their designated representatives receiving it, its going to happen anyway.  It happens currently.  I would suggest some language be added in (c) that NIST recommend when, over vendor protest, the software, the source should be distributed.  I’m not sure that such a case exists but someone ought to think that one through carefully.
MALE SPEAKER 49:	Well, I guess, looking at, basically as this is written this would allow Brit Williams in Georgia, who has a Diebold state owned system, statewide to force ES&S to hand over their source code.
MR. BERGER:		Or the other example would be if the election official from Cut and Shoot, Texas, had a brother-in-law who has a competing company, he could get everyone else’s source code under this.  I don’t think that’s our intent.

FEMALE SPEAKER 28:	I think that’s the point of being sure that we specify what the conditions are and we didn’t finish the sentence that you started so we might want to finish that before Alan gets lost in the ether. 
For the use in evaluating systems for purchase something or something.
ALAN:		For purchase, certification and I forget what the third one was.
DR. SMERGIAN:	Let me ask another question.  It seems to me what I’m hearing is, this is current, the current practice is accomplishing what you want to accomplish.
MALE SPEAKER 44:	Yes.  It’s a little stronger than that.  The current standard specifies that the vendors shall submit the source code with their technical data package when they submit the system for review.
DR. SMERGIAN:		Its in the 2002 – so then, if that is there, and if we don’t want to change the procedures or the requirements or the criteria, is the idea, Dr. Revest, I guess maybe this is addressed to you, is the idea to have a resolution that publicly states that this is what happens, we don’t have black boxes, that these are contractually as well as part of the requirements of 2002 VSS that vendors provide, you know, source code for whatever purposes, either for, you know, safekeeping or whatever and that this practice should continue.  I’m trying to see, when you start saying draft, you direct NIST to draft requirements, I’m not sure what we are developing requirements for.  I’m trying to understand what it is that I would ask our people to do in response to such a resolution.

DR. REVEST (via telephone):	(Undecipherable) standards merely codify what is common practice and I think we are essentially in agreement here that its common practice for source code to be available to election officials and their team to evaluate for the purposes of prospective purchase or maybe to evaluate what’s going on with the system they’ve already purchased and so on.  I think that this (undecipherable) a situation where we are essentially codifying common practice.  It doesn’t go as far as the Clinton Boxer proposal did which would have made source code public, available to the public with no control whatsoever.  They tried to draw reasonable boundaries on who would have access to the source code.  That boundary corresponds to common practice.
MALE SPEAKER 48:	I believe the standards are useful sometimes even when the do correspond to (undecipherable).
DR. SMERGIAN:		I agree with that but what I’m told is that its already in the standards.  Its in the 2002 VSS.
DR. REVEST:		(Undecipherable) in the standard for making it available to the ITA.  I don’t believe its in the standard to make it available prospective purchasers.
MALE SPEAKER 44:	And, some software purchasers don’t want it.  Those that do want it, get it because you are a purchaser they are going to give you what you tell them you need.
MALE SPEAKER 50:	I wouldn’t have any objection to wording that said purchasers.
FEMALE SPEAKER 28:	You said what?
MALE SPEAKER 50:	To purchasing the system.  This says anybody whether you are purchasing it or just want to look at it or what.
MALE SPEAKER 51:	That’s your objection.  It’s a system you haven’t purchased.
FEMALE SPEAKER 28:	Yeah.  So purchasers (undecipherable).

MALE SPEAKER 44:	I think I’m going to have to speak against this because I don’t think, I was thinking that, you know, we could do a somewhat friendly amendment but, I think this goes outside of the scope of voting system evaluation.  It goes outside the scope of the standards.  It goes more to a national policy of making source code available on a wider basis than is needed for evaluating and certifying systems.
DR. SMERGIAN:	Mr. Berger and then Secretary Davidson.
MR. BERGER:	Actually I think we’ve brought up another issue that’s probably worth looking at and that is, are we, in changing the system, because currently the source code is never given to any Federal entity.  In the system we are constructing is this now given to a Federal entity and can it be protected even under the assumptions all of us would assume, against a request under Freedom of Information.  That’s probably worth a look to make sure that someone’s not inadvertently making their source code publicly available.
DR. SMERGIAN:	Secretary Davidson.  Just a minute.
SECRETARY DAVIDSON:	Number one, I think there’s two problems that I really see.  I don’t want to leave it in the hands of the locals and leave it there because those are the ones that programming the elections and if you ever wanted to tamper, that gives them an edge.  
Number two, why the Federal.  The state is running the Federal election so, who are you going to give it to at the Federal level?
DR. SMERGIAN:	We don’t want it.  Yes, J.R., you had a question or comment.
J.R.:		Yes, I was going to suggest to our committee chairman that (undecipherable) will be evolving the standards for some time.  Let’s get past the (undecipherable) and then re-examine the issue if he still thinks its needed.
DR. SMERGIAN:	Dr. Revest would you –

DR. REVEST (via telephone):	Yes, I think it is a long term issue in some sense.  I think, you know, it is my opinion that the earlier the better just to make sure the source code is available for review.  I think the issue that Dr. Berger raised on the (undecipherable) issue is one I hadn’t considered.  That’s an interesting point.  That might be sufficient cause for (undecipherable) on that score but I think, and I’m looking (undecipherable) J.R. (undecipherable).  The point (undecipherable) the transparency should be as high as possible.  I think there’s a lot of mistrust among the public at large for voting officials and if the source code is not available and we have attempted to mitigation that.  The goal here is to increase confidence and trust in the voting systems and if the source code is not available except to (undecipherable) only doing a limited review I think, in my mind that’s something that (undecipherable).  I feel its an important point (undecipherable) this committee could take a stand on this point (undecipherable) and I appreciate the (undecipherable) to try to word it in a way that would make it most acceptable.
DR. SMERGIAN:	So, would you withdraw this and, perhaps, consider some of the comments made here and if you want to bring up at a future meeting, it would still be timely.
DR. REVEST (via telephone):	I appreciate the comments.  As I said, I think the one about the FOIA is one I would not want to have a trap like that sprung on us.  So, yes, I’m going to withdraw it at this meeting and proceed to take the comments under advisement and (undecipherable) to TGDC for wording changes and (undecipherable) that can be brought into this.
DR. SMERGIAN:		Thank you.  Did you want to make a comment, Ms. Purcell?
MS. PURCELL:		I was just going to follow up on what Secretary Davidson said.  I don’t believe the local officials want that responsibility either.  We have the same issues of public records law that come into play a great deal.

DR. SMERGIAN:		Well, yes, Secretary Davidson.
SECRETARY DAVIDSON:	I think, just to add, every state has different open record acts.  They are very different throughout the United States and that, I mean, its going to take lot of research.
DR. SMERGIAN:		Okay.  Then, we considered this Resolution withdrawn and Dr. Revest will reconsider it, you know, especially in light of the comments and see if he wants to bring it up again.
Well, believe it or not, that completes our tasks for today.  A job well done, I must say.  Certainly the Resolution adopted resolutions and the ideas expressed, adopted at this plenary meeting further instruct NIST staff on the research and drafting of standards recommendations.  The adopted motions provide essential policy guidance on relevant voting standard issues and NIST staff, in cooperation with the TGDC members will continue to make best efforts to accomplish the critical tasks most urgently needed by election community as part of their recommendations to the EAS.
I appreciate the participation today of all of the committee members.  I want you to notice that all of the members were here at least part of the or participated at least part of the meeting and I very much appreciate that participation of all the members.  I think that tells you how seriously each one of you take this task and we look forward to working with you in the coming month.  My contact information is in your meeting notebook so if you have any specific issues that you want to discuss with me privately, you know where I can be reached.  Its in the list of membership. 

I also want to thank the NIST scientists for their efforts to make this meeting a success.  I’m sure, its not just from the weight of the paper that you received but I think from the quality of the work that they have been working very, very hard and, as acting Director of NIST I’m certainly extremely proud of the effort that they have put in.  So, thank you, great job.
Our next plenary session of the TGDC is scheduled for April 20 and 21.  I actually have a conflict that has developed and I was wondering if instead of 20 and 21, which is a Wednesday and Thursday, if we could move it to 21 and 22 which is Thursday and Friday.  Would that be a problem?  Dr. Williams.
DR. WILLIAMS:	(Cannot hear what is being said).
DR. SMERGIAN:	Well, that sounds like maybe we’ll have to leave it alone.  Is everybody going to be available so far as you know?
MALE SPEAKER 49:	Wednesday and Thursday?
DR. SMERGIAN:	Okay.  Well, then we’ll stick with that.  Again, thank you to the TGDC members and thank you to all the other folks who participated and listened in and thank you to Commissioner Martinez and Commissioner DeGregorio for sitting in.  With that I adjourn this meeting of the Technical Guidelines Development Committee.  Thank you very much.
(END OF AUDIOTAPE)
	*		*		*		*		*

	CERTIFICATE OF AGENCY

	I, Carol J. Schwartz, President of Carol J. Thomas Stenotype Reporting Services, Inc., do hereby certify we were authorized to transcribe the submitted cassette tapes, and that thereafter these proceedings were transcribed under our supervision, and I further certify that the forgoing transcription contains a full, true and correct transcription of the cassettes furnished, to the best of our ability.

					_____________________________
						  CAROL J. SCHWARTZ
							 PRESIDENT