% -*-text-*-
% $Id: Documentation.tex 509 2009-03-25 16:00:13Z dflater $

\documentclass[11pt]{article}
\usepackage[pdftex]{graphicx}
\usepackage{sssubsection}
% \usepackage{timestamp}
\usepackage{draftfoot}
\usepackage{blockquote}
\usepackage{ltxtable}
\usepackage{lscape}
\usepackage[pdftex,bookmarks,colorlinks=true,citecolor=blue,linkcolor=blue,urlcolor=blue]{hyperref}

\oddsidemargin      0 in
\evensidemargin     0 in
\topmargin          0 in
\headheight         0 in
\headsep            0 in
\textheight         9 in
\textwidth        6.5 in
\parindent          0 pt
\parskip            6 pt plus 6 pt
\partopsep         -6 pt

\pdfpagewidth     8.5 in
\pdfpageheight     11 in
\pdfcompresslevel   9

\setcounter{tocdepth}{2}

\renewcommand{\tabularxcolumn}[1]{m{#1}}
\renewcommand{\sectionautorefname}{Section}
\renewcommand{\subsectionautorefname}{Section}
\renewcommand{\subsubsectionautorefname}{Section}
\renewcommand{\paragraphautorefname}{Section}

% Make autorefs to constraints work.  Changing \theenumi gets the
% autorefs to use Roman numbers but it also affects the behavior of
% the enumerate environment.
\renewcommand{\Itemautorefname}{Constraint}
\renewcommand{\theenumi}{\Roman{enumi}}

\newenvironment{constraints}{\begin{list}{\Roman{enumi}.}{
\leftmargin 0.4 in
\labelwidth 0.4 in
\usecounter{enumi}}}{\end{list}}

% Work around the redefined \theenumi.
\renewenvironment{enumerate}{\begin{list}{\arabic{enumi}.}{
\leftmargin 0.25 in
\labelwidth 0.25 in
\usecounter{enumi}}}{\end{list}}

\newtheorem{assertion}{Assertion}
\newcommand{\FIXME}{{\Huge FIXME}}
\newcommand{\capsep}{\vspace{6pt}}
\newcommand{\bigcapsep}{\vspace{10pt}}


\begin{document}

\title{Votetest:  Voting system logic testing for the VVSG-NI}
\author{Draft --- Version 1.0}
% \date{\timestamp}
\date{April 1, 2009}
\maketitle


% \section*{Notes to reviewers}
%
% % \subsection{Welcome to Votetest!}
%
% What you have received is an early review version---essentially a first
% draft---of Votetest.  Your comments on this early review version will shape
% future developments to better meet the needs of Voting System Testing
% Laboratories.
%
% Votetest consists of this documentation and a set of data files that are
% bundled with it.  This package is distributed as an archive file in either
% zip or compressed tar format.  A high-level description of the contents of
% the Votetest distribution is given in \autoref{WhatItIs}; a more detailed
% description that identifies the various data files is given in
% \autoref{DistContents} and \autoref{FileListing}.
%
% % \subsection{Feedback}
%
% Comments and suggestions on Votetest should be directed to David Flater,
% NIST, by e-mail at dflater@nist.gov.  If you wish to embed comments within
% this documentation, you may do so using the free Adobe Reader software
% available from Adobe.  The following detailed instructions for commenting the
% PDF file are current as of 2009-03-25 and Adobe Reader version 9.1.0.
% Versions 8.1.X are also usable.
%
% \begin{enumerate}
%
% \item Ensure that Adobe Reader is installed on your computer.  Adobe Reader
% may be obtained from \url{http://get.adobe.com/reader/}.
%
% \item Open Documentation.pdf in Adobe Reader.
%
% \item There should be a menu on the toolbar labelled \textbf{Comment} or
% \textbf{Review \& Comment}.  Select \textbf{Show Comment \& Markup Toolbar}
% from that menu to get a new toolbar that includes the \textbf{Sticky
% Note} tool, the \textbf{Text Edits} tool, and others.  (These tools can also
% be accessed via Tools $\rightarrow$ Comment \& Markup.)
%
% \item To insert a comment someplace in the document, go to that page and use
% the \textbf{Sticky Note} tool.  Once the text of the comment has been
% entered, the yellow note icon can be dragged to place it near the text in
% question.
%
% \item To indicate desired textual changes, use the \textbf{Text Edits}
% tool to insert, delete, or replace text.
%
% \item Save your changes using File $\rightarrow$ Save.
%
% \end{enumerate}
%
% Thanks for reviewing!
%
% \newpage


% \setcounter{section}{-2}
\section*{Legal notices}

\subsection*{Test materials disclaimer}
\label{disclaimertxt}

This document and associated files have been prepared by the National Institute
of Standards and Technology (NIST) and represent draft test materials for the
Election Assistance Commission's next iteration of the VVSG\@.  It is a
preliminary draft and does not represent a consensus view or recommendation
from NIST, nor does it represent any policy positions of NIST.


\subsection*{Software disclaimer}
\label{COPYING}

This software was developed at the National Institute of Standards and
Technology by employees of the Federal Government in the course of
their official duties.  Pursuant to Title 17 Section 105 of the United
States Code this software is not subject to copyright protection and
is in the public domain.  This software is an experimental system.
NIST assumes no responsibility whatsoever for its use by other
parties, and makes no guarantees, expressed or implied, about its
quality, reliability, or any other characteristic.  We would
appreciate acknowledgement if the software is used.

This software can be redistributed and/or modified freely provided
that any derivative works bear some notice that they are derived from
it, and any modified versions bear some notice that they have been
modified.


\subsection*{Commercial products disclaimer}

Specific computer hardware and software products are identified in this paper
to support reproducibility of results.  Such identification does not imply
recommendation or endorsement by the National Institute of Standards and
Technology, nor does it imply that the products identified are necessarily
the best available for the purpose.


\newpage

\tableofcontents
\listoffigures
\listoftables


\newpage

\section{Introduction}


\subsection{What Votetest Is}
\label{WhatItIs}

The Votetest distribution, or simply ``Votetest'' for short, is a package of
public domain data, software, and documentation that the National Institute of
Standards and Technology (NIST) is developing.  Its purpose is to provide
conformance test materials for use by Voting System Testing Laboratories.
These test materials are used to assess conformity to the Voluntary Voting
System Guidelines (VVSG) \cite{VVSGNG}.

Votetest defines abstract tests that exercise every phase of the voting
process from election definition through report generation.  The scope of
functionality that they cover to some extent is therefore quite broad.  The
abstract tests are ``realized'' according to the specifics of the voting
system being tested.

In addition to the abstract tests, Votetest includes the expected results for
each test, a means to derive the expected results for newly developed tests,
a test generator, and detailed documentation.


\subsection{What Votetest Is Not}

Votetest CANNOT simply be plugged into a voting system to obtain a verdict on
conformity.  The tests that are defined abstractly by Votetest must be
translated into concrete tests that can actually be run on a given voting
system.  This task requires the same expertise and diligence that test labs
have employed in the past when developing their own tests from scratch.

Votetest, by itself, DOES NOT provide complete coverage of the VVSG\@.
Votetest is only one tool that is used in one part of the conformity
assessment process.  It is designed not to be used in isolation, but rather
to complement the other testing activities, which include the physical
configuration audit, documentation and design reviews, electromagnetic
compatibility and environmental testing, security reviews, usability and
accessibility assessments, the volume test (mock election), and the
evaluation of reliability, accuracy, and misfeed rates.  Test labs should
consult Part~3 of the VVSG \cite{VVSGNG} regarding the full scope of testing
to be performed.

Votetest DOES NOT address ``fitness for use.''  Its scope is strictly
limited to conformity assessment.  It is not a substitute for Voting
System Testing Laboratories' diligent assessments of the full scope of
voting system functionality (including vendor-specific functionality),
nor does it obviate the need for acceptance testing by jurisdictions.


\subsection{Structure of this document}

\autoref{Background} provides additional background of general interest.
\autoref{TestMaterials} provides detailed technical documentation on the
portions of Votetest that are essential to performing conformity assessment.
\autoref{Schema} and \autoref{TestSuite} provide detailed technical
documentation on the portions of Votetest that could be used to develop
additional test cases.  Finally, \autoref{UseCase} walks through the process
of defining a new test case based on an example election.


\section{Background}
\label{Background}

By authorization of the 2002 Help America Vote Act (HAVA), NIST is assisting
the Election Assistance Commission (EAC) with the implementation of Voluntary
Voting System Guidelines (VVSG) for states and local governments conducting
Federal elections.  The EAC's Technical Guidelines Development Committee
(TGDC) in collaboration with NIST researchers has developed a draft of the
next iteration of the VVSG\@.  The draft document is a set of detailed
technical requirements addressing core requirements, human factors, privacy,
security, and transparency of the next generation of voting systems.  The EAC
plans to issue the next VVSG after receiving and reviewing public comments.

NIST is developing a set of uniform public test suites to be used as part of
the EAC's Testing and Certification Program.  Test labs will be able to use
these freely available test suites to help determine that VVSG requirements
are met by voting systems.  The test suites address human factors, security
and core functionality requirements for voting systems as specified in the
VVSG\@.  Use of the public test suites will produce consistent results and
promote transparency of the testing process. The test suites can also assist
manufacturers in the development of conforming products by providing precise
test specifications.  Also, they will help reduce the cost of testing since
each test lab would no longer need to develop its own test suites.  Finally,
a uniform set of public test suites can increase election officials' and
voters' confidence that voting systems conform to VVSG requirements.


\section{Test materials}
\label{TestMaterials}

\subsection{Overview of test materials}
\label{DistContents}

This is the documentation for the test materials portion of the Votetest
distribution.  Its intended audience is test labs accredited by the Election
Assistance Commission to perform voting system certification testing.  The
test lab is assumed to have competence with the following:

\begin{itemize}
\item The Voluntary Voting System Guidelines (VVSG) \cite{VVSGNG};
\item Data modelling;
\item Structured Query Language (SQL) \cite{SQL}.
\end{itemize}

Votetest includes the following components:

\begin{itemize}

\item The data model (\autoref{DataModel}) documents the world view
inherent in the schema and test suite.  It is general enough to
support arbitrary combinations of all of the voting variations defined
in the VVSG\@.

\item The basic schema (\autoref{Schema1}) is an SQL realization
of the data model.

\item The basic test suite (\autoref{BasicTestSuite}) is a set of abstract
test cases expressed in SQL using the basic schema.  The test cases are
delivered as separate files in the Votetest distribution.  These tests trace
primarily to the requirements of \cite[Part~1 Chapter 7]{VVSGNGdraft};
traceability details will be provided in \autoref{ReqTrace}.  However,
because they exercise every phase of the voting process from election
definition through report generation, the scope of functionality that they
cover to some extent is quite broad.

\item The expected results (\autoref{ExpectedResults}) are text files
providing the vote totals that a conforming voting system should produce
for each test case.

\item The advanced schema (\autoref{Schema}) is an SQL realization of the logic
model of the VVSG \cite[Part~1 Section~8.3]{VVSGNGdraft}, which specifies the
results that voting systems are required to report.

\item The advanced test development environment (\autoref{TestSuite}) includes
the infrastructure and tooling used to develop test cases and determine
their expected results.

\end{itemize}

While Votetest includes both basic and advanced materials, only the data model
(\autoref{DataModel}), basic schema (\autoref{Schema1}), basic test suite
(\autoref{BasicTestSuite}) and expected results (\autoref{ExpectedResults}) are
essential to the conformance testing process.  \autoref{HowToUse} explains
in more detail how Votetest is used in that process.


\subsection{File listing}
\label{FileListing}

The Votetest distribution is provided as a zip file named
votetest-\emph{YYYYMMDD}.zip and alternately as a compressed tar file named
votetest-\emph{YYYYMMDD}.tar.bz2, where \emph{YYYYMMDD} is a sequence of
digits indicating the date of the release.  The files contained in the
distribution are described in \autoref{FileListTable}.

\begin{table}[htbp]
\caption{Basic files in the Votetest distribution}
\capsep
\begin{tabularx}{\textwidth}{|l|X|l|}
\multicolumn{1}{c}{\bf Files} & \multicolumn{1}{c}{\bf Description} & \multicolumn{1}{c}{\bf Details} \\ \hline

disclaimer.txt & Text file containing the test materials disclaimer. & \hyperref[disclaimertxt]{Legal notices} \\ \hline
COPYING & Text file containing the software disclaimer. & \hyperref[COPYING]{Legal notices} \\ \hline

doc/ & Subdirectory containing the source and PDF of this documentation. & N/A \\ \hline

1-basic-\emph{description}.sql & SQL, abstract test cases of the basic test suite (92 files). & \autoref{BasicTestSuite} \\ \hline

sample\_output/ & Subdirectory containing text files that provide the expected results for each test case and each integrity check (106 files). & \autoref{ExpectedResults} \\ \hline
sample\_output\_kill-overvotes/ & Same, but with overvoting suppressed (106 files). & \autoref{ExpectedResults} \\ \hline

ChangeLog & Text file containing the change log for the Votetest distribution. & N/A \\ \hline

... \emph{other files} ... & Files pertaining to the advanced test development environment. & \autoref{AdvancedFileListTable} \\ \hline

\end{tabularx}
\label{FileListTable}
\end{table}


\subsection{How to use Votetest}
\label{HowToUse}

The steps listed in the following subsections fit within the context of the
overall conformity assessment process that is described in \cite[Part~3
Chapter~2]{VVSGNGdraft}.  Thus, in practice, the steps will not necessarily
follow directly one to the next as is implied here, because there may be
other conformity assessment activities occurring in between or parallel to
them.


\subsubsection{Determine relevant voting variation classes---system level}

The Technical Data Package received from the manufacturer is required to
include an \emph{implementation statement}, which is specified by the
Conformance Clause of the VVSG \cite[Part~1 Section~2.4]{VVSGNGdraft}.  This
implementation statement includes the manufacturer's classification of the
voting system as a whole and of the different devices of which it is
comprised.  For Votetest, the relevant voting variation classes are
determined by examining the implementation statement.  (The completeness and
correctness of the manufacturer's classification should have been established
in a preceding documentation and design review; see \cite[Part~3
Req.~4.2-C]{VVSGNGdraft}\@.)  The system and device classes that the
manufacturer may reference are defined in the Conformance Clause \cite[Part~1
Section~2.5]{VVSGNGdraft}.

Usually, multiple classes will apply to a given voting system or voting
device.  For example, if a DRE supports both N-of-M and ranked order voting
variations, it will be classified as both an \emph{N-of-M voting device} and
a \emph{Ranked order voting device} in addition to being classified as a
\emph{DRE}\@.  This multiple classification allows multiple sets of tests to
apply so that all of the supported functionality will be tested.

At this stage, what is relevant are the system-level classes.  If the system
as a whole does not support a particular voting variation, then support for
that variation in isolated devices is irrelevant.


\subsubsection{Determine applicable tests}

\autoref{Level1Table} (coming up in \autoref{BasicTestSuite}) lists the
tests in the basic Votetest test suite and indicates the voting variations
that are used by each test.  The set of tests applicable to a given voting
system is the set of all tests that \textbf{do not use} any voting variations
that the voting system \textbf{does not support}.  In other words, every test
is applicable unless it is specifically excluded, and a test is excluded only
if it uses a voting variation that, according to the reviewed and accepted
implementation statement, the system does not implement.

For example, if a voting system \emph{did not} support ranked order voting,
the following tests would be excluded:

\begin{tabular}{l}
 1-basic-NoBallots-RankedOrder.sql \\
 1-basic-RankedOrder-1.sql \\
 1-basic-RankedOrder-2.sql \\
 1-basic-StraightParty-RankedOrder.sql \\
 1-basic-Primary-RankedOrder.sql \\
 1-basic-BallotRotation-RankedOrder.sql \\
 1-basic-AbsenteeByCategory-RankedOrder.sql \\
 1-basic-Cumulative-RankedOrder.sql \\
 1-basic-SplitPrecinct-RankedOrder.sql \\
 1-basic-RankedOrder-WriteIns.sql \\
 1-basic-RankedOrder-NofM.sql \\
 1-basic-RankedOrder-Provisional.sql \\
 1-basic-AbsenteeBySpecialPrecinct-RankedOrder.sql
\end{tabular}

There is a special case regarding absentee voting that will be addressed in
\autoref{BasicTestSuite}.


\subsubsection{Multiply tests for different interfaces and devices}

At this point the test lab has the beginnings of a test plan with a list
of the tests that are applicable to the voting system as a whole.  That
plan must now be expanded down to the device level.

When a voting system offers multiple paths by which voting can occur, such as
a touchscreen interface and an audio ballot on one device plus optical
scanning of paper ballots by another device, each of these paths must be
tested.  It is entirely possible for a failure to be specific to a particular
user interface.  For example, some controls that can be activated using a
touchscreen might be incorrectly skipped over if an input device with a
navigational model (next option, previous option) is used.

Several different strategies could be used to divide or replicate the testing
among the available interfaces and devices.  However, since most tests in the
basic test suite are small by design, with relatively few ballots, the
recommended approach is to repeat each test on each device and interface
separately.  For large tests such as the volume test (mock election), one
would instead divide the ballots among the available devices and interfaces
using a distribution that imitates a realistic election.

Absentee voting is again a special case.  In systems where absentee ballots
follow a separate path from other ballots (e.g., a dedicated central count
optical scanner), it makes sense to repeat tests over the absentee interface.
However, the basic test suite also contains tests specifically designated for
absentee voting, that mix absentee ballots with non-absentee ballots.  These
designated tests would be executed using both the absentee and non-absentee
paths at the same time, not repeated on each path separately.

As noted in \cite[Part~1 Section~6.2]{VVSGNGdraft}, it is not necessarily the
case that every device in the system would support every voting variation
claimed at the system level.  Thus there may be cases where it is not
possible to multiply certain tests onto certain devices.  However, other
devices that do pass those tests must be present to enable satisfaction of
the system-level claim.



\subsubsection{Perform testing}

Once the test plan has been approved, work on implementing the plan can
begin.  The steps that the test lab must do for each applicable Votetest test
are summarized in \autoref{PerformTestingFigure} and explained in more detail
in the following subsections.


\paragraph{Translate test from Votetest model to voting system}

Since interfaces to voting systems vary widely, the test lab must use the
basic test suite as input from which to generate system-specific tests.  For
each applicable test, the test lab translates the abstract, SQL version
included in the basic test suite (\autoref{BasicTestSuite}) into a concrete
test for the voting system under test, thereby ``realizing'' the test.  The
data model (\autoref{DataModel}) and basic schema (\autoref{Schema1}) are
essential to understanding the abstract tests so that they can be
realized correctly.

\autoref{SQLFigure} shows an example of a Votetest test expressed in SQL\@.
It specifies both the election definition and the ballots and votes to be
cast.

Things to be determined for the voting system under test would include:

\begin{itemize}

\item Initial state:  Determine how the voting system should be set and reset
to a starting state prior to each test.

\item Election definition:

\begin{itemize}

\item Read the Voting Equipment User Documentation supplied by the
manufacturer to understand how election definition is performed using the
Election Management System.

\item Determine how the definitions of the ballot styles, contests, choices,
and reporting contexts used in the Votetest basic test suite would be
implemented in the voting system under test.

\end{itemize}

\autoref{EMSFigure} shows an example of entering part of the election
definition from \autoref{SQLFigure} into a hypothetical voting system.  In an
actual voting system, entering the complete election definition might require
many separate interactive steps, such as creating the election, enumerating
political parties, creating
precincts, creating contests, creating choices within those contests, adding
ballot text, setting options, etc.  Additionally, the test lab may need to
synthesize content for system-specific details that are not specified by
Votetest.  This example only shows the creation of a contest.

\item Voting:  Determine how the pattern of votes used in each test would be
cast in the voting system under test.  The system may support numerous
different voting interfaces, each of which would have a different process for
casting votes.  For example, if a DRE is to be tested, scripts for ``test
voters'' to follow must be produced.  The scripts instruct the test voters on
what votes to cast using the electronic interface of the DRE\@.  If an
optical scanner is to be tested, physical test ballots must be produced.
This might be accomplished by handing a similar script to test voters to
instruct them on how to fill in the paper ballots, or the test engineer might
simply mark the ballots personally.

\autoref{SimBallotFigure} shows an example of how the ballot with ID 2 in
\autoref{SQLFigure} would translate for the precinct count optical scan
interface of the hypothetical voting system.  \autoref{SimScriptFigure} shows
an example of how the same ballot would translate for the DRE interface of
the hypothetical voting system.

\item Reporting:  Determine how to generate pre-election audit reports,
system readiness audit reports, in-process audit reports, and vote data
reports (i.e., reports of ballot counts and vote totals) in the voting system
under test.

\end{itemize}

\begin{figure}[hbp]
\caption{How to use a test}
\capsep
\includegraphics[width=6.5in]{Basic.pdf}
\label{PerformTestingFigure}
\end{figure}

\begin{figure}[p]
\caption{Example Votetest test}
\capsep
\includegraphics[width=6.5in]{1-basic-NofM.pdf}
\label{SQLFigure}
\end{figure}

\begin{figure}[htbp]
\caption{Example EMS interaction}
\capsep
\includegraphics[width=6.5in]{EMS.png}
\label{EMSFigure}
\end{figure}

\begin{figure}[htbp]
\caption{Example optical scan ballot}
\capsep
\centering
\includegraphics[width=4in]{ExampleBallotVoted.pdf}
\label{SimBallotFigure}
\end{figure}

\begin{figure}[htbp]
\caption{Example DRE voting script}
\capsep

\emph{If at any step you make a mistake performing the \textbf{Action for
tester}, go directly to the Botched Test Checklist.}
\capsep

\emph{If at any step the DRE does not respond as indicated under
\textbf{Expected DRE response}, go directly to the Anomaly
Checklist.}
\capsep

\centering
\begin{tabularx}{6 in}{|X|X|}
\multicolumn{1}{c}{\bf Action for tester} & \multicolumn{1}{c}{\bf Expected DRE response} \\ \hline
1. Insert the card that the poll worker gave you into the slot on the
front of the DRE\@. & The DRE should show the text ``Ballot activated''
followed by instructions to voters. \\ \hline
2. Touch NEXT on the touch-screen. & The DRE should advance to the PARKING COMMITTEE contest. \\ \hline
3. Touch the box to the left of \textbf{Beeso Tu}. & The box should turn
green. \\ \hline
4. Touch the box to the left of \textbf{Tayra Tree}. & The box should turn
green. \\ \hline
5. Touch NEXT on the touch-screen. & The DRE should advance to the summary
screen. \\ \hline
6. Touch CAST BALLOT on the touch-screen. & The DRE should display ``Please
remove the card and deposit it in the shoe box near the exit.'' and ``Thank you for voting.'' \\ \hline
7. Remove the card. & The DRE should return to its welcome screen, which
shows the text ``Precinct 1'' and ``Please insert card.'' \\ \hline
\end{tabularx}
\label{SimScriptFigure}
\end{figure}

\clearpage
\paragraph{Execute test}

For each test case, the test lab executes the ``realized'' test case within
the voting system under test following the general test template specified in
\cite[Part~3 Section~5.2.2.1]{VVSGNGdraft}:

\begin{enumerate}
\item Establish initial state (clean out data from previous tests, verify
resident software/firmware);
\item Program election and prepare ballots and/or ballot styles;
\item Generate pre-election audit reports;
\item Configure voting devices;
\item Run system readiness tests;
\item Generate system readiness audit reports;
\item Precinct count only:
  \begin{enumerate}
  \item Open poll;
  \item Run precinct count test ballots; and
  \item Close poll.
  \end{enumerate}
\item Run central count test ballots (central count / absentee ballots only);
\item Generate in-process audit reports;
\item Generate data reports for the specified reporting contexts;
\item Inspect ballot counters; and
\item Inspect reports.
\end{enumerate}

This template essentially puts the voting system through the technical paces
of a very small election for each test.  Although the election definition is
simple and the number of ballots small, the process from beginning to end
exercises all of the voting system functions that would be used in an
election.  The surrounding procedures and environment are, of course, quite
different, and vastly simplified in the laboratory testing environment, so it
is not a mock election in the sense that the volume test is.

Also see \cite[Part~3 Section~2.5]{VVSGNGdraft} regarding the test
conditions, test fixtures, test data requirements, and test practices that
apply.


\paragraph{Compare observed and expected results}
\label{ExpectedResults}

As part of the test execution, the voting system under test is instructed to
generate vote data reports.  These contain the ``observed results'' with
respect to ballot counts and vote totals.  \autoref{ExampleObservedResults}
shows an example report from the hypothetical voting system.

The ``expected results'' for all of the test cases are saved as text files in
two subdirectories of the Votetest distribution.  The subdirectory
sample\_output contains the results expected from the test cases as written,
while the subdirectory sample\_output\_kill-overvotes contains the results
expected from the test cases with overvotes changed to undervotes.

Each results file has the following organization:

\begin{itemize}
\item Test header with timestamp and description of the test case.
\item Some details that are not relevant to conformity assessment:
\begin{itemize}
\item Results of data integrity checks.  These are checks of the integrity
of the test cases themselves and do not correspond to requirements on
voting systems.
\item View materialization log.
\end{itemize}
\item Reports, for one or more \hyperref[ReportingContext]{ReportingContexts},
that include the vote totals that voting systems are expected to produce.
\item Test footer including the report total volume and timestamp.
\end{itemize}

An example is shown in \autoref{ExampleExpectedResults}.  Comparing the
``observed results'' in \autoref{ExampleObservedResults} and the ``expected
results'' in \autoref{ExampleExpectedResults}, it is clear that the
quantities reported for candidate vote totals are identical, even though the
form of the reports is somewhat different.  The test lab would find no
discrepancy in those results.  However, the observed results shown in
\autoref{ExampleObservedResults} are not in and of themselves sufficient to
satisfy all of the VVSG requirements on reporting.  For example, \cite[Part~1
Req.~7.8.3.2-A through D]{VVSGNGdraft} specify additional information that
the voting system under test must provide in some report if the system is to
be found conforming.

The other reports generated as part of the test (pre-election reports,
etc.)\ would also be inspected to look for anomalies and to assess conformity to the
L\&A testing and auditability requirements of the VVSG\@.  Similarly, any
unexpected behavior of the voting system observed by the tester would be
assessed by the resident expert (possibly but not necessarily the same
person) to determine if a nonconformity was demonstrated.

Some notes and cautions apply:

\begin{enumerate}

\item Many test cases contain overvotes.  However, in some voting systems,
overvoting is prevented.  Votetest handles this possibility through test
transformation.  For each ballot that overvotes a particular contest, an SQL
script (\hyperref[AdvancedFileListing]{Infrastructure-KillOvervotes.sql})
deletes that ballot's votes in that contest, with the effect that the contest
is undervoted instead of overvoted.  Expected results are provided both for
the test cases as originally written (in sample\_output) and for the test
cases transformed to remove overvotes (in sample\_output\_kill-overvotes).

\item For {\em Ballot rotation} tests, the test lab must check to make
sure that the voting system does in fact produce and utilize rotated
ballots as specified in the VVSG \cite{VVSGNG}.  The assignment of
ballot choices to specific ballot positions is abstracted out of the
Votetest data model, so the rotation behavior is not represented in
the ``expected results.''

\item Ranked order logic is not normative.  The algorithm used to derive the
``expected results'' for ranked order contests is only one example of
conforming behavior.  This algorithm is not recommended or endorsed by the
National Institute of Standards and Technology for use in elections and it is
probably not the best algorithm available for the purpose.  It is used in
Votetest only to provide output for comparison in simple cases where the
implementation-dependent details have no impact.
\end{enumerate}

\begin{figure}[htbp]
\caption{``Observed results'' for example}
\capsep
\begin{center}
NO-SUCH JURISDICTION \\
GENERAL ELECTION \\
MAY 16, 2008 \\
\capsep
REPORT GENERATED AT 2008-05-16 16:12:23 \\
\capsep \capsep
\begin{tabular}{lrrrrr}
 & Total & \% & Election Day & Early & Provisional \\
Precincts Counted & 1 & 100.00 & & & \\
Registered Voters & 9 & & & & \\
Ballots Cast / Turnout & 9 & 100.00 & 9 & 0 & 0 \\
& & & & & \\ \hline
\multicolumn{6}{l}{PARKING COMMITTEE (Vote for at most 2)} \\ \hline
FOWER, CAR TAY & 4 & 40.00 & 4 & 0 & 0 \\
TREE, TAYRA & 3 & 30.00 & 3 & 0 & 0 \\
TU, BEESO & 2 & 20.00 & 2 & 0 & 0 \\
WON, OONA & 1 & 10.00 & 1 & 0 & 0 \\
ZAYRO, NADA & 0 & 0.00 & 0 & 0 & 0 \\
Write-Ins (combined) & 0 & 0.00 & 0 & 0 & 0 \\
Overvotes & 2 & & 2 & 0 & 0 \\
Undervotes & 6 & & 6 & 0 & 0
\end{tabular}
\end{center}
\label{ExampleObservedResults}
\end{figure}

\begin{figure}[p]
\caption{``Expected results'' for example}
\capsep
\includegraphics[width=6.5in]{ExampleExpectedResults.pdf}
\label{ExampleExpectedResults}
\end{figure}

\clearpage
\subsection{Applicability to VVSG 2005}
\label{Usefor05}

The 2005 Voluntary Voting System Guidelines (VVSG 2005) \cite{VVSG05} have
several important differences from the next VVSG \cite{VVSGNG} that impact
the application of Votetest.

\begin{itemize}

\item VVSG 2005 does not include a logic model \cite[Part~1
Section~8.3]{VVSGNGdraft} to define precisely what values are expected to be
reported for votes, overvotes, and undervotes.  While the draft of the next
VVSG is believed to be consistent with the intent of VVSG 2005, it explicitly
rules out certain alternative interpretations that might have been found
compliant to the less precise wording of VVSG 2005.  Use of Votetest to
assess conformity to VVSG 2005 might therefore require clarification and/or
interpretation of requirements by the EAC.

\item VVSG 2005 does not include specific functional requirements to define
what it means to ``support'' a given voting variation such as N-of-M voting.
The manufacturer is only required to document \emph{how} a voting variation
is supported \cite[Volume~I Section~2.1.7.2]{VVSG05}.  This weakens the
traceability of Votetest test cases to requirements and may require
additional EAC interpretation decisions.

\item Votetest is structured on the assumption that manufacturers would
formally state their claims of having implemented particular voting
variations in an implementation statement \cite[Part~1
Section~2.4]{VVSGNGdraft} and that both the testing and the certification of
the affected systems would be scoped according to those claims.  The VVSG
2005 conformance clause requires manufacturers to ``identify optional
features and capabilities supported by the voting system'' \cite[Volume~I
Section~1.6.4]{VVSG05}, but those optional features and capabilities are not
required to be drawn from the list of voting variations defined in the next
VVSG\@.  To the extent that they are not, the applicability of Votetest test
cases will be subject to interpretation.

\item VVSG 2005 identifies ``partisan offices,'' ``non-partisan offices,''
``primary presidential delegation nominations,'' and ``recall issues, with
options'' as voting variations.  In the drafting of the next VVSG, these were
all found to be subsumed by other voting variations with appropriately
configured ballot text, so they are not called out explicitly in the draft
VVSG\@.  A test lab wishing to test specifically to the 2005 list of voting
variations would need to create new tests and/or modify existing ones to
exercise these special cases.

\end{itemize}

There is no technical incompatibility that would outright prevent the use of
any of the Votetest test cases in a conformity assessment for VVSG 2005,
provided that the draft next VVSG accurately reflects the intent of the
requirements of VVSG 2005.  A focused effort to develop test cases for VVSG
2005 instead of the next VVSG would not necessarily arrive at a different set
of test cases.


\subsection{Data model}
\label{DataModel}

\begin{figure}
\caption{Vote data model for core requirements}
\capsep
\includegraphics[width=6.5in]{VoteDataModel-20070305-diagram.pdf}
\label{diagram}
\end{figure}

The data model for voting system core requirements is described in
\autoref{diagram} by a Unified Modeling Language (UML) class diagram
\cite{UML15}.  Following sections explain the diagram.


\subsubsection{Assumptions}

All entities in this data model are implicitly scoped by an election.
It is assumed that different elections are stored in different
databases, and any reuse of definitions from one
election to another is accomplished by copying over the relevant data.

This data model is constructed from an integrated, top-level
viewpoint.  In practice, different portions of the system will deal
with only a portion of the data at any given time.  It is expected
that test labs will project and extract data from the integrated schema
as needed to support these limited viewpoints for testing.

The results of tabulation and reporting are derived from the content
of the data model, but those results are themselves outside the scope
of the model.


\subsubsection{POD (Plain Old Data) types}

\begin{description}

\item[BallotCategory] \label{BallotCategory} (enum) Arbitrary tag that may be
applied to \hyperref[Ballot]{Ballots}; e.g., Early, Regular, InPerson,
Absentee, Provisional, Challenged, NotRegistered, WrongPrecinct,
IneligibleVoter.  Categories are jurisdiction-defined but are likely to
include several classes of provisional.

\item[Boolean]  Normal true/false data type.

\item[ContestCountingLogic] (enum) N-of-M, Cumulative, Ranked order,
or Straight party selection.  (1-of-M is a special case of N-of-M\@.)
The tabulation logic for a straight party selection
\hyperref[Contest]{Contest} is implicitly 1-of-M, but with
side-effects for other \hyperref[Contest]{Contests}.

\item[NaturalNumber]  Integer greater than zero.

\item[Text] Normal character string.

\item[WholeNumber] Integer greater than or equal to zero.

\end{description}


\subsubsection{Classes}
\label{Classes}

\paragraph{Ballot}
\label{Ballot}

The undefined primitive in all elections.  The \hyperref[Contest]{Contests} that appear on
a particular \hyperref[Ballot]{Ballot} are defined by its
\hyperref[BallotStyle]{BallotStyle}.  The applicable
\hyperref[ReportingContext]{ReportingContexts} include all those
specified for its \hyperref[BallotStyle]{BallotStyle}, but additional
\hyperref[ReportingContext]{ReportingContexts} may be specified for the individual \hyperref[Ballot]{Ballot}.

Attributes of \hyperref[Ballot]{Ballot}:

\begin{description}

\item[Categories]  Arbitrary, jurisdiction-defined tags applied to the
\hyperref[Ballot]{Ballot}; e.g., Early, Regular, InPerson,
Absentee, Provisional, Challenged, NotRegistered, WrongPrecinct,
IneligibleVoter.

\item[Accepted]  True if the \hyperref[Ballot]{Ballot} should be counted, false if not (e.g.,
for a provisional \hyperref[Ballot]{Ballot} that was not accepted).

\end{description}


\paragraph{BallotStyle}
\label{BallotStyle}

Set of \hyperref[Contest]{Contests} and
\hyperref[ReportingContext]{ReportingContexts} that is inherited by
all \hyperref[Ballot]{Ballots} of that style.  Depending on the type
of election and local practices, a jurisdiction would define a
separate \hyperref[BallotStyle]{BallotStyle} for each precinct, each
split within a precinct, and/or for each political party.

\cite{VVSGNG} defines both ``ballot style'' and ``ballot
configuration.''  This model assumes that the association between
styles and configurations is 1-to-1 and represents the combined
concept with \hyperref[BallotStyle]{BallotStyle}.

Attributes of \hyperref[BallotStyle]{BallotStyle}:

\begin{description}
\item[Name]  Human-readable identifier.
\end{description}


\paragraph{Choice}
\label{Choice}

One of the things you can vote on in a \hyperref[Contest]{Contest},
such as a candidate, a political party, or yes or no.
\hyperref[Choice]{Choice} is scoped by \hyperref[Contest]{Contest}, so
even if the same person runs as a candidate in two or more
\hyperref[Contest]{Contests}, those separate candidacies are
represented by separate \hyperref[Choice]{Choices}.
\hyperref[Choice]{Choices} do not map 1:1 with ballot positions---a
\hyperref[Choice]{Choice} uniquely identifies a candidate, while a
given ballot position might just be a generic write-in slot.

Attributes of \hyperref[Choice]{Choice}:

\begin{description}

\item[Name] Human-readable identifier.  (In a real system,
\hyperref[Choice]{Choices} could have complex descriptive data
associated with them that must be displayed to the user somehow, but
for logic testing purposes a single field suffices.)

\item[IsWriteIn] True if the \hyperref[Choice]{Choice} is a write-in
candidate, false if not.

\end{description}


\paragraph{Contest}
\label{Contest}

Subdivision of a \hyperref[Ballot]{Ballot} corresponding to a single
question being put before the voters, consisting of header text, a
discrete set of \hyperref[Choice]{Choices}, and possibly write-in opportunities.  It is
possible for a \hyperref[Contest]{Contest} to have zero
\hyperref[Choice]{Choices}, e.g., if there are no registered
candidates but write-ins are being accepted.
\hyperref[Choice]{Choices} corresponding to the candidates written in
would be added later.

Attributes of \hyperref[Contest]{Contest}:

\begin{description}

\item[Description]  Human-readable header text.

\item[CountingLogic] Identifies the tabulation method used for the
\hyperref[Contest]{Contest}.

\item[N] For CountingLogic other than ranked order, N is the maximum number of
votes that may be cast in the \hyperref[Contest]{Contest} by a given voter.
In an N-of-M \hyperref[Contest]{Contest}, the voter may cast at most one vote
for each \hyperref[Choice]{Choice}, so N is equal to the maximum number of
\hyperref[Choice]{Choices} that the voter may select without
overvoting.\footnote{The value of M, for N-of-M voting, is simply the number
of \hyperref[Choice]{Choices} associated with the \hyperref[Contest]{Contest}
and is not explicitly modelled.}  In a cumulative
\hyperref[Contest]{Contest}, there is no such constraint---the voter may cast
more than one vote for a given \hyperref[Choice]{Choice}.

Typically, N also is the number of winners for the
\hyperref[Contest]{Contest}, but not necessarily.  The voting system only
needs to gather votes and report the totals; the picking of winners may be an
external process impacted by election law, late-breaking judicial rulings,
etc.  However, for ranked order \hyperref[Contest]{Contests}, N \emph{is}
specifically the number of \hyperref[Choice]{Choices} to be elected,
and has no other meaning.

\item[MaxWriteIns] The number of ballot positions allocated for
write-ins; the maximum number of candidates that the voter may write
in.  Any value between zero and N is possible.  Zero would mean that
write-ins are not allowed; N would mean that write-ins are allowed; a
number in between would mean that write-ins must be approved and the
number of approved write-in candidates is less than N\@.

\item[Rotate] True if the ordering of \hyperref[Choice]{Choices}
within the \hyperref[Contest]{Contest} should be rotated, false if
not.

\end{description}


\paragraph{ElectionDistrict}
\label{ElectionDistrict}

Surrogate for real-world entity that may have associated
\hyperref[ReportingContext]{ReportingContexts}.  Any relationship
between \hyperref[ElectionDistrict]{ElectionDistricts} and
\hyperref[Contest]{Contests} appearing on the ballot in those districts is
implemented by \hyperref[BallotStyle]{BallotStyles}.


\paragraph{Party}
\label{Party}

Surrogate for real-world political party.

Attributes of \hyperref[Party]{Party}:

\begin{description}

\item[Name] Unique human-readable identifier.

\end{description}


\paragraph{Precinct}
\label{Precinct}

Surrogate for real-world entity that may have associated
\hyperref[ReportingContext]{ReportingContexts}.  Any relationship
between \hyperref[Precinct]{Precincts} and \hyperref[Contest]{Contests}
appearing on the ballot in those \hyperref[Precinct]{Precincts} is implemented
by \hyperref[BallotStyle]{BallotStyles}.

\paragraph{ReportingContext}
\label{ReportingContext}

Particular scope within which the system must be capable of generating
reports.  E.g., to support reporting at the precinct level, there must
be a \hyperref[ReportingContext]{ReportingContext} for each precinct.
The association between \hyperref[ReportingContext]{ReportingContexts}
and individual tabulators, precincts, election districts,
political parties, ballot categories, or other
arbitrary scopes of reporting is jurisdiction-defined and
jurisdiction-managed, mostly using
\hyperref[BallotStyle]{BallotStyles}.  The ways in which
\hyperref[ReportingContext]{ReportingContexts} overlap or include one
another is entirely determined by the assignment of multiple
\hyperref[ReportingContext]{ReportingContexts} to
\hyperref[BallotStyle]{BallotStyles} and \hyperref[Ballot]{Ballots}.

Attributes of \hyperref[ReportingContext]{ReportingContext}:

\begin{description}
\item[Name] Human-readable identifier.
\end{description}


\paragraph{Tabulator}
\label{Tabulator}

Surrogate for real-world entity, e.g., a DRE or optical scanner, that may
have associated \hyperref[ReportingContext]{ReportingContexts}.


\subsubsection{Named associations}

\paragraph{Affiliation}
\label{Affiliation}

Identifies the \hyperref[Party]{Party} to which a candidate claims
allegiance.  Does not necessarily have anything to do with
\hyperref[Endorsement]{Endorsements}.


\paragraph{Alias}
\label{Alias}

Identifies an alternative \hyperref[Choice]{Choice} that for
tabulation purposes is considered equivalent to a particular canonical
\hyperref[Choice]{Choice}.  \hyperref[Alias]{Aliases} will normally be
variant spellings of a candidate's name that appeared in write-in
positions.


\paragraph{Endorsement}
\label{Endorsement}

Identifies a voter response that would be implied by a straight party
vote for the endorsing \hyperref[Party]{Party}.  Does not necessarily
have anything to do with \hyperref[Affiliation]{Affiliation}.

Attributes of \hyperref[Endorsement]{Endorsement}:

\begin{description}

\item[Value] Analogous to \hyperref[VoterInput]{VoterInput} Value,
this is the vote recommended by the endorser.

\end{description}

In a 1-of-M or N-of-M \hyperref[Contest]{Contest}, an
\hyperref[Endorsement]{Endorsement} with Value = 1 would exist for the single
\hyperref[Choice]{Choice} or for each of the \hyperref[Choice]{Choices}
endorsed by the \hyperref[Party]{Party}.

In a Cumulative \hyperref[Contest]{Contest}, Value may take on values greater
than 1.  For example, if the \hyperref[Party]{Party} recommended that voters
cast two votes for the first \hyperref[Choice]{Choice} and one vote for the
second, an \hyperref[Endorsement]{Endorsement} with Value = 2 would exist for
the first \hyperref[Choice]{Choice} and an
\hyperref[Endorsement]{Endorsement} with Value = 1 would exist for the second
\hyperref[Choice]{Choice}.

In a Ranked order \hyperref[Contest]{Contest}, Value contains the ranking
that the \hyperref[Party]{Party} recommends that voters assign to each
\hyperref[Choice]{Choice}.


\paragraph{VoterInput}
\label{VoterInput}

The response that a particular \hyperref[Ballot]{Ballot} provides for
a particular \hyperref[Choice]{Choice}.

Attributes of \hyperref[VoterInput]{VoterInput}:

\begin{description}

\item[Value] The response of the voter in some ballot position.  The
absence of a response is equivalent to a Value of 0 except in ranked
order contests, where the behavior is implementation-defined.

\end{description}

In a 1-of-M or N-of-M \hyperref[Contest]{Contest}, a
\hyperref[VoterInput]{VoterInput} with Value = 1 would exist for the single
\hyperref[Choice]{Choice} or for each of the \hyperref[Choice]{Choices}
for which the voter voted.

In a Cumulative \hyperref[Contest]{Contest}, Value may take on values greater
than 1.  For example, if a voter cast two votes for the first
\hyperref[Choice]{Choice} and one vote for the second, a
\hyperref[VoterInput]{VoterInput} with Value = 2 would exist for the first
\hyperref[Choice]{Choice} and a \hyperref[VoterInput]{VoterInput} with
Value = 1 would exist for the second \hyperref[Choice]{Choice}.

In a Ranked order \hyperref[Contest]{Contest}, Value contains the ranking
that the voter assigns to each \hyperref[Choice]{Choice}.


\subsubsection{Constraints}
\label{Constraints}

\begin{constraints}

\item \label{VoterInputValueConstraint}
For N-of-M and straight party selection \hyperref[Contest]{Contests}, the Value
attribute of \hyperref[VoterInput]{VoterInput} or
\hyperref[Endorsement]{Endorsement} must be 1.  For cumulative \hyperref[Contest]{Contests}, $1 \leq \mbox{Value} \leq \mbox{N}$.  (Deliberately, there is no analogous constraint
for ranked order \hyperref[Contest]{Contests}.)

\item \label{MinimumNConstraint}
$\mbox{N} > 0$.

\item \label{MaxWriteInsConstraint}
$0 \leq \mbox{MaxWriteIns} \leq \mbox{N}$.

\item \label{StraightPartyContestParametersConstraint}
In \hyperref[Contest]{Contests} with CountingLogic = Straight
party selection, N = 1 and MaxWriteIns = 0.

\item \label{UnreportedBallotsConstraint}
Every \hyperref[Ballot]{Ballot} must be associated with at least
one \hyperref[ReportingContext]{ReportingContext} either directly or
through its \hyperref[BallotStyle]{BallotStyle}.  (Otherwise the \hyperref[Ballot]{Ballot}
would never be reported.)

\item \label{ExtraneousVotesConstraint}
 A \hyperref[Ballot]{Ballot} cannot have a
\hyperref[VoterInput]{VoterInput} for a \hyperref[Choice]{Choice} in a
\hyperref[Contest]{Contest} that does not appear in its
\hyperref[BallotStyle]{BallotStyle}.

\item \label{MoreThanOneStraightPartyContestConstraint}
A given \hyperref[BallotStyle]{BallotStyle} may contain at most
one \hyperref[Contest]{Contest} with CountingLogic = Straight party
selection.

\item \label{CircularEndorsementsConstraint}
A \hyperref[Contest]{Contest} with CountingLogic = Straight
party selection cannot be straight-party-votable (i.e., there can be
no \hyperref[Endorsement]{Endorsements} referring to its
\hyperref[Choice]{Choices}).

\item \label{StraightPartyChoiceNamesConstraint}
In \hyperref[Contest]{Contests} with CountingLogic = Straight
party selection, the Names of the \hyperref[Choice]{Choices} must
match the Names of \hyperref[Party]{Parties}.

\item \label{PartyNamesUniqueConstraint}
\hyperref[Party]{Party} names must be unique.

\item \label{AliasDoubleVotesConstraint}
A \hyperref[Ballot]{Ballot} may not simultaneously have
\hyperref[VoterInput]{VoterInput} for a \hyperref[Choice]{Choice} and
an \hyperref[Alias]{Alias} of that \hyperref[Choice]{Choice}.  (The
handling of double votes for a given candidate resulting from write-in
reconciliation is deliberately unspecified in the VVSG, so for testing
purposes it is considered an error.)

\item \label{StraightPartyOverridesConstraint}
A \hyperref[Ballot]{Ballot} may not simultaneously have
\hyperref[VoterInput]{VoterInput} in a straight-party-votable
\hyperref[Contest]{Contest} and a straight party vote that implies
votes in that same \hyperref[Contest]{Contest}.  (Resolution of
straight party overrides is deliberately unspecified in the VVSG, so for testing
purposes they are considered to be errors.)

\item \label{DoubleIndirectAliasesConstraint}
The \hyperref[Choice]{Choice} that an \hyperref[Alias]{Alias}
cites as canonical cannot be aliased.  (Corollary:  There can be
no cycles or self-referential \hyperref[Alias]{Aliases}.)

\item \label{CrossContestAliasesConstraint}
The \hyperref[Choice]{Choice} that an \hyperref[Alias]{Alias}
cites as canonical must be in the same \hyperref[Contest]{Contest}.

\item \label{EndorsedAliasesConstraint}
The \hyperref[Choice]{Choice} referenced by an
\hyperref[Endorsement]{Endorsement} must be canonical (it cannot be an
\hyperref[Alias]{Alias}).

\item \label{TooManyWriteInsConstraint}
A \hyperref[Ballot]{Ballot} cannot have
\hyperref[VoterInput]{VoterInput} for more write-in
\hyperref[Choice]{Choices} in a given \hyperref[Contest]{Contest} than
is allowed by the MaxWriteIns attribute of the
\hyperref[Contest]{Contest}.

\end{constraints}


\subsubsection{Usage for all standard voting variations}

\paragraph{In-person voting}

No special requirements.

\paragraph{Absentee voting}
\label{ABCdefinition}

Absentee voting is implemented in several different ways in practice,
and it can be implemented in several different ways using this model.

\begin{enumerate}

\item Absentee \hyperref[Ballot]{Ballots} can be tagged with the Absentee category and
otherwise mingled with other \hyperref[Ballot]{Ballots}.

\item A separate \hyperref[ReportingContext]{ReportingContext} can be
created for absentee \hyperref[Ballot]{Ballots} and applied to the individual absentee
\hyperref[Ballot]{Ballots}.

\item A separate \hyperref[BallotStyle]{BallotStyle} can be used for
absentee \hyperref[Ballot]{Ballots}.

\end{enumerate}

While the first option is the least invasive, absentee \hyperref[Ballot]{Ballots} are in
practice sometimes processed as a separate precinct, which usually
means both a separate \hyperref[ReportingContext]{ReportingContext}
and a separate \hyperref[BallotStyle]{BallotStyle}.


\paragraph{Review-required ballots}

Use Categories and Accepted attributes of \hyperref[Ballot]{Ballot} as
needed.

\paragraph{Write-ins}

The number of write-ins permitted is an attribute of the
\hyperref[Contest]{Contest}.  If the write-in is new, a new
\hyperref[Choice]{Choice} is created for it (with IsWriteIn = true).
Votes are then associated with that \hyperref[Choice]{Choice}.
\hyperref[Alias]{Alias} associations are created as applicable during
write-in reconciliation.

\paragraph{Split precincts}

\hyperref[Ballot]{Ballots} are associated with the
\hyperref[ReportingContext]{ReportingContexts} pertaining to the
applicable \hyperref[Precinct]{Precinct} and
\hyperref[ElectionDistrict]{ElectionDistrict}.  If different
\hyperref[BallotStyle]{BallotStyles} are used for each split, the
associations can be made on the \hyperref[BallotStyle]{BallotStyles}.
Otherwise, each \hyperref[Ballot]{Ballot} must be individually
associated.

\paragraph{Straight party voting}

A single \hyperref[Contest]{Contest} is created with CountingLogic = Straight party
selection and \hyperref[Choice]{Choice} Names being equal to the
Names of the available \hyperref[Party]{Parties}.  In every other
\hyperref[Contest]{Contest} that is straight-party-votable, the straight party behaviors
are configured by creating \hyperref[Endorsement]{Endorsement}
associations between the \hyperref[Choice]{Choices} and the
\hyperref[Party]{Parties}.

\paragraph{Cross-party endorsement}

See straight party voting.  Create additional
\hyperref[Endorsement]{Endorsement} associations as needed for
multiply endorsed \hyperref[Choice]{Choices}.

\paragraph{Ballot rotation}

Rotate is a Boolean attribute of \hyperref[Contest]{Contest}.  The
implementation of variable mapping between \hyperref[Choice]{Choices}
and ballot positions is out of scope because ballot positions are
abstracted out of the model.  However, in paper-based systems,
rotation may involve a proliferation of
\hyperref[BallotStyle]{BallotStyles} that would have to be added.

\paragraph{Primary elections}

Create \hyperref[BallotStyle]{BallotStyles} and
\hyperref[ReportingContext]{ReportingContexts} as needed to support
the different political parties and unaffiliated voters.  Non-party-specific
\hyperref[Contest]{Contests} appear in all \hyperref[BallotStyle]{BallotStyles} while
party-specific \hyperref[Contest]{Contests} only appear in those
\hyperref[BallotStyle]{BallotStyles} applicable to the relevant
\hyperref[Party]{Party}.

\paragraph{Closed primaries}

Assignment of \hyperref[BallotStyle]{BallotStyles} to voters is
procedural and out of scope.

\paragraph{Open primaries}

Assignment of \hyperref[BallotStyle]{BallotStyles} to voters is
procedural and out of scope.

\paragraph{Provisional / challenged ballots}

Use Categories and Accepted attributes of \hyperref[Ballot]{Ballot} as
needed.

\paragraph{1-of-M voting}

Set ContestCountingLogic = N-of-M and set N = 1.

\paragraph{N-of-M voting}

Set ContestCountingLogic = N-of-M and set N appropriately.

\paragraph{Cumulative voting}

Set ContestCountingLogic = Cumulative and set N appropriately.

\paragraph{Ranked order voting}

Set ContestCountingLogic = Ranked order and set N appropriately.
\hyperref[VoterInput]{VoterInput} Values specify the rankings as
provided on each \hyperref[Ballot]{Ballot}.


\subsection{Basic schema}
\label{Schema1}

The following transforms were used to render the UML model as SQL.

\begin{enumerate}

\item At the most basic level, a table represents a class, the columns
of the table represent the attributes of that class, and the rows of
the table represent the instances of that class.

\item Object identity ({\em haecceity}) is implemented either by using
an existing identifier as primary key or by using a synthetic
identifier of integer type, as convenient.

\item Associations to at most 1 instance of another class are
implemented using foreign keys within the relevant table with a
not-null constraint if the minimum multiplicity is 1.  Associations of
higher multiplicity are reified as separate tables.

\item Attributes of multiplicity greater than 1 are treated as
associations and reified as separate tables.

\item Enums are implemented using the names of the enum values as
identifiers.  Integrity is maintained by creating a table containing
the enum values and making attributes of that enum type into foreign
keys on that table.

\end{enumerate}

The classes \hyperref[Tabulator]{Tabulator},
\hyperref[Precinct]{Precinct} and
\hyperref[ElectionDistrict]{ElectionDistrict} are not represented.
They were modelled only to clarify how the more general concept
\hyperref[ReportingContext]{ReportingContext} relates to the real
world and are not needed by the test suite.

\begin{verbatim}
-- enum
create table BallotCategory (
  Name  Text  primary key
);
insert into BallotCategory values
  ('Early'), ('Regular'), ('InPerson'), ('Absentee'), ('Provisional'),
  ('Challenged'), ('NotRegistered'), ('WrongPrecinct'), ('IneligibleVoter');


-- enum
create table ContestCountingLogic (
  Name  Text  primary key
);
insert into ContestCountingLogic values
  ('N-of-M'), ('Cumulative'), ('Ranked order'), ('Straight party selection');


-- class
create table ReportingContext (
  Name  Text  primary key
);


-- class
create table Party (
  Name  Text  primary key
);


-- class
create table Contest (
  ContestId      Integer  primary key,
  Description    Text     not null,
  CountingLogic  Text     not null references ContestCountingLogic,
  N              Integer  not null check (N > 0),
  MaxWriteIns    Integer  not null check (MaxWriteIns between 0 and N),
  Rotate         Boolean  not null,

  -- Straight party selections must be 1-of-M with no write-ins.
  check (CountingLogic <> 'Straight party selection' or
          (N = 1 and MaxWriteIns = 0))
);


-- class
create table Choice (
  ChoiceId     Integer  primary key,
  ContestId    Integer  not null references Contest,
  Name         Text     not null,
  Affiliation  Text     references Party,    -- named association
  IsWriteIn    Boolean  not null
);


-- class
create table BallotStyle (
  StyleId  Integer  primary key,
  Name     Text     not null
);


-- class
create table Ballot (
  BallotId  Integer  primary key,
  StyleId   Integer  not null references BallotStyle,
  Accepted  Boolean  not null
);


-- attribute Ballot::Categories
create table BallotCategoryAssociation (
  BallotId  Integer  references Ballot,
  Category  Text     references BallotCategory,
  primary key (BallotId, Category)
);


-- association class
create table VoterInput (
  BallotId  Integer  references Ballot,
  ChoiceId  Integer  references Choice,
  Value     Integer  not null check (Value > 0),
  primary key (BallotId, ChoiceId)
);


-- association class
create table Endorsement (
  Party     Text     references Party,
  ChoiceId  Integer  references Choice,
  Value     Integer  not null check (Value > 0),
  primary key (Party, ChoiceId)
);


-- named association
create table Alias (
  AliasId   Integer  primary key references Choice,  -- The unwanted alias
  ChoiceId  Integer  not null references Choice,     -- The canonical choice
  check (ChoiceId <> AliasId)                 -- Circular aliases are no good
);


-- unnamed association
create table BallotStyleContestAssociation (
  StyleId    Integer  references BallotStyle,
  ContestId  Integer  references Contest,
  primary key (StyleId, ContestId)
);


-- unnamed association
create table BallotStyleReportingContextAssociation (
  StyleId           Integer  references BallotStyle,
  ReportingContext  Text     references ReportingContext,
  primary key (StyleId, ReportingContext)
);


-- unnamed association
create table BallotReportingContextAssociation (
  BallotId          Integer  references Ballot,
  ReportingContext  Text     references ReportingContext,
  primary key (BallotId, ReportingContext)
);
\end{verbatim}


\subsection{Basic test suite}
\label{BasicTestSuite}

The basic test suite contains 92 tests that exercise different voting
variations in small, simple scenarios to isolate the conditions under which
failures occur.  The basic test suite is complemented by the volume test
specified in Part 3 of the VVSG \cite{VVSGNG}, which exercises all features
together in a large, complex scenario where a significant volume of ballots
is processed.

The variations identified in the test suite as AbsenteeVoting and
AbsenteeByCategories both conform to the voting variation defined in the VVSG
as {\em Absentee voting}.  Tests tagged as AbsenteeByCategories require
support for a capability that might not be present in all systems, while
AbsenteeVoting tests use a more procedural approach to achieve the same goal.
The documented assumption attached to test case
1-basic-AbsenteeByCategory.sql and other tests using the
absentee-ballots-by-categories approach means that it is not applicable to
{\em Absentee voting} systems that are limited to the procedural approach.
See \autoref{ABCdefinition} for further discussion on this issue.

\autoref{Level1Table} provides the full list of tests, which breaks down as
follows:

\begin{itemize}
\item 3 baseline tests that require support for no optional voting variations.

\item 19 single-variation tests covering the 12 optional voting variations
identified in the basic test suite.

\item 66 two-variation tests covering 63 combinations of two voting
variations.  The other 3 combinations are not meaningful:

\begin{enumerate}

\item AbsenteeByCategories plus AbsenteeVoting:  Support for
AbsenteeByCategories implies support for AbsenteeVoting.

\item CrossPartyEndorsement plus StraightPartyVoting:  Support for
CrossPartyEndorsement implies support for StraightPartyVoting.

\item CrossPartyEndorsement plus RankedOrderVoting:  In ranked order contests,
a straight party vote manifests as a particular ranking of choices that is
specified by the party.  Since all choices must be ranked in any event, there
is no additional cross-party functionality to test.

\end{enumerate}

\item 1 three-variation test.

\item 3 tests that use ballot configurations based on actual sample ballots
contributed to the test effort \cite{samples}.  These tests do not attempt to
replicate the reporting structures of the relevant jurisdictions, which include
several additional levels of districting and many more ballot configurations.

\end{itemize}

Several voting variations do not have designated tests:

\begin{itemize}

\item {\em In-person voting} is tested incidentally.  Unless a system is
exclusively for use with absentee \hyperref[Ballot]{Ballots}, most scenarios
will include simulated in-person voting.

\item {\em Review-required ballots} would be tested using a variation
of 1-basic-Provisional.sql, 1-basic-WriteIns.sql, or similar.  The
reasons for which a \hyperref[Ballot]{Ballot} might be flagged or separated for review
are jurisdiction-dependent, but provisional \hyperref[Ballot]{Ballots} and \hyperref[Ballot]{Ballots} with
write-ins are likely candidates.

\item {\em Closed primaries} and {\em Open primaries} are special
cases of {\em Primary elections}.  They are distinguished only by the
procedure for assigning \hyperref[BallotStyle]{BallotStyles} to
voters, which is beyond the scope of the test suite.  They would be
tested using the same scenarios as {\em Primary elections}, but
additional requirements on the behavior of the system would apply.

\end{itemize}


\begin{landscape}
\footnotetext[2]{Assumption:  System supports categorization of
\hyperref[Ballot]{Ballots}.  This test is not applicable to systems that require the
creation of distinct ballot configurations to implement
absentee voting.}
\LTXtable{9in}{1-basic}
\end{landscape}
\setcounter{footnote}{2}

\subsection{Required test cases not included in the basic Votetest test suite}
\label{RequiredNotIncluded}

Some requirements that might logically be tested during the phase when Votetest
is used do not have associated test cases because the behaviors in question are
orthogonal to the Votetest data model and/or so dependent on the specifics of
the implementation that only an abstract test script could be provided.  Those
requirements and descriptions of the needed test cases are shown in
\autoref{othertests}.

\LTXtable{6.5in}{other-tests.tex}


\subsection{Requirements trace}
\label{ReqTrace}

This section lists the tests that are traceable to each requirement in
Chapter~7 of the VVSG \cite{VVSGNGdraft}.

\input{RequirementsTrace}


\section{Advanced schema}
\label{Schema}

The schema for core requirements is designed only for testing.  It does not
respond to the security, privacy, accessibility, or usability requirements of
the VVSG\@.

The schema is built in five layers:

\begin{enumerate}

\item Translation of the data model.  This layer contains all of the
tables and data.  The other layers are comprised entirely of views.

\item Conveniences defined over the data model.

\item Adaptation layer.  This layer translates the raw voter inputs
per the data model into the effective voter inputs required by the
logic model.

\item Data integrity checks.

\item Translation of the logic model.

\end{enumerate}

The schema was developed with PostgreSQL \cite{postgres} running on
a GNU/Linux operating system.  It uses extensions to the SQL standard
\cite{SQL} that might not function as intended with other databases.

Layer 1, the translation of the data model, was covered in \autoref{Schema1}.
The following subsections cover the other four layers.


\subsection{Conveniences}

All assertions have as a precondition the assumption that all of
the constraints are satisfied (see \autoref{Constraints}).

\subsubsection{FilteredBallotContestAssociation}

The view FilteredBallotContestAssociation identifies all
\hyperref[Contest]{Contests} that
appear on a given
\hyperref[Ballot]{Ballot}, excluding ranked order \hyperref[Contest]{Contests}.

\begin{verbatim}
create view FilteredBallotContestAssociation (BallotId, ContestId) as
  select BallotId, ContestId
    from Ballot
      natural join BallotStyleContestAssociation
      natural join Contest
    where CountingLogic <> 'Ranked order';
\end{verbatim}

\begin{assertion}
\label{FilteredBallotContestAssociation-Assertion}
For each \hyperref[Ballot]{Ballot}, FilteredBallotContestAssociation contains exactly one
row for each non-ranked-order \hyperref[Contest]{Contest} appearing in the \hyperref[BallotStyle]{BallotStyle}
identified by Ballot.StyleId, and zero rows for any ranked order \hyperref[Contest]{Contests}.
\end{assertion}

\begin{enumerate}

\item For each \hyperref[Ballot]{Ballot} in table Ballot, the result of Ballot natural
join BallotStyleContestAssociation contains one row for each \hyperref[Contest]{Contest}
appearing in the \hyperref[BallotStyle]{BallotStyle} identified by Ballot.StyleId.
The
primary key constraints on Ballot and BallotStyleContestAssociation
ensure that there cannot be more than one such row.

\item The primary key constraint on table Contest and the foreign key
constraint on table BallotStyleContestAssociation ensure that the
natural join of the previous result with table Contest will not drop
rows.  Consequently, for each \hyperref[Ballot]{Ballot}, there is still
one row for each \hyperref[Contest]{Contest} appearing in the \hyperref[BallotStyle]{BallotStyle} identified by
Ballot.StyleId.

\item The where clause removes rows pertaining to ranked order
\hyperref[Contest]{Contests}.
\end{enumerate}


\begin{assertion}
If the \hyperref[BallotStyle]{BallotStyle} contains zero \hyperref[Contest]{Contests},
BallotStyleContestAssociation contains no rows with that BallotId.
\end{assertion}

If BallotStyleContestAssociation contains no rows with a StyleId
matching Ballot.StyleId, meaning that the \hyperref[BallotStyle]{BallotStyle} contains no
\hyperref[Contest]{Contests}, then the natural join of Ballot and
BallotStyleContestAssociation produces no rows for that \hyperref[Ballot]{Ballot}.  The
subsequent natural join with Contest cannot add back a BallotId that
was dropped, so FilteredBallotContestAssociation correctly contains no
rows for that \hyperref[Ballot]{Ballot}.


\subsubsection{ReportingContextAssociationMerge}

The view ReportingContextAssociationMerge merges \hyperref[ReportingContext]{ReportingContexts}
inherited from the \hyperref[BallotStyle]{BallotStyle} with \hyperref[ReportingContext]{ReportingContexts} specified on
\hyperref[Ballot]{Ballot} instances.  Duplicates are suppressed.

\begin{verbatim}
create view ReportingContextAssociationMerge (BallotId, ReportingContext) as
    select BallotId, ReportingContext
      from BallotReportingContextAssociation
  union
    select BallotId, ReportingContext
      from Ballot natural join BallotStyleReportingContextAssociation;
\end{verbatim}

\begin{assertion}
\label{ReportingContextAssociationMerge-Assertion}
For each \hyperref[Ballot]{Ballot}, ReportingContextAssociationMerge contains exactly one
row for each relevant \hyperref[ReportingContext]{ReportingContext}.
\end{assertion}

\begin{enumerate}

\item The only two ways by which a \hyperref[ReportingContext]{ReportingContext} may be relevant
to a \hyperref[Ballot]{Ballot} are by association with a \hyperref[Ballot]{Ballot}'s \hyperref[BallotStyle]{BallotStyle} or by association
with the \hyperref[Ballot]{Ballot} itself.

\item If a \hyperref[ReportingContext]{ReportingContext} is associated with the \hyperref[BallotStyle]{BallotStyle}, a
corresponding (BallotId, ReportingContext) tuple will be projected from the
result of Ballot natural join BallotStyleReportingContextAssociation.

\item If a \hyperref[ReportingContext]{ReportingContext} is associated with the \hyperref[Ballot]{Ballot} itself, a
corresponding (BallotId, ReportingContext) tuple will be selected from
BallotReportingContextAssociation.

\item Duplicate (BallotId, ReportingContext) tuples are suppressed by
the union operator.
\end{enumerate}

\begin{assertion}
\label{ReportingContextAssociationMerge-ExistsAssertion}
For each \hyperref[Ballot]{Ballot}, ReportingContextAssociationMerge contains at least
one row.
\end{assertion}

The case where no \hyperref[ReportingContext]{ReportingContext} is
relevant to a given \hyperref[Ballot]{Ballot} is prohibited by
\autoref{UnreportedBallotsConstraint} and is detected by the integrity
view UnreportedBallots (see \autoref{IntegrityViews}).


\subsubsection{VotableChoices}

The view VotableChoices identifies all canonical
\hyperref[Choice]{Choices} for which a valid
\hyperref[VoterInput]{VoterInput} could exist (those contained in the
applicable \hyperref[BallotStyle]{BallotStyles}), excluding \hyperref[Alias]{Aliases}.

\begin{verbatim}
create view VotableChoices (BallotId, ChoiceId) as
  select BallotId, ChoiceId
    from Ballot
      natural join BallotStyleContestAssociation
      natural join Choice
    where ChoiceId not in
      (select AliasId from Alias);
\end{verbatim}

\begin{assertion}
\label{VotableChoices-Assertion}
For each \hyperref[Ballot]{Ballot}, VotableChoices contains exactly one row for each
canonical \hyperref[Choice]{Choice} for which the \hyperref[Ballot]{Ballot} could contain a vote.
\end{assertion}

\begin{enumerate}

\item For each \hyperref[Ballot]{Ballot} in table Ballot, the result of Ballot natural
join BallotStyleContestAssociation contains one row for each \hyperref[Contest]{Contest}
appearing in the \hyperref[BallotStyle]{BallotStyle} identified by Ballot.StyleId.  The
primary key constraints on Ballot and BallotStyleContestAssociation
ensure that there cannot be more than one such row.

\item For each \hyperref[Ballot]{Ballot}, joining Choice with the previous result
produces exactly one row for each \hyperref[Choice]{Choice} pertaining to any \hyperref[Contest]{Contest}
appearing on the \hyperref[Ballot]{Ballot}.  The primary key constraint on Choice ensures
that there cannot be more than one such row.

\item By definition, a canonical \hyperref[Choice]{Choice} does not appear in the AliasId
column of table Alias, so the where clause does not eliminate any canonical
\hyperref[Choice]{Choices}.

\end{enumerate}

\begin{assertion}
VotableChoices contains no rows pertaining to any non-canonical \hyperref[Choice]{Choices}
(\hyperref[Alias]{Aliases}).
\end{assertion}

The where clause eliminates rows pertaining to non-canonical \hyperref[Choice]{Choices}
(those appearing in the AliasId column of table Alias).

\begin{assertion}
If the \hyperref[BallotStyle]{BallotStyle} of a given \hyperref[Ballot]{Ballot} contains zero \hyperref[Contest]{Contests},
VotableChoices contains no rows with that BallotId.
\end{assertion}

If BallotStyleContestAssociation contains no rows with a StyleId
matching Ballot.StyleId, meaning that the \hyperref[BallotStyle]{BallotStyle} contains no
\hyperref[Contest]{Contests}, then the natural join of Ballot and
BallotStyleContestAssociation produces no rows for that \hyperref[Ballot]{Ballot}.  The
subsequent natural join with Choice cannot add back a BallotId that
was dropped, so VotableChoices correctly contains no
rows for that \hyperref[Ballot]{Ballot}.

\begin{assertion}
If the \hyperref[BallotStyle]{BallotStyle} of a given \hyperref[Ballot]{Ballot} contains only \hyperref[Contest]{Contests}
having zero \hyperref[Choice]{Choices}, VotableChoices contains no rows with that BallotId.
\end{assertion}

If the \hyperref[BallotStyle]{BallotStyle} of a given \hyperref[Ballot]{Ballot} contains only \hyperref[Contest]{Contests}
having zero \hyperref[Choice]{Choices}, then the natural join with Choice will eliminate
all rows containing that BallotId.


\subsubsection{ReportingContextContestAssociation}

The view ReportingContextContestAssociation identifies all
\hyperref[Contest]{Contests} that are relevant in a given
\hyperref[ReportingContext]{ReportingContext}.  This includes those
appearing in a \hyperref[BallotStyle]{BallotStyle} associated with the
\hyperref[ReportingContext]{ReportingContext} and those appearing in a \hyperref[Ballot]{Ballot} associated
with the \hyperref[ReportingContext]{ReportingContext}.  A \hyperref[BallotStyle]{BallotStyle} association
can make a \hyperref[Contest]{Contest} relevant even if there are no
applicable \hyperref[Ballot]{Ballots}.

\begin{verbatim}
create view ReportingContextContestAssociation (ReportingContext, ContestId) as
    select ReportingContext, ContestId
      from BallotStyleReportingContextAssociation
        natural join BallotStyleContestAssociation
  union
    select ReportingContext, ContestId
      from BallotReportingContextAssociation
        natural join Ballot
        natural join BallotStyleContestAssociation;
\end{verbatim}

\begin{assertion}
\label{ReportingContextContestAssociation-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, ReportingContextContestAssociation contains exactly one row for each \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}.
\end{assertion}

\begin{enumerate}

\item The relevance of a \hyperref[Contest]{Contest} within a
\hyperref[ReportingContext]{ReportingContext} happens through an
intermediary \hyperref[BallotStyle]{BallotStyle}.  The association
of \hyperref[Contest]{Contests} with \hyperref[BallotStyle]{BallotStyles} is specified by the table
BallotStyleContestAssociation.

\item
A
\hyperref[BallotStyle]{BallotStyle} can become relevant to a
\hyperref[ReportingContext]{ReportingContext} in two ways:
directly, via BallotStyleReportingContextAssociation, or indirectly,
via BallotReportingContextAssociation and Ballot.StyleId.

\item If a \hyperref[BallotStyle]{BallotStyle} is directly relevant to
a \hyperref[ReportingContext]{ReportingContext}, a corresponding
(ReportingContext, ContestId) tuple will be projected from the natural
join of BallotStyleReportingContextAssociation and
BallotStyleContestAssociation.

\item If a \hyperref[BallotStyle]{BallotStyle} is indirectly relevant
to a \hyperref[ReportingContext]{ReportingContext}, there must exist a
\hyperref[Ballot]{Ballot} having that \hyperref[BallotStyle]{BallotStyle} that is
associated with that \hyperref[ReportingContext]{ReportingContext} via
BallotReportingContextAssociation.  BallotReportingContextAssociation
natural join Ballot will therefore contain one row identifying that \hyperref[Ballot]{Ballot}
and that \hyperref[ReportingContext]{ReportingContext}.  The subsequent natural join with
BallotStyleContestAssociation is on the column StyleId; constraints
ensure that this join will not drop rows.  Consequently,
a (ReportingContext, ContestId) tuple corresponding to the indirect
association between the \hyperref[ReportingContext]{ReportingContext} and the \hyperref[Contest]{Contest} will be
projected from the three-way join.

\item Duplicate (ReportingContext, ContestId) tuples are suppressed by
the union operator.
\end{enumerate}


\subsubsection{FilteredContextContestAssociation}

The view FilteredContextContestAssociation is the same as
ReportingContextContestAssociation except it excludes ranked order \hyperref[Contest]{Contests}.

\begin{verbatim}
create view FilteredContextContestAssociation (ReportingContext, ContestId) as
  select ReportingContext, ContestId
    from ReportingContextContestAssociation
      natural join Contest
    where CountingLogic <> 'Ranked order';
\end{verbatim}

\begin{assertion}
\label{FilteredContextContestAssociation-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, FilteredContextContestAssociation contains
exactly one row for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in
that \hyperref[ReportingContext]{ReportingContext}, and zero rows for each ranked-order \hyperref[Contest]{Contest}.
\end{assertion}

Per \hyperref[ReportingContextContestAssociation-Assertion]{Assertion
\ref{ReportingContextContestAssociation-Assertion}}, for each
\hyperref[ReportingContext]{ReportingContext}, ReportingContextContestAssociation contains exactly
one row for each \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}.
Constraints ensure that the natural join with Contest will not drop
rows.  The only difference from ReportingContextContestAssociation
therefore is the elimination of ranked order \hyperref[Contest]{Contests} by the where
clause.



\subsubsection{FilteredContextChoiceAssociation}

The view FilteredContextChoiceAssociation identifies all
\hyperref[Choice]{Choices} that are relevant in a given
\hyperref[ReportingContext]{ReportingContext}, excluding \hyperref[Alias]{Aliases} and
\hyperref[Choice]{Choices} from ranked order \hyperref[Contest]{Contests}.

\begin{verbatim}
create view FilteredContextChoiceAssociation (ReportingContext, ChoiceId) as
  select ReportingContext, ChoiceId
    from FilteredContextContestAssociation
      natural join Choice
    where ChoiceId not in
      (select AliasId from Alias);
\end{verbatim}

\begin{assertion}
\label{FilteredContextChoiceAssociation-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, FilteredContextChoiceAssociation contains
exactly one row for each canonical \hyperref[Choice]{Choice} in each non-ranked-order
\hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, and zero rows for
any other \hyperref[Choice]{Choice}.
\end{assertion}

Per \hyperref[FilteredContextContestAssociation-Assertion]{Assertion
\ref{FilteredContextContestAssociation-Assertion}}, for each \hyperref[ReportingContext]{ReportingContext}, FilteredContextContestAssociation contains exactly one row
for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}.  For each \hyperref[ReportingContext]{ReportingContext}, the result of
FilteredContextContestAssociation natural join Choice contains exactly
one row for each \hyperref[Choice]{Choice} in each non-ranked-order \hyperref[Contest]{Contest} that is
relevant in that \hyperref[ReportingContext]{ReportingContext}.  The where clause eliminates rows
pertaining to non-canonical \hyperref[Choice]{Choices} (\hyperref[Alias]{Aliases}).


\subsubsection{BallotCounts}
\label{BallotCountsSection}

The view BallotCounts produces the count of the number of read and
counted \hyperref[Ballot]{Ballots} for each
\hyperref[ReportingContext]{ReportingContext}.

\begin{verbatim}
create view BallotCounts (ReportingContext, Read, Counted) as
  select Name, count(BallotId), count (nullif (Accepted, false))
    from Ballot
      natural join ReportingContextAssociationMerge
      right outer join ReportingContext on (Name = ReportingContext)
    group by Name;
\end{verbatim}

\begin{assertion}
For each \hyperref[ReportingContext]{ReportingContext}, BallotCounts
contains exactly one row giving the number of relevant \hyperref[Ballot]{Ballots}
(in the Read column) and the number of accepted \hyperref[Ballot]{Ballots} (in the Counted
column).
\end{assertion}

\begin{enumerate}

\item Using
\hyperref[ReportingContextAssociationMerge-Assertion]{Assertion
\ref{ReportingContextAssociationMerge-Assertion}}, for each \hyperref[ReportingContext]{ReportingContext}, the natural join of Ballot and
ReportingContextAssociationMerge contains one row for each relevant
\hyperref[Ballot]{Ballot}.

\item For each \hyperref[ReportingContext]{ReportingContext} having zero relevant \hyperref[Ballot]{Ballots}, the right outer
join with ReportingContext generates a single row having a nulls in
the BallotId and Accepted columns.  In all other cases, the right
outer join has the effect of duplicating the ReportingContext column
in the Name column and leaving the other columns unchanged.

\item In the three-way join, there is at least one row for each
\hyperref[ReportingContext]{ReportingContext}.  Grouping by Name therefore yields one row for each
\hyperref[ReportingContext]{ReportingContext}.

\item The count operation does not include nulls.  Therefore, for each
\hyperref[ReportingContext]{ReportingContext} having one or more relevant \hyperref[Ballot]{Ballots}, count(BallotId) yields
the number of such \hyperref[Ballot]{Ballots}, and count (nullif (Accepted, false))
yields the number of those that were accepted.

\item For each \hyperref[ReportingContext]{ReportingContext} having zero relevant \hyperref[Ballot]{Ballots}, the three-way
join contains a single row with nulls in both the BallotId and
Accepted columns, so both counts yield 0.
\end{enumerate}


\subsubsection{BallotCountsByConfiguration}
\label{BallotCountsByConfigurationSection}

The view BallotCountsByConfiguration produces the count of the number
of read and counted \hyperref[Ballot]{Ballots} broken down by \hyperref[BallotStyle]{BallotStyle} within each
\hyperref[ReportingContext]{ReportingContext}.  Rows pertaining to
combinations of \hyperref[ReportingContext]{ReportingContext} and \hyperref[BallotStyle]{BallotStyle} that have no applicable
\hyperref[Ballot]{Ballots} are suppressed.

\begin{verbatim}
create view BallotCountsByConfiguration (ReportingContext, StyleId,
                                         Read, Counted) as
  select ReportingContext, StyleId, count(*), count (nullif (Accepted, false))
    from Ballot natural join ReportingContextAssociationMerge
    group by ReportingContext, StyleId;
\end{verbatim}

\begin{assertion}
For each combination of \hyperref[ReportingContext]{ReportingContext}
and \hyperref[BallotStyle]{BallotStyle} having relevant \hyperref[Ballot]{Ballots},
BallotCountsByConfiguration contains exactly one row giving the number
of relevant \hyperref[Ballot]{Ballots} (in the Read column) and the number of accepted
\hyperref[Ballot]{Ballots} (in the Counted column).
\end{assertion}

\begin{enumerate}
\item Using
\hyperref[ReportingContextAssociationMerge-Assertion]{Assertion
\ref{ReportingContextAssociationMerge-Assertion}}, for each \hyperref[ReportingContext]{ReportingContext}, the natural join of Ballot and
ReportingContextAssociationMerge contains one row for each relevant
\hyperref[Ballot]{Ballot}.

\item Grouping by ReportingContext and StyleId yields exactly one row
for each combination of \hyperref[ReportingContext]{ReportingContext}
and \hyperref[BallotStyle]{BallotStyle} having relevant \hyperref[Ballot]{Ballots}.

\item count(*) and count (nullif (Accepted, false)) yield the number
of \hyperref[Ballot]{Ballots} and the number of accepted \hyperref[Ballot]{Ballots} within each group.
\end{enumerate}

\begin{assertion}
BallotCountsByConfiguration contains no rows for combinations of
\hyperref[ReportingContext]{ReportingContext}
and \hyperref[BallotStyle]{BallotStyle} that have no relevant \hyperref[Ballot]{Ballots}.
\end{assertion}

The natural join of Ballot with ReportingContextAssociationMerge
cannot generate a row with a particular combination of
\hyperref[ReportingContext]{ReportingContext} and
\hyperref[BallotStyle]{BallotStyle} unless there exists a \hyperref[Ballot]{Ballot} of
that \hyperref[BallotStyle]{BallotStyle} that is relevant in that
\hyperref[ReportingContext]{ReportingContext}.


\subsubsection{BallotCountsByCategory}
\label{BallotCountsByCategorySection}

\begin{verbatim}
create view BallotCountsByCategory (ReportingContext, Category,
                                    Read, Counted) as
  select ReportingContext, Category, count(*), count (nullif (Accepted, false))
    from Ballot
      natural join ReportingContextAssociationMerge
      natural join BallotCategoryAssociation
    group by ReportingContext, Category;
\end{verbatim}

\begin{assertion}
For each combination of \hyperref[ReportingContext]{ReportingContext}
and \hyperref[BallotCategory]{BallotCategory} having relevant \hyperref[Ballot]{Ballots},
BallotCountsByCategory contains exactly one row giving the number
of relevant \hyperref[Ballot]{Ballots} (in the Read column) and the number of accepted
\hyperref[Ballot]{Ballots} (in the Counted column).
\end{assertion}

\begin{enumerate}
\item Using
\hyperref[ReportingContextAssociationMerge-Assertion]{Assertion
\ref{ReportingContextAssociationMerge-Assertion}}, for each \hyperref[ReportingContext]{ReportingContext}, the natural join of Ballot and
ReportingContextAssociationMerge contains one row for each relevant
\hyperref[Ballot]{Ballot}.

\item For each \hyperref[ReportingContext]{ReportingContext}, for each
relevant
\hyperref[Ballot]{Ballot}, the natural join of the previous result with BallotCategoryAssociation yields a row for each associated \hyperref[BallotCategory]{BallotCategory}.

\item Grouping by ReportingContext and Category yields exactly one row
for each combination of \hyperref[ReportingContext]{ReportingContext}
and \hyperref[BallotCategory]{BallotCategory} having relevant \hyperref[Ballot]{Ballots}.

\item count(*) and count (nullif (Accepted, false)) yield the number
of \hyperref[Ballot]{Ballots} and the number of accepted \hyperref[Ballot]{Ballots} within each group.
\end{enumerate}

\begin{assertion}
BallotCountsByCategory contains no rows for combinations of
\hyperref[ReportingContext]{ReportingContext}
and \hyperref[BallotCategory]{BallotCategory} that have no relevant \hyperref[Ballot]{Ballots}.
\end{assertion}

The natural join of (Ballot natural join
ReportingContextAssociationMerge) with BallotCategoryAssociation
cannot generate a row with a particular combination of
\hyperref[ReportingContext]{ReportingContext} and
\hyperref[BallotCategory]{BallotCategory} unless there exists a
\hyperref[Ballot]{Ballot} of that
\hyperref[BallotCategory]{BallotCategory} that is relevant in that
\hyperref[ReportingContext]{ReportingContext}.


\subsubsection{BallotCountsByCategoryAndConfiguration}

\begin{verbatim}
create view BallotCountsByCategoryAndConfiguration (ReportingContext, StyleId,
                                                    Category, Read, Counted) as
  select ReportingContext, StyleId, Category, count(*),
         count (nullif (Accepted, false))
    from Ballot
      natural join ReportingContextAssociationMerge
      natural join BallotCategoryAssociation
    group by ReportingContext, StyleId, Category;
\end{verbatim}

The assertions and discussion for
BallotCountsByCategoryAndConfiguration are analogous to those of
\hyperref[BallotCountsByConfigurationSection]{BallotCountsByConfiguration} and
\hyperref[BallotCountsByCategorySection]{BallotCountsByCategory}.


\subsubsection{BlankBallot}

\begin{verbatim}
create view BlankBallot (BallotId, StyleId, Accepted) as
  select BallotId, StyleId, Accepted
    from Ballot natural left outer join VoterInput
    where Value is null;
\end{verbatim}

\begin{assertion}
BlankBallot contains exactly one row for each \hyperref[Ballot]{Ballot} having no
associated votes.
\end{assertion}

Ballot natural left outer join VoterInput produces one row with a
non-null Value column for each vote (possibly many such rows with the
same BallotId), and exactly one row with a null Value column for each
\hyperref[Ballot]{Ballot} having no associated votes.  The where clause selects only the
latter rows.


\subsubsection{BlankBallotCounts}

\begin{verbatim}
create view BlankBallotCounts (ReportingContext, Read, Counted) as
  select Name, count(BallotId), count (nullif (Accepted, false))
    from BlankBallot
      natural join ReportingContextAssociationMerge
      right outer join ReportingContext on (Name = ReportingContext)
    group by Name;
\end{verbatim}

The assertion and discussion for BlankBallotCounts are parallel to
those of \hyperref[BallotCountsSection]{BallotCounts}, substituting
BlankBallot for Ballot.


\subsubsection{BlankBallotCountsByConfiguration}

\begin{verbatim}
create view BlankBallotCountsByConfiguration (ReportingContext, StyleId,
                                              Read, Counted) as
  select ReportingContext, StyleId, count(*), count (nullif (Accepted, false))
    from BlankBallot natural join ReportingContextAssociationMerge
    group by ReportingContext, StyleId;
\end{verbatim}

The assertions and discussion for BlankBallotCountsByConfiguration are
parallel to those of
\hyperref[BallotCountsByConfigurationSection]{BallotCountsByConfiguration},
substituting BlankBallot for Ballot.


\subsection{Adaptation}

Converting the raw voter inputs into the effective voter inputs
required by the logic model involves \hyperref[Alias]{Alias} reconciliation,
implementation of straight party voting, and generation of default (0)
values for ballot positions that were not voted.

The VoterInput table has a primary key on (BallotId, ChoiceId), so
there is at most one row for any given ballot position on any given
\hyperref[Ballot]{Ballot}.  Deliberately, the adaptation views do not preserve this
constraint in the event that double votes result from \hyperref[Alias]{Alias}
reconciliation or straight party voting.  Both of these cases are
treated as errors for testing purposes, and the errors are most easily
located by looking for duplicate keys.  This is done by the integrity
view DoubleVotes (see \autoref{IntegrityViews}).

All assertions have as a precondition the assumption that all of
the constraints are satisfied (see \autoref{Constraints}).


\subsubsection{AntiAliasedVoterInput}

AntiAliasedVoterInput provides a view of VoterInput in which all
\hyperref[Choice]{Choices} have been ``canonicalized.''

\begin{verbatim}
create view AntiAliasedVoterInput (BallotId, ChoiceId, Value) as
  select BallotId, coalesce (Alias.ChoiceId, VoterInput.ChoiceId), Value
    from VoterInput left outer join Alias
      on VoterInput.ChoiceId = Alias.AliasId;
\end{verbatim}

\begin{assertion}
\label{AntiAliasedVoterInput-Assertion}
AntiAliasedVoterInput contains exactly one row for each row in VoterInput,
with any non-canonical \hyperref[Choice]{Choices} replaced by the associated canonical \hyperref[Choice]{Choices}.
\end{assertion}

\begin{enumerate}
\item Alias.AliasId is a primary key, so there can be at most one row
in Alias matching any given row of VoterInput on VoterInput.ChoiceId =
Alias.AliasId.

\item If there is a row in Alias matching a row of VoterInput, the
left outer join generates exactly one row with the canonical equivalent
of VoterInput.ChoiceId in the column Alias.ChoiceId.

\item If there is no row in Alias matching a row of VoterInput, the
left outer join generates exactly one row with a null in the
column Alias.ChoiceId.

\item coalesce (Alias.ChoiceId, VoterInput.ChoiceId) substitutes the
canonical \hyperref[Choice]{Choice} in the case where a matching row in Alias exists, and
retains the original \hyperref[Choice]{Choice} in the case where no such row exists
and Alias.ChoiceId is null.
\end{enumerate}


\subsubsection{VoterInputMerge}
\label{VoterInputMergeSection}

VoterInputMerge provides a view over AntiAliasedVoterInput in which
the side-effects implied by straight party votes have been
incorporated.  If a straight party selection \hyperref[Contest]{Contest} is overvoted, it has no
side-effects.

VoterInputMerge depends on two intervening views,
ValidStraightPartyVotes and ImpliedStraightPartyVotes.

\begin{verbatim}
create view ValidStraightPartyVotes (BallotId, Party) as
  select BallotId, max(Name)      -- There can be only one.  See below.
    from AntiAliasedVoterInput
      natural join Choice
      natural join Contest
    where CountingLogic = 'Straight party selection'
    group by BallotId
    having sum(Value) = 1;        -- There can be only one.
\end{verbatim}

\begin{assertion}
ValidStraightPartyVotes contains exactly one row for each \hyperref[Ballot]{Ballot}
containing a vote in a straight party selection \hyperref[Contest]{Contest}, excluding
those that overvoted the straight party selection \hyperref[Contest]{Contest}.
\end{assertion}

\begin{enumerate}
\item The primary key on VoterInput and
\hyperref[AntiAliasedVoterInput-Assertion]{Assertion
\ref{AntiAliasedVoterInput-Assertion}} ensure that
AntiAliasedVoterInput natural join Choice will have the same number of
rows as VoterInput.

\item The natural join of the previous result with Contest again adds
columns but yields the same number of rows as VoterInput.

\item The where clause eliminates all rows pertaining to \hyperref[Contest]{Contests}
that are not straight party selection \hyperref[Contest]{Contests}.

\item \autoref{MoreThanOneStraightPartyContestConstraint} states that
a given \hyperref[BallotStyle]{BallotStyle} may contain at most
one \hyperref[Contest]{Contest} with CountingLogic = Straight party
selection.

\item Straight party \hyperref[Contest]{Contests} are implicitly 1-of-M \hyperref[Contest]{Contests}.

\item \autoref{VoterInputValueConstraint} states that any remaining
rows must have a VoterInput.Value of 1.

\item The having clause therefore eliminates rows pertaining to straight party
\hyperref[Contest]{Contests} that were overvoted (having two or more associated votes).

\item There can be at most one row remaining for a given BallotId.  If
such a row exists for a given \hyperref[Ballot]{Ballot}, max(Name) retrieves the name of
the Party that was selected in the straight party selection \hyperref[Contest]{Contest}.
(The name cannot be selected directly due to the intervening group by
operation.)

\end{enumerate}

Note that ValidStraightPartyVotes does not filter out \hyperref[Ballot]{Ballots}
that are not accepted.

\begin{verbatim}
create view ImpliedStraightPartyVotes (BallotId, ChoiceId, Value) as
  select BallotId, ChoiceId, Value
    from VotableChoices
      natural join Endorsement
      natural join ValidStraightPartyVotes;
\end{verbatim}

\begin{assertion}
\label{ImpliedStraightPartyVotes-Assertion}
ImpliedStraightPartyVotes contains exactly one row for each vote that
is implied by a straight party vote.
\end{assertion}

\begin{enumerate}

\item Per \hyperref[VotableChoices-Assertion]{Assertion
\ref{VotableChoices-Assertion}}, for each \hyperref[Ballot]{Ballot}, VotableChoices
contains exactly one row for each canonical \hyperref[Choice]{Choice} for which the
\hyperref[Ballot]{Ballot} could contain a vote.

\item The primary key on Endorsement ensures that there is at most one
row for a given combination of Party and ChoiceId.

\item \autoref{EndorsedAliasesConstraint} states that the
\hyperref[Choice]{Choice} referenced by an
\hyperref[Endorsement]{Endorsement} must be canonical.

\item For each \hyperref[Ballot]{Ballot}, VotableChoices natural join Endorsement
contains one row for each vote implied by any possible straight party vote.

\item The natural join of the previous result with
ValidStraightPartyVotes eliminates all rows except those pertaining to
actual, non-overvoted straight party votes.

\end{enumerate}

Note that ImpliedStraightPartyVotes does not do anything about the
possibility that endorsements may be ill-formed (e.g., containing
overvotes).

VoterInputMerge simply concatenates the contents of the previous
two views.
VoterInputMerge does not inherently prevent there from being more than one row
with the same (BallotId, ChoiceId).  This case violates
\autoref{StraightPartyOverridesConstraint} and is detected by
the integrity view StraightPartyOverrides (see \autoref{IntegrityViews}).

\begin{verbatim}
create view VoterInputMerge (BallotId, ChoiceId, Value) as
    select BallotId, ChoiceId, Value from AntiAliasedVoterInput
  union all
    select BallotId, ChoiceId, Value from ImpliedStraightPartyVotes;
\end{verbatim}

\begin{assertion}
VoterInputMerge contains exactly one row for each row in VoterInput,
with any non-canonical \hyperref[Choice]{Choices} replaced by the associated canonical
\hyperref[Choice]{Choices}, plus exactly one row for each vote that
is implied by a straight party vote.
\end{assertion}

The assertion follows from
\hyperref[AntiAliasedVoterInput-Assertion]{Assertion
\ref{AntiAliasedVoterInput-Assertion}},
\hyperref[ImpliedStraightPartyVotes-Assertion]{Assertion
\ref{ImpliedStraightPartyVotes-Assertion}} and the definition of the
union all operator.

\begin{assertion}
\label{VoterInputMerge-Key-Assertion}
VoterInputMerge
contains at most one row for a given combination of (BallotId, ChoiceId).
The Value column contains either (1) the Value derived from a vote
for that \hyperref[Choice]{Choice}, (2) the Value derived from a vote for an \hyperref[Alias]{Alias}
of that \hyperref[Choice]{Choice}, or (3) the Value derived
from a vote for that \hyperref[Choice]{Choice} implied by a straight party vote.
\end{assertion}

\begin{enumerate}
\item By
\hyperref[AntiAliasedVoterInput-Assertion]{Assertion
\ref{AntiAliasedVoterInput-Assertion}},
\hyperref[ImpliedStraightPartyVotes-Assertion]{Assertion
\ref{ImpliedStraightPartyVotes-Assertion}},
\autoref{StraightPartyOverridesConstraint}
and
\autoref{CrossContestAliasesConstraint}, it is not possible
for AntiAliasedVoterInput and ImpliedStraightPartyVotes to both contain
votes in a given \hyperref[Contest]{Contest} for a given \hyperref[Ballot]{Ballot}.

\item Table Choice is defined such that a \hyperref[Choice]{Choice} is uniquely
associated with exactly one \hyperref[Contest]{Contest}.

\item Consequently, AntiAliasedVoterInput and
ImpliedStraightPartyVotes cannot both contain votes with a given
combination of (BallotId, ChoiceId).

\item If a given combination of (BallotId, ChoiceId) appears in
AntiAliasedVoterInput, the Value column of VoterInputMerge
will contain the Value derived from a vote
for the \hyperref[Choice]{Choice} or an \hyperref[Alias]{Alias} of that \hyperref[Choice]{Choice}, satisfying (1) or (2).

\item If a given combination of (BallotId, ChoiceId) appears in ImpliedStraightPartyVotes, the Value column of VoterInputMerge will contain the Value
derived from a vote for that \hyperref[Choice]{Choice} implied by a straight party vote,
satisfying (3).
\end{enumerate}


\subsubsection{EffectiveInput}
\label{EffectiveInputSection}

Finally, the view EffectiveInput generates zeroes for ballot positions
that were not voted.  This is appropriate for all
\hyperref[Contest]{Contest} types except ranked order.  Ranked
order tabulators should instead access
\hyperref[VoterInputMergeSection]{VoterInputMerge}
directly.

\begin{verbatim}
create view EffectiveInput (BallotId, ChoiceId, Value) as
  select BallotId, ChoiceId, coalesce (Value, 0)
    from VotableChoices natural left outer join VoterInputMerge;
\end{verbatim}

\begin{assertion}
\label{EffectiveInput-Assertion}
For each \hyperref[Ballot]{Ballot}, EffectiveInput contains exactly one row for each
canonical \hyperref[Choice]{Choice} for which the \hyperref[Ballot]{Ballot} could contain a vote.
The Value column contains either (1) the Value derived from a vote
for that \hyperref[Choice]{Choice}, (2) the Value derived from a vote for an \hyperref[Alias]{Alias}
of that \hyperref[Choice]{Choice}, (3) the Value derived
from a vote for that \hyperref[Choice]{Choice} implied by a straight party vote, or
(4) the default value of zero.
\end{assertion}

\begin{enumerate}
\item Per \hyperref[VotableChoices-Assertion]{Assertion \ref{VotableChoices-Assertion}}, for each \hyperref[Ballot]{Ballot}, VotableChoices contains exactly one row for each
canonical \hyperref[Choice]{Choice} for which the \hyperref[Ballot]{Ballot} could contain a vote.

\item Per \hyperref[VoterInputMerge-Key-Assertion]{Assertion \ref{VoterInputMerge-Key-Assertion}}, VoterInputMerge
contains at most one row for a given combination of (BallotId, ChoiceId).

\item Consequently, VotableChoices natural left outer join VoterInputMerge
contains the same number of rows as VotableChoices.

\item Since VotableChoices does not have a Value column, the Value column
of the join will come from VoterInputMerge when a matching row exists and
be null when no matching row exists.

\item If Value is not null, then it came from VoterInputMerge.  Per
\hyperref[VoterInputMerge-Key-Assertion]{Assertion \ref{VoterInputMerge-Key-Assertion}}, this Value will satisfy one of (1), (2), or (3).

\item coalesce (Value, 0) has the effect of replacing nulls with zeroes,
satisfying (4).
\end{enumerate}


\subsection{Integrity checks}
\label{IntegrityViews}

For those integrity constraints that are too complex to code directly as SQL
constraints within the tables, a series of views exists to look for problems.
All of the integrity checking views should always be empty.  If data appear
in any of the views, the input was invalid and the results of the model will
be invalid.  The integrity views are listed in \autoref{IntegrityViewsTable}
but their definitions have been elided.  Interested readers can find them in
the Votetest distribution, in the file Infrastructure-VoteSchema.sql.

\begin{table}
\caption{Integrity checks}
\capsep
\begin{center}
\begin{tabular}{|l|l|}
\multicolumn{1}{c}{\bf Constraint} & \multicolumn{1}{c}{\bf Integrity view(s)} \\ \hline
\autoref{VoterInputValueConstraint} & OutOfRangeVoterInputs, OutOfRangeEndorsements \\ \hline
\autoref{MinimumNConstraint} & N/A, enforced by SQL check constraint \\ \hline
\autoref{MaxWriteInsConstraint} & N/A, enforced by SQL check constraint \\ \hline
\autoref{StraightPartyContestParametersConstraint} & N/A, enforced by SQL check constraint \\ \hline
\autoref{UnreportedBallotsConstraint} & UnreportedBallots \\ \hline
\autoref{ExtraneousVotesConstraint} & ExtraneousInputs \\ \hline
\autoref{MoreThanOneStraightPartyContestConstraint} & MoreThanOneStraightPartyContest \\ \hline
\autoref{CircularEndorsementsConstraint} & CircularStraightPartyEndorsements \\ \hline
\autoref{StraightPartyChoiceNamesConstraint} & NonExistentParties \\ \hline
\autoref{PartyNamesUniqueConstraint} & N/A, enforced by SQL primary key constraint \\ \hline
\autoref{AliasDoubleVotesConstraint} & DoubleVotes \\ \hline
\autoref{StraightPartyOverridesConstraint} & StraightPartyOverrides \\ \hline
\autoref{DoubleIndirectAliasesConstraint} & DoubleIndirectAliases \\ \hline
\autoref{CrossContestAliasesConstraint} & CrossContestAliases \\ \hline
\autoref{EndorsedAliasesConstraint} & EndorsedAliases \\ \hline
\autoref{TooManyWriteInsConstraint} & TooManyWriteIns \\ \hline
\end{tabular}
\end{center}
\label{IntegrityViewsTable}
\end{table}


\subsection{Translation of logic model}
\label{TopLevelViews}

The following transforms are used to render the logic model as SQL.

\begin{enumerate}

\item Each function is replaced by a view in which the parameters form
the primary key and the last column is the value of the function.

\item Time parameters ($t$) are factored out.  All views implicitly
project results for the time $t$ corresponding to the current state of
the database.

\item When a function takes both a \hyperref[Contest]{Contest} and a \hyperref[Choice]{Choice} as parameters,
the \hyperref[Contest]{Contest} parameter is omitted.  With the data model used here, the
\hyperref[Contest]{Contest} can be inferred from the
\hyperref[Choice]{Choice}.

\item Logic is translated into those SQL constructs that are most
transparently equivalent.

\item Ranked order \hyperref[Contest]{Contests}, which are not handled by the logic model,
are suppressed.

\item Irrelevant values, such as zero tallies for \hyperref[Choice]{Choices} that do not
appear in the applicable \hyperref[BallotStyle]{BallotStyle} or \hyperref[Contest]{Contests} that are not relevant
in the applicable \hyperref[ReportingContext]{ReportingContext}, are suppressed.

\end{enumerate}

The following subsections first quote relevant portions of the logic
model, then describe their analogs in the schema.  Some terms from the
logic model are elided from this discussion.  For complete information
on the logic model, please refer to the VVSG \cite{VVSGNG}.

All assertions have as a precondition the assumption that all of
the constraints are satisfied (see \autoref{Constraints}).


\subsubsection{\texorpdfstring{$S(c,r,t,v)$}{S(c,r,t,v)}}

\begin{quote}
Ballot $v$'s vote with respect to contest choice $c$ in contest
$r$ as of time $t$.  For checkboxes and the like, the value is 1
(selected) or 0 (not selected).  For cumulative voting, the value is
the number of votes that $v$ gives to contest choice $c$ in
contest $r$.  If the applicable ballot style does not include contest
$r$, $S(c,r,t,v) = 0$.
\end{quote}

\begin{assertion}
\label{S4-Assertion}
The relevant case of the quaternary function $S$ is implemented by the view
\hyperref[EffectiveInputSection]{EffectiveInput}.  For each \hyperref[Ballot]{Ballot},
for each canonical \hyperref[Choice]{Choice} appearing on that \hyperref[Ballot]{Ballot}, column Value contains the
value specified by the definition of $S(c,r,t,v)$ for $c$ = ChoiceId,
$r$ = Choice.ContestId, $t$ as of the state of the database, and $v$ =
BallotId.
\end{assertion}

Follows from \hyperref[EffectiveInput-Assertion]{Assertion
\ref{EffectiveInput-Assertion}}.  Rows are not generated for the
irrelevant case where $S(c,r,t,v)$ is defined to be zero.  Note that
EffectiveInput does not eliminate inputs for ranked order \hyperref[Contest]{Contests}.


\subsubsection{\texorpdfstring{$S(r,t,v)$}{S(r,t,v)}}

\begin{quote}
The total number of votes that ballot $v$ has in contest $r$ as of time $t$.
\[ S(r,t,v) = \!\!\!\!\!\! \sum_{c \in C(r,t)} \!\!\!\!\! S(c,r,t,v) \]
\end{quote}

The ternary function $S$ is implemented by the view S\@.  The current
value of $S(r,t,v)$ is obtained by selecting S\_val where ContestId =
$r$ and BallotId = $v$.  The view S contains rows only for \hyperref[Contest]{Contests}
that actually appear on the \hyperref[Ballot]{Ballot} according to its \hyperref[BallotStyle]{BallotStyle}.  All
others are defined to be 0.

\begin{verbatim}
create view S (ContestId, BallotId, S_val) as
  select ContestId, BallotId, coalesce (sum (Value), 0)
    from FilteredBallotContestAssociation
      natural left outer join Choice
      natural left outer join EffectiveInput
    group by ContestId, BallotId;
\end{verbatim}

\begin{assertion}
\label{S-Assertion}
For each \hyperref[Ballot]{Ballot}, for each non-ranked-order \hyperref[Contest]{Contest}
appearing on that \hyperref[Ballot]{Ballot}, column S\_val of S contains the
value specified by the definition of $S(r,t,v)$ for
$r$ = ContestId, $t$ as of the state of the database, and
$v$ = BallotId.
\end{assertion}

\begin{enumerate}

\item Per
\hyperref[FilteredBallotContestAssociation-Assertion]{Assertion
\ref{FilteredBallotContestAssociation-Assertion}}, for each \hyperref[Ballot]{Ballot},
FilteredBallotContestAssociation contains exactly one row for each
non-ranked-order \hyperref[Contest]{Contest} appearing in the \hyperref[BallotStyle]{BallotStyle} identified by
Ballot.StyleId, and zero rows for any ranked order \hyperref[Contest]{Contests}.

\item Per
\hyperref[EffectiveInput-Assertion]{Assertion \ref{EffectiveInput-Assertion}},
for each \hyperref[Ballot]{Ballot}, EffectiveInput contains exactly one row for each
canonical \hyperref[Choice]{Choice} for which the \hyperref[Ballot]{Ballot} could contain a vote.

\item For each \hyperref[Ballot]{Ballot}, FilteredBallotContestAssociation
      natural left outer join Choice contains exactly one row for
each \hyperref[Choice]{Choice} in each non-ranked-order \hyperref[Contest]{Contest} on the \hyperref[Ballot]{Ballot}, plus
exactly one row for each non-ranked-order \hyperref[Contest]{Contest} on the \hyperref[Ballot]{Ballot} that has
zero associated \hyperref[Choice]{Choices}.

\item For each \hyperref[Ballot]{Ballot}, the natural left outer join of the previous
result with EffectiveInput contains (1) exactly one row for each
canonical \hyperref[Choice]{Choice} in each non-ranked-order \hyperref[Contest]{Contest} on the \hyperref[Ballot]{Ballot}, with
Value from EffectiveInput, plus (2) exactly one row for each
non-canonical \hyperref[Choice]{Choice} in each non-ranked-order \hyperref[Contest]{Contest} on the \hyperref[Ballot]{Ballot},
with null Value, plus (3) exactly one row for each non-ranked-order
\hyperref[Contest]{Contest} on the \hyperref[Ballot]{Ballot} that has zero associated \hyperref[Choice]{Choices}, with null
Value.

\item For \hyperref[Contest]{Contests} having \hyperref[Choice]{Choices}, the value specified by the
definition of $S(r,t,v)$ follows directly by summing the Value column
while grouping by ContestId and BallotId.  The null values for
non-canonical \hyperref[Choice]{Choices} are ignored by the summation operator.
\autoref{DoubleIndirectAliasesConstraint} and
\autoref{CrossContestAliasesConstraint} prevent any \hyperref[Contest]{Contest} from
having only \hyperref[Alias]{Aliases} as \hyperref[Choice]{Choices}.

\item For \hyperref[Contest]{Contests} with zero associated \hyperref[Choice]{Choices}, the summation of a
single null value returns null and coalesce (sum (Value), 0)
substitutes the value zero.

\end{enumerate}


VotesByContestAndContext is a convenience to retrieve all of the
S\_val vote counts for each relevant combination of \hyperref[ReportingContext]{ReportingContext} and
\hyperref[Contest]{Contest}.  For each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest} that
contains no \hyperref[Ballot]{Ballots}, there is a single row with nulls in the last
three columns.

\begin{verbatim}
create view VotesByContestAndContext (ContestId, N, ReportingContext,
                                      BallotId, Accepted, S_val) as
  select ContestId, N, ReportingContext, BallotId, Accepted, S_val
    from FilteredContextContestAssociation
      natural join Contest
      natural left outer join (S natural join ReportingContextAssociationMerge)
      natural left outer join Ballot;
\end{verbatim}

The discussion for the following two assertions is combined.

\begin{assertion}
\label{VotesByContestAndContext-Assertion1}
For each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest} (excluding ranked order), VotesByContestAndContext contains a row for each \hyperref[Ballot]{Ballot} that contains that
\hyperref[Contest]{Contest} and that is reported in that \hyperref[ReportingContext]{ReportingContext}.  Column Accepted is as specified
in the table Ballot.  Column S\_val contains the
value specified by the definition of $S(r,t,v)$ for
$r$ = ContestId, $t$ as of the state of the database, and
$v$ = BallotId.
\end{assertion}

\begin{assertion}
\label{VotesByContestAndContext-Assertion2}
For each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest} (excluding ranked order)
where there does not exist a \hyperref[Ballot]{Ballot} that contains that
\hyperref[Contest]{Contest} and that is reported in that \hyperref[ReportingContext]{ReportingContext}, VotesByContestAndContext
contains a single row with nulls in the BallotId, Accepted, and S\_val
columns.
\end{assertion}

\begin{enumerate}
\item Per \hyperref[FilteredContextContestAssociation-Assertion]{Assertion \ref{FilteredContextContestAssociation-Assertion}},
for each \hyperref[ReportingContext]{ReportingContext}, FilteredContextContestAssociation contains
exactly one row for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in
that \hyperref[ReportingContext]{ReportingContext}, and zero rows for each ranked-order \hyperref[Contest]{Contest}.

\item The natural join with Contest adds the column N but does not change
the number of rows.

\item Per \hyperref[ReportingContextAssociationMerge-Assertion]{Assertion \ref{ReportingContextAssociationMerge-Assertion}}, for each \hyperref[Ballot]{Ballot}, ReportingContextAssociationMerge contains exactly one
row for each relevant \hyperref[ReportingContext]{ReportingContext}.  Equivalently, for each \hyperref[ReportingContext]{ReportingContext}, ReportingContextAssociationMerge lists every \hyperref[Ballot]{Ballot} that is
reported in that \hyperref[ReportingContext]{ReportingContext}.

\item Per \hyperref[S-Assertion]{Assertion \ref{S-Assertion}}, for
each \hyperref[Ballot]{Ballot}, the view S contains exactly one row for each non-ranked-order \hyperref[Contest]{Contest}
appearing on that \hyperref[Ballot]{Ballot}, with S\_val as described above.

\item For each \hyperref[Ballot]{Ballot}, S natural join ReportingContextAssociationMerge
yields the Cartesian product of the S rows (one for each
non-ranked-order \hyperref[Contest]{Contest} appearing on that \hyperref[Ballot]{Ballot}) with the
ReportingContextAssociationMerge rows (one for each \hyperref[ReportingContext]{ReportingContext}
relevant to that \hyperref[Ballot]{Ballot}).

\item For each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest} (excluding
ranked order), the natural left outer join of
(FilteredContextContestAssociation natural join Contest) and (S
natural join ReportingContextAssociationMerge) contains a row for each
\hyperref[Ballot]{Ballot} that contains that \hyperref[Contest]{Contest} and that is reported in that
\hyperref[ReportingContext]{ReportingContext}.  For each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest}
(excluding ranked order) where there does not exist a \hyperref[Ballot]{Ballot} that
contains that \hyperref[Contest]{Contest} and that is reported in that \hyperref[ReportingContext]{ReportingContext}, the result
contains a single row with nulls in the BallotId and S\_val
columns.

\item The natural left outer join with Ballot adds the Accepted column
but does not change the number of rows.  Where BallotId is null, Accepted
is also null.
\end{enumerate}



\subsubsection{\texorpdfstring{$S'(c,r,t,v)$}{S'(c,r,t,v)}}

\begin{quote}
Ballot $v$'s vote with respect to contest choice $c$ in contest
$r$ as accepted for counting purposes (i.e., valid votes only), as of
time $t$.
\[ t \geq t_{E} \rightarrow S'(c,r,t,v) = \left\{ \begin{array}{ll} S(c,r,D(v),v)\;\; & \mbox{if $S(r,D(v),v) \leq N(r) \wedge A(t,v)$} \\ 0 & \mbox{otherwise} \end{array} \right. \]
\end{quote}

The quaternary function $S'$ is implemented by the view SPrime.  The
current value of $S'(c,r,t,v)$ is obtained by selecting SPrime\_val
where ChoiceId = $c$ and BallotId = $v$.

\begin{verbatim}
create view SPrime (ChoiceId, BallotId, SPrime_val) as
  select ChoiceId, BallotId,
    case
      when S_val <= N and Accepted then Value
      else 0
    end
    from EffectiveInput
      natural join Choice
      natural join Contest
      natural join Ballot
      natural join S;
\end{verbatim}

\begin{assertion}
\label{SPrime-Assertion}
For each \hyperref[Ballot]{Ballot}, for each canonical \hyperref[Choice]{Choice} for which that \hyperref[Ballot]{Ballot} could contain a vote, excluding ranked order \hyperref[Contest]{Contests},
column SPrime\_val of SPrime contains the value specified by the definition
of $S'(c,r,t,v)$
for $c$ = ChoiceId, 
$r$ = Choice.ContestId, $t$ as of the state of the database, and
$v$ = BallotId.
\end{assertion}

\begin{enumerate}
\item Per \hyperref[S4-Assertion]{Assertion \ref{S4-Assertion}}, the quaternary function $S$ is implemented by the view EffectiveInput.
Per \hyperref[EffectiveInput-Assertion]{Assertion \ref{EffectiveInput-Assertion}},
EffectiveInput contains exactly one row for each
canonical \hyperref[Choice]{Choice} for which the \hyperref[Ballot]{Ballot} could contain a vote.

\item The natural joins with Choice, then Contest, then Ballot add
columns but do not change the number of rows.

\item The natural join with S eliminates rows pertaining to \hyperref[Choice]{Choices} in ranked order
\hyperref[Contest]{Contests} and adds the S\_val column (providing $S(r,t,v)$) to the others.

\item For each \hyperref[Ballot]{Ballot}, for each canonical \hyperref[Choice]{Choice} for which the \hyperref[Ballot]{Ballot}
could contain a vote, the case statement generates the value of
$S'(c,r,t,v)$ as specified above.

\end{enumerate}


\subsubsection{\texorpdfstring{$T(c,j,r,t)$}{T(c,j,r,t)}}

\begin{quote}
The vote total for contest choice $c$ in contest $r$ and
reporting context $j$ as of time $t$.  This does not include votes
that are invalid due to overvoting or votes from ballots for which
$A(t,v)$ is false.
\[ t \geq t_{E} \rightarrow T(c,j,r,t) = \!\!\!\!\!\!\! \sum_{v \in V(j,t_{E})} \!\!\!\!\!\!\! S'(c,r,t_{E},v) \]
\end{quote}

The quaternary function $T$ is implemented by the view T\@.  The current
value of $T(c,j,r,t)$ is obtained by selecting T\_val where ChoiceId =
$c$ and ReportingContext = $j$.

\begin{verbatim}
create view T (ChoiceId, ReportingContext, T_val) as
  select ChoiceId, ReportingContext, coalesce (sum (SPrime_val), 0)
    from FilteredContextChoiceAssociation
      natural left outer join
        (SPrime natural join ReportingContextAssociationMerge)
    group by ChoiceId, ReportingContext;
\end{verbatim}

\begin{assertion}
For each \hyperref[ReportingContext]{ReportingContext}, for each canonical \hyperref[Choice]{Choice} in each non-ranked-order
\hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, column T\_val of
T contains the value specified by the definition of $T(c,j,r,t)$ for
$c$ = ChoiceId, $j$ = ReportingContext, $r$ = Choice.ContestId, and $t$ as of the state of the database.
\end{assertion}

\begin{enumerate}
\item Per \hyperref[FilteredContextChoiceAssociation-Assertion]{Assertion \ref{FilteredContextChoiceAssociation-Assertion}}, for each \hyperref[ReportingContext]{ReportingContext}, FilteredContextChoiceAssociation contains
exactly one row for each canonical \hyperref[Choice]{Choice} in each non-ranked-order
\hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}.

\item Per \hyperref[SPrime-Assertion]{Assertion \ref{SPrime-Assertion}},
for each \hyperref[Ballot]{Ballot}, for each canonical \hyperref[Choice]{Choice} for which that \hyperref[Ballot]{Ballot} could contain a vote, excluding ranked order \hyperref[Contest]{Contests},
column SPrime\_val of SPrime contains the value specified by the definition
of $S'(c,r,t,v)$.

\item Per \hyperref[ReportingContextAssociationMerge-Assertion]{Assertion \ref{ReportingContextAssociationMerge-Assertion}}, for each \hyperref[Ballot]{Ballot}, ReportingContextAssociationMerge contains exactly one
row for each relevant \hyperref[ReportingContext]{ReportingContext}.

\item For each \hyperref[Ballot]{Ballot}, SPrime natural join ReportingContextAssociationMerge 
yields the Cartesian product of the SPrime rows with
the ReportingContextAssociationMerge rows.

\item For each \hyperref[ReportingContext]{ReportingContext}, the natural left outer join of
FilteredContextChoiceAssociation with the previous result adds a single row
with a null in the SPrime\_val column for each canonical \hyperref[Choice]{Choice} in
each non-ranked-order \hyperref[Contest]{Contest} for which there were no relevant
\hyperref[Ballot]{Ballots}.  The rows already existing from the previous result are
not changed.

\item For each \hyperref[ReportingContext]{ReportingContext}, for each canonical \hyperref[Choice]{Choice} in each
non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}
and for which at least one relevant \hyperref[Ballot]{Ballot} exists,
the value specified by the definition of $T(c,j,r,t)$ follows directly
by summing the SPrime\_val column while grouping by ChoiceId and
ReportingContext.

\item For the case in which no relevant \hyperref[Ballot]{Ballot} exists, the null that
is returned by the summation operation on the single null value is
changed to zero by coalesce (sum (SPrime\_val), 0).

\end{enumerate}

TSum is a convenience that sums T\_val by \hyperref[Contest]{Contest}.
Note that TSum does not eliminate ranked order contests but rather
provides a value of zero for them in the TSum\_val column.

\begin{verbatim}
create view TSum (ContestId, ReportingContext, TSum_val) as
  select ContestId, ReportingContext, coalesce (sum (T_val), 0)
    from ReportingContextContestAssociation
      natural left outer join Choice
      natural left outer join T
    group by ContestId, ReportingContext;
\end{verbatim}

\begin{assertion}
\label{TSum-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, column TSum\_val of TSum contains the value
$\sum_{c \in C(r,t)} T(c,j,r,t)$ for $j$ = ReportingContext, $r$ = ContestId and $t$ as of the state of the database.
\end{assertion}

\begin{enumerate}
\item Per \hyperref[ReportingContextContestAssociation-Assertion]{Assertion \ref{ReportingContextContestAssociation-Assertion}}, for each \hyperref[ReportingContext]{ReportingContext}, ReportingContextContestAssociation contains exactly one row for each \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}.

\item For each \hyperref[ReportingContext]{ReportingContext}, ReportingContextContestAssociation
      natural left outer join Choice contains one row for each \hyperref[Choice]{Choice}
in each \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, plus one row
with null in the ChoiceId column for each \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext} that has no associated \hyperref[Choice]{Choices}.

\item The natural left outer join of the previous result with T, using both the ChoiceId
and ReportingContext columns, has the same number of rows as the previous
result.  Each row pertaining to a \hyperref[Choice]{Choice} in some non-ranked-order \hyperref[Contest]{Contest} acquires T\_val supplying the value of $T(c,j,r,t)$.  Each row pertaining to a ranked order \hyperref[Contest]{Contest} or a \hyperref[Contest]{Contest} with
no associated \hyperref[Choice]{Choices} acquires T\_val containing a null value.

\item \sloppypar For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext} and that has at least one associated
\hyperref[Choice]{Choice}, the value specified by $\sum_{c \in C(r,t)} T(c,j,r,t)$ follows directly by summing the
T\_val column while grouping by ContestId and ReportingContext.

\item For ranked order \hyperref[Contest]{Contests} and \hyperref[Contest]{Contests} having no associated \hyperref[Choice]{Choices}, the null that
is returned by the summation operation on the single null value is
changed to zero by coalesce (sum (T\_val), 0).
\end{enumerate}


\subsubsection{\texorpdfstring{$O(j,r,t)$}{O(j,r,t)}}

\begin{quote}
For a given contest and reporting context, the number of overvotes in
read ballots for which $A(t,v)$ is true as of time $t$.  Each ballot
in which contest $r$ is overvoted contributes $N(r)$ to $O(j,r,t)$.
\[ t \geq t_{E} \rightarrow O(j,r,t) = \!\!\!\!\!\!\! \sum_{v \in V(j,t_{E})} \left\{ \begin{array}{ll} N(r)\;\; & \mbox{if $S(r,D(v),v) > N(r) \wedge A(t,v)$} \\ 0 & \mbox{otherwise} \end{array} \right. \]
\end{quote}

The ternary function $O$ is implemented by the view O\@.  The current
value of $O(j,r,t)$ is obtained by selecting O\_val where ContestId =
$r$ and ReportingContext = $j$.

\begin{verbatim}
create view O (ContestId, ReportingContext, O_val) as
  select ContestId, ReportingContext, sum (
      case
        when S_val > N and Accepted then N
        else 0
      end                                 )
    from VotesByContestAndContext
    group by ContestId, ReportingContext;
\end{verbatim}

\begin{assertion}
\label{O-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, column O\_val of O contains the value
specified by the definition of $O(j,r,t)$ for
$j$ = ReportingContext, $r$ = ContestId, and $t$ as of the state of the database.
\end{assertion}

\begin{enumerate}
\item Per \hyperref[VotesByContestAndContext-Assertion1]{Assertion \ref{VotesByContestAndContext-Assertion1}}, for each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest} (excluding ranked order), VotesByContestAndContext contains a row for each \hyperref[Ballot]{Ballot} that contains that
\hyperref[Contest]{Contest} and that is reported in that \hyperref[ReportingContext]{ReportingContext}.  Column Accepted is as specified
in the table Ballot.  Column S\_val contains the
value specified by the definition of $S(r,t,v)$ for
$r$ = ContestId, $t$ as of the state of the database, and
$v$ = BallotId.

\item Per \hyperref[VotesByContestAndContext-Assertion2]{Assertion \ref{VotesByContestAndContext-Assertion2}}, for each relevant combination of \hyperref[ReportingContext]{ReportingContext} and \hyperref[Contest]{Contest} (excluding ranked order)
where there does not exist a \hyperref[Ballot]{Ballot} that contains that
\hyperref[Contest]{Contest} and that is reported in that \hyperref[ReportingContext]{ReportingContext}, VotesByContestAndContext
contains a single row with nulls in the BallotId, Accepted, and S\_val
columns.

\item For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest}
that is relevant in that \hyperref[ReportingContext]{ReportingContext}, the value specified by the
definition of $O(j,r,t)$ follows directly by summing the result of the
case statement for each row while grouping by ContestId and
ReportingContext.  In the case where Accepted and S\_val are null,
i.e., where there does not exist a \hyperref[Ballot]{Ballot} that contains that \hyperref[Contest]{Contest}
and that is reported in that \hyperref[ReportingContext]{ReportingContext}, the case statement returns 0.
\end{enumerate}


\subsubsection{\texorpdfstring{$U(j,r,t)$}{U(j,r,t)}}

\begin{quote}
For a given contest and reporting context, the number of undervotes in
read ballots for which $A(t,v)$ is true as of time $t$.  A given
ballot contributes at most $N(r)$ to $U(j,r,t)$.  Ballot styles that
do not include contest $r$ do not contribute to this total.
\[ t \geq t_{E} \rightarrow U(j,r,t) = \!\!\!\!\!\!\! \sum_{v \in V(j,t_{E})} \left\{ \begin{array}{ll} N(r) - S(r,D(v),v)\;\; & \mbox{if $S(r,D(v),v) \leq N(r) \wedge A(t,v)$} \\ 0 & \mbox{otherwise} \end{array} \right. \]
\end{quote}

The ternary function $U$ is implemented by the view U\@.  The current
value of $U(j,r,t)$ is obtained by selecting U\_val where ContestId =
$r$ and ReportingContext = $j$.

\begin{verbatim}
create view U (ContestId, ReportingContext, U_val) as
  select ContestId, ReportingContext, sum (
      case
        when S_val <= N and Accepted then N - S_val
        else 0
      end                                 )
    from VotesByContestAndContext
    group by ContestId, ReportingContext;
\end{verbatim}

\begin{assertion}
\label{U-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, column U\_val of U contains the value
specified by the definition of $U(j,r,t)$ for
$j$ = ReportingContext, $r$ = ContestId, and $t$ as of the state of the database.
\end{assertion}

The argument is parallel to that of \hyperref[O-Assertion]{Assertion \ref{O-Assertion}}, substituting U for O.


\subsubsection{\texorpdfstring{$K(j,r,t)$}{K(j,r,t)}}

\begin{quote}
For a given contest and reporting context, the number of read ballots
for which $A(t,v)$ is true as of time $t$ (i.e., the number of ballots
that should be counted).  Ballot styles that do not include contest
$r$ do not contribute to this total.
\end{quote}

The ternary function $K$ is implemented by the view K\@.  The current
value of $K(j,r,t)$ is obtained by selecting K\_val where ContestId =
$r$ and ReportingContext = $j$.

\begin{verbatim}
create view K (ContestId, ReportingContext, K_val) as
  select ContestId, ReportingContext,
    (select count(*)
       from Ballot
         natural join ReportingContextAssociationMerge
         natural join BallotStyleContestAssociation
       where ReportingContextAssociationMerge.ReportingContext
         = FilteredContextContestAssociation.ReportingContext
       and BallotStyleContestAssociation.ContestId
         = FilteredContextContestAssociation.ContestId
       and Accepted)
    from FilteredContextContestAssociation;
\end{verbatim}

\begin{assertion}
\label{K-Assertion}
For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, column K\_val of K contains the value
specified by the definition of $K(j,r,t)$ for
$j$ = ReportingContext, $r$ = ContestId, and $t$ as of the state of the database.
\end{assertion}

\begin{enumerate}

\item Per
\hyperref[FilteredContextContestAssociation-Assertion]{Assertion
\ref{FilteredContextContestAssociation-Assertion}}, for each
\hyperref[ReportingContext]{ReportingContext}, FilteredContextContestAssociation contains exactly one row
for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}, and zero rows for each ranked-order \hyperref[Contest]{Contest}.

\item Per \hyperref[ReportingContextAssociationMerge-Assertion]{Assertion \ref{ReportingContextAssociationMerge-Assertion}}, for each \hyperref[Ballot]{Ballot}, ReportingContextAssociationMerge contains exactly one
row for each relevant \hyperref[ReportingContext]{ReportingContext}.

\item For each \hyperref[Ballot]{Ballot}, Ballot natural join
ReportingContextAssociationMerge contains exactly one row for each
relevant \hyperref[ReportingContext]{ReportingContext}.

\item For each \hyperref[Ballot]{Ballot}, for each relevant \hyperref[ReportingContext]{ReportingContext}, the
natural join of the previous result with BallotStyleContestAssociation
contains exactly one row for each \hyperref[Contest]{Contest} appearing on the \hyperref[Ballot]{Ballot}.
If the \hyperref[BallotStyle]{BallotStyle} has no associated \hyperref[Contest]{Contests}, there are zero such rows.

\item For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest}
that is relevant in that \hyperref[ReportingContext]{ReportingContext}, the value specified by the
definition of $K(j,r,t)$ follows directly from counting the number of
rows in the previous three-way join that have a matching \hyperref[ReportingContext]{ReportingContext} and ContestId, and where Accepted is true.

\end{enumerate}


\subsubsection{Balance}

\begin{quote}
Every vote must be accounted for.
\[ t \geq t_{E} \rightarrow \!\!\!\! \sum_{c \in C(r,t)} \!\!\!\!\! T(c,j,r,t) + O(j,r,t) + U(j,r,t) = K(j,r,t) \times N(r) \]
\end{quote}

A check for this assertion is implemented by the view Balance.
The current difference between $\sum_{c \in C(r,t)} T(c,j,r,t) +
O(j,r,t) + U(j,r,t)$ and $K(j,r,t) \times N(r)$ is obtained by
selecting Discrepancy where ContestId = $r$ and ReportingContext =
$j$.  Discrepancy should always be zero.

\begin{verbatim}
create view Balance (ContestId, ReportingContext, Discrepancy) as
  select ContestId, ReportingContext, K_val * N - (TSum_val + O_val + U_val)
    from K
      natural join TSum
      natural join O
      natural join U
      natural join Contest;
\end{verbatim}

\begin{assertion}
\sloppypar
For each \hyperref[ReportingContext]{ReportingContext}, for each
non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext},
column Discrepancy of Balance contains the value $K(j,r,t) \times N(r)
- \left( \sum_{c \in C(r,t)} T(c,j,r,t) + O(j,r,t) + U(j,r,t) \right)$
for $j$ = ReportingContext, $r$ = ContestId, and $t$ as of the state
of the database.
\end{assertion}

\begin{enumerate}
\item Per
\hyperref[TSum-Assertion]{Assertion \ref{TSum-Assertion}},
\hyperref[O-Assertion]{Assertion \ref{O-Assertion}},
\hyperref[U-Assertion]{Assertion \ref{U-Assertion}} and
\hyperref[K-Assertion]{Assertion \ref{K-Assertion}},
the views K, TSum, O and U each provide one of the needed values for
each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest} that is relevant in that \hyperref[ReportingContext]{ReportingContext}.  (TSum additionally provides rows for ranked order
\hyperref[Contest]{Contests} that will be eliminated.)

\item The successive natural joins of K, TSum, O and U all occur on
the columns ContestId and ReportingContext.  Each join adds a column
but does not change the number of rows.  The rows in TSum pertaining to
ranked order contests are eliminated by the first join.

\item The natural join with Contest (on the column ContestId) adds the
column N but does not change the number of rows.

\item For each \hyperref[ReportingContext]{ReportingContext}, for each non-ranked-order \hyperref[Contest]{Contest}
that is relevant in that \hyperref[ReportingContext]{ReportingContext}, the value specified above
follows directly from the expression K\_val * N $-$ (TSum\_val + O\_val
+ U\_val).

\end{enumerate}


\section{Advanced test development environment}
\label{TestSuite}

\subsection{Software prerequisites}

All software, tools and materials were developed on a GNU/Linux operating
system.

The following packages are required in order to build and run the
programs in Votetest:

\begin{tabular}{llll}
Name                        & Short name & Version tested & Source \\ \hline
PostgreSQL                  & postgres   & 8.3.7\footnotemark[4] & \cite{postgres} \\
Class Library for Numbers   & CLN        & 1.2.2          & \cite{CLN} \\
GNU Compiler Collection     & g++        & 4.3.3          & \cite{g++} \\
Flex: Fast Lexical Analyzer & flex       & 2.5.35         & \cite{flex} \\
Bison: GNU parser generator & bison      & 2.3            & \cite{bison} \\
\end{tabular}

\footnotetext[4]{Some planner problems with outer joins that interfered with
the operation of the schema were fixed in version 8.2.5 (see Bug \#3426).}
\setcounter{footnote}{4}

Votetest uses extensions to the SQL standard \cite{SQL} and the C++
standard \cite{C++} that might not function as intended with other
databases and compilers.

Some help in configuring PostgreSQL is provided in \autoref{PostgresHelp}.


\subsection{Hardware prerequisites}
\label{hardware}

Votetest was developed on a PC having a 3.6 GHz Pentium 4 processor, 1 GiB of
RAM, and an 80 GB SATA hard drive.  The resources of this PC were more than
adequate for all tests in the basic test suite, and if necessary a lesser
configuration should be usable.  Performance limitations became obvious only
with the large and complex scenarios generated for the scalability testing
described in \autoref{Scalability}.


\subsection{File listing}
\label{AdvancedFileListing}

Files pertaining to the advanced test development environment are described
in \autoref{AdvancedFileListTable}.

\LTXtable{6.5in}{FileListing}


\subsection{Installation}
\label{INSTALL}

Votetest is packaged with GNU automake \cite{automake}, so all
usual GNU tricks should work.  Help on configuration options can be
found in the INSTALL file or obtained by entering\
\verb|./configure --help|.

Normally, one should only need to do the following to compile
\hyperref[ReportGenerator]{ReportGenerator}
and
\hyperref[TestGenerator]{TestGenerator}.

\begin{verbatim}
bash-3.1$ ./configure
bash-3.1$ make
\end{verbatim}

However, in the event that PostgreSQL and/or CLN are installed in
nonstandard locations, an invocation such as the following might be
required.

\begin{verbatim}
bash-3.1$ ./configure \
> CPPFLAGS="-I/usr/local/pgsql/include -I/usr/local/cln-1.1.13/include" \
> LDFLAGS="-L/usr/local/pgsql/lib -L/usr/local/cln-1.1.13/lib"
bash-3.1$ make
\end{verbatim}


\subsection{Infrastructure}
\label{Infrastructure}


\subsubsection{runTest}
\label{runTest}

The script runTest is used to run an SQL test case against the database
and report the results from the Votetest environment.

A test case is executed by changing the current working directory to the
directory containing the test suite and invoking the runTest script with the
file name of the test case.  The runTest script resets the database to an
initial state and then feeds the test case to the SQL interpreter.  No database
named votetest other than the one created by the test suite should exist or it
will be destroyed without warning.

\begin{verbatim}
Usage: ./runTest [--kill-overvotes] test-file-name.sql
\end{verbatim}

Some test cases involve overvoting, which means they cannot be executed as-is
on a system that prevents overvoting.  If the --kill-overvotes switch is used,
the test case is transformed into one that can be executed on a system that
prevents overvoting.  Otherwise, overvotes are processed and reported as they
would be in a system that supports overvoting.

If a ballot overvotes a contest, --kill-overvotes removes all of that ballot's
votes in that contest, effectively converting overvotes into undervotes.  This
of course changes the expected results of the test case.  The expected results
in the normal configuration and with --kill-overvotes enabled are saved in the
subdirectories sample\_output and sample\_output\_kill-overvotes respectively.


\subsubsection{runAllTests}
\label{runAllTests}

The script runAllTests invokes runTest for each test case and directs the
output into a file in the subdirectory named ``output.''  It then invokes
runTest --kill-overvotes for each test case and directs the output into a file
in the subdirectory named ``output\_kill-overvotes.''  These results may then
be compared with the contents of the sample\_output and
sample\_output\_kill-overvotes subdirectories to ensure that the test suite is
operating as expected.


\subsubsection{ReportGenerator}
\label{ReportGenerator}

Usage:  \verb|ReportGenerator [-v] context-name [context-name...]|.  A no-frills,
plain-ASCII post-voting report for each specified
\hyperref[ReportingContext]{ReportingContext} is sent to standard
output.  As a convenience to test labs, the report total volume needed
for the accuracy test protocol of the VVSG is also calculated and
reported.  A sample report is shown in \autoref{ReportFigure}.

The verbose flag (-v) is only useful in ranked order contests.  It
causes the state of the ranked order logic (with many ballot images)
to be output for each round of voting.

\begin{figure}
\caption{Sample report}
\capsep
% FIXME: find a way to center verbatim properly.
\begin{verbatim}
         Report for context Precinct 1 generated 2007-03-21 09:19-0400

                                          BALLOT COUNTS

         Configuration                                  Read    Counted
         -------------                                  ----    -------
         Total                                            13         13
                                            Blank          1          1
         Precinct 1 Style                                 13         13
                                            Blank          1          1

                                           VOTE TOTALS

         Straight party, vote for at most 1
         Bipartisan Party                                  1
         Moderate Party                                    1
         Overvotes                                         1
         Undervotes                                       10
         Counted ballots                                  13
         Balance                                           0

         President, vote for at most 1
         Car Tay Fower                                     4
         Tayra Tree                                        3
         Beeso Tu (Moderate Party)                         2
         Oona Won (Bipartisan Party)                       1
         Nada Zayro                                        0
         Overvotes                                         1
         Undervotes                                        2
         Counted ballots                                  13
         Balance                                           0

         Report total volume: 108
           - Includes optional reporting of blank ballots.
           - Excludes separate reporting of ballots cast vs. read.
\end{verbatim}
\label{ReportFigure}
\end{figure}

ReportGenerator is normally invoked by individual test cases and
need not be used directly.  If it is invoked from a shell script, the
codes that it returns to the shell are listed in
\autoref{ReturnCodeTable}.  A return of 0 indicates success; other
values indicate one or more problems as encoded by individual bits.
Consult the standard error output of the program for additional
details on the failure or failures that occurred.

\begin{table}
\caption{ReportGenerator return codes}
\capsep
\begin{center}
\begin{tabular}{|l|l|}
\multicolumn{1}{c}{\bf Bit} & \multicolumn{1}{c}{\bf Meaning} \\ \hline
00001 & Incorrect usage \\ \hline
00010 & No such reporting context \\ \hline
00100 & Exception on attempt to connect to database \\ \hline
01000 & Exception while connected \\ \hline
10000 & Exception on attempt to disconnect from database \\ \hline
\end{tabular}
\end{center}
\label{ReturnCodeTable}
\end{table}

If an error similar to the following occurs when ReportGenerator
is invoked:

\begin{verbatim}
ReportGenerator/ReportGenerator: error while loading shared libraries:
  libecpg.so.5: cannot open shared object file: No such file or directory
\end{verbatim}

The solution is to add a command like the following to
$\sim$/.bash\_profile or another script that is always executed,
specifying the location of the library that was not found.

\begin{verbatim}
export LD_LIBRARY_PATH=/usr/local/pgsql/lib
\end{verbatim}

The algorithm used for ranked order contests is only one example of
conforming behavior.  This algorithm is not recommended or endorsed by the
National Institute of Standards and Technology for use in elections and it is
probably not the best algorithm available for the purpose.  It is used in
Votetest only to provide output for comparison in simple cases where the
implementation-dependent details have no impact.

\begin{itemize}
\item The quota is Hagenbach-Bischoff plus epsilon.
\item Surpluses are transferred via the Gregory method using unlimited precision rational numbers.
\item No special cases are handled.  This means:  every \hyperref[Choice]{Choice} must be ranked on every \hyperref[Ballot]{Ballot}; every \hyperref[Choice]{Choice} must be assigned a different rank; only one \hyperref[Choice]{Choice} is elected or eliminated at a time; and if a tie occurs, the algorithm halts.
\end{itemize}


\subsubsection{Features}
\label{Features}

Infrastructure-Features.sql creates the following table, which maps test cases
to system capabilities (VVSG classes):

\begin{verbatim}
create table Features (
  Test                           Text     primary key,

  PrimaryElections               Boolean  not null default false,
  AbsenteeVoting                 Boolean  not null default false,
  AbsenteeByCategories           Boolean  not null default false,
  SplitPrecincts                 Boolean  not null default false,
  BallotRotation                 Boolean  not null default false,
  WriteIns                       Boolean  not null default false,
  CumulativeVoting               Boolean  not null default false,
  NofMVoting                     Boolean  not null default false,
  RankedOrderVoting              Boolean  not null default false,
  ProvisionalChallengedBallots   Boolean  not null default false,
  StraightPartyVoting            Boolean  not null default false,
  CrossPartyEndorsement          Boolean  not null default false,

  check (StraightPartyVoting or not CrossPartyEndorsement),
  check (AbsenteeVoting or not AbsenteeByCategories)
);
\end{verbatim}

One could find the set of test cases applicable to a system that lacks support
for certain features (e.g., ranked order voting and straight party voting) as
follows:

\begin{verbatim}
bash-3.1$ psql votetest
[... PostgreSQL interactive terminal starts ...]

votetest=# \i Infrastructure-Features.sql
[... Features table is created ...]

votetest=# select Test from Features where not RankedOrderVoting
votetest-#   and not StraightPartyVoting;
[... Test cases are listed ...]
\end{verbatim}


\subsubsection{PairsCoverage}
\label{PairsCoverage}

While the purpose of Features is to recall all test cases that apply for a
given set of supported features, the purpose of PairsCoverage is to establish
that test cases exist for every pair of features.  In PairsCoverage, subclassed
voting variations are treated separately to establish that a test case exists
for both cases.  Since all meaningful pairings are now covered, the
PairsCoverage tool is no longer being maintained or updated as new test cases
are added.

PairsCoverage is prepared and run as follows:

\begin{verbatim}
bash-3.1$ psql votetest < Infrastructure-PairsCoverage.sql
bash-3.1$ ./Infrastructure-PairsCoverage > matrix.html
\end{verbatim}

Infrastructure-PairsCoverage outputs the content of \autoref{PairsTable} in
HTML form.  A key to the row and column headings is provided in
\autoref{PairsKey}.

\begin{table}
\caption{Output of Infrastructure-PairsCoverage}
\bigcapsep
\begin{center}
\begin{tabular}{ccccccccccccc}
& {\bf PE} & {\bf AV} & {\bf ABC} & {\bf SP} & {\bf BR} & {\bf WI} & {\bf CV} & {\bf NMV} & {\bf ROV} & {\bf PCB} & {\bf SPV} & {\bf CPE} \\
{\bf PE} & 12 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\
{\bf AV} & 1 & 13 & N/A & 1 & 1 & 2 & 1 & 1 & 1 & 1 & 1 & 1 \\
{\bf ABC} & 1 & N/A & 12 & 1 & 1 & 2 & 1 & 1 & 1 & 1 & 1 & 1 \\
{\bf SP} & 1 & 1 & 1 & 13 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\
{\bf BR} & 1 & 1 & 1 & 1 & 12 & 1 & 1 & 1 & 1 & 1 & 1 & 1 \\
{\bf WI} & 1 & 2 & 2 & 1 & 1 & 16 & 1 & 2 & 1 & 1 & 1 & 1 \\
{\bf CV} & 1 & 1 & 1 & 1 & 1 & 1 & 12 & 1 & 1 & 1 & 1 & 1 \\
{\bf NMV} & 1 & 1 & 1 & 1 & 1 & 2 & 1 & 13 & 1 & 1 & 1 & 1 \\
{\bf ROV} & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 12 & 1 & 1 & N/A \\
{\bf PCB} & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 12 & 1 & 1 \\
{\bf SPV} & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 11 & N/A \\
{\bf CPE} & 1 & 1 & 1 & 1 & 1 & 1 & 1 & 1 & N/A & 1 & N/A & 10
\end{tabular}
\end{center}
\label{PairsTable}
\end{table}

\begin{table}
\caption{Key to headings appearing in \autoref{PairsTable}}
\bigcapsep
\begin{center}
\begin{tabular}{lll}
\multicolumn{1}{c}{\bf Heading} & \multicolumn{1}{c}{\bf Test suite ID} & \multicolumn{1}{c}{\bf VVSG name} \\
PE & PrimaryElections & {\em Primary elections} \\
AV & AbsenteeVoting & {\em Absentee voting} \\
ABC & AbsenteeByCategories & {\em Absentee voting} \\
SP & SplitPrecincts & {\em Split precincts} \\
BR & BallotRotation & {\em Ballot rotation} \\
WI & WriteIns & {\em Write-ins} \\
CV & CumulativeVoting & {\em Cumulative voting} \\
NMV & NofMVoting & {\em N of M voting} \\
ROV & RankedOrderVoting & {\em Ranked order voting} \\
PCB & ProvisionalChallengedBallots & {\em Provisional / challenged ballots} \\
SPV & StraightPartyVoting & {\em Straight party voting} \\
CPE & CrossPartyEndorsement & {\em Cross-party endorsement}
\end{tabular}
\end{center}
\label{PairsKey}
\end{table}


\subsection{Test suite self-tests}
\label{BIST}

Self-tests are for the purpose of validating the test suite itself, not for
testing voting systems.  The integrity and self-testing features of a voting
system may or may not bear any resemblance to the integrity and self-testing
features of the test suite.

\subsubsection{Baseline}

The test case 0-integrity-Baseline.sql verifies that the integrity
views show no false positives on the base state for integrity tests.

\subsubsection{Constraint violations}

The operation of schema constructs designed to detect violations of
the constraints specified in \autoref{Constraints} is verified by test
cases that deliberately violate them.  The test cases are listed in
\autoref{ConstraintViolationTable}.

\begin{table}
\caption{Constraint violation tests}
\capsep
\begin{center}
\begin{tabular}{|l|l|}
\multicolumn{1}{c}{\bf Constraint} & \multicolumn{1}{c}{\bf Test case(s)} \\ \hline
\autoref{VoterInputValueConstraint} & 0-integrity-OutOfRangeInput.sql,
0-integrity-OutOfRangeEndorsement.sql \\ \hline
\autoref{MinimumNConstraint} & N/A, enforced by SQL check constraint \\ \hline
\autoref{MaxWriteInsConstraint} & N/A, enforced by SQL check constraint \\ \hline
\autoref{StraightPartyContestParametersConstraint} & N/A, enforced by SQL check constraint \\ \hline
\autoref{UnreportedBallotsConstraint} & 0-integrity-UnreportedBallots.sql \\ \hline
\autoref{ExtraneousVotesConstraint} & 0-integrity-ExtraneousInput.sql \\ \hline
\autoref{MoreThanOneStraightPartyContestConstraint} & 0-integrity-MoreThanOneStraightPartyContest.sql \\ \hline
\autoref{CircularEndorsementsConstraint} & 0-integrity-CircularEndorsement.sql \\ \hline
\autoref{StraightPartyChoiceNamesConstraint} & 0-integrity-NonExistentParties.sql \\ \hline
\autoref{PartyNamesUniqueConstraint} & N/A, enforced by SQL primary key constraint \\ \hline
\autoref{AliasDoubleVotesConstraint} & 0-integrity-AliasDoubleVotes.sql \\ \hline
\autoref{StraightPartyOverridesConstraint} & 0-integrity-StraightPartyOverrides.sql \\ \hline
\autoref{DoubleIndirectAliasesConstraint} & 0-integrity-DoubleIndirectAlias.sql \\ \hline
\autoref{CrossContestAliasesConstraint} & 0-integrity-CrossContestAliases.sql \\ \hline
\autoref{EndorsedAliasesConstraint} & 0-integrity-EndorsedAlias.sql \\ \hline
\autoref{TooManyWriteInsConstraint} & 0-integrity-TooManyWriteIns.sql \\ \hline
\end{tabular}
\end{center}
\label{ConstraintViolationTable}
\end{table}


\subsection{TestGenerator}
\label{TestGenerator}

The test generator is an extra testing tool that may be useful in the
creation of additional tests beyond those of the basic test suite, for
example, for volume testing.  However, it is not part of the basic
test suite.  A different approach to generating test data is detailed
in \autoref{LenTestDataGenerator}.

\begin{verbatim}
Usage:  TestGenerator input-filename > output.sql
\end{verbatim}

The input to the test generator is a plain text file having the
following format:

\begin{tabular}{ll}
Input: & ElectionSpec ContestSpec* \\
ElectionSpec: & NameValuePair+ \\
ContestSpec: & ( NameValuePair+ ) \\
NameValuePair: & Name = Value
\end{tabular}

Whitespace, /* C */ and // C++ style comments are ignored.

The permissible names and values are detailed below.

\subsubsection{Election specification}

The following values must be specified using name-value pairs:

\begin{itemize}
\item ballots = number of ballots
\item districts = number of districts
\item precincts = number of precincts
\end{itemize}

The following fields have default values and may be omitted:

\begin{itemize}
\item ballotDistribution = UniformRandom (default) or Even.
\item precinctDistribution = UniformRandom (default) or Even.
\end{itemize}

The ballot distribution controls the assignment of ballots to
precincts.  The precinct distribution controls the assignment of
precincts to districts.

\begin{quote}

{\bf UniformRandom}.  $X$ are assigned randomly to $Y$ such that on
average all $Y$ would get the name number of $X$; however, no specific
number is aimed for and it is unlikely that all $Y$ will in fact
receive exactly the same number of $X$.  The $X$ pertaining to each
$Y$ are not grouped (i.e., they do not appear with consecutive numbers
in the generated test).

{\bf Even}.  $X$ are deterministically divided as evenly as possible
across $Y$ and are grouped by $Y$.

\end{quote}

\subsubsection{Contest specification}

The following values must be specified using name-value pairs:

\begin{itemize}
\item N = per definition of N in \autoref{Contest} ($> 0$).
\item M = number of choices to generate ($\geq N$).
\end{itemize}

The following fields have default values and may be omitted:

\begin{itemize}
\item level = S (system extent, default), D (district), or P (precinct).
\item logic = N (N-of-M, default), C (cumulative), or R (ranked order).
\item distribution = UniformRandom (default) or Triangle.
\item W = number of write-in choices to generate ($0 \leq W \leq M$, default 0).
\end{itemize}

If a \hyperref[Contest]{Contest} is declared as system extent level,
one \hyperref[Contest]{Contest} that appears on every
\hyperref[Ballot]{Ballot} is generated.  If a
\hyperref[Contest]{Contest} is declared as district level, a separate
\hyperref[Contest]{Contest} is generated for each district and appears
only on \hyperref[Ballot]{Ballots} for that district.  If a
\hyperref[Contest]{Contest} is declared as precinct level, a separate
\hyperref[Contest]{Contest} is generated for each precinct and appears
only on \hyperref[Ballot]{Ballots} for that precinct.  The assignment
of precincts to districts is random.

Distribution controls how votes from individual ballots are distributed
to choices.

\begin{quote}

{\bf UniformRandom}.  For N-of-M and Cumulative contests, votes are
distributed randomly to contest choices such that on average they would
get the same number of votes; however, no specific tally is aimed for
and it is unlikely that all choices will in fact receive exactly the
same number of votes.  For ranked order contests, every ballot ranks
all of the ballot choices in random order.

{\bf Triangle}.  For N-of-M and Cumulative contests, votes are
deterministically assigned to contest choices such that, for some
integer $X \geq 0$, the first contest choice will receive $X$ votes,
the second contest choice will receive $2X$ votes, and so on.  $X$ is
made as large as possible for the available number of votes and
ballots (in an N-of-M contest, the tally for any contest choice cannot
exceed the number of ballots).  Any surplus votes are left as
undervotes.  For ranked order contests, every ballot ranks all of the
ballot choices in reverse order so that the highest numbered contest
choice is elected in the first round of voting, the second highest
numbered contest choice is elected in the second round of voting, etc.

\end{quote}

If $W$ is nonzero, the first $W$ choices by number become write-ins.
Thus, in a Triangle distribution, write-ins get fewer votes than other
choices.


\subsubsection{Example}
\label{TestSpecs}

\begin{verbatim}
ballots=200 districts=2 precincts=4
(level=S logic=N N=1 M= 7 distribution=Triangle)
(level=S logic=N N=1 M= 6 distribution=Triangle)
(level=D logic=N N=1 M= 4 distribution=Triangle)
(level=D logic=N N=1 M= 4 distribution=Triangle)
(level=D logic=N N=4 M=10 distribution=Triangle)
(level=P logic=N N=4 M=16 distribution=Triangle)
\end{verbatim}

In this example, the distribution of votes is deterministic, but the
assignment of ballots to precincts and precincts to districts is
random.  This randomness affects the number of votes and ballots
available in each contest, so the results in each contest will vary as
the Triangle distributions scale up or down accordingly.

Additional examples can be found in the test\_specs subdirectory of
the Votetest distribution.


\subsection{On performance and scalability}
\label{Scalability}

Votetest was designed with a preference for transparency of logic over
performance and scalability.  With the exception of ranked order,
tabulation logic is implemented in SQL and mirrors the logic model
defined in the VVSG\@.  This limits opportunities to introduce faults
but incurs a considerable performance penalty.

To improve performance in large and complex test cases,
ReportGenerator builds temporary tables corresponding to views that
are in or near the top level of the schema.  This avoids repeated
computation of views that are accessed many times during report
generation.  Nevertheless, for sufficiently large and complex test
cases, the construction of these temporary tables becomes I/O bound
and the time to generate reports therefore becomes quite long.

\autoref{ScaleTable} shows the run time for several test cases of significant
size and complexity as observed on the computer described in
\autoref{hardware}.  The parameters used in these examples are believed to
exceed the limits likely to be used in conformity assessment.  Performance
will vary by PostgreSQL version and configuration, by hardware configuration,
and by workload.

\begin{table}[bh]
\caption{Scalability figures}
\capsep
\begin{tabular}{rrrrrrrr}
Ballots & Districts & Precincts & Contests & Ballot positions & Votes & Run time \\
2000000 & 1 & 1 & 1 & 10000000 & 6000000 & 27 min \\
2000000 & 5 & 25 & 1 & 10000000 & 6000000 & 32 min \\
2000000 & 1 & 1 & 3 & 30000000 & 6000000 & 1 hr 6 min \\
120000  & 4 & 60 & $15 + 4 \times 4 + 60 = 91$ & 9600000 & 2400000 & 26 min \\
\end{tabular}
\label{ScaleTable}
\end{table}

Within reasonable limits, slow execution should not be a barrier to testing.
Test cases are typically executed by Votetest only once to obtain results for
comparison with those from actual voting system products.  The output from
every test case provided as part of the distribution is saved in the
sample\_output and sample\_output\_kill-overvotes subdirectories.  Any new
tests that are developed by test labs can similarly be executed in advance and
the results saved for future reference.

N.B., The --kill-overvotes test transformation option was not used during the
scalability tests.  Additional optimization may be required if it is necessary
to transform large test cases.


\subsection{PostgreSQL configuration help}
\label{PostgresHelp}

The following is intended to help someone unfamiliar with PostgreSQL
reach a usable configuration with minimal effort.  More complete
information is available in the PostgreSQL documentation.

The steps to reaching a usable configuration are:

\begin{enumerate}
\item Create the Unix account `postgres.'
\item Install the PostgreSQL software.
\item Initialize the database.
\item Customize the database configuration.
\item Start the database d\ae mon.
\item Grant database access to the testing account.
\end{enumerate}

\subsubsection{Create the Unix account `postgres'}

This step is performed according to the procedures of the Unix
operating system being used.  Commonly there is a shell script in
/usr/sbin for this purpose, that must be run as root.

The postgres account should be locked to prevent interactive log-ins:

\begin{verbatim}
bash-3.1# passwd -l postgres
Password changed.
\end{verbatim}

\subsubsection{Install the PostgreSQL software}

If a pre-built PostgreSQL package is not available, PostgreSQL may be
built from source code.

\begin{verbatim}
bash-3.1$ ./configure --prefix=/usr/local
bash-3.1$ make
bash-3.1$ su
bash-3.1# make install
\end{verbatim}

\subsubsection{Initialize the database}

Assuming that the database should go in /usr/share/data:

\begin{verbatim}
bash-3.1# mkdir /usr/share/data
bash-3.1# chown postgres. /usr/share/data
bash-3.1# su postgres
bash-3.1$ initdb -D /usr/share/data
\end{verbatim}

\subsubsection{Customize the database configuration}

The database configuration is customized by modifying the file
`postgresql.conf' in the data directory (/usr/share/data in this
example).

The following changes to the default configuration are recommended for
a database being used exclusively for Votetest.  They would not
necessarily be appropriate for a database that is also used for other
purposes.

\begin{itemize}
\item fsync = off
\item full\_page\_writes = off
\item checkpoint\_segments = 50
\end{itemize}

PostgreSQL supports several different approaches to access control.
If the database is to be used in a multi-user or networked environment,
please consult the PostgreSQL documentation to determine which access
control approach is optimal for your configuration.

\subsubsection{Start the database d\ae mon}

\begin{verbatim}
bash-3.1# touch /var/log/postgres
bash-3.1# chown postgres. /var/log/postgres
bash-3.1# su postgres -c "nohup /usr/local/bin/postgres -D /usr/share/data >&
/var/log/postgres < /dev/null &"
\end{verbatim}

The final command should be added to an /etc/rc.d script to automate
restarting the database after each system reboot.

\subsubsection{Grant database access to the testing account}

As user postgres, use the PostgreSQL command-line script `createuser' to
grant database access to the Unix account that will be running
Votetest (in this example, johndoe).

\begin{verbatim}
bash-3.1$ createuser johndoe
Shall the new role be a superuser? (y/n) y
CREATE ROLE
\end{verbatim}


\subsection{Votetest under Cygwin}
\label{Cygwin}

It is possible to compile and run the programs of the Votetest
distribution under Microsoft Windows using Cygwin \cite{cygwin}.
However, this configuration has known problems and has not been
thoroughly tested.

Procedures for installing and troubleshooting PostgreSQL under
Cygwin, along with some known problems, are documented in the
Installing PostgreSQL on Windows Using Cygwin FAQ \cite{cygfaq}.

Note also the following:

\begin{itemize}

\item The version of PostgreSQL that is available as a binary package for
Cygwin is usually too old, so the latest version must be built from source.

\item If the results of the configure script are used without modification, the
PostgreSQL server reports the error {\tt FATAL: setsid() failed: Operation not
permitted} at nondeterministic intervals, and the test cases that are in
progress at the time fail.  To avoid this problem, the file pg\_config.h
that is produced by the configure script must be patched as follows before
PostgreSQL is compiled.

\begin{verbatim}
sed --in-place --expression="s/#define HAVE_SETSID 1/#undef HAVE_SETSID/" \
    src/include/pg_config.h
\end{verbatim}

\item The PostgreSQL lib directory must be added to PATH\@.  Adding it
to LD\_LIBRARY\_PATH does not work.

\item In addition to running cygserver as described in the Installing
PostgreSQL on Windows Using Cygwin FAQ, it is necessary to set the
environment variable CYGWIN to the value server ({\tt export
CYGWIN=server}).

\item Some versions of the PostgreSQL server
encountered many errors of the form {\tt could not remove file or
directory "base/55958": Directory not empty} and leaked directories
under (data\_directory)/base at a significant rate.  As a workaround,
it is advisable to re-initialize the entire database at regular
intervals to recover the leaked resources.

\item According to the Installing PostgreSQL on Windows Using Cygwin
FAQ, Cygwin emulates local Unix sockets using Internet sockets that
are visible on the network, creating a security hazard.

\end{itemize}


\section{New test case walk-through}
\label{UseCase}

\subsection{Introduction}

To help in understanding the Votetest model, this section presents an
example election scenario, maps that example to both the Votetest data model
and the database schema that realizes it, and describes how one could proceed
to use that example as a testing scenario.

\autoref{DataModel} defines an abstract UML model to represent a potential
election and the vote counting capabilities of a voting system.  Portions of
the abstract model will be used at appropriate times to help in understanding
its relationship to the example presented herein.  The concrete realization
of that model as a relational database schema provided in
\autoref{Schema1} will also be used later in this section.

This section begins with a use case that may be analogous to the basic
requirements of a local election district and shows how Votetest might be
used to help create specific test cases relevant to that election district.

\autoref{LenSection2} presents an example election in layman's
terms. \autoref{LenSection3} explains how the features of that example get
represented in the Votetest abstract model. \autoref{LenSection4} shows how
the details of the example might be represented in the Votetest physical
model as tables in an SQL relational database. \autoref{LenSection5} gives
some assistance in how to use the Votetest model to generate specific test
cases, \autoref{LenSection6} presents several of the reports generated by
Votetest for the sample elections, and \autoref{LenSection7} draws some
general conclusions.


\subsection{Example election}
\label{LenSection2}

Consider a mid-sized county that serves as the election district for a
general election that includes federal, state and local contests. Suppose the
county is small enough that the federal contests are the same throughout the
county but large enough that the state and local contests vary.  Some state
contests may be larger than just the county, some may be included entirely
within the county, and some may be split across county lines. The local
contests are all within the county, but there may be political units within
the county that cross voting precincts and have different ballot styles.

The county needs to prepare for several different kinds of contests, including political offices where only a single candidate is the winner, political offices where there are multiple candidates and multiple winners (e.g., County Council seats), and ballot initiatives that are voted either up or down. A small municipality within the county allows cumulative voting for its City Council members, so the voting system for the county will also have to support cumulative voting for that contest. The Votetest model also supports straight-party voting and ranked order voting, but this county currently does not allow either of those voting styles in any contests. The county will have to support primary elections in the future with several different ballot styles within each precinct, but for simplicity this example is a general election with exactly one ballot style in each precinct.

Individual candidates may be affiliated with a political party and local law requires that for specific offices the party of the candidate be listed alongside their name on the ballot. It's also possible that local political parties may endorse candidates for offices that are considered non-political. In the Votetest model, party affiliations and party endorsements are modelled independently, the first to support labeling of party affiliation on the ballot and the second to support the straight-party voting style popular in some election districts. The independence of these two features allows conflicting affiliations versus endorsements, but that is a feature rather than a bug in the model since cross-party endorsements occur in some elections. This example election does not have any straight-party voting contests, and endorsements will not be printed on the ballot, but the county desires to track such endorsements for possible reporting purposes.

The county has a number of voting precincts and is required to report results by precinct as well as by state delegate districts that fall within the county. In some cases it may desire to report the results of ballot initiatives by geographic or political sub-divisions of the county. Each precinct has a number of voting machines, but all results are totalled within the precinct by a single precinct tabulator. For simplicity, we assume that precincts are the smallest unit requiring result reporting; however, the data model (\autoref{DataModel}) is capable of handling precincts split across political units.

By a fluke of election law, write-ins are not supported for federal contests
or for some state-wide contests, but they are usually allowed for local
offices, including multiple write-ins for offices that have multiple winners.
(This fluke is contrived to increase the variety of contest types in the
example.)  Additionally, the interpretation of write-ins and the crediting of
valid write-in votes to persons eligible to hold that office is deferred
until after the close of polls, in accordance with jurisdiction policies and
procedures.


\subsubsection{Federal and statewide contests}

\begin{itemize}
\item U.S. President---Three candidates, each registered by a recognized political party, no write-ins allowed.

\item U.S. Senate---Two candidates, each registered by one of the parties appearing in the presidential contest, no write-ins allowed.

\item U.S. House---Three candidates, two registered by the same parties as
for U.S. Senate, but the third registered with a 4\textsuperscript{th} party
different from any of those above, no write-ins allowed.

\item State Senate---Three candidates, registered with the same three parties as for U.S. House, no write-ins allowed.

\item State House---Multiple house districts within the county identified below.
\end{itemize}


\subsubsection{Local single winner contests}

\begin{itemize}
\item State House District \#1---Three candidates, registered with the same parties as for U.S. House, ballot must accommodate single potential write-in. District completely within the county.

\item State House District \#2---Three candidates, two registered with the same parties as U.S. Senate, but the third registered as an Independent, ballot must accommodate single potential write-in. District overlaps with another county so reporting by district (with county tabulators) only gives totals for this county.

\item State House District \#3---One candidate running unopposed, registered as an Independent. Ballot must accommodate single potential write-in. District completely within the county.
\end{itemize}

In each of the above contests, if the voter requests a write-in candidate for
a specific contest, the voting system will assist the voter to create the
write-in choice. The voting system makes no attempt to validate write-in
candidates for eligibility, but may appropriately restrict the number of
write-ins allowed for a given contest. Validation of write-ins will be
accomplished by a separate mechanism.

\subsubsection{Local County Council contest}

There are 10 candidates who have satisfied the requirements to be listed on the County Council ballot. Seven of the candidates are registered with one of the 4 political parties active in the county, one is a registered independent and two are not registered with any political party. The two not registered candidates are NOT registered Independents, so the ballot position allocated for political party affiliation for those two candidates must be left blank.

Each voter is allowed to vote for 4 candidates and may, if they wish, add up
to 4 write-in candidates. The 4 candidates with the 4 highest vote totals are
the winners. As with the single winner contests, there is no provision in
local election law for handling ties!

\subsubsection{Local ballot initiatives}

There are four ballot initiatives with Yes or No votes possible, but not all initiatives are county wide! The first two initiatives are county-wide and apply to every precinct. The third initiative applies only to precincts in State House District \#2 (the one that overlaps with another county). The fourth initiative applies only to two precincts that are their own municipality. The municipality crosses between State House districts 1 and 2.

\subsubsection{Municipal council contest}

There is a single municipality in the county that is holding a City Council
contest with 5 candidates who have satisfied the requirements to be listed on
the ballot. The selection process to appear on the ballot was done in a way
that precludes write-ins, so these five candidates are the only possible
candidates for two new City Council positions. In addition, this is a
non-political contest, so candidate affiliations are not modelled or shown on
the ballot; however, candidates may be endorsed, or not, by the political
parties. Each voter is allowed two votes, but the cumulative voting variation
is used, meaning that both votes could be cast for the same candidate. The two
candidates with the two highest vote totals are the winners. As with the
other elections above, there is no provision in local election law for
handling ties.

\subsubsection{Political parties}
\label{LenSection2.6}

The county requires that the voting system keep track of all political
parties for which a legal candidate for office has a declared affiliation and
for which there is a provision in election law to carry that affiliation on
the ballot. From the above it can be determined that there are five political
parties that must be represented in any voting system used by the county. We
assume that the five political parties have unique identifiers and unique
names as shown in \autoref{LenPartyTable}.

\begin{table}[htbp]
\caption{Party information}
\capsep
\begin{center}
\begin{tabular}{|l|l|l|}
\multicolumn{1}{c}{\bf PartyID} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf FormalName} \\ \hline
PP\_NAP & Action & National Action Party \\ \hline
PP\_SCP & Conservation & State Conservation Party \\ \hline
PP\_SRI & Independent & State Registered Independent \\ \hline
PP\_NMP & Moderate & National Moderate Party \\ \hline
PP\_NPP & Progressive & National Progressive Party \\ \hline
\end{tabular}
\end{center}
\label{LenPartyTable}
\end{table}

Local law provides that the Name of the recognized political party is the name that will be placed on the ballot when required by election law. The state Registered Independent Party is treated slightly differently by state law in that any person may register as a State Independent, any candidate may have that affiliation appear for them on a ballot, but state law prohibits Registered Independents from forming any organization that endorses candidates or takes positions on issues.

The National Moderate Party and National Progressive Party are the two political parties that have an affiliated candidate for each national and statewide contest. The National Action Party is the affiliation of the third party candidate in the Presidential contest and the State Conservation Party is the affiliation of the third party candidate in the U.S. House and State Senate seats. Two State House candidates are registered Independents.

Three County Council candidates have Progressive affiliation, two have Moderate affiliation, two have Conservation affiliation, one is a registered Independent, and two have no affiliation. Although it is theoretically possible for a candidate to be affiliated with more than one political party, that phenomenon is so rare as to not be captured in the Votetest abstract model. If a candidate is a member of more than one party, then the candidate will have to choose which single affiliation gets printed on the ballot.

The Moderate, Progressive and Conservation parties have each endorsed a
relatively full slate of candidates, including four County Council
candidates. Since not every party has 4 County Council candidates, they may
sometimes endorse a candidate from a rival party. The National Action party
has only endorsed a partial set of candidates, but for County Council has
endorsed the Independent candidate, the two Conservation candidates, and one
non-affiliated candidate. These endorsements will not be visible on the
ballot, which does not include straight-party voting, but the county would
like to record them in the election definition anyway.  The Moderate and
Progressive parties have endorsed positions on the countywide ballot
initiatives but not on local ballot initiatives. The Conservation party has
endorsed candidates and positions on all of the local contests and local
ballot initiatives.

\subsubsection{Precincts}
\label{LenSection2.7}

There are ten election precincts in the county distributed across the State
House districts, with four in H1, three in H2 and three in H3. Two of the
precincts make up the municipality that crosses districts 1 and 2.
\autoref{LenPrecinctTable} gives the relationships among the precincts, the
single State Senate district, the three State House districts, the single
municipality, and the four ballot initiatives.

\begin{table}[htbp]
\caption{Precinct relationships}
\capsep
\begin{center}
\begin{tabular}{|l|l|l|l|l|}
\multicolumn{1}{c}{\bf PrecinctID} & \multicolumn{1}{c}{\bf SS\_District} & \multicolumn{1}{c}{\bf SH\_District} & \multicolumn{1}{c}{\bf Municipality} & \multicolumn{1}{c}{\bf BallotInitiatives} \\ \hline
P01 & S1 & H1 &  & B1, B2 \\ \hline
P02 & S1 & H1 &  & B1, B2 \\ \hline
P03 & S1 & H1 &  & B1, B2 \\ \hline
P04 & S1 & H1 & M1 & B1, B2, B4 \\ \hline
P05 & S1 & H2 & M1 & B1, B2, B3, B4 \\ \hline
P06 & S1 & H2 &  & B1, B2, B3 \\ \hline
P07 & S1 & H2 &  & B1, B2, B3 \\ \hline
P08 & S1 & H3 &  & B1, B2 \\ \hline
P09 & S1 & H3 &  & B1, B2 \\ \hline
P10 & S1 & H3 &  & B1, B2 \\ \hline
\end{tabular}
\end{center}
\label{LenPrecinctTable}
\end{table}


\subsubsection{Reporting requirements}
\label{LenSection2.8}

State law requires that election results be reported by State Senate
districts, State House districts, counties, and precincts.  Local law
requires that results for all municipalities be reported separately, so the
single municipality that crosses State House districts 1 and 2 must be
reported separately.  Election law also requires election districts to
maintain statistics by ballot category, e.g., Absentee, Provisional,
Challenged, etc.


\subsection{Modelling the election in Votetest}
\label{LenSection3}

The first step in using Votetest to create test cases for this county
election is to model the candidates and county requirements for this election
as presented in \autoref{LenSection2}. Beginning with the Votetest abstract
model presented in \autoref{DataModel}, this section looks at subsets
of that model to capture the above example information.

\subsubsection{Ballot styles}
\label{LenSection3.1}

\autoref{LenPrecinctTable} indicates that there will need to be at least five
different ballot styles in order to capture the different ballot requirements
for the different contests in each of the precincts; one style for precincts
P1 through P3, a second and third styles for the unique requirements of
precincts P4 and P5, a fourth style for precincts P6 and P7, and a fifth
style for precincts P8 through P10. Some election districts may choose to
have as many ballot styles as they have precincts, reasoning that the
precinct number may be printed on the ballot, thereby giving each precinct a
unique ballot style. However, the election board of this county chooses to
model only the five ballot styles minimally necessary to represent all the
contests of this election.

The model subset shown in \autoref{LenFigure1} captures the required
information for a ballot style.
The BallotStyle class represents the structure of an
unvoted ballot. In this example, the BallotStyle class will have five
instances, each with a unique name. The county election board decides to
identify the ballot styles and give them unique names as shown in
\autoref{LenBallotStyleTable}.

\begin{figure}[htbp]
\caption{Model subset for ballot styles}
\bigcapsep
\centering
\includegraphics[scale=1.2]{LenFigure1.pdf}
\label{LenFigure1}
\end{figure}

\begin{table}[htbp]
\caption{Ballot styles}
% \capsep
\begin{center}
\begin{tabular}{|l|l|l|}
\multicolumn{1}{c}{\bf Local} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\
\multicolumn{1}{c}{\bf Id} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf Description} \\ \hline
S1 & H1B1B2 & House District \#1 countywide initiatives only \\ \hline
S2 & H1B1B2M1B4 & House District \#1 countywide and municipality contests \\ \hline
S3 & H2B1B2B3 & House District \#2 countywide and district initiatives \\ \hline
S4 & H2B1B2B3M1B4 & House District \#2 countywide, district and municipality contests \\ \hline
S5 & H3B1B2 & House District \#3 countywide initiatives only \\ \hline
\end{tabular}
\end{center}
\label{LenBallotStyleTable}
\end{table}

Unique identifiers for BallotStyles are implicit in the abstract Votetest
model, via object identity; the concrete database schema uses explicit
integer identifiers to achieve the same effect.

The many-to-many association between BallotStyle and Contest indicates that a contest may exist without being assigned to a ballot style, a ballot style consists of one or more contests, and a contest may appear on multiple different ballot styles.

The Contest class represents each of the contests described in
\autoref{LenSection2}.  In total there will be 13 contest instances: one for
President, one for U.S. Senate, one for U.S. House, one for State Senate,
three for State House, one for County Council, one for municipal City
Council, and four for ballot initiatives.  Ballot style S1 will be associated
with 8 contests:  one for President, one for U.S. Senate, one for U.S. House,
one for State Senate, one for State House, one for County Council, and two
for ballot initiatives.  Similarly, ballot style S2 will be associated with
10 contests, S3 with 9 contests, S4 with 11 contests, and S5 with 8 contests.

The Choice class represents the voting alternatives for each contest. The single winner candidate contests (e.g., President, Senate, House) will each have from 1 to 3 choices:  one choice for the unopposed candidate in State House District \#3 and two or three choices for each of the other single winner candidate contests. Note that the model allows a contest to have 0 choices; this is possible in the situation where a contest exists, no candidates have satisfied the requirements to be placed on the ballot, but write-in votes are allowed. Each write-in vote may result in the creation of a new choice instance, with the IsWriteIn attribute set to \emph{true}, but that possibility is not part of the original ballot definition.

The County Council contest will have 10 choices, one for each candidate whose
name appears on the ballot. Similarly, the municipal City Council contest
will have 5 choices. The four ballot initiative contests will each have two
choices, one for a Yes vote and one for a No vote.

The association between Contest and Choice is one-to-many; the solid diamond
represents that each choice instance is tightly bound to its parent
contest. It is not possible for a choice to exist apart from its parent class
or in more than one Contest.  If the same person is a candidate in more than
one contest, then two separate Choices exist, i.e., one for each Contest.

The many-to-one Affiliation association between Choice and Party maintains an optional affiliation for each candidate choice. A candidate may or may not have a party affiliation; if the affiliation exists and local law allows the affiliation to be printed on the ballot, it will be printed next to the candidate's name. Note that a Party instance may exist even if no candidate is affiliated with that party. Also note that in real life a candidate may have multiple party affiliations, but the model restricts the affiliation to at most one because there is no known election board that allows multiple affiliations to be listed on a ballot. Non-candidate choices (e.g., ballot initiatives) would normally not have any affiliation associations; this could be enforced, if desired, by a separate constraint.  However, ballot initiatives could have endorsements by political parties (c.f.\ \autoref{LenSection3.4}).


\subsubsection{Contest attributes}

The Contest class has five significant attributes. In addition, each instance of the class will have an instance identifier separate from the five attributes. The Description attribute identifies the contest, e.g., U.S. President, State Senate District \#5, State House District \#3, County Council, Municipal City Council, or Ballot Initiative \#2. For ballot initiatives the Description may also include a summary of the question being voted.

The CountingLogic attribute consists of one of the four enumeration values
indicated in \autoref{diagram} for ContestCountingLogic. All of the contests
except municipal City Council in this example are N-of-M; they may be
specifically 1-of-1, 1-of-2, 1-of-3, or 4-of-10, but all are identified as
N-of-M by this attribute.  The municipal City Council contest will have
CountingLogic set to Cumulative, since the candidates may receive multiple
votes on each ballot. The other two counting logics are not part of this
example, but are defined in the VVSG \cite{VVSGNG}.

For N-of-M and Cumulative contests, N identifies the number of votes that the
voter may allocate without overvoting.  Typically this is also the number of
winners in that contest, but not necessarily.  (For N-of-M and Cumulative
contests, the voting system only needs to gather votes and report the totals;
the picking of winners may be an external process impacted by election law,
late-breaking judicial rulings, etc.)

In this example, N will be set to 1 for all of the single-winner contests and
all of the ballot initiatives, to 4 for the County Council contest, and to 2
for the municipal City Council contest.

The MaxWriteIns attribute determines the provision that must be provided for write-ins by a voting system. In this example, all candidate contests will have this attribute set to 0, 1, 2, or 4, depending on whether or not write-ins are allowed for that contest. The candidate contests where write-ins are not allowed and all four of the ballot initiatives will have this attribute set to 0. The County Council contest will have it set to 4, the municipal City Council contest will have it set to 2, and all of the remaining candidate contests will have it set to 1.

The Rotate attribute determines whether or not the choices for a contest will have their positions on the ballot rotated during printing or display of the actual ballot instances. Note that ballot instances are represented in the model by the Ballot class (not yet discussed). This attribute is not relevant in the logic model of the VVSG, so for this example it could be set to \emph{true} or \emph{false} with no effect on the reporting requirements of the model.


\subsubsection{Voted ballots}
\label{LenSection3.3}

The Votetest abstract model makes a distinction between ballot structure and a voted ballot. Ballot structure is represented by the BallotStyle class whereas a voted ballot is represented by the Ballot class. Each Ballot instance is tied to exactly one ballot style and to one or more reporting contexts. The model subset in \autoref{LenFigure2} shows the attributes of a voted ballot and its potential relationships with other items.

\begin{figure}[htbp]
\caption{Model subset for ballots}
\bigcapsep
\centering
\includegraphics[scale=1.2]{LenFigure2.pdf}
\label{LenFigure2}
\end{figure}

A Ballot instance is associated with exactly one ballot style. The ballot
style determines the contests and the choices that will be presented to the
voter. For each contest on the ballot, the voter may select one or more of
the canonical choices, create a new write-in choice if that is permitted, or
not vote in that contest. A voter is not required to make any choices and
could turn in an unmarked ballot. The VoterInput class is included in the
abstract model to be able to handle cumulative and ranked order voting. In
this example, for all contests except municipal City Council, for each choice
selected on a ballot, the Value attribute on VoterInput for a given
association between Ballot and Choice carries Value = 1. In the municipal
City Council contest, since a voter may cast up to two votes for the same
candidate, for each choice selected on the ballot, the Value attribute on
VoterInput for a given association between Ballot and Choice could be set to
either 1 or 2.

A Ballot instance is required to be associated with at least one reporting
context. This is to guarantee that every ballot is accounted for and reported
in at least one context. In our example, Ballot instances are created by the
voting systems in each precinct, and precinct is the smallest reporting unit,
so there will be 10 precinct instances for the ReportingContext class and
each ballot will be associated with the relevant precinct. A ballot may also
be indirectly associated with different reporting contexts through its parent
ballot style. Other reporting contexts, both direct and indirect, are
presented in \autoref{LenSection3.5}.

A Ballot instance may be tagged with zero or more ballot categories chosen
from the tags appearing in the BallotCategory enumeration given in
\autoref{diagram}. The ballot category tags are not necessarily mutually
exclusive or collectively exhaustive; new tags may be added to the list to
support future acceptance or reporting rules. The purpose of the ballot
category tags is to clearly distinguish those ballots that may be handled or
reported according to a different set of rules. In our example, a regular
ballot will have an empty BallotCategory attribute, an absentee ballot will
carry an Absentee tag, and a provisional ballot will carry a Provisional tag.
The conditions under which categories can legally be added or deleted from a
ballot are specified by election law; controls on adding or deleting
categories may be implemented by the voting system or they may be implemented
procedurally.  Either way, the voting system must be able to produce reports
that properly reflect the categories that a ballot carries at the time the
report is produced.  In this section, only Absentee and Provisional
categories will be used.

The Accepted attribute on a ballot instance indicates whether or not a ballot
is to be counted. If Accepted is \emph{true}, as it normally is for ballots having
no unusual issues, then the ballot will be counted. If Accepted is \emph{false},
then the ballot may be read and accounted for by the system but no choices
for any contest will be counted. The Accepted attribute may be set by the
voting system according to rules embedded in the voting system itself, or
this attribute may be changed later by election officials after an analysis of
provisional or challenged ballots.  A voting system is generally not
responsible for the current state of the Accepted attribute for provisional
and challenged ballots; it must simply be able to produce reports that count,
or not, all read ballots, and produce accurate reports depending on how this
attribute is set at the time of reporting.

The Alias association on the Choice class is mostly used for write-in
reconciliation.  In this example, the voting system makes no attempt to
interpret write-ins on the fly:  Whenever a voter writes in a choice, the
system invariably creates a new Choice instance with IsWriteIn = \emph{true}.
Later, through a separate process with humans in the loop, write-ins are
``reconciled'' to determine how their votes are to be credited.  Each time a
write-in is found to refer to a previously defined Choice, an instance of the
Alias relationship is created to credit that write-in vote to the previously
defined Choice.  If a write-in is found to be completely new, no alias is
created for it, and it will be reported as a separate choice.

In this example, without Alias relationships, every single write-in would be
counted and reported as a distinct choice.  However, this is only one valid
approach.  In a different jurisdiction with different equipment, policies and
procedures, it would be perfectly valid, e.g., for an electronic voting
system to create only one Choice instance for each distinct string of
characters that some voter has ``written in'' and to refer to that same
Choice instance the second and subsequent times that that string of
characters appears.  This reduces the amount of reconciliation that must be
done later:  Every occurrence of the same string of characters will be
treated identically by the voting system.  If two different spellings of a
candidate's name were found to be equivalent, only one instance of the Alias
relationship would be needed to credit all of the affected votes
appropriately.

Rarely, aliases may be used to merge the counts for non-write-in choices.
For example, if election law forces there to be separate choices on the
ballot for each political party that endorsed a given candidate, these may be
aliased in order to report a single, consolidated total for that candidate.


\subsubsection{Provisions for endorsements}
\label{LenSection3.4}

The Endorsement class in the Votetest model exists for the purpose of
modelling elections that allow straight-party voting. This example election
does not include straight-party selections; however, for completeness, the
election board has decided to record the endorsements noted in
\autoref{LenSection2.6}.

The model subset shown in \autoref{LenFigure3} represents the independent
concepts of party affiliation and party endorsement.

\begin{figure}[htbp]
\caption{Model subset for affiliations and endorsements}
\bigcapsep
\centering
\includegraphics[scale=1.2]{LenFigure3.pdf}
\label{LenFigure3}
\end{figure}

Party and Choice classes and the Affiliation association were presented and
discussed in \autoref{LenSection3.1}.  The Value attribute in the Endorsement
class is similar to the Value attribute in the VoterInput class discussed in
the previous section, but instead of specifying the vote of a particular
voter for a particular choice, it specifies the vote that a particular party
\emph{recommends} that voters make.

For this example, there is a contest that uses Cumulative counting logic, but
only the Conservation party has decided to weight its endorsement for that
contest. All of the other party endorsements are simple endorsements of
choices without weighting. Thus the Value attribute will be set to 1 for all
associations between Party and Choice, except for the single association
between the Conservation party and its weighted choice for municipal City
Council. That association will have Value = 2.

Each of the four local parties have endorsed a full slate of candidates for County Council, have endorsed candidates with their own affiliation when that affiliation is known, and have endorsed a Yes or No vote on the two county-wide ballot initiatives.  None of the mainstream parties has taken a position on the non-county-wide ballot initiatives or has endorsed any of the municipal City Council candidates; however, the Conservation party did recommend a vote on all 4 ballot initiatives and endorsed a single candidate with double weight for the municipal City Council contest.

Conceptually, focusing only on the two council contests and the ballot initiatives, these endorsements might look like \autoref{LenEndorsementTable}.

\begin{table}[htbp]
\caption{Endorsements}
\capsep
\begin{center}
\begin{tabular}{|l|l|l|l|}
\multicolumn{1}{c}{\bf Party} & \multicolumn{1}{c}{\bf Contest} & \multicolumn{1}{c}{\bf Choice (Affiliation)} & \multicolumn{1}{c}{\bf Value} \\ \hline
Action & County Council & Candidate \#8 (Independent) & 1 \\ \hline
Action & County Council & Candidate \#4 (Conservation) & 1 \\ \hline
Action & County Council & Candidate \#9 (Conservation) & 1 \\ \hline
Action & County Council & Candidate \#6 (empty) & 1 \\ \hline
Conservation & County Council & Candidate \#4 (Conservation) & 1 \\ \hline
Conservation & County Council & Candidate \#9 (Conservation) & 1 \\ \hline
Conservation & County Council & Candidate \#7 (Moderate) & 1 \\ \hline
Conservation & County Council & Candidate \#5 (Progressive) & 1 \\ \hline
Conservation & Initiative \#1 & Yes & 1 \\ \hline
Conservation & Initiative \#2 & No & 1 \\ \hline
Conservation & Initiative \#3 & Yes & 1 \\ \hline
Conservation & Initiative \#4 & No & 1 \\ \hline
Conservation & Municipal Council & Candidate \#3 (Independent) & 2 \\ \hline
Moderate & County Council & Candidate \#1 (Moderate) & 1 \\ \hline
Moderate & County Council & Candidate \#9 (Conservation) & 1 \\ \hline
Moderate & County Council & Candidate \#2 (Progressive) & 1 \\ \hline
Moderate & County Council & Candidate \#7 (Moderate) & 1 \\ \hline
Moderate & Initiative \#1 & Yes & 1 \\ \hline
Moderate & Initiative \#2 & No & 1 \\ \hline
Progressive & County Council & Candidate \#10 (Progressive) & 1 \\ \hline
Progressive & County Council & Candidate \#5 (Progressive) & 1 \\ \hline
Progressive & County Council & Candidate \#2 (Progressive) & 1 \\ \hline
Progressive & County Council & Candidate \#7 (Moderate) & 1 \\ \hline
Progressive & Initiative \#1 & No & 1 \\ \hline
Progressive & Initiative \#2 & Yes & 1 \\ \hline
\end{tabular}
\end{center}
\label{LenEndorsementTable}
\end{table}

The physical representation of these endorsements as a normalized relational
database table is given in \autoref{LenEndorsementTableTable}.  Note that the
Contest column and the Affiliation in parentheses appearing in
\autoref{LenEndorsementTable} are both superfluous since each Choice is
linked to a unique Contest and to at most one Party; these convenience
details go away in the physical representation.


\subsubsection{Reporting of election results}
\label{LenSection3.5}

As stated in \autoref{LenSection2.8}, the county election board in this
example is required to report results by U.S. congressional district, by
State Senate and State House districts, by county, by municipality, by
precinct, and report ballot counts for at least three ballot categories,
i.e., Regular, Absentee and Provisional.  Some additional reporting may be
required later to account for other potential ballot categories, but
initially only these three are considered.  Reporting by county will produce
the same results as reporting by U.S. congressional district and reporting by
State Senate district, so only county reporting is considered in this
example.  The model subset shown in \autoref{LenFigure4} represents the basic
reporting requirements for this election.

\begin{figure}[htbp]
\caption{Model subset for reporting}
\bigcapsep
\centering
\includegraphics[scale=1.2]{LenFigure4.pdf}
\label{LenFigure4}
\end{figure}

It's already established that for this example there are ten precincts, one
election district, i.e., the county, and one tabulator for each precinct. All
reporting requirements by election district, or by tabulators, are satisfied
by including ReportingContext instances for one county (C1), 10 precincts
(P01 through P10), three State House districts (H1, H2, H3), and one
municipality (M1).  The report generator automatically breaks down ballot
counts by category in each reporting context, so there is no need to list the
ballot categories as separate reporting contexts.  The election board decides
to consider the Regular ballot category as implicit, so the Categories
attribute is left empty for every Regular ballot. The ReportingContext class
will have 15 explicit instances, each with a unique Name.

The voting system will record explicit associations between Ballot and ReportingContext for each Precinct. As stated earlier, this ensures that every accepted ballot will be counted in its home precinct.

The remaining reporting contexts, i.e., County, one municipality and 3 State House districts, are associated with ballots only indirectly through BallotStyle. \autoref{LenBSRCATable} records the explicit associations between ReportingContext and BallotStyle using the names of ballot styles from \autoref{LenSection3.1}. In addition, the County reporting context (C1) will be linked to all five ballot styles.

\begin{table}[htbp]
\caption{Ballot style---reporting context associations}
\capsep
\begin{center}
\begin{tabular}{|l|l|}
\multicolumn{1}{c}{\bf ReportingContext} & \multicolumn{1}{c}{\bf BallotStyle} \\ \hline
StateHouseD1 & H1B1B2 \\ \hline
StateHouseD1 & H1B1B2M1B4 \\ \hline
StateHouseD2 & H2B1B2B3 \\ \hline
StateHouseD2 & H2B1B2B3M1B4 \\ \hline
StateHouseD3 & H3B1B2 \\ \hline
Municipality1 & H1B1B2M1B4 \\ \hline
Municipality1 & H2B1B2B3M1B4 \\ \hline
\end{tabular}
\end{center}
\label{LenBSRCATable}
\end{table}


\subsection{Representing the election in the database}
\label{LenSection4}

\autoref{Schema1} translates the classes and associations of the abstract model into tables in the concrete relational database representation. In general, classes map to relational tables with the same attributes, many-to-one associations are represented as a reference attribute in the table, and many-to-many associations map to tables with two columns, representing the source and target instances of the association. Some associations have a third attribute to capture the Value attribute from the VoterInput or Endorsement classes in the abstract model. If a class has an obvious unique attribute, e.g., unique Name, it may be used as the primary key of the table; otherwise, a table primary key is added with an integer data type.

The Ballot table is initially empty, and so are any association tables where
a ballot is either the source or the target of the association. These tables
are filled dynamically during the election as ballots are cast. The initial,
non-empty tables, before voting, are shown in
\autoref{LenBallotStyleTableTable} through
\autoref{LenEndorsementTableTable}. Most tables contain a MyDescription
attribute not present in the Votetest model and not used during test
generation; it simply explains the attributes of the model in terms of the
example.

\begin{longtable}{|l|l|l|}
\caption{BallotStyle} \\
\multicolumn{1}{c}{\bf Style} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\
\multicolumn{1}{c}{\bf Id} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
\label{LenBallotStyleTableTable}
1 & H1B1B2 & S1---House District \#1 countywide initiatives only \\ \hline
2 & H1B1B2M1B4 & S2---House District \#1 countywide and municipality initiatives \\ \hline
3 & H2B1B2B3 & S3---House District \#2 countywide and district initiatives \\ \hline
4 & H2B1B2B3M1B4 & S4---House District \#2 countywide, district and municipality initiatives \\ \hline
5 & H3B1B2 & S5---House District \#3 countywide initiatives only \\ \hline
\end{longtable}


\begin{longtable}{|l|l|l|l|l|l|}
\caption{Contest} \\
\multicolumn{1}{c}{\bf ContestId} & \multicolumn{1}{c}{\bf Description} & \multicolumn{1}{c}{\bf CountingLogic} & \multicolumn{1}{c}{\bf N} & \multicolumn{1}{c}{\bf MaxWriteIns} & \multicolumn{1}{c}{\bf Rotate} \\ \hline
1 & President & N-of-M & 1 & 0 & Yes \\ \hline
2 & U.S. Senate & N-of-M & 1 & 0 & Yes \\ \hline
3 & U.S. House & N-of-M & 1 & 0 & Yes \\ \hline
4 & State Senate & N-of-M & 1 & 0 & Yes \\ \hline
5 & State House \#1 & N-of-M & 1 & 1 & Yes \\ \hline
6 & State House \#2 & N-of-M & 1 & 1 & Yes \\ \hline
7 & State House \#3 & N-of-M & 1 & 1 & Yes \\ \hline
8 & County Council & N-of-M & 4 & 4 & Yes \\ \hline
% FIXME KLUDGE FOR LONGTABLE LOSSAGE
9 & Ballot Initiative 1 & N-of-M & 1 & 0 & No \\ \hline % \pagebreak \hline
10 & Ballot Initiative 2 & N-of-M & 1 & 0 & No \\ \hline
11 & Ballot Initiative 3 & N-of-M & 1 & 0 & No \\ \hline
12 & Ballot Initiative 4 & N-of-M & 1 & 0 & No \\ \hline
13 & Muni City Council & Cumulative & 2 & 0 & No \\ \hline
\end{longtable}

% FIXME
\pagebreak
\begin{longtable}{|l|l|l|}
\caption{BallotStyleContestAssociation} \\
\multicolumn{1}{c}{\bf StyleId} & \multicolumn{1}{c}{\bf ContestId} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
1 & 1 & H1B1B2---Pres \\ \hline
1 & 2 & H1B1B2---US Senate \\ \hline
1 & 3 & H1B1B2---US House \\ \hline
1 & 4 & H1B1B2---State Senate \\ \hline
1 & 5 & H1B1B2---State House \\ \hline
1 & 8 & H1B1B2---Council \\ \hline
1 & 9 & H1B1B2---initiative B1 \\ \hline
1 & 10 & H1B1B2---initiative B2 \\ \hline
2 & 1 & H1B1B2M1B4---Pres \\ \hline
2 & 2 & H1B1B2M1B4---US Senate \\ \hline
2 & 3 & H1B1B2M1B4---US House \\ \hline
2 & 4 & H1B1B2M1B4---State Senate \\ \hline
2 & 5 & H1B1B2M1B4---State House H1 \\ \hline
2 & 8 & H1B1B2M1B4---Council \\ \hline
2 & 9 & H1B1B2M1B4---initiative B1 \\ \hline
2 & 10 & H1B1B2M1B4---initiative B2 \\ \hline
2 & 12 & H1B1B2M1B4---initiative B4 \\ \hline
2 & 13 & H1B1B2M1B4---City Council \\ \hline
3 & 1 & H2B1B2B3---Pres \\ \hline
3 & 2 & H2B1B2B3---US Senate \\ \hline
3 & 3 & H2B1B2B3---US House \\ \hline
3 & 4 & H2B1B2B3---State Senate \\ \hline
3 & 6 & H2B1B2B3---State House H2 \\ \hline
3 & 8 & H2B1B2B3---Council \\ \hline
3 & 9 & H2B1B2B3---initiative B1 \\ \hline
3 & 10 & H2B1B2B3---initiative B2 \\ \hline
3 & 11 & H2B1B2B3---initiative B3 \\ \hline
4 & 1 & H2B1B2B3M1B4---Pres \\ \hline
4 & 2 & H2B1B2B3M1B4---US Senate \\ \hline
4 & 3 & H2B1B2B3M1B4---US House \\ \hline
4 & 4 & H2B1B2B3M1B4---State Senate \\ \hline
4 & 6 & H2B1B2B3M1B4---State House H2 \\ \hline
4 & 8 & H2B1B2B3M1B4---Council \\ \hline
4 & 9 & H2B1B2B3M1B4---initiative B1 \\ \hline
4 & 10 & H2B1B2B3M1B4---initiative B2 \\ \hline
4 & 11 & H2B1B2B3M1B4---initiative B3 \\ \hline
4 & 12 & H2B1B2B3M1B4---initiative B4 \\ \hline
4 & 13 & H2B1B2B3M1B4---City Council \\ \hline
5 & 1 & H3B1B2---Pres \\ \hline
5 & 2 & H3B1B2---US Senate \\ \hline
5 & 3 & H3B1B2---US House \\ \hline
5 & 4 & H3B1B2---State Senate \\ \hline
5 & 7 & H3B1B2---State House H3 \\ \hline
5 & 8 & H3B1B2---Council \\ \hline
5 & 9 & H3B1B2---initiative B1 \\ \hline
5 & 10 & H3B1B2---initiative B2 \\ \hline
\end{longtable}


\begin{longtable}{|l|l|l|l|l|l|}
\caption{Choice (before write-ins)} \\
\multicolumn{1}{c}{\bf Choice} & \multicolumn{1}{c}{\bf Contest} & \multicolumn{2}{c}{} & \multicolumn{1}{c}{\bf Is} & \multicolumn{1}{c}{} \\
\multicolumn{1}{c}{\bf Id} & \multicolumn{1}{c}{\bf Id} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf Affiliation} & \multicolumn{1}{c}{\bf WriteIn} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
\label{LenSection4.4}
1 & 1 & PresidentC1 & Moderate & No & US President Candidate 1 \\ \hline
2 & 1 & PresidentC2 & Progressive & No & US President Candidate 2 \\ \hline
3 & 1 & PresidentC3 & Action & No & US President Candidate 3 \\ \hline
4 & 2 & USSenateC1 & Moderate & No & US Senate Candidate 1 \\ \hline
5 & 2 & USSenateC2 & Progressive & No & US Senate Candidate 2 \\ \hline
6 & 3 & USHouseC1 & Moderate & No & US House Candidate 1 \\ \hline
7 & 3 & USHouseC2 & Progressive & No & US House Candidate 2 \\ \hline
8 & 3 & USHouseC3 & Conservation & No & US House Candidate 3 \\ \hline
9 & 4 & SSenateC1 & Moderate & No & State House Candidate 1 \\ \hline
10 & 4 & SSenateC2 & Progressive & No & State House Candidate 2 \\ \hline
11 & 4 & SSenateC3 & Conservation & No & State House Candidate 3 \\ \hline
12 & 5 & SHouseD1C1 & Moderate & No & State House D1 Candidate 1 \\ \hline
13 & 5 & SHouseD1C2 & Progressive & No & State House D1 Candidate 2 \\ \hline
14 & 5 & SHouseD1C3 & Conservation & No & State House D1 Candidate 3 \\ \hline
15 & 6 & SHouseD2C1 & Moderate & No & State House D2 Candidate 1 \\ \hline
16 & 6 & SHouseD2C2 & Progressive & No & State House D2 Candidate 2 \\ \hline
17 & 6 & SHouseD2C3 & Independent & No & State House D2 Candidate 3 \\ \hline
18 & 7 & SHouseD3C1 & Independent & No & State House D3 Candidate 1 \\ \hline
19 & 8 & CCouncilC01 & Moderate & No & Council Candidate 1 \\ \hline
20 & 8 & CCouncilC02 & Progressive & No & Council Candidate 2 \\ \hline
21 & 8 & CCouncilC03 &  & No & Council Candidate 3 \\ \hline
22 & 8 & CCouncilC04 & Conservation & No & Council Candidate 4 \\ \hline
23 & 8 & CCouncilC05 & Progressive & No & Council Candidate 5 \\ \hline
24 & 8 & CCouncilC06 &  & No & Council Candidate 6 \\ \hline
25 & 8 & CCouncilC07 & Moderate & No & Council Candidate 7 \\ \hline
26 & 8 & CCouncilC08 & Independent & No & Council Candidate 8 \\ \hline
27 & 8 & CCouncilC09 & Conservation & No & Council Candidate 9 \\ \hline
28 & 8 & CCouncilC10 & Progressive & No & Council Candidate 10 \\ \hline
29 & 9 & Yes &  & No & Yes on Ballot Initiative 1 \\ \hline
30 & 9 & No &  & No & No on Ballot Initiative 1 \\ \hline
31 & 10 & Yes &  & No & Yes on Ballot Initiative 2 \\ \hline
32 & 10 & No &  & No & No on Ballot Initiative 2 \\ \hline
33 & 11 & Yes &  & No & Yes on Ballot Initiative 3 \\ \hline
34 & 11 & No &  & No & No on Ballot Initiative 3 \\ \hline
35 & 12 & Yes &  & No & Yes on Ballot Initiative 4 \\ \hline
36 & 12 & No &  & No & No on Ballot Initiative 4 \\ \hline
37 & 13 & MCCouncilC1 &  & No & \\ \hline
38 & 13 & MCCouncilC2 &  & No & \\ \hline
39 & 13 & MCCouncilC3 &  & No & \\ \hline
40 & 13 & MCCouncilC4 &  & No & \\ \hline
41 & 13 & MCCouncilC5 &  & No & \\ \hline
\end{longtable}


\begin{longtable}{|l|l|}
\caption{Party} \\
\multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
Action & National Action Party \\ \hline
Conservation & State Conservation Party \\ \hline
Independent & State Registered Independent \\ \hline
Moderate & National Moderate Party \\ \hline
Progressive & National Progressive Party \\ \hline
\end{longtable}


\begin{longtable}{|l|l|}
\caption{ReportingContext} \\
\multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
County1 & County \\ \hline
StateHouseD1 & State House District \#1 \\ \hline
StateHouseD2 & State House District \#2 \\ \hline
StateHouseD3 & State House District \#3 \\ \hline
P01 & Precinct 1 \\ \hline
P02 & Precinct 2 \\ \hline
P03 & Precinct 3 \\ \hline
P04 & Precinct 4 \\ \hline
P05 & Precinct 5 \\ \hline
P06 & Precinct 6 \\ \hline
P07 & Precinct 7 \\ \hline
P08 & Precinct 8 \\ \hline
P09 & Precinct 9 \\ \hline
P10 & Precinct 10 \\ \hline
Municipality1 & Municipality \\ \hline
\end{longtable}


\begin{longtable}{|l|l|l|}
\caption{BallotStyleReportingContextAssociation} \\
\multicolumn{1}{c}{\bf StyleId} & \multicolumn{1}{c}{\bf ReportingContext} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
1 & County1 & County---S1 \\ \hline
2 & County1 & County---S2 \\ \hline
3 & County1 & County---S3 \\ \hline
4 & County1 & County---S4 \\ \hline
5 & County1 & County---S5 \\ \hline
1 & StateHouse1 & House District \#1---Not in municipality---S1 \\ \hline
2 & StateHouse1 & House District \#1---In municipality---S2 \\ \hline
3 & StateHouse2 & House District \#2---Not in municipality---S3 \\ \hline
4 & StateHouse2 & House District \#2---In municipality---S4 \\ \hline
5 & StateHouse3 & House District \#3---S5 \\ \hline
2 & Municipality1 & Municipality---House District 1---S2 \\ \hline
4 & Municipality1 & Municipality---House District 2---S4 \\ \hline
\end{longtable}


\begin{longtable}{|l|l|l|l|}
\caption{Endorsement} \\
\multicolumn{1}{c}{\bf Party} & \multicolumn{1}{c}{\bf ChoiceId} & \multicolumn{1}{c}{\bf Value} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
\label{LenEndorsementTableTable}
Action & 3 & 1 & US President Candidate 3---Action \\ \hline
Action & 4 & 1 & US Senate Candidate 1---Moderate \\ \hline
Action & 7 & 1 & US House Candidate 2---Progressive \\ \hline
Action & 11 & 1 & State Senate Candidate 3---Conservation \\ \hline
Action & 22 & 1 & Council Candidate 4---Conservation \\ \hline
Action & 24 & 1 & Council Candidate 6 \\ \hline
Action & 26 & 1 & Council Candidate 8---Independent \\ \hline
Action & 27 & 1 & Council Candidate 9---Conservation \\ \hline
Action & 30 & 1 & No on Ballot Initiative 1 \\ \hline
Action & 31 & 1 & Yes on Ballot Initiative 2 \\ \hline
Conservation & 8 & 1 & US House Candidate 3---Conservation \\ \hline
Conservation & 11 & 1 & State Senate Candidate 3---Conservation \\ \hline
Conservation & 14 & 1 & State House D1 Candidate 3---Conservation \\ \hline
Conservation & 17 & 1 & State House D2 Candidate 3---Independent \\ \hline
Conservation & 22 & 1 & Council Candidate 4---Conservation \\ \hline
Conservation & 23 & 1 & Council Candidate 5---Progressive \\ \hline
Conservation & 25 & 1 & Council Candidate 7---Moderate \\ \hline
Conservation & 27 & 1 & Council Candidate 9---Conservation \\ \hline
Conservation & 29 & 1 & Yes on Ballot Initiative 1 \\ \hline
Conservation & 32 & 1 & No on Ballot Initiative 2 \\ \hline
Conservation & 33 & 1 & Yes on Ballot Initiative 3 \\ \hline
Conservation & 36 & 1 & No on Ballot Initiative 4 \\ \hline
Conservation & 39 & 2 & Municipal Council Candidate 3 \\ \hline
Moderate & 1 & 1 & US President Candidate 1---Moderate \\ \hline
Moderate & 4 & 1 & US Senate Candidate 1---Moderate \\ \hline
Moderate & 6 & 1 & US House Candidate 1---Moderate \\ \hline
Moderate & 9 & 1 & State Senate Candidate 1---Moderate \\ \hline
Moderate & 12 & 1 & State House D1 Candidate 1---Moderate \\ \hline
Moderate & 15 & 1 & State House D2 Candidate 1---Moderate \\ \hline
Moderate & 18 & 1 & State House D3 Candidate 1---Independent \\ \hline
Moderate & 19 & 1 & Council Candidate 1---Moderate \\ \hline
Moderate & 20 & 1 & Council Candidate 2---Progressive \\ \hline
Moderate & 25 & 1 & Council Candidate 7---Moderate \\ \hline
Moderate & 27 & 1 & Council Candidate 9---Conservation \\ \hline
Moderate & 30 & 1 & No on Ballot Initiative 1 \\ \hline
Moderate & 31 & 1 & Yes on Ballot Initiative 2 \\ \hline
Progressive & 2 & 1 & US President Candidate 2---Progressive \\ \hline
Progressive & 5 & 1 & US Senate Candidate 2---Progressive \\ \hline
Progressive & 7 & 1 & US House Candidate 2---Progressive \\ \hline
Progressive & 10 & 1 & State Senate Candidate 2---Progressive \\ \hline
Progressive & 13 & 1 & State House D1 Candidate 2---Progressive \\ \hline
Progressive & 16 & 1 & State House D2 Candidate 2---Progressive \\ \hline
Progressive & 18 & 1 & State House D3 Candidate 1---Independent \\ \hline
Progressive & 20 & 1 & Council Candidate 2---Progressive \\ \hline
Progressive & 23 & 1 & Council Candidate 5---Progressive \\ \hline
Progressive & 25 & 1 & Council Candidate 7---Moderate \\ \hline
Progressive & 28 & 1 & Council Candidate 10---Progressive \\ \hline
Progressive & 29 & 1 & Yes on Ballot Initiative 1 \\ \hline
Progressive & 32 & 1 & No on Ballot Initiative 2 \\ \hline
\end{longtable}


% \clearpage
\subsection{Generating relevant test cases}
\label{LenSection5}

The purpose of this subsection is to describe how to use the tables
identified in \autoref{LenSection4}, together with the Votetest model, to
produce test cases relevant to the characteristics of the election example
presented.  The tester may use tests already generated by Votetest to test
the general validity of a voting system. Additional tests generated here
serve to exercise the voting system in an environment that most closely
approximates a specific election with specific ballot styles and reporting
requirements. The following subsections describe the steps taken to assign
relevant probability distributions and to create a representative number of
test ballots. In general, a tester would follow these same steps to generate
test ballots for a specific election.

\subsubsection{Populate the tables}

The first step is to use local election characteristics to prepare content
for all of the non-empty tables identified in \autoref{LenSection4}. The
MyDescription attribute may be omitted as it is not used by Votetest. In
addition, the Endorsement table is optional because in this election the
straight-party voting variation is not used.

\subsubsection{Assign Ballot number and general Ballot distributions}
\label{LenSection5.2}

\autoref{LenCategoryDistributionTable} indicates the distribution of sample
ballots over the Categories and Accepted attributes of the Ballot class
(presented in \autoref{LenSection3.3}) that the tester in this example wants
to generate.  A probability is given for each category in the BallotCategory
table; in addition, the probability of a ballot being rejected or blank is
included.  The total number of ballots that the tester wants to generate is
2000.

\begin{table}[htbp]
\caption{BallotCategory, rejected and blank ballot distributions}
\capsep
\begin{center}
\begin{tabular}{|l|r|}
\multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf Distribution} \\ \hline
RejectedBallots & 1~\% \\ \hline
BlankBallots & 1~\% \\ \hline
Absentee & 13~\% \\ \hline
Challenged & 0~\% \\ \hline
Early & 0~\% \\ \hline
IneligibleVoter & 0~\% \\ \hline
InPerson & 0~\% \\ \hline
NotRegistered & 0~\% \\ \hline
Provisional & 6~\% \\ \hline
Regular & 0~\% \\ \hline
WrongPrecinct & 0~\% \\ \hline
\end{tabular}
\end{center}
\label{LenCategoryDistributionTable}
\end{table}

Each of the named probability distributions is treated as an independent, random binomial event. So we expect approximately 1~\% of the 2000 generated ballots to have the Accepted attribute set to \emph{false}, 1~\% to be a blank ballot, 13~\% to have Absentee in the Categories attribute, and 6~\% to have Provisional in the Categories attribute. None of the other category entries in this table will occur at all.

Note that all four of these potential events are treated as independent. Thus it is possible for a Provisional ballot to be an Absentee ballot. It is also possible that a Rejected ballot be Provisional, Absentee, Blank, or any combination of the three. If desired, a testing facility could set up a more sophisticated ballot distribution with conditional probabilities for those events that have interdependencies.

The Ballot table is created first with 2000 rows. Then the Accepted attribute is set to \emph{true} or \emph{false} depending on the RejectedBallots distribution of 1~\%. Next the BallotCategoryAssociation table is populated---first with 13~\% Absentee associations and then, independently, with 6~\% Provisional distributions. Then 1~\% of the ballots are independently tagged as blank ballots so that no associations will ever be created from a blank ballot to a Choice instance.


\subsubsection{Assign Precinct and BallotStyle distributions}
\label{LenSection5.3}

Most elections will have pre-determined relationships among the precincts, reporting contexts, and ballot styles. Using the precinct relationships information in \autoref{LenSection2.7}, \autoref{LenOtherDistributionTable} shows the probability distributions for assigning ballots to precincts. The other distributions are then derived from the precinct distribution.

\begin{table}[htbp]
\caption{Precinct and BallotStyle distributions}
\capsep
\begin{center}
\begin{tabular}{|l|r|l|r|l|r|l|r|}
\multicolumn{1}{c}{\bf Precinct} & \multicolumn{1}{c}{\bf P\_Dist} & \multicolumn{1}{c}{\bf District} & \multicolumn{1}{c}{\bf D\_Dist} & \multicolumn{1}{c}{\bf Munic} & \multicolumn{1}{c}{\bf M\_Dist} & \multicolumn{1}{c}{\bf BallotStyle} & \multicolumn{1}{c}{\bf BS\_Dist} \\ \hline
P01 & 5~\% & H1 & 28~\% &  &  & H1B1B2 & 24~\% \\ \hline
P02 & 12~\% & H1 & 28~\% &  &  & H1B1B2 & 24~\% \\ \hline
P03 & 7~\% & H1 & 28~\% &  &  & H1B1B2 & 24~\% \\ \hline
P04 & 4~\% & H1 & 28~\% & M1 & 12~\% & H1B1B2M1B4 & 4~\% \\ \hline
P05 & 8~\% & H2 & 36~\% & M1 & 12~\% & H2B1B2B3M1B4 & 8~\% \\ \hline
P06 & 13~\% & H2 & 36~\% &  &  & H2B1B2B3 & 28~\% \\ \hline
P07 & 15~\% & H2 & 36~\% &  &  & H2B1B2B3 & 28~\% \\ \hline
P08 & 6~\% & H3 & 36~\% &  &  & H3B1B2 & 36~\% \\ \hline
P09 & 11~\% & H3 & 36~\% &  &  & H3B1B2 & 36~\% \\ \hline
P10 & 19~\% & H3 & 36~\% &  &  & H3B1B2 & 36~\% \\ \hline
\end{tabular}
\end{center}
\label{LenOtherDistributionTable}
\end{table}

The general abstract model in \autoref{DataModel} supports precincts that may not be the smallest reporting unit. If that is the case, then a testing facility could assign probability distributions to the smallest reporting units and then have those probabilities distributed over all of the other reporting units and ballot styles.

The precinct probability distributions should add up to 100~\%. The assignment of ballots to precincts is not considered to be a collection of independent events. Instead, the assignment of precincts is handled as a single, random multinomial event. Using the precinct probability distributions given in \autoref{LenOtherDistributionTable} one is able to populate the BallotReportingContextAssociation table between Ballot and ReportingContext to satisfy the requirement that each ballot be assigned to at least one reporting context, i.e., its precinct.


\subsubsection{Assign undervote distributions}

The Contest table contains a list of potential contests for each ballot. In
practice, some ballots may leave a given contest unvoted. It is possible to
add one additional column to the Contest table to assign a probability that
the contest will be left unvoted on a given ballot. Contests like a national
presidential contest may have a very low probability of being left unvoted,
whereas some complex ballot initiatives may have a relatively high
probability of being left unvoted. One could assume different probabilities
for each contest, depending on which ballot style is being used, or in which
political unit a ballot is cast. For simplicity, this section assumes that
all ballots will have the same probability of leaving a specific contest
unvoted.

\autoref{LenUndervoteDistributionTable} carries the undervote probability distributions for each contest. The undervote distribution is independent of the CountingLogic, N, MaxWriteIns, or Rotate attributes of a contest, so those columns are elided.

\begin{table}[htbp]
\caption{Undervote distributions}
\capsep
\begin{center}
\begin{tabular}{|l|l|r|}
\multicolumn{1}{c}{\bf ContestId} & \multicolumn{1}{c}{\bf Description} & \multicolumn{1}{c}{\bf Unvoted} \\ \hline
1 & President & 1~\% \\ \hline
2 & U.S. Senate & 2~\% \\ \hline
3 & U.S. House & 3~\% \\ \hline
4 & State Senate & 4~\% \\ \hline
5 & State House \#1 & 5~\% \\ \hline
6 & State House \#2 & 5~\% \\ \hline
7 & State House \#3 & 22~\% \\ \hline
8 & County Council & 3~\% \\ \hline
9 & Ballot Initiative 1 & 15~\% \\ \hline
10 & Ballot Initiative 2 & 15~\% \\ \hline
11 & Ballot Initiative 3 & 20~\% \\ \hline
12 & Ballot Initiative 4 & 25~\% \\ \hline
13 & Muni City Council & 30~\% \\ \hline
\end{tabular}
\end{center}
\label{LenUndervoteDistributionTable}
\end{table}

Each time a contest appears on a ballot, whether or not it is left unvoted is treated as a random, independent binomial event with the above probabilities. Each ballot is linked to a ballot style, and each ballot style is linked to a set of contests. Thus a join of these tables, with the elimination of blank ballots, produces a link between a specific ballot and the contests relevant to that ballot. Each row of that table is considered as an independent event for being voted or unvoted and rows are tagged as appropriate. Ballots linked to an unvoted contest will not be linked to any choices for that contest.


\subsubsection{Assign Choice distributions}
\label{LenSection5.5}

The Choice table as initialized in \autoref{LenSection4.4} consists of a
collection of choices for each contest.  Since a number of
candidates are already known to be running specifically as write-in
candidates, choices for these candidates may also be created in advance of
the election.  The tester prepares a list of likely write-ins for each
contest as shown in \autoref{LenChoiceDistributionTable1}.

\begin{table}[htbp]
\caption{Choices for anticipated write-ins}
\capsep
\begin{center}
\begin{tabular}{|l|l|l|l|l|l|}
\multicolumn{1}{c}{\bf ChoiceId} & \multicolumn{1}{c}{\bf ContestId} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf Affiliation} & \multicolumn{1}{c}{\bf IsWriteIn} & \multicolumn{1}{c}{\bf MyDescription} \\ \hline
1001 & 5 & SHouseD1 Writein1 &  & Yes & Canonical \\ \hline
1002 & 5 & SHouseD1 Writein2 &  & Yes & Canonical \\ \hline
1003 & 5 & Invalid Writein &  & Yes & Indeterminate \\ \hline
1004 & 6 & SHouseD2 Writein1 &  & Yes & Canonical \\ \hline
1005 & 6 & Invalid Writein &  & Yes & Indeterminate \\ \hline
1006 & 7 & SHouseD3 Writein1 &  & Yes & Canonical \\ \hline
1007 & 7 & SHouseD3 Writein2 &  & Yes & Canonical \\ \hline
1008 & 7 & SHouseD3 Writein3 &  & Yes & Canonical \\ \hline
1009 & 7 & Invalid Writein &  & Yes & Indeterminate \\ \hline
1010 & 8 & CCouncil Writein1 &  & Yes & Canonical \\ \hline
1011 & 8 & CCouncil Writein2 &  & Yes & Canonical \\ \hline
1012 & 8 & CCouncil Writein3 &  & Yes & Canonical \\ \hline
1013 & 8 & CCouncil Writein4 &  & Yes & Canonical \\ \hline
1014 & 8 & CCouncil Writein5 &  & Yes & Canonical \\ \hline
1015 & 8 & Invalid Writein &  & Yes & Indeterminate \\ \hline
\end{tabular}
\end{center}
\label{LenChoiceDistributionTable1}
\end{table}

None of the choices listed in \autoref{LenChoiceDistributionTable1} will ever
be referenced directly by a given ballot. Instead, each time a write-in is
chosen on a ballot, the new Choice created for it will be aliased to one
of these ``canonical'' write-ins, or to one of the invalid write-ins if the
given write-in is indeterminate or illegal.  If a valid write-in were received
that did not match anything on this list, then the Choice instance created
for it would not be aliased to anything.  See \autoref{LenSection3.3} for
more discussion on the use of aliasing in this example.

The tester now needs to assign a probability distribution to each of
the possible choices, including potential write-ins.  The result is shown in
\autoref{LenChoiceDistributionTable2}.

% FIXME
\pagebreak
\begin{longtable}{|l|l|l|l|l|l|r|}
\caption{Canonical choice and write-in distribution} \\
% \multicolumn{1}{c}{\bf ChoiceId} & \multicolumn{1}{c}{\bf ContestId} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf Affiliation} & \multicolumn{1}{c}{\bf IsWriteIn} & \multicolumn{1}{c}{\bf MyDescription} & \multicolumn{1}{c}{\bf Dist} \\ \hline
\multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{\bf IsW} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\
\multicolumn{1}{c}{\bf Choi} & \multicolumn{1}{c}{\bf Cont} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{\bf rite} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\
\multicolumn{1}{c}{\bf ceId} & \multicolumn{1}{c}{\bf estId} & \multicolumn{1}{c}{\bf Name} & \multicolumn{1}{c}{\bf Affiliation} & \multicolumn{1}{c}{\bf In} & \multicolumn{1}{c}{\bf MyDescription} & \multicolumn{1}{c}{\bf Dist} \\ \hline
\label{LenChoiceDistributionTable2}
1 & 1 & PresidentC1 & Moderate & No & US President Cand 1 & 41~\% \\ \hline
2 & 1 & PresidentC2 & Progressive & No & US President Cand 2 & 41~\% \\ \hline
3 & 1 & PresidentC3 & Action & No & US President Cand 3 & 18~\% \\ \hline
4 & 2 & USSenateC1 & Moderate & No & US Senate Cand 1 & 50~\% \\ \hline
5 & 2 & USSenateC2 & Progressive & No & US Senate Cand 2 & 50~\% \\ \hline
6 & 3 & USHouseC1 & Moderate & No & US House Cand 1 & 33~\% \\ \hline
7 & 3 & USHouseC2 & Progressive & No & US House Cand 2 & 34~\% \\ \hline
8 & 3 & USHouseC3 & Conservation & No & US House Cand 3 & 33~\% \\ \hline
9 & 4 & SSenateC1 & Moderate & No & State House Cand 1 & 34~\% \\ \hline
10 & 4 & SSenateC2 & Progressive & No & State House Cand 2 & 33~\% \\ \hline
11 & 4 & SSenateC3 & Conservation & No & State House Cand 3 & 33~\% \\ \hline
12 & 5 & SHouseD1C1 & Moderate & No & State House D1 Cand 1 & 36~\% \\ \hline
13 & 5 & SHouseD1C2 & Progressive & No & State House D1 Cand 2 & 36~\% \\ \hline
14 & 5 & SHouseD1C3 & Conservation & No & State House D1 Cand 3 & 21~\% \\ \hline
1001 & 5 & SHouseD1 Writein1 &  & Yes & Canonical & 3~\% \\ \hline
1002 & 5 & SHouseD1 Writein2 &  & Yes & Canonical & 2~\% \\ \hline
1003 & 5 & Invalid Writein &  & Yes & Indeterminate & 2~\% \\ \hline
15 & 6 & SHouseD2C1 & Moderate & No & State House D2 Cand 1 & 42~\% \\ \hline
16 & 6 & SHouseD2C2 & Progressive & No & State House D2 Cand 2 & 42~\% \\ \hline
17 & 6 & SHouseD2C3 & Independent & No & State House D2 Cand 3 & 10~\% \\ \hline
1004 & 6 & SHouseD2 Writein1 &  & Yes & Canonical & 5~\% \\ \hline
1005 & 6 & Invalid Writein &  & Yes & Indeterminate & 1~\% \\ \hline
18 & 7 & SHouseD3C1 & Independent & No & State House D3 Cand 1 & 40~\% \\ \hline
1006 & 7 & SHouseD3 Writein1 &  & Yes & Canonical & 45~\% \\ \hline
1007 & 7 & SHouseD3 Writein2 &  & Yes & Canonical & 8~\% \\ \hline
1008 & 7 & SHouseD3 Writein3 &  & Yes & Canonical & 4~\% \\ \hline
1009 & 7 & Invalid Writein &  & Yes & Indeterminate & 3~\% \\ \hline
19 & 8 & CCouncilC01 & Moderate & No & Council Candidate 1 & 25~\% \\ \hline
20 & 8 & CCouncilC02 & Progressive & No & Council Candidate 2 & 20~\% \\ \hline
21 & 8 & CCouncilC03 &  & No & Council Candidate 3 & 45~\% \\ \hline
22 & 8 & CCouncilC04 & Conservation & No & Council Candidate 4 & 15~\% \\ \hline
23 & 8 & CCouncilC05 & Progressive & No & Council Candidate 5 & 30~\% \\ \hline
24 & 8 & CCouncilC06 &  & No & Council Candidate 6 & 20~\% \\ \hline
25 & 8 & CCouncilC07 & Moderate & No & Council Candidate 7 & 40~\% \\ \hline
26 & 8 & CCouncilC08 & Independent & No & Council Candidate 8 & 20~\% \\ \hline
27 & 8 & CCouncilC09 & Conservation & No & Council Candidate 9 & 20~\% \\ \hline
28 & 8 & CCouncilC10 & Progressive & No & Council Candidate 10 & 25~\% \\ \hline
1010 & 8 & CCouncil Writein1 &  & Yes & Canonical & 28~\% \\ \hline
1011 & 8 & CCouncil Writein2 &  & Yes & Canonical & 15~\% \\ \hline
1012 & 8 & CCouncil Writein3 &  & Yes & Canonical & 10~\% \\ \hline
1013 & 8 & CCouncil Writein4 &  & Yes & Canonical & 10~\% \\ \hline
1014 & 8 & CCouncil Writein5 &  & Yes & Canonical & 5~\% \\ \hline
1015 & 8 & Invalid Writein &  & Yes & Indeterminate & 3~\% \\ \hline
29 & 9 & Yes &  & No & Yes on Ballot Initiative 1 & 60~\% \\ \hline
30 & 9 & No &  & No & No on Ballot Initiative 1 & 40~\% \\ \hline
31 & 10 & Yes &  & No & Yes on Ballot Initiative 2 & 50~\% \\ \hline
32 & 10 & No &  & No & No on Ballot Initiative 2 & 50~\% \\ \hline
33 & 11 & Yes &  & No & Yes on Ballot Initiative 3 & 40~\% \\ \hline
34 & 11 & No &  & No & No on Ballot Initiative 3 & 60~\% \\ \hline
35 & 12 & Yes &  & No & Yes on Ballot Initiative 4 & 51~\% \\ \hline
36 & 12 & No &  & No & No on Ballot Initiative 4 & 49~\% \\ \hline
37 & 13 & MCCouncilC1 &  & No &  & 5~\% \\ \hline
38 & 13 & MCCouncilC2 &  & No &  & 35~\% \\ \hline
39 & 13 & MCCouncilC3 &  & No &  & 10~\% \\ \hline
40 & 13 & MCCouncilC4 &  & No &  & 20~\% \\ \hline
41 & 13 & MCCouncilC5 &  & No &  & 30~\% \\ \hline
\end{longtable}

A test facility may use the above distributions in a variety of ways. For example, suppose a single-winner contest has 2 candidates each with a 50~\% probability distribution. The tester could regard this contest as a single, random binomial event for each ballot, or it could regard the contest as two independent, random binomial events for each ballot. In the first case, each candidate would have a 50~\% chance of being selected on the ballot and there would be no undervotes and no overvotes. In the second case there would be a 25~\% chance of neither candidate receiving a vote, a 50~\% chance of exactly one candidate receiving a vote, and a 25~\% chance of both candidates receiving a vote. The second case would have 25~\% undervotes and 25~\% overvotes. The process would be different, but each candidate would receive, on average, the same number of votes.

For this example, we assume that each single-winner contest, i.e., each 1-of-M contest, whose choice probabilities add to 100~\%, is treated as a single, random multinomial event with exactly one candidate receiving a vote on that ballot. There will be no overvotes for these contests. However, there may still be some undervotes because of the previous allocation for blank ballots and unvoted contests. This technique still works even if the choice probabilities add to less than 100~\%; the difference between the sum and 100~\% would just add to the undervote population.

For multiple-winner N-of-M contests, it is often more realistic to consider the contest to be a sequence of M independent, binomial events, with each candidate having its given probability of receiving a vote. We apply this technique to the County Council contest. Note that the above probabilities for choices in the County Council contest (ContestId = 8) sum to 331~\%. This means that on average, a non-blank, voted ballot for this contest will select only 3.31 council candidates; however, there will be a distribution from 0 to 16 of the candidates being selected, likely resulting in a substantial number of undervotes and overvotes. One could modify the distribution to add or subtract from the sum, thereby adding to the number of undervoted or overvoted ballots.

For multiple winner N-of-M contests, it is also possible to consider the contest as a sequence of N independent multinomial events, with each candidate having its given probability of receiving a vote on each event. We apply this technique to the municipal City Council contest (ContestId = 13) where the counting logic is Cumulative, up to 2 votes allowed, and the 5 given probabilities add to exactly 100~\%. Each candidate will have 2 independent chances to receive a vote, so some will receive 0 votes, some exactly 1 vote, and some exactly 2 votes. Since this contest has cumulative counting logic, there will be no overvotes from this approach and no new undervotes. There will still be some undervotes because of the previous allocation for blank ballots and unvoted contests.

\subsubsection{Generate a simulated election}
\label{LenTestDataGenerator}

It's now time to vote! We simulate an election based on the above probability distributions by taking the following actions:

\begin{enumerate}
\item Populate the Ballot table with 2000 ballots. Do not yet enforce the integrity constraint to BallotStyle.
\item Assign an appropriate value to the Accepted attribute, based upon the binomial distribution for RejectedBallots defined in \autoref{LenSection5.2}.
\item Populate the BallotCategoryAssociation table according to the binomial distributions for Categories defined in \autoref{LenSection5.2}.
\item Populate the BallotReportingContextAssociation table according to the Precinct probability distributions defined in \autoref{LenSection5.3}.
\item Update the Ballot table to assign a value to the StyleId attribute using the Precincts just generated for each ballot to link to ReportingContext and the existing BallotStyleReportingContextAssociation table to identify the appropriate BallotStyle ID\@. Turn on the referential integrity check from Ballot to BallotStyle.
\item Populate the Choice table with new choices for the write-ins identified in \autoref{LenChoiceDistributionTable1}.
\item Populate the Choice table with a new choice for each write-in in any contest where a write-in choice is generated by the distributions given in \autoref{LenChoiceDistributionTable2}.
\item Populate the Alias table to have each write-in choice linked to the appropriate ``canonical'' write-in. Recall that the canonical write-ins in the Choice table do not identify a choice on any ballot; instead the dynamically generated write-in choices are now linked to a canonical write-in if one exists.
\item Populate the VoterInput table for all 1-of-M contests making sure not to add choices for ballots that have been tagged as blank ballots or for ballot-contest combinations that have been tagged as unvoted contests.
\item Populate the VoterInput table for the County Council contest with the same provisions for blank ballots and unvoted contests.
\item Populate the VoterInput table for the municipal City Council contest with the same provisions for blank ballots and unvoted contests.
\end{enumerate}

A Microsoft Access database implementing this example appears
in the TestGenerator subdirectory of the Votetest distribution with file name
WorkedExampleDataGenerator.mdb.  The database contains all of the relational
database tables, plus macros that implement the steps of the test data
generation process.  The READ\_ME base table contains additional documentation
specific to the Access database.  A tester could modify the data and rerun
the macros to generate test data for example elections with different
contests and probability distributions.

N.B., An alternative test generator was documented in
\autoref{TestGenerator} as part of the Votetest advanced test development
environment.


\subsubsection{Input ballots to a specific voting system}

Use the simulated ballots generated in the previous subsection as input to a specific voting system under test. This may take some effort to define a suitable interface to the voting system, since no standard test interface is required by the VVSG\@.

Cast all of the ballots and then compare the results from the voting system
to the reports generated by Votetest.  They should be identical.


\subsection{Reports generated by Votetest}
\label{LenSection6}

The database tables generated in \autoref{LenSection5} were transferred to
the advanced test development environment of Votetest (discussed in
\autoref{TestSuite}) by exporting them to a tab-delimited text format and
then importing them with the PostgreSQL COPY command.  The Votetest
infrastructure for integrity checking and reporting was then invoked as for
any other test case.  The results are presented in the following subsections.

To conserve space, not all reports are fully presented below. However, the
Votetest integrity check report, the county report, the three district
reports, the municipality report, and representative examples of the precinct
reports are presented in full.


\subsubsection{Integrity checks}

\begin{verbatim}
######################################################################
           BEGIN TEST CASE OUTPUT   2008-03-11 15:47:35-04
######################################################################

$Id: Documentation.tex 509 2009-03-25 16:00:13Z dflater $

----- Begin integrity check output -- All results should be empty -----

 Out Of Range Voter Inputs
 ballotid | choiceid | value
----------+----------+-------
(0 rows)

Out Of Range Endorsements
 party | choiceid | value
-------+----------+-------
(0 rows)

 Extraneous Inputs
 ballotid | choiceid | value
----------+----------+-------
(0 rows)

Unreported Ballots
 ballotid
----------
(0 rows)

 Double Votes
 ballotid | choiceid | count | sum
----------+----------+-------+-----
(0 rows)

 Cross Contest Aliases
 aliasid | aliasname | aliascontestid | choiceid | choicename | choicecontestid
---------+-----------+----------------+----------+------------+-----------------
(0 rows)

 Double Indirect Aliases
 aliasid | aliasedaliasid
---------+----------------
(0 rows)

Ballot Styles With More Than One Straight Party Contest
 styleid
---------
(0 rows)

Non-Existent Parties In Straight Party Contest
 contestid | name
-----------+------
(0 rows)

Circular Straight Party Endorsements
 party | choiceid
-------+----------
(0 rows)

 Endorsed Aliases
 party | choiceid | value
-------+----------+-------
(0 rows)

 Straight Party Overrides
 ballotid | contestid | choiceid | value
----------+-----------+----------+-------
(0 rows)

 Too Many Write-Ins
 ballotid | contestid | writeinscount
----------+-----------+---------------
(0 rows)

----- End integrity check output ---- All results should be empty -----

-------------------------------------------------------------------------------

Report total volume: 155990
  - Includes optional reporting of blank ballots.
  - Excludes separate reporting of ballots cast vs. read.

######################################################################
\end{verbatim}


\subsubsection{Countywide results}

\begin{verbatim}
Report for context County1 generated 2008-03-11 15:48:07-0400

                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                          2000       1979
                                Absentee        254        251
                             Provisional        120        119
                                   Blank         19         19

H1B1B2                                          479        475
                                Absentee         68         67
                             Provisional         30         30
                                   Blank          5          5

H1B1B2M1B4                                       87         87
                                Absentee         11         11
                             Provisional          6          6
                                   Blank          2          2

H2B1B2B3                                        547        540
                                Absentee         61         61
                             Provisional         33         32
                                   Blank          6          6

H2B1B2B3M1B4                                    148        146
                                Absentee         21         20
                             Provisional          7          7
                                   Blank          1          1

H3B1B2                                          739        731
                                Absentee         93         92
                             Provisional         44         44
                                   Blank          5          5


                                  VOTE TOTALS

President
PresidentC1 (Moderate)                          796
PresidentC2 (Progressive)                       778
PresidentC3 (Action)                            370
Overvotes                                         0
Undervotes                                       35
Counted ballots                                1979
Balance                                           0

U.S. Senate
USSenateC2 (Progressive)                        968
USSenateC1 (Moderate)                           964
Overvotes                                         0
Undervotes                                       47
Counted ballots                                1979
Balance                                           0

U.S. House
USHouseC2 (Progressive)                         678
USHouseC1 (Moderate)                            625
USHouseC3 (Conservation)                        616
Overvotes                                         0
Undervotes                                       60
Counted ballots                                1979
Balance                                           0

State Senate
SSenateC2 (Progressive)                         647
SSenateC1 (Moderate)                            627
SSenateC3 (Conservation)                        620
Overvotes                                         0
Undervotes                                       85
Counted ballots                                1979
Balance                                           0

State House #1
SHouseD1C1 (Moderate)                           200
SHouseD1C2 (Progressive)                        193
SHouseD1C3 (Conservation)                       108
SHouseD1 Writein1 (write-in)                     12
SHouseD1 Writein2 (write-in)                     11
Invalid Writein (write-in)                        7
Overvotes                                         0
Undervotes                                       31
Counted ballots                                 562
Balance                                           0

State House #2
SHouseD2C1 (Moderate)                           269
SHouseD2C2 (Progressive)                        266
SHouseD2C3 (Independent)                         72
SHouseD2 Writein1 (write-in)                     44
Invalid Writein (write-in)                        2
Overvotes                                         0
Undervotes                                       33
Counted ballots                                 686
Balance                                           0

State House #3
SHouseD3 Writein1 (write-in)                    266
SHouseD3C1 (Independent)                        221
SHouseD3 Writein2 (write-in)                     41
SHouseD3 Writein3 (write-in)                     20
Invalid Writein (write-in)                       14
Overvotes                                         0
Undervotes                                      169
Counted ballots                                 731
Balance                                           0

County Council
CCouncilC03                                     595
CCouncilC07 (Moderate)                          511
CCouncilC05 (Progressive)                       378
CCouncil Writein1 (write-in)                    330
CCouncilC10 (Progressive)                       329
CCouncilC01 (Moderate)                          322
CCouncilC06                                     248
CCouncilC02 (Progressive)                       241
CCouncilC08 (Independent)                       233
CCouncilC09 (Conservation)                      230
CCouncilC04 (Conservation)                      176
CCouncil Writein2 (write-in)                    165
CCouncil Writein3 (write-in)                    104
CCouncil Writein4 (write-in)                    102
CCouncil Writein5 (write-in)                     62
Invalid Writein (write-in)                       30
Overvotes                                      1584
Undervotes                                     2276
Counted ballots                                1979
Balance                                           0

Ballot Initiative 1
Yes                                             987
No                                              663
Overvotes                                         0
Undervotes                                      329
Counted ballots                                1979
Balance                                           0

Ballot Initiative 2
No                                              843
Yes                                             832
Overvotes                                         0
Undervotes                                      304
Counted ballots                                1979
Balance                                           0

Ballot Initiative 3
No                                              311
Yes                                             207
Overvotes                                         0
Undervotes                                      168
Counted ballots                                 686
Balance                                           0

Ballot Initiative 4
Yes                                              93
No                                               86
Overvotes                                         0
Undervotes                                       54
Counted ballots                                 233
Balance                                           0

Muni City Council
MCCouncilC2                                     115
MCCouncilC5                                      91
MCCouncilC4                                      65
MCCouncilC3                                      34
MCCouncilC1                                      13
Overvotes                                         0
Undervotes                                      148
Counted ballots                                 233
Balance                                           0
\end{verbatim}

\subsubsection{Precinct results}

\begin{verbatim}
Report for context P01 generated 2008-03-11 15:48:07-0400

                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                            93         93
                                Absentee         19         19
                             Provisional          6          6
                                   Blank          0          0

H1B1B2                                           93         93
                                Absentee         19         19
                             Provisional          6          6


                                  VOTE TOTALS

President
PresidentC2 (Progressive)                        42
PresidentC1 (Moderate)                           37
PresidentC3 (Action)                             14
Overvotes                                         0
Undervotes                                        0
Counted ballots                                  93
Balance                                           0

U.S. Senate
USSenateC1 (Moderate)                            47
USSenateC2 (Progressive)                         46
Overvotes                                         0
Undervotes                                        0
Counted ballots                                  93
Balance                                           0

U.S. House
USHouseC2 (Progressive)                          32
USHouseC1 (Moderate)                             31
USHouseC3 (Conservation)                         29
Overvotes                                         0
Undervotes                                        1
Counted ballots                                  93
Balance                                           0

State Senate
SSenateC1 (Moderate)                             35
SSenateC3 (Conservation)                         31
SSenateC2 (Progressive)                          27
Overvotes                                         0
Undervotes                                        0
Counted ballots                                  93
Balance                                           0

State House #1
SHouseD1C1 (Moderate)                            33
SHouseD1C2 (Progressive)                         27
SHouseD1C3 (Conservation)                        17
SHouseD1 Writein2 (write-in)                      3
Invalid Writein (write-in)                        3
SHouseD1 Writein1 (write-in)                      1
Overvotes                                         0
Undervotes                                        9
Counted ballots                                  93
Balance                                           0

County Council
CCouncilC03                                      27
CCouncilC05 (Progressive)                        23
CCouncilC07 (Moderate)                           17
CCouncilC02 (Progressive)                        16
CCouncilC10 (Progressive)                        14
CCouncil Writein1 (write-in)                     13
CCouncilC01 (Moderate)                           12
CCouncilC06                                      12
CCouncilC08 (Independent)                        11
CCouncilC09 (Conservation)                       11
CCouncil Writein2 (write-in)                      9
CCouncil Writein4 (write-in)                      7
CCouncilC04 (Conservation)                        6
CCouncil Writein3 (write-in)                      5
CCouncil Writein5 (write-in)                      3
Invalid Writein (write-in)                        1
Overvotes                                        80
Undervotes                                      105
Counted ballots                                  93
Balance                                           0

Ballot Initiative 1
Yes                                              40
No                                               34
Overvotes                                         0
Undervotes                                       19
Counted ballots                                  93
Balance                                           0

Ballot Initiative 2
Yes                                              46
No                                               36
Overvotes                                         0
Undervotes                                       11
Counted ballots                                  93
Balance                                           0



Report for context P04 generated 2008-03-11 15:48:08-0400

                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                            87         87
                                Absentee         11         11
                             Provisional          6          6
                                   Blank          2          2


H1B1B2M1B4                                       87         87
                                Absentee         11         11
                             Provisional          6          6
                                   Blank          2          2


                                  VOTE TOTALS

President
PresidentC1 (Moderate)                           29
PresidentC2 (Progressive)                        29
PresidentC3 (Action)                             22
Overvotes                                         0
Undervotes                                        7
Counted ballots                                  87
Balance                                           0

U.S. Senate
USSenateC2 (Progressive)                         44
USSenateC1 (Moderate)                            40
Overvotes                                         0
Undervotes                                        3
Counted ballots                                  87
Balance                                           0

U.S. House
USHouseC2 (Progressive)                          37
USHouseC3 (Conservation)                         25
USHouseC1 (Moderate)                             20
Overvotes                                         0
Undervotes                                        5
Counted ballots                                  87
Balance                                           0

State Senate
SSenateC2 (Progressive)                          30
SSenateC3 (Conservation)                         28
SSenateC1 (Moderate)                             25
Overvotes                                         0
Undervotes                                        4
Counted ballots                                  87
Balance                                           0

State House #1
SHouseD1C1 (Moderate)                            31
SHouseD1C2 (Progressive)                         26
SHouseD1C3 (Conservation)                        20
SHouseD1 Writein1 (write-in)                      3
SHouseD1 Writein2 (write-in)                      3
Invalid Writein (write-in)                        0
Overvotes                                         0
Undervotes                                        4
Counted ballots                                  87
Balance                                           0

County Council
CCouncilC03                                      25
CCouncilC07 (Moderate)                           23
CCouncilC05 (Progressive)                        17
CCouncil Writein1 (write-in)                     15
CCouncilC08 (Independent)                        14
CCouncilC10 (Progressive)                        13
CCouncilC01 (Moderate)                           12
CCouncilC09 (Conservation)                       12
CCouncilC06                                       8
CCouncilC02 (Progressive)                         7
CCouncilC04 (Conservation)                        7
CCouncil Writein2 (write-in)                      4
CCouncil Writein3 (write-in)                      4
CCouncil Writein4 (write-in)                      4
CCouncil Writein5 (write-in)                      3
Invalid Writein (write-in)                        2
Overvotes                                        88
Undervotes                                       90
Counted ballots                                  87
Balance                                           0

Ballot Initiative 1
Yes                                              41
No                                               34
Overvotes                                         0
Undervotes                                       12
Counted ballots                                  87
Balance                                           0

Ballot Initiative 2
Yes                                              43
No                                               30
Overvotes                                         0
Undervotes                                       14
Counted ballots                                  87
Balance                                           0

Ballot Initiative 4
No                                               34
Yes                                              33
Overvotes                                         0
Undervotes                                       20
Counted ballots                                  87
Balance                                           0

Muni City Council
MCCouncilC2                                      47
MCCouncilC5                                      30
MCCouncilC4                                      28
MCCouncilC3                                      10
MCCouncilC1                                       5
Overvotes                                         0
Undervotes                                       54
Counted ballots                                  87
Balance                                           0



Report for context P05 generated 2008-03-11 15:48:08-0400


                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                           148        146
                                Absentee         21         20
                             Provisional          7          7
                                   Blank          1          1

H2B1B2B3M1B4                                    148        146
                                Absentee         21         20
                             Provisional          7          7
                                   Blank          1          1


                                  VOTE TOTALS

President
PresidentC2 (Progressive)                        63
PresidentC1 (Moderate)                           53
PresidentC3 (Action)                             29
Overvotes                                         0
Undervotes                                        1
Counted ballots                                 146
Balance                                           0

U.S. Senate
USSenateC1 (Moderate)                            70
USSenateC2 (Progressive)                         69
Overvotes                                         0
Undervotes                                        7
Counted ballots                                 146
Balance                                           0

U.S. House
USHouseC2 (Progressive)                          53
USHouseC1 (Moderate)                             46
USHouseC3 (Conservation)                         42
Overvotes                                         0
Undervotes                                        5
Counted ballots                                 146
Balance                                           0

State Senate
SSenateC3 (Conservation)                         51
SSenateC1 (Moderate)                             48
SSenateC2 (Progressive)                          42
Overvotes                                         0
Undervotes                                        5
Counted ballots                                 146
Balance                                           0

State House #2
SHouseD2C1 (Moderate)                            61
SHouseD2C2 (Progressive)                         58
SHouseD2C3 (Independent)                         11
SHouseD2 Writein1 (write-in)                     10
Invalid Writein (write-in)                        1
Overvotes                                         0
Undervotes                                        5
Counted ballots                                 146
Balance                                           0

County Council
CCouncilC07 (Moderate)                           44
CCouncilC03                                      39
CCouncilC05 (Progressive)                        34
CCouncilC09 (Conservation)                       25
CCouncilC10 (Progressive)                        24
CCouncil Writein1 (write-in)                     22
CCouncilC01 (Moderate)                           21
CCouncilC02 (Progressive)                        19
CCouncilC04 (Conservation)                       16
CCouncil Writein2 (write-in)                     16
CCouncilC08 (Independent)                        15
CCouncilC06                                      14
CCouncil Writein4 (write-in)                      8
CCouncil Writein3 (write-in)                      4
CCouncil Writein5 (write-in)                      3
Invalid Writein (write-in)                        3
Overvotes                                       120
Undervotes                                      157
Counted ballots                                 146
Balance                                           0

Ballot Initiative 1
Yes                                              78
No                                               41
Overvotes                                         0
Undervotes                                       27
Counted ballots                                 146
Balance                                           0

Ballot Initiative 2
No                                               66
Yes                                              60
Overvotes                                         0
Undervotes                                       20
Counted ballots                                 146
Balance                                           0

Ballot Initiative 3
No                                               69
Yes                                              48
Overvotes                                         0
Undervotes                                       29
Counted ballots                                 146
Balance                                           0

Ballot Initiative 4
Yes                                              60
No                                               52
Overvotes                                         0
Undervotes                                       34
Counted ballots                                 146
Balance                                           0

Muni City Council
MCCouncilC2                                      68
MCCouncilC5                                      61
MCCouncilC4                                      37
MCCouncilC3                                      24
MCCouncilC1                                       8
Overvotes                                         0
Undervotes                                       94
Counted ballots                                 146
Balance                                           0



Report for context P08 generated 2008-03-11 15:48:08-0400


                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                           129        129
                                Absentee         17         17
                             Provisional          5          5
                                   Blank          0          0

H3B1B2                                          129        129
                                Absentee         17         17
                             Provisional          5          5


                                  VOTE TOTALS

President
PresidentC1 (Moderate)                           55
PresidentC2 (Progressive)                        46
PresidentC3 (Action)                             27
Overvotes                                         0
Undervotes                                        1
Counted ballots                                 129
Balance                                           0

U.S. Senate
USSenateC1 (Moderate)                            66
USSenateC2 (Progressive)                         62
Overvotes                                         0
Undervotes                                        1
Counted ballots                                 129
Balance                                           0

U.S. House
USHouseC3 (Conservation)                         44
USHouseC1 (Moderate)                             40
USHouseC2 (Progressive)                          40
Overvotes                                         0
Undervotes                                        5
Counted ballots                                 129
Balance                                           0


State Senate
SSenateC1 (Moderate)                             43
SSenateC2 (Progressive)                          41
SSenateC3 (Conservation)                         41
Overvotes                                         0
Undervotes                                        4
Counted ballots                                 129
Balance                                           0

State House #3
SHouseD3C1 (Independent)                         42
SHouseD3 Writein1 (write-in)                     40
SHouseD3 Writein2 (write-in)                      6
SHouseD3 Writein3 (write-in)                      4
Invalid Writein (write-in)                        4
Overvotes                                         0
Undervotes                                       33
Counted ballots                                 129
Balance                                           0

County Council
CCouncilC03                                      44
CCouncilC05 (Progressive)                        32
CCouncilC07 (Moderate)                           28
CCouncilC10 (Progressive)                        28
CCouncilC01 (Moderate)                           19
CCouncilC02 (Progressive)                        17
CCouncil Writein1 (write-in)                     17
CCouncilC08 (Independent)                        14
CCouncilC06                                      10
CCouncilC09 (Conservation)                       10
CCouncil Writein2 (write-in)                     10
CCouncilC04 (Conservation)                        8
CCouncil Writein3 (write-in)                      7
CCouncil Writein4 (write-in)                      4
CCouncil Writein5 (write-in)                      4
Invalid Writein (write-in)                        3
Overvotes                                       116
Undervotes                                      145
Counted ballots                                 129
Balance                                           0

Ballot Initiative 1
Yes                                              64
No                                               49
Overvotes                                         0
Undervotes                                       16
Counted ballots                                 129
Balance                                           0

Ballot Initiative 2
Yes                                              55
No                                               49
Overvotes                                         0
Undervotes                                       25
Counted ballots                                 129
Balance                                           0
\end{verbatim}


\subsubsection{Statehouse District results}

\begin{verbatim}
Report for context StateHouseD1 generated 2008-03-11 15:48:09-0400

                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                           566        562
                                Absentee         79         78
                             Provisional         36         36
                                   Blank          7          7
H1B1B2                                          479        475
                                Absentee         68         67
                             Provisional         30         30
                                   Blank          5          5
H1B1B2M1B4                                       87         87
                                Absentee         11         11
                             Provisional          6          6
                                   Blank          2          2

                                  VOTE TOTALS

President
PresidentC1 (Moderate)                          224
PresidentC2 (Progressive)                       212
PresidentC3 (Action)                            112
Overvotes                                         0
Undervotes                                       14
Counted ballots                                 562
Balance                                           0

U.S. Senate
USSenateC1 (Moderate)                           279
USSenateC2 (Progressive)                        271
Overvotes                                         0
Undervotes                                       12
Counted ballots                                 562
Balance                                           0

U.S. House
USHouseC2 (Progressive)                         202
USHouseC3 (Conservation)                        175
USHouseC1 (Moderate)                            171
Overvotes                                         0
Undervotes                                       14
Counted ballots                                 562
Balance                                           0

State Senate
SSenateC3 (Conservation)                        185
SSenateC2 (Progressive)                         177
SSenateC1 (Moderate)                            175
Overvotes                                         0
Undervotes                                       25
Counted ballots                                 562
Balance                                           0

State House #1
SHouseD1C1 (Moderate)                           200
SHouseD1C2 (Progressive)                        193
SHouseD1C3 (Conservation)                       108
SHouseD1 Writein1 (write-in)                     12
SHouseD1 Writein2 (write-in)                     11
Invalid Writein (write-in)                        7
Overvotes                                         0
Undervotes                                       31
Counted ballots                                 562
Balance                                           0

County Council
CCouncilC03                                     191
CCouncilC07 (Moderate)                          130
CCouncilC05 (Progressive)                       111
CCouncilC10 (Progressive)                       102
CCouncilC01 (Moderate)                           88
CCouncil Writein1 (write-in)                     85
CCouncilC08 (Independent)                        75
CCouncilC06                                      74
CCouncilC09 (Conservation)                       72
CCouncilC02 (Progressive)                        71
CCouncil Writein2 (write-in)                     47
CCouncilC04 (Conservation)                       45
CCouncil Writein3 (write-in)                     31
CCouncil Writein4 (write-in)                     27
CCouncil Writein5 (write-in)                     23
Invalid Writein (write-in)                        6
Overvotes                                       372
Undervotes                                      698
Counted ballots                                 562
Balance                                           0

Ballot Initiative 1
Yes                                             260
No                                              193
Overvotes                                         0
Undervotes                                      109
Counted ballots                                 562
Balance                                           0

Ballot Initiative 2
Yes                                             247
No                                              225
Overvotes                                         0
Undervotes                                       90
Counted ballots                                 562
Balance                                           0

Ballot Initiative 4
No                                               34
Yes                                              33
Overvotes                                         0
Undervotes                                       20
Counted ballots                                  87
Balance                                           0

Muni City Council
MCCouncilC2                                      47
MCCouncilC5                                      30
MCCouncilC4                                      28
MCCouncilC3                                      10
MCCouncilC1                                       5
Overvotes                                         0
Undervotes                                       54
Counted ballots                                  87
Balance                                           0



Report for context StateHouseD2 generated 2008-03-11 15:48:09-0400

                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                           695        686
                                Absentee         82         81
                             Provisional         40         39
                                   Blank          7          7
H2B1B2B3                                        547        540
                                Absentee         61         61
                             Provisional         33         32
                                   Blank          6          6
H2B1B2B3M1B4                                    148        146
                                Absentee         21         20
                             Provisional          7          7
                                   Blank          1          1

                                  VOTE TOTALS

President
PresidentC2 (Progressive)                       290
PresidentC1 (Moderate)                          267
PresidentC3 (Action)                            118
Overvotes                                         0
Undervotes                                       11
Counted ballots                                 686
Balance                                           0

U.S. Senate
USSenateC1 (Moderate)                           334
USSenateC2 (Progressive)                        331
Overvotes                                         0
Undervotes                                       21
Counted ballots                                 686
Balance                                           0

U.S. House
USHouseC2 (Progressive)                         246
USHouseC1 (Moderate)                            211
USHouseC3 (Conservation)                        208
Overvotes                                         0
Undervotes                                       21
Counted ballots                                 686
Balance                                           0

State Senate
SSenateC2 (Progressive)                         230
SSenateC1 (Moderate)                            220
SSenateC3 (Conservation)                        207
Overvotes                                         0
Undervotes                                       29
Counted ballots                                 686
Balance                                           0

State House #2
SHouseD2C1 (Moderate)                           269
SHouseD2C2 (Progressive)                        266
SHouseD2C3 (Independent)                         72
SHouseD2 Writein1 (write-in)                     44
Invalid Writein (write-in)                        2
Overvotes                                         0
Undervotes                                       33
Counted ballots                                 686
Balance                                           0

County Council
CCouncilC07 (Moderate)                          199
CCouncilC03                                     197
CCouncilC05 (Progressive)                       126
CCouncil Writein1 (write-in)                    114
CCouncilC01 (Moderate)                          110
CCouncilC10 (Progressive)                       106
CCouncilC02 (Progressive)                        93
CCouncilC08 (Independent)                        85
CCouncilC09 (Conservation)                       81
CCouncilC06                                      73
CCouncilC04 (Conservation)                       66
CCouncil Writein2 (write-in)                     64
CCouncil Writein4 (write-in)                     39
CCouncil Writein3 (write-in)                     34
CCouncil Writein5 (write-in)                     20
Invalid Writein (write-in)                       11
Overvotes                                       560
Undervotes                                      766
Counted ballots                                 686
Balance                                           0

Ballot Initiative 1
Yes                                             353
No                                              217
Overvotes                                         0
Undervotes                                      116
Counted ballots                                 686
Balance                                           0

Ballot Initiative 2
No                                              303
Yes                                             280
Overvotes                                         0
Undervotes                                      103
Counted ballots                                 686
Balance                                           0

Ballot Initiative 3
No                                              311
Yes                                             207
Overvotes                                         0
Undervotes                                      168
Counted ballots                                 686
Balance                                           0

Ballot Initiative 4
Yes                                              60
No                                               52
Overvotes                                         0
Undervotes                                       34
Counted ballots                                 146
Balance                                           0

Muni City Council
MCCouncilC2                                      68
MCCouncilC5                                      61
MCCouncilC4                                      37
MCCouncilC3                                      24
MCCouncilC1                                       8
Overvotes                                         0
Undervotes                                       94
Counted ballots                                 146
Balance                                           0



Report for context StateHouseD3 generated 2008-03-11 15:48:09-0400

                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                           739        731
                                Absentee         93         92
                             Provisional         44         44
                                   Blank          5          5
H3B1B2                                          739        731
                                Absentee         93         92
                             Provisional         44         44
                                   Blank          5          5

                                  VOTE TOTALS

President
PresidentC1 (Moderate)                          305
PresidentC2 (Progressive)                       276
PresidentC3 (Action)                            140
Overvotes                                         0
Undervotes                                       10
Counted ballots                                 731
Balance                                           0

U.S. Senate
USSenateC2 (Progressive)                        366
USSenateC1 (Moderate)                           351
Overvotes                                         0
Undervotes                                       14
Counted ballots                                 731
Balance                                           0

U.S. House
USHouseC1 (Moderate)                            243
USHouseC3 (Conservation)                        233
USHouseC2 (Progressive)                         230
Overvotes                                         0
Undervotes                                       25
Counted ballots                                 731
Balance                                           0

State Senate
SSenateC2 (Progressive)                         240
SSenateC1 (Moderate)                            232
SSenateC3 (Conservation)                        228
Overvotes                                         0
Undervotes                                       31
Counted ballots                                 731
Balance                                           0

State House #3
SHouseD3 Writein1 (write-in)                    266
SHouseD3C1 (Independent)                        221
SHouseD3 Writein2 (write-in)                     41
SHouseD3 Writein3 (write-in)                     20
Invalid Writein (write-in)                       14
Overvotes                                         0
Undervotes                                      169
Counted ballots                                 731
Balance                                           0

County Council
CCouncilC03                                     207
CCouncilC07 (Moderate)                          182
CCouncilC05 (Progressive)                       141
CCouncil Writein1 (write-in)                    131
CCouncilC01 (Moderate)                          124
CCouncilC10 (Progressive)                       121
CCouncilC06                                     101
CCouncilC02 (Progressive)                        77
CCouncilC09 (Conservation)                       77
CCouncilC08 (Independent)                        73
CCouncilC04 (Conservation)                       65
CCouncil Writein2 (write-in)                     54
CCouncil Writein3 (write-in)                     39
CCouncil Writein4 (write-in)                     36
CCouncil Writein5 (write-in)                     19
Invalid Writein (write-in)                       13
Overvotes                                       652
Undervotes                                      812
Counted ballots                                 731
Balance                                           0

Ballot Initiative 1
Yes                                             374
No                                              253
Overvotes                                         0
Undervotes                                      104
Counted ballots                                 731
Balance                                           0

Ballot Initiative 2
No                                              315
Yes                                             305
Overvotes                                         0
Undervotes                                      111
Counted ballots                                 731
\end{verbatim}

\subsubsection{Municipality results}

\begin{verbatim}
Report for context Municipality1 generated 2008-03-11 15:48:07-0400


                                 BALLOT COUNTS

Configuration                                  Read    Counted
-------------                                  ----    -------
Total                                           235        233
                                Absentee         32         31
                             Provisional         13         13
                                   Blank          3          3

H1B1B2M1B4                                       87         87
                                Absentee         11         11
                             Provisional          6          6
                                   Blank          2          2

H2B1B2B3M1B4                                    148        146
                                Absentee         21         20
                             Provisional          7          7
                                   Blank          1          1


                                  VOTE TOTALS

President
PresidentC2 (Progressive)                        92
PresidentC1 (Moderate)                           82
PresidentC3 (Action)                             51
Overvotes                                         0
Undervotes                                        8
Counted ballots                                 233
Balance                                           0

U.S. Senate
USSenateC2 (Progressive)                        113
USSenateC1 (Moderate)                           110
Overvotes                                         0
Undervotes                                       10
Counted ballots                                 233
Balance                                           0

U.S. House
USHouseC2 (Progressive)                          90
USHouseC3 (Conservation)                         67
USHouseC1 (Moderate)                             66
Overvotes                                         0
Undervotes                                       10
Counted ballots                                 233
Balance                                           0

State Senate
SSenateC3 (Conservation)                         79
SSenateC1 (Moderate)                             73
SSenateC2 (Progressive)                          72
Overvotes                                         0
Undervotes                                        9
Counted ballots                                 233
Balance                                           0

State House #1
SHouseD1C1 (Moderate)                            31
SHouseD1C2 (Progressive)                         26
SHouseD1C3 (Conservation)                        20
SHouseD1 Writein1 (write-in)                      3
SHouseD1 Writein2 (write-in)                      3
Invalid Writein (write-in)                        0
Overvotes                                         0
Undervotes                                        4
Counted ballots                                  87
Balance                                           0

State House #2
SHouseD2C1 (Moderate)                            61
SHouseD2C2 (Progressive)                         58
SHouseD2C3 (Independent)                         11
SHouseD2 Writein1 (write-in)                     10
Invalid Writein (write-in)                        1
Overvotes                                         0
Undervotes                                        5
Counted ballots                                 146
Balance                                           0

County Council
CCouncilC07 (Moderate)                           67
CCouncilC03                                      64
CCouncilC05 (Progressive)                        51
CCouncilC09 (Conservation)                       37
CCouncilC10 (Progressive)                        37
CCouncil Writein1 (write-in)                     37
CCouncilC01 (Moderate)                           33
CCouncilC08 (Independent)                        29
CCouncilC02 (Progressive)                        26
CCouncilC04 (Conservation)                       23
CCouncilC06                                      22
CCouncil Writein2 (write-in)                     20
CCouncil Writein4 (write-in)                     12
CCouncil Writein3 (write-in)                      8
CCouncil Writein5 (write-in)                      6
Invalid Writein (write-in)                        5
Overvotes                                       208
Undervotes                                      247
Counted ballots                                 233
Balance                                           0

Ballot Initiative 1
Yes                                             119
No                                               75
Overvotes                                         0
Undervotes                                       39
Counted ballots                                 233
Balance                                           0

Ballot Initiative 2
Yes                                             103
No                                               96
Overvotes                                         0
Undervotes                                       34
Counted ballots                                 233
Balance                                           0

Ballot Initiative 3
No                                               69
Yes                                              48
Overvotes                                         0
Undervotes                                       29
Counted ballots                                 146
Balance                                           0

Ballot Initiative 4
Yes                                              93
No                                               86
Overvotes                                         0
Undervotes                                       54
Counted ballots                                 233
Balance                                           0

Muni City Council
MCCouncilC2                                     115
MCCouncilC5                                      91
MCCouncilC4                                      65
MCCouncilC3                                      34
MCCouncilC1                                      13
Overvotes                                         0
Undervotes                                      148
Counted ballots                                 233
Balance                                           0
\end{verbatim}

\subsection{Conclusion}
\label{LenSection7}

Any person interested in simulating potential results of a specific election
and testing voting systems for use in that election may follow the procedure
described in this section to generate sample ballots.  This procedure helps
to identify the features of a specific election that are relevant under the
logic model of the VVSG and provides guidance in representing those features
in the abstract and physical models of Votetest.  The database representation
can then be augmented with probability distributions that are most relevant
to the local election and sample ballots generated.

The sample ballots can be input to the voting system in use for the local
election to verify that those systems operate as expected for that election.
In addition, Votetest reporting results for the same sample ballots can be
used to verify that the voting system being tested produces correct results
for that election as required by the VVSG\@.


\bibliographystyle{unsrt}
\bibliography{Documentation}
\end{document}
