\begin{longtable}{|X|X|}

\caption{Additional required test cases} \label{othertests} \\

\multicolumn{1}{c}{\bf Requirements} & \multicolumn{1}{c}{\bf Test description} \\ \hline

Realistic ballot styles \cite[Part~1 Req.\ 7.2-A.1, A.4 and A.5]{VVSGNGdraft}
& Test a variety of sample ballot styles, such as those available on the NIST
web site \cite{samples}, to ensure that the voting system can handle the text
of long ballot questions, complicated choice labels (such as for primary
presidential delegation nominations), miscellaneous ballot text, and overall
formatting and layout of ballot styles as used in practice. \\
\hline

Optical scan accuracy \cite[Part~1 Req.\ 6.3.2-B and
Section~7.7.5]{VVSGNGdraft} & When test cases are realized for optical scan
devices, the marks used should exercise the range of reliably detectable marks
as defined by the vendor, from ideal marks to the standard mark specified in
\cite[Part~1 Req.\ 7.7.5-D]{VVSGNGdraft}.  However, to ensure that the tests
are defensible, the relative frequency with which the different types of marks
appear should be realistic.  While both ideal marks and standard marks should
be tested repeatedly, it would not be realistic for either extreme to dominate
the input.  The average mark should be typical of what an average voter would
make.  The ability of the scanner to ignore extraneous marks should also
be tested (\cite[Part~1 Req.\ 7.7.5-E]{VVSGNGdraft}).\\ \hline

Marginal marks \cite[Part~1 Req.\ 3.2.2.2-E and 7.7.3-C]{VVSGNGdraft} & Test
marks that are clearly within the marginal zone as defined by the vendor to
verify that the behavior on marginal marks is as specified.  Do not test marks
that are near the boundaries, as the uncertainty of the boundaries is of no
consequence (see \cite[Part~1 Section~7.7.5.1]{VVSGNGdraft}). \\ \hline

Respecting limits \cite[Part~1 Req.\ 2.4-A.e, Req.\ 6.4.1.8-A.1 and
Section~7.5.6]{VVSGNGdraft}\footnote{Note on pending erratum.  \cite[Part~1 Section 7.1]{VVSGNGdraft} is missing a requirement saying that the central EMS shall prevent the defining of any election that would cause the voting system to exceed design limits such as maximum number of choices in a contest, maximum number of contests on a ballot, etc.  That requirement should be cited here if and when it is added.} & Construct test cases as needed to satisfy
\cite[Part~3 Req.\ 5.2.3-B and C]{VVSGNGdraft}.  The test generator
(\autoref{TestGenerator}) may be of use. \\ \hline

Definition reuse \cite[Part~1 Req.\ 7.2-C]{VVSGNGdraft} & Choose two test cases that use similar ballot styles.  Execute the first as usual, but instead of clearing out all definitions between tests, attempt to construct the ballot styles for the second test based on the definitions retained from the first. \\ \hline

Ballot style mismatches \cite[Part~1 Req.\ 7.7.1-A]{VVSGNGdraft} & Attempt to
execute a test case with mismatched ballot styles loaded on different devices
and verify that the system prevents incorrect reports from being generated.
(It may prevent mismatched ballot styles from being loaded in the first
place.) \\ \hline

Ballot orientation \cite[Part~1 Req.\ 7.7.1-B]{VVSGNGdraft} & If the vendor
specifies that the tabulator is capable of scanning ballots in more than one
orientation, allow the orientation of fed ballots to vary during the execution
of all test cases.  If the vendor specifies one or more orientations as being
incorrect, attempt to feed ballots in the unsupported orientations and verify
that the ballots are rejected. \\ \hline

Validation of input \cite[Part~1 Req.\ 6.4.1.8-A and A.1]{VVSGNGdraft} & Since
poor validation of input often creates an attack vector, this should be covered
as part of security testing and evaluation. \\
\hline

Miscellaneous capabilities \cite[Part~1 Req.\ 7.2-A.1, 7.2-A.5, 7.3.1-B,
7.7.3-A.3, 7.8.3.3-B.1 and
many others]{VVSGNGdraft} & The VVSG contains numerous requirements for
miscellaneous voting system capabilities that are orthogonal to the Votetest
data model.  Since the general test template specified in
\cite[Part~3 Section~5.2.1.1]{VVSGNGdraft} treats each test case as an entire
election, many such capabilities will be tested incidentally during the
execution of the basic test suite.  Use the Voting Equipment User Documentation
to determine how to exercise any capabilities that are not tested incidentally
and then verify that they satisfy the applicable VVSG requirements. \\ \hline

System-specific functions & These must be tested in accordance with
\cite[Part~3 Req.\ 5.2.3-G]{VVSGNGdraft}. \\ \hline

\end{longtable}
