Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Is the Urn Well-Mixed? Uncovering False Cofactor Homogeneity Assumption in Evaluation

Published

Author(s)

Ross J. Micheals, T E. Boult

Abstract

Measuring system performance is conceptually straightforward; it is the interpretation of the results and their use as predictors of future performance that are the exceptional challenges in system evaluation and the experimentation in general. Good experimental design is critical in evaluation, but there have been very few techniques that a scientist may use to check their design for either overlooked associations or weak assumptions. For biometric and vision system evaluation, the complexity of the systems make a thorough exploration of the problem space impossible. This lack of verifiability in experimental design is a serious issue. In this paper, we present a new evaluation methodology that aids the researcher in discovering false assumptions about the homogeneity of cofactors - when the data is not well mixed. The new methodology is then applied in the context of a biometric system evaluation.
Citation
NIST Interagency/Internal Report (NISTIR) - 7156
Report Number
7156

Keywords

biometrics, evaluation, performance characterization, sample design, variance estimation

Citation

Micheals, R. and Boult, T. (2004), Is the Urn Well-Mixed? Uncovering False Cofactor Homogeneity Assumption in Evaluation, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=150259 (Accessed March 28, 2024)
Created October 1, 2004, Updated February 19, 2017