Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Measures, Uncertainties, and Significance Test in Operational ROC Analysis

Published

Author(s)

Jin Chu Wu, Alvin F. Martin, Raghu N. Kacker

Abstract

In operational ROC (receiver operating characteristic) analysis of fingerprint-image matching algorithms on large datasets, the measures and their accuracies are investigated in the three scenarios: 1) the true accept rate (TAR) of genuine scores at a specified false accept rate (FAR) of impostor scores, 2) the TAR and FAR at a given threshold, and 3) the equal error rate (EER). The key issue is how to compute the accuracy. The accuracy is calculated using the nonparametric two-sample bootstrap based on our extensive studies of bootstrap variability on large datasets. The ultimate goal is to perform the comparison. Using the standard errors computed, the significance test is carried out to determine whether the difference between the performance of one algorithm and a hypothesized value, or the difference between the performances of two algorithms where the correlation is taken into account is statistically significant. In the case that the alternative hypothesis is accepted, the sign of the difference is employed to determine which is better than the other. Examples are provided.
Citation
Journal of Research (NIST JRES) -
Volume
116
Issue
1

Keywords

Receiver operating characteristic (ROC) analysis, Fingerprint, Biometrics, Nonparametric bootstrap, Standard error, Confidence interval, Significance test

Citation

, J. , Martin, A. and Kacker, R. (2011), Measures, Uncertainties, and Significance Test in Operational ROC Analysis, Journal of Research (NIST JRES), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=903816 (Accessed November 4, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created January 31, 2011, Updated February 19, 2017