Measures, Uncertainties, and Significance Test in Operational ROC Analysis
Jin Chu Wu, Alvin F. Martin, Raghu N. Kacker
In operational ROC (receiver operating characteristic) analysis of fingerprint-image matching algorithms on large datasets, the measures and their accuracies are investigated in the three scenarios: 1) the true accept rate (TAR) of genuine scores at a specified false accept rate (FAR) of impostor scores, 2) the TAR and FAR at a given threshold, and 3) the equal error rate (EER). The key issue is how to compute the accuracy. The accuracy is calculated using the nonparametric two-sample bootstrap based on our extensive studies of bootstrap variability on large datasets. The ultimate goal is to perform the comparison. Using the standard errors computed, the significance test is carried out to determine whether the difference between the performance of one algorithm and a hypothesized value, or the difference between the performances of two algorithms where the correlation is taken into account is statistically significant. In the case that the alternative hypothesis is accepted, the sign of the difference is employed to determine which is better than the other. Examples are provided.
, Martin, A.
and Kacker, R.
Measures, Uncertainties, and Significance Test in Operational ROC Analysis, Journal of Research (NIST JRES), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=903816
(Accessed December 3, 2023)