An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Bootstrap method versus analytical approach for estimating uncertainty of measure in ROC analysis on large datasets
Published
Author(s)
Jin Chu Wu, Alvin F. Martin, Gregory A. Sanders, Raghu N. Kacker
Abstract
The nonparametric two-sample bootstrap is employed to estimate uncertainties of statistics of interest in receiver operating characteristic (ROC) analysis on large datasets with/without data dependency due to multiple use of the same subjects in many disciplines, based on our studies of bootstrap variability. On the other hand, it would seem that the analytical method might be used for the same purpose. However, comparing these two methods, the differences are noteworthy. (1) The bootstrap method takes account of how genuine scores and impostor scores are distributed, which is associated with how the matching system works; but the analytical method does not. (2) If datasets involve data dependency, the bootstrap method can take it into account; but the analytical method cannot. (3) The covariance term that occurred in the speaker recognition evaluations while estimating the standard error of the measure can be taken into account intrinsically by the bootstrap method; but it is very hard to estimate analytically. (4) The analytical method generally underestimates the uncertainties of statistics of interest; but the bootstrap method estimates them more conservatively. To demonstrate these observations, the data used in this article were generated from a variety of sources: speaker recognition evaluations, biometrics evaluations, simulated data with normal distributions, and simulated data with nonparametric distributions.
, J.
, Martin, A.
, Sanders, G.
and Kacker, R.
(2018),
Bootstrap method versus analytical approach for estimating uncertainty of measure in ROC analysis on large datasets, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.IR.8218
(Accessed November 12, 2024)