Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Measurement Uncertainties in Speaker Recognition Evaluation



Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker


The speaker recognition evaluation is an ongoing series of evaluations conducted by NIST. A detection cost function is computed over the sequence of trials provided and used for all speaker detection tests while measuring speaker detection performance. The detection cost function is defined as a weighted sum of probabilities of type I error and type II error. The sampling variability can result in measurement uncertainties of the detection cost function. Hence, while evaluating and comparing the performances of speaker recognition systems, the measurement uncertainties must be taken into account. In this article, the uncertainties of measures of detection cost functions in terms of standard errors (SE) and confidence intervals are computed using the nonparametric two-sample bootstrap methods based on our extensive bootstrap variability studies on large datasets conducted before. The data independence is assumed because the bootstrap results of SEs matched very well with the analytical results of SEs using the Mann-Whitney statistic if the metric of area under ROC curve is employed. Examples are provided.
NIST Interagency/Internal Report (NISTIR) - 7722
Report Number


speaker recognition evaluation, biometrics, bootstrap, uncertainty, standard error, confidence interval


, J. , Martin, A. , Greenberg, C. and Kacker, R. (2010), Measurement Uncertainties in Speaker Recognition Evaluation, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], (Accessed February 24, 2024)
Created September 15, 2010, Updated February 19, 2017