Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Measurement Uncertainties of Three Score Distributions and Two Thresholds with Data Dependency



Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker


The National Institute of Standards and Technology conducts an ongoing series of Speaker Recognition Evaluations (SRE). Recently a new paradigm was adopted to evaluate the performance of speaker recognition systems, in which three distributions of scores, i.e., target, known non-target, and unknown non-target scores, as well as two thresholds, were employed. The new detection cost function was defined to be an average of the two weighted sums of the probabilities of type I and type II errors corresponding to the two thresholds. In addition, data dependency due to multiple use of the same subjects is also involved. The data were reorganized into a two-layer structure in view of the probability theory, and then the uncertainties of the detection cost functions were computed using the nonparametric three-sample two-layer bootstrap method. Comparing these results with those calculated by using all the raw data and the nonparametric three-sample bootstrap method with the i.i.d. assumption, the measurement accuracies, i.e., the detection cost functions, have changed little; but the measurement uncertainties, i.e., the standard errors of the detection cost function, have improved as a result of taking account of the data dependency. Forty speaker recognition systems were taken as examples.
NIST Interagency/Internal Report (NISTIR) - 8025
Report Number


metrology, measure, accuracy, uncertainty, bootstrap, data dependency, speaker recognition


, J. , Martin, A. , Greenberg, C. and Kacker, R. (2014), Measurement Uncertainties of Three Score Distributions and Two Thresholds with Data Dependency, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], (Accessed June 12, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created September 18, 2014, Updated November 10, 2018