Measurement Uncertainties of Three Score Distributions and Two Thresholds with Data Dependency
Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker
The National Institute of Standards and Technology conducts an ongoing series of Speaker Recognition Evaluations (SRE). Recently a new paradigm was adopted to evaluate the performance of speaker recognition systems, in which three distributions of scores, i.e., target, known non-target, and unknown non-target scores, as well as two thresholds, were employed. The new detection cost function was defined to be an average of the two weighted sums of the probabilities of type I and type II errors corresponding to the two thresholds. In addition, data dependency due to multiple use of the same subjects is also involved. The data were reorganized into a two-layer structure in view of the probability theory, and then the uncertainties of the detection cost functions were computed using the nonparametric three-sample two-layer bootstrap method. Comparing these results with those calculated by using all the raw data and the nonparametric three-sample bootstrap method with the i.i.d. assumption, the measurement accuracies, i.e., the detection cost functions, have changed little; but the measurement uncertainties, i.e., the standard errors of the detection cost function, have improved as a result of taking account of the data dependency. Forty speaker recognition systems were taken as examples.
, Martin, A.
, Greenberg, C.
and Kacker, R.
Measurement Uncertainties of Three Score Distributions and Two Thresholds with Data Dependency, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], https://doi.org/10.6028/NIST.IR.8025
(Accessed October 2, 2022)