NIST logo

Publication Citation: Significance Test with Data Dependency in Speaker Recognition Evaluation

NIST Authors in Bold

Author(s): Jin Chu Wu; Alvin Martin; Craig S. Greenberg; Raghu N. Kacker; Vincent M. Stanford;
Title: Significance Test with Data Dependency in Speaker Recognition Evaluation
Published: July 25, 2013
Abstract: To evaluate the performance of speaker recognition systems, a detection cost function defined as a weighted sum of the probabilities of type I and type II errors is employed. The speaker datasets may have data dependency due to multiple uses of the same subjects. Using the standard errors of the detection cost function computed by means of the two-layer nonparametric two-sample bootstrap method, a significance test is performed to determine whether the difference between the measured performance levels of two speaker recognition algorithms is statistically significant. While conducting the significance test, the correlation coefficient between two systems‰ detection cost functions is taken into account. Examples are provided.
Conference: SPIE Defense Security Sensing, 2013
Proceedings: Proc. of SPIE Vol. 8734
Location: Baltimore, MD
Dates: May 1-2, 2013
Keywords: Significance test; Data dependency; Speaker recognition evaluation; Measurement uncertainty; Standard error; ROC analysis; Bootstrap; Biometrics.
Research Areas: Data and Informatics, Measurements, Uncertainty Analysis
PDF version: PDF Document Click here to retrieve PDF version of paper (387KB)