Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Significance Test with Data Dependency in Speaker Recognition Evaluation



Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker, Vincent M. Stanford


To evaluate the performance of speaker recognition systems, a detection cost function defined as a weighted sum of the probabilities of type I and type II errors is employed. The speaker datasets may have data dependency due to multiple uses of the same subjects. Using the standard errors of the detection cost function computed by means of the two-layer nonparametric two-sample bootstrap method, a significance test is performed to determine whether the difference between the measured performance levels of two speaker recognition algorithms is statistically significant. While conducting the significance test, the correlation coefficient between two systems’ detection cost functions is taken into account. Examples are provided.
Proceedings Title
Proc. of SPIE Vol. 8734
Conference Dates
May 1-2, 2013
Conference Location
Baltimore, MD
Conference Title
SPIE Defense Security Sensing, 2013


Significance test, Data dependency, Speaker recognition evaluation, Measurement uncertainty, Standard error, ROC analysis, Bootstrap, Biometrics.


, J. , Martin, A. , Greenberg, C. , Kacker, R. and Stanford, V. (2013), Significance Test with Data Dependency in Speaker Recognition Evaluation, Proc. of SPIE Vol. 8734, Baltimore, MD, [online], (Accessed June 24, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created July 25, 2013, Updated February 19, 2017