NOTICE: Due to a lapse in annual appropriations, most of this website is not being updated. Learn more.
Form submissions will still be accepted but will not receive responses at this time. Sections of this site for programs using non-appropriated funds (such as NVLAP) or those that are excepted from the shutdown (such as CHIPS and NVD) will continue to be updated.
An official website of the United States government
Here’s how you know
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Data Dependency on Measurement Uncertainties in Speaker Recognition Evaluation
Published
Author(s)
Jin Chu Wu, Alvin F. Martin, Craig S. Greenberg, Raghu N. Kacker
Abstract
The National Institute of Standards and Technology conducts an ongoing series of Speaker Recognition Evaluations (SRE). Speaker detection performance is measured using a detection cost function defined as a weighted sum of the probabilities of type I and type II errors. The sampling variability can result in measurement uncertainties. In our prior study, the data independency was assumed in using the nonparametric two-sample bootstrap method to compute the standard errors (SE) of the detection cost function based on our extensive bootstrap variability studies in ROC analysis on large datasets. In this article, the data dependency caused by multiple uses of the same subjects is taken into account. The data are grouped into target sets and non-target sets, and each set contains multiple scores. One-layer and two-layer bootstrap methods are proposed based on whether the two-sample bootstrap resampling takes place only on target sets and non-target sets, or subsequently on target scores and non-target scores within the sets, respectively. The SEs of the detection cost function using these two methods along with those with the assumption of data independency are compared. It is found that the data dependency increases both estimated SEs and the variations of SEs. Some suggestions regarding the test design are provided.
Proceedings Title
Proc. of SPIE Vol. 8382, Active and Passive Signatures III
, J.
, Martin, A.
, Greenberg, C.
and Kacker, R.
(2012),
Data Dependency on Measurement Uncertainties in Speaker Recognition Evaluation, Proc. of SPIE Vol. 8382, Active and Passive Signatures III, Baltimore, MD, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=911725
(Accessed October 2, 2025)