U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

PUBLICATIONS

Results of The 2015 NIST Language Recognition Evaluation

Published

Author(s)

Hui Zhao, Desire Banse, G R. Doddington, Craig Greenberg, Audrey N. Tong, John M. Howard, Alvin F. Martin, Jaime Hernandez-Cordero, Lisa Mason, Douglas A. Reynolds, Elliot Singer

Abstract

In 2015, NIST conducted the most recent in an ongoing series of Language Recognition Evaluations (LRE) meant to foster research in language recognition. The 2015 Language Recognition Evaluation (LRE15) featured 20 target languages grouped into 6 language clusters. The evaluation was focused on distinguishing languages within each cluster, without disclosing which cluster a test language belongs to. Different from prior LRE's, LRE15 introduced several key new aspects, such as using limited and specified training data and a wider range of durations for test segments. Unlike in past LRE's, system were not expected to output hard decisions for each test language and test segment, instead systems were required to provide a score vector of log likelihood ratios for each the test segment. A total of 24 research organizations participated in this four-month long evaluation and submitted 167 systems to be evaluated. The evaluation results showed that top performing systems exhibited similar performance and there were wide variations in performance based on language clusters and within cluster language pairs. Among the 6 clusters, the French cluster was the hardest to recognize, with near random performance, and the Slavic cluster was the easiest to recognize.
Proceedings Title
17th Annual Conference of the International Speech Communication Association (INTERSPEECH 2016): Understanding Speech Processing in Humans and Machines
Conference Dates
September 8-12, 2016
Conference Location
San Frisco, CA, US
Conference Title
Interspeech
Pub Type
Conferences

Keywords

language recognition, language detection, language identification, NIST LRE, NIST evaluation
Materials characterization

Citation

Zhao, H. , Banse, D. , Doddington, G. , Greenberg, C. , Tong, A. , Howard, J. , Martin, A. , Hernandez-Cordero, J. , Mason, L. , Reynolds, D. and Singer, E. (2016), Results of The 2015 NIST Language Recognition Evaluation, 17th Annual Conference of the International Speech Communication Association (INTERSPEECH 2016): Understanding Speech Processing in Humans and Machines, San Frisco, CA, US (Accessed November 30, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created September 11, 2016, Updated October 12, 2021