Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Applying Automated Metrics to Speech Translation Dialogs



Sherri Condon, Jon Phillips, Christy Doran, John Aberdeen, Dan Parvaz, Beatrice Oshika, Gregory A. Sanders, Craig I. Schlenoff


Over the past five years, the Defense Advanced Research Projects Agency (DARPA) has funded development of speech translation systems for tactical applications. A key component of the research program has been extensive system evaluation, with dual objectives of assessing progress overall and comparing among systems. This paper describes evaluation methods and measures, and presents results from the January 2007 evaluation of English-Iraqi Arabic systems. The results demonstrate the value of obtaining multiple perspectives from a variety of analyses. A companion report details the methods used to obtain BLEU, TER, and METEOR scores for the systems and discusses issues concerning the relation between development data and evaluation methods.
Proceedings Title
Proceedings of the 2008 LREC (Language Resources and Evaluation Conference)
Conference Dates
May 28-30, 2008
Conference Location
Morocco, US


TRANSTAC, speech to speech translation, DARPA, Arabic


Condon, S. , Phillips, J. , Doran, C. , Aberdeen, J. , Parvaz, D. , Oshika, B. , Sanders, G. and Schlenoff, C. (2008), Applying Automated Metrics to Speech Translation Dialogs, Proceedings of the 2008 LREC (Language Resources and Evaluation Conference), Morocco, US, [online], (Accessed July 13, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created June 29, 2008, Updated October 12, 2021