Applying Automated Metrics to Speech Translation Dialogs

Published

June 30, 2008

Author(s)

Sherri Condon, Jon Phillips, Christy Doran, John Aberdeen, Dan Parvaz, Beatrice Oshika, Gregory A. Sanders, Craig I. Schlenoff

Abstract

Over the past five years, the Defense Advanced Research Projects Agency (DARPA) has funded development of speech translation systems for tactical applications. A key component of the research program has been extensive system evaluation, with dual objectives of assessing progress overall and comparing among systems. This paper describes evaluation methods and measures, and presents results from the January 2007 evaluation of English-Iraqi Arabic systems. The results demonstrate the value of obtaining multiple perspectives from a variety of analyses. A companion report details the methods used to obtain BLEU, TER, and METEOR scores for the systems and discusses issues concerning the relation between development data and evaluation methods.

Proceedings Title

Proceedings of the 2008 LREC (Language Resources and Evaluation Conference)

Conference Dates

May 28-30, 2008

Conference Location

Morocco, US

Pub Type

Conferences

Download Paper

Local Download

Keywords

TRANSTAC, speech to speech translation, DARPA, Arabic

Citation

Condon, S. , Phillips, J. , Doran, C. , Aberdeen, J. , Parvaz, D. , Oshika, B. , Sanders, G. and Schlenoff, C. (2008), Applying Automated Metrics to Speech Translation Dialogs, Proceedings of the 2008 LREC (Language Resources and Evaluation Conference), Morocco, US, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=824596 (Accessed December 1, 2024)