Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Performance Evaluation of Speech Translation Systems



Brian A. Weiss, Craig I. Schlenoff, Gregory A. Sanders, Michelle P. Steves, Sherri Condon, Jon Phillips, Dan Parvaz


One of the most challenging tasks for uniformed service personnel serving in foreign countries is effective verbal communication with the local population. To remedy this problem, several companies and academic institutions have been funded to develop machine translation systems as part of the DARPA TRANSTAC (Spoken Language Communication and Translation System for Tactical Use) program. The goal of this program is to demonstrate capabilities to rapidly develop and field free-form, two-way translation systems that would enable speakers of different languages to communicate with one another in real-world tactical situations. DARPA has mandated that each TRANSTAC technology be evaluated numerous times throughout the life of the program and has tasked the National Institute of Standards and Technology (NIST) to lead this effort. This paper describes the experimental design methodology and test procedures from the most recent evaluation, conducted in July 2007, which focused on English to/from Iraqi Arabic.
Proceedings Title
Proceedings of the 6th edition of the Language Resources and Evaluation Conference
Conference Dates
May 28-30, 2008
Conference Location
Marrakech, MA
Conference Title
6th edition of the Language Resources and Evaluation Conference


Performance Evaluation, SCORE Evaluation Framework, Utility, Machine Translation


Weiss, B. , Schlenoff, C. , Sanders, G. , Steves, M. , Condon, S. , Phillips, J. and Parvaz, D. (2008), Performance Evaluation of Speech Translation Systems, Proceedings of the 6th edition of the Language Resources and Evaluation Conference, Marrakech, MA, [online], (Accessed April 15, 2024)
Created May 28, 2008, Updated February 19, 2017