Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Evaluation Methodology and Metrics Employed to Assess the TRANSTAC Two-way, Speech-to-Speech Translation Systems



Gregory A. Sanders, Brian A. Weiss, Craig I. Schlenoff, Michelle P. Steves, Sherri Condon


One of the most difficult challenges that military personnel face when operating in foreign countries is clear and successful communication with the local population. To address this issue, the Defense Advanced Research Projects Agency (DARPA) is funding academic institutions and industrial organizations through the Spoken Language Communication and Translation System for Tactical Use (TRANSTAC) program to develop practical machine translation systems. The goal of the TRANSTAC program is to demonstrate capabilities to rapidly develop and field free-form, two-way, speech-to-speech translation systems that enable speakers of different languages to communicate with one another in real-world tactical situations without an interpreter. Evaluations of these technologies are a significant part of the program and DARPA has asked the National Institute of Standards and Technology (NIST) to lead this effort. This article presents the experimental design of the TRANSTAC evaluations and the metrics, both quantitative and qualitative, that were captured to comprehensively assess the systems’ performance.
Computer Speech and Language


Evaluatio, Performance Metrics, Speech-to-Speech Translation, TRANSTAC


Sanders, G. , Weiss, B. , Schlenoff, C. , Steves, M. and Condon, S. (2013), Evaluation Methodology and Metrics Employed to Assess the TRANSTAC Two-way, Speech-to-Speech Translation Systems, Computer Speech and Language, [online], (Accessed June 15, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created February 1, 2013, Updated November 10, 2018