Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Utility Assessment in TRANSTAC: Using a set of complementary methods



Michelle P. Steves, Emile L. Morse


This paper describes the methods used during formative utility assessments for speech-to-speech, real-time translation systems intended for tactical use. The systems, while still prototypical, had hardware platforms that could be exercised indoors or outdoors by study participants with pertinent backgrounds and skills that are similar to the intended target users. English-speaking and target-language subjects participated in exercises of the systems exchanging information without the aid of a human interpreter. Feedback on subjects’ experiences during the dialog interactions was collected via two primary methods, i.e., survey questionnaires and semi-structured interviews. In this paper, we describe our human-centered approach to utility assessment, how we combined the use of two feedback gathering methods, and discuss findings from use of this approach.
Proceedings Title
Performance Metrics for Intelligent Systems Workshop 2009
Conference Dates
September 21-23, 2009
Conference Location
Gaithersburg, MD


Speech-to-speech translation systems, evaluation, questionnaire, semi-structured interview, utility assessments, complementary assessment methods


Steves, M. and Morse, E. (2010), Utility Assessment in TRANSTAC: Using a set of complementary methods, Performance Metrics for Intelligent Systems Workshop 2009, Gaithersburg, MD (Accessed July 22, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created February 8, 2010, Updated February 19, 2017