Utility Assessment in TRANSTAC: Using a set of complementary methods

Published

February 8, 2010

Author(s)

Michelle P. Steves, Emile L. Morse

Abstract

This paper describes the methods used during formative utility assessments for speech-to-speech, real-time translation systems intended for tactical use. The systems, while still prototypical, had hardware platforms that could be exercised indoors or outdoors by study participants with pertinent backgrounds and skills that are similar to the intended target users. English-speaking and target-language subjects participated in exercises of the systems exchanging information without the aid of a human interpreter. Feedback on subjects’ experiences during the dialog interactions was collected via two primary methods, i.e., survey questionnaires and semi-structured interviews. In this paper, we describe our human-centered approach to utility assessment, how we combined the use of two feedback gathering methods, and discuss findings from use of this approach.

Proceedings Title

Performance Metrics for Intelligent Systems Workshop 2009

Conference Dates

September 21-23, 2009

Conference Location

Gaithersburg, MD

Pub Type

Conferences

Keywords

Speech-to-speech translation systems, evaluation, questionnaire, semi-structured interview, utility assessments, complementary assessment methods

Citation

Steves, M. and Morse, E. (2010), Utility Assessment in TRANSTAC: Using a set of complementary methods, Performance Metrics for Intelligent Systems Workshop 2009, Gaithersburg, MD (Accessed April 26, 2024)