The Rich Transcription 2007 Meeting Recognition Evaluation

Author(s)

Jonathan G. Fiscus, Jerome G. Ajot, John S. Garofolo

Abstract

We present the design and results of the Spring 2007 (RT-07) Rich Transcription Meeting Recognition Evaluation; the fifth in a series of community-wide evaluations of language technologies in the meeting domain. For 2007, we supported three evaluation tasks: Speech-To-Text (STT) transcription, ?Who Spoke When? Diarization (SPKR), and Speaker Attributed Speech-To-Text (SASTT). The SASTT task, which combines STT and SPKR tasks, was a new evaluation task. The test data consisted of three test sets: Conference Meetings, Lecture Meetings, and Coffee Breaks from lecture meetings. The Coffee Break data was included as a new test set this year. Twenty-one re-search sites materially contributed to the evaluation by providing data or build-ing systems. The lowest STT word error rates with up to four simultaneous speakers in the multiple distant microphone condition were 40.6 %, 49.8 %, and 48.4 % for the conference, lecture, and coffee break test sets respectively. For the SPKR task, the lowest diarization error rates for all speech in the multiple distant microphone condition were 8.5 %, 25.8 %, and 25.5 % for the conference, lecture, and coffee break test sets respectively. For the SASTT task, the lowest speaker attributed word error rates for segments with up to three simultaneous speakers in the multiple distant microphone condition were

Proceedings Title

The Joint Proceedings of the 2006 CLEAR and RT Evaluations

Pub Type

Conferences

Keywords

Language Technology, Rich Transcription, Speech-To-Text

Citation

Fiscus, J. , Ajot, J. and Garofolo, J. (1970), The Rich Transcription 2007 Meeting Recognition Evaluation, The Joint Proceedings of the 2006 CLEAR and RT Evaluations (Accessed April 25, 2024)