The Rich Transcription evaluation series is implemented to promote and gauge advances in the state-of-the-art in several automatic speech recognition technologies. The goal of the evaluation series is to create recognition technologies that will produce transcriptions which are more readable by humans and more useful for machines. As such, a set of research tasks has been defined which are broadly categorized as either Speech-to-Text Transcription (STT) tasks and Metadata Extraction (MDE) tasks.

The evaluation series was started in 2002 and continues to this day.  The latest evaluation, "Rich Transcription Spring 2007 Meeting Recognition", evaluation will be conducted during February-May of 2007.  The meeting recognition community is expanding the scope of the RT evaluations to include multimodal research including audio and video.

Rich Transcription Evaluation History

The Rich Transcription 2002 Evaluation (RT-02), the first in the RT evaluation series, was implemented in April 2002. It included STT tasks for broadcast news speech, conversational telephone speech, and meeting room speech in English. The evaluation also included one MDE task speaker diarization for broadcast news and conversational telephone speech also in English.

The Rich Transcription Spring 2003 Evaluation (RT-03S), implemented in March-April 2003, focused on STT tasks. The STT tasks not only included the different data domains but also extended to different languages in addition to English. Unlike the RT-02 STT tasks, the RT-03S STT tasks did not include the meeting data domain because that domain was moved to another RT evaluation. The RT-03S evaluation also included one MDE task, "Who Spoke When" speaker diarization for broadcast news speech and conversational telephone speech in English.

The Rich Transcription Fall 2003 Evaluation (RT-03F), implemented in October 2003, focused on MDE tasks. The MDE tasks included "Who Said What" speaker diarization, sentence boundary detection, and disfluencies detection for broadcast news speech and conversational telephone speech in English.

The Rich Transcription Spring 2004 Evaluation (RT-04S) will be implemented in 2004. It includes separate STT and MDE tasks as well as integrated STT and MDE tasks for the various domains and languages.

The Rich Transcription Fall 2004 Evaluation (RT-04F), implemented in October 2004, included STT tasks for broadcast news speech and conversational telephone speech in English, Mandarin, and Arabic and MDE tasks also for broadcast news speech and conversational telephone speech but only for English.

The Rich Transcription Spring 2005 Evaluation (RT-05F) will be a Meeting Domain evaluation of English meetings. There will be four evaluation tasks: STT, MDE Speaker Diarization, MDE Speech Activity Detection, and MDE Source Localization. The cross site evaluation corpus will include "conference" room meetings and "lecture" room meetings. Consult the RT-05S evaluation page for details.

The RT-02, RT-03S, RT-03F, RT-04S, RT-04F, RT-05S, RT-06S, and RT-07 pages contain the documentation and tools for implementing the tasks for a given evaluation.

The Rich Transcription evaluation series is sponsored by NIST and is open to all interested participants.

Contact Information

If you would like to participate in the evaluation or obtain additional information, send an email to our staff.

NIST RT website comments and corrections should be emailed to our webmaster