TREC-6 1997 Spoken Document Retrieval Track Overview and Results

Published

October 25, 1999

Author(s)

John S. Garofolo, Ellen M. Voorhees, Vincent M. Stanford, K Sparck Jones

Abstract

This paper describes the 1997 TREC-6 Spoken Document Retrieval (SDR) Track which implemented a first evaluation of retrieval of broadcast news excerpts using a combination of automatic speech recognition and information retrieval technologies. The motivations behind the SDR Track and background regarding its development and implementation are discussed. The SDR evaluation collection and topics are described and summaries and analyses of the results of the track are presented. Finally, plans for future SDR tracks are described.Since this was the first implementation of an evaluation of SDR, the evaluation itself as well as the evaluated technology should be considered experimental. The results of the first SDR Track were very encouraging and showed us that SDR could be successfully implemented and evaluated. However, the results of the SDR Track should be considered preliminary since the 50-hour spoken document collection used was very small for retrieval experiments (even though it was considered extremely large for speech recognition purposes.) Nonetheless, with thirteen groups participating in the TREC-6 SDR Track, a considerable amount of experience was gained in implementing and evaluating the SDR task. This experience will greatly benefit the next 1998 TREC-7 SDR Track.

Proceedings Title

Proceedings of the 1998 Workshop

Conference Dates

July 9, 1998

Conference Title

DARPA Broadcast News Transcription and Understanding Workshop

Pub Type

Conferences

Download Paper

Local Download

Keywords

automatic speech recognition, broadcast news, evaluation, Hub-4, information retrieval, spoken document retrieval, TREC

Citation

Garofolo, J. , Voorhees, E. , Stanford, V. and Sparck, K. (1999), TREC-6 1997 Spoken Document Retrieval Track Overview and Results, Proceedings of the 1998 Workshop, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=151373 (Accessed October 4, 2024)