1998 Broadcast News Benchmark Test Results: English and Non-English Word Error Rate Performance Measures

Published

October 20, 1998

Author(s)

D S. Pallett, Jonathan G. Fiscus, John S. Garofolo, Alvin F. Martin, Mark A. Przybocki

Abstract

This paper documents the use of Broadcast News test materials in DARPA-sponsored Automatic Speech Recognition (ASR) Benchmark Tests conducted late in 1998. As in last year s tests, statistical selection procedures were used in selecting test materials. Two test epochs were used, each yielding (nominally) one and one-half hours of test material. One of the test sets was drawn from the same test epoch as was used for last year s tests, and the other was drawn from a more recent period. Results are reported for two types of systems: one (the Hub, or baseline systems) for which there were no limits on computational resources, and another (the less than 10X real-time spoke systems) for systems that ran in less than 10 times real time. The lowest word error rate reported this year for the Hub systems was 13.5 %, contrasting with last year's lowest word error rate of 16.2 %. For the less than 10X real-time spoke systems, the lowest reported word error rate was 16.1 %.Results are also reported, for the second year, on non-English language Broadcast News materials in Spanish and Mandarin.

Citation

DARPA Broadcast News Transcription and Understanding Workshop

Pub Type

Journals

Download Paper

Local Download

Keywords

automatic speech recognition, benchmark tests, broadcast news

Citation

Pallett, D. , Fiscus, J. , Garofolo, J. , Martin, A. and Przybocki, M. (1998), 1998 Broadcast News Benchmark Test Results: English and Non-English Word Error Rate Performance Measures, DARPA Broadcast News Transcription and Understanding Workshop, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=151417 (Accessed October 15, 2024)