Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Overview of the TREC-9 Question Answering Track

Published

Author(s)

Ellen M. Voorhees

Abstract

The TREC question answering track is an effort to bring the benefits of large-scale evaluation to bear on the question answering problem. The track has run twice so far, where the goal both times was to retrieve small snippets of text that contain the actual answer to a question rather than the document lists traditionally returned by text retrieval systems. The best performing system in TREC-9, the Falcon system from Southern Methodist University, was able to answer about 65 % of the questions (compared to approximately 42 % of the questions for the next best systems) by combining abductive reasoning with various natural language processing techniques. The 65 % score is slightly less than the best scores for TREC-8 in absolute terms, but it represents a very significant improvement in question answering systems. The TREC-9 task was considerably harder than the TREC-8 task because the TREC-9 track used actual users' questions rather than questions constructed specifically for the track.
Citation
Overview of the TREC-9 Question Answering Track

Keywords

question answering, TREC

Citation

Voorhees, E. (2001), Overview of the TREC-9 Question Answering Track, Overview of the TREC-9 Question Answering Track, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=151529 (Accessed March 28, 2024)
Created September 3, 2001, Updated February 17, 2017