Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Overview of the TREC 2005 Robust Retrieval Track

Published

Author(s)

Ellen M. Voorhees

Abstract

The robust retrieval track explores methods for improving the consistency of retrieval technology by focusing on poorly performing topics. The retrieval task in the track is a traditional ad~hoc retrieval task where the evaluation methodology emphasizes a system's least effective topics. The 2005 edition of the track used 50 topics that had been demonstrated to be difficult on one document collection, and ran those topics on a different document collection. Relevance information from the first collection could be exploited in producing a query for the second collection, if desired. The main measure for evaluating system effectiveness is "gmap", a variant of the traditional MAP measure that uses a geometric mean rather than an arithmetic mean to average individual topic results. As in previous years, the most effective retrieval strategy was to expand queries using terms derived from additional corpora. The relative difficulty of topics differed across the two document sets.
Conference Title
Text Retrieval Conference (TREC)

Keywords

evaluation, search engines, TREC

Citation

Voorhees, E. (2006), Overview of the TREC 2005 Robust Retrieval Track, Text Retrieval Conference (TREC), [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=150643 (Accessed March 29, 2024)
Created October 16, 2006, Updated February 17, 2017