U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Search Publications by:

Search Title, Abstract, Conference, Citation, Keyword or Author
Displaying 1 - 25 of 64

TREC 2020 News Track Overview

May 21, 2021
Author(s)
Ian Soboroff, Shudong Huang, Donna Harman
The News track focuses on information retrieval in the service of help- ing people read the news. In 2018, in cooperation with the Washington Post1, we released a new collection of nearly 600,000 news articles, and crafted two tasks related to how news is

DUC in Context

November 21, 2007
Author(s)
Paul D. Over, Hoa T. Dang, Donna K. Harman
Recent years have seen increased interest in text summarization with emphasis on evaluation of prototype ststems. Many factors can affect the design of such evaluations, requiring choices among competing alternatives. This paper examines several major

The Importance of Focused Evaluations: A Case Study of TREC and DUC

January 10, 2007
Author(s)
Donna K. Harman
Evaluation has always been an important part of scientific research, and in information retrieval, this evaluation has mostly been done using test collections. In 1992 a new test collection was built at the National Institute of Standards and Technology

The Fifth Text Retrival Conference [TREC-5]

October 30, 2006
Author(s)
Ellen M. Voorhees, Donna K. Harman
This paper is the track report for the TREC-5 confusion track. For TREC-5, retrieval from corrupted data was studied through retrieval of specific target documents from a corpus that was corrupted by applying OCR techniques to page images of varying

TREC: An Overview

February 17, 2006
Author(s)
Donna K. Harman, Ellen M. Voorhees
The Text REtrieval Conference (TREC) is a workshop series designed to build the infrastructure necessary for large-scale evaluation of text retrieval technology. Participants in the workshops (over 100 groups in the latest TREC) have been drawn from the

The History of IDF and its Influences on IR and Other Fields

December 21, 2005
Author(s)
Donna K. Harman
The surprisingly simple IDF measure developed in 1972 by Karen Sparck Jones has continued to dominate the term weighting metrics used in information retrieval, despite several efforts to develop more complex measures of term distribution. It has been

Novelty Detection: The TREC Experience

October 1, 2005
Author(s)
Ian M. Soboroff, Donna K. Harman
A challenge for search systems is to detect not only when an item is relevant to the user's information need, but also when it contains something new which the user has not seen before. In the TREC novelty track, the task was to highlight sentences

Beyond English

April 1, 2005
Author(s)
Donna K. Harman
This chapter summarizes TREC work on retrieval for language other than English. TREC has explored a variety of tasks including both single language tasks (for example, retrieving Chinese documents using Chinese queries) and cross-language tasks (such as

The TREC Ad Hoc Experiments

April 1, 2005
Author(s)
Donna K. Harman
Ad hoc retrieval is the prototypical search engine task: searching a static set of documents with a previouslyunseen query. The ad hoc task was one of the first two tasks tackled in TREC and was run for eight years, representing hundreds of experiments

The TREC Test Collections

April 1, 2005
Author(s)
Donna K. Harman
The creation of a set of large, unbiased test collections has been critical to the success of TREC. This chapteris the documentation for the TREC collections. It reviews the motivation for building the collections, describes the methods used to create them

IT--The Twelfth Text Retrieval Conference, TREC 2003

October 25, 2004
Author(s)
Ellen M. Voorhees, Donna K. Harman
This chapter provides an executive summary of the TREC workshop series and the remainder of thevolume. It explains the motivation for TREC and highlights TREC's accomplishments in improving retrievaleffectiveness and fostering technology transfer.

The Effects of Human Variation in DUC Summarization Evaluation

July 1, 2004
Author(s)
Donna K. Harman, Paul D. Over
There is a long history of research in automatic text summarization systems by both the text retrieval and the natural language processing communities, but evaluation of such systems' output has always presented problems. One critical problem remains how
Was this page helpful?