NIST Special Database 23

NIST TREC Document Database: Disk 5 has been Discontinued.

Price: $90.00 for disk 5 PLACE ORDER

NIST TREC Document Databases are distributed for the development and testing of information retrieval (IR) systems and related natural language processing research. The document collections consist of the full text of various newspaper and newswire articles plus government proceedings. The documents have been used to develop a series of large IR test collections known as the TREC collections (https://trec.nist.gov/). An IR test collection consists of three parts: a set of documents, a set of questions (called topics in TREC) that can be answered by some of the documents, and the right answers (called relevance judgments) that list the documents that are relevant to each question. The topics and relevance judgments for the TREC collections are available separately at (https://trec.nist.gov/data.html). Three other disks of documents, known as the "Tipster" disks were used in early TREC workshops. The TIPSTER disks may be purchased from the Linguistic Data Consortium (LDC) (< www.ldc.upenn.edu).

The format of the documents on the TREC disks use a labeled bracketing expressed in the style of SGML (Standard Generalized Markup Language). SGML DTD's are included on each disk. The different datasets on the disks have identical major structures but have different minor structures. Every document is bracketed by <DOC></DOC> tags and has a unique document identifier, bracketed by <DOCNO></DOCNO> tags. The datasets have all been compressed using the UNIX compress utility and are stored in chunks of about 1 megabyte each (uncompressed size).

The contents of the disks are as follows:

TREC Disk 5:

Data provided from the Foreign Broadcast Information Service

approx. 130,000 documents

approx. 470 MB

Los Angeles Times (randomly selected articles from 1989 & 1990)

approx. 130,000 documents

approx. 475 MB

Some of the data on the disks is copyrighted by the data providers. The data providers have granted permission to use the data for research purposes only.

A signed hard copy of the data use permission organizational form or individual form must be received by NIST before a disk can be shipped.

SYSTEM REQUIREMENTS: CD-ROM drive with software to read ISO-9660 format. Software to uncompress files (e.g., UNIX "uncompress", gunzip).

Further questions, please contact:
Standard Reference Data Program
National Institute of Standards and Technology 
100 Bureau Drive, Stop 6410
Gaithersburg, MD 20899
(844) 975-2200 (Toll Free)

Scientific Contact:
Ellen Voorhees
National Institute of Standards and Technology
Gaithersburg, MD 20899
Ph: (301) 975-3761
ellen.voorhees [at] nist.gov (ellen[dot]voorhees[at]nist[dot]gov)

Keywords: Information retrieval, text processing, test collections.