Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Understanding TREC Results - the Role of Statistics



Walter S. Liggett Jr, Paul D. Over


The challenge in empirical development of information retrieval systems at TREC is obtaining general conclusions from IR system responses to a sample of perhaps 50 topics, that is, 50 statements of information need. Treating such a sample as a simple random sample and using a univariate performance measure to describe each search result does not provide sufficient sensitivity for many purposes. In this paper, we contend that greater sensitivity can be gained through study of the natural language basis for the dependence of system performance on topic.
Proceedings Title
Proceedings of the 53rd Session of the International Statistical Institute
Conference Dates
August 22-29, 2001
Conference Location
Seoul, KO
Conference Title
International Statistical Institute


information retrieval, multidimensional scaling, query formation, rank correlation, search engines, system evaluation


Liggett, W. and Over, P. (2001), Understanding TREC Results - the Role of Statistics, Proceedings of the 53rd Session of the International Statistical Institute, Seoul, KO (Accessed July 21, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created August 1, 2001, Updated February 17, 2017