Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Blind Men and Elephants: Six Approaches to TREC Data



D L. Banks, Paul D. Over, Nien F. Zhang


The paper reviews six recent efforts to better understand performance measurements on information retrieval (IR) within the framework of the Text REtrieval Conferences (TREC): analysis of variance, cluster analyses, rank correlations, beadplots, miltidimensional scaling, and item response analysis. None of this work has yielded any substantial new insights. Prospects that additional work along these lines will yield more interesting results vary but are in general, not promising. Some suggestions are made for paying greater attention to richer descriptions of IR system behavior but within smaller, better controlled settings.
Information Retrieval


analysis of variance, clustering, information retreival, item response, multidimensional scaling, rank correlations, statistical analysis, TREC


Banks, D. , Over, P. and Zhang, N. (1998), Blind Men and Elephants: Six Approaches to TREC Data, Information Retrieval, [online], (Accessed May 29, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created October 28, 1998, Updated February 17, 2017