Blind Men and Elephants: Six Approaches to TREC Data
D L. Banks, Paul D. Over, Nien F. Zhang
The paper reviews six recent efforts to better understand performance measurements on information retrieval (IR) within the framework of the Text REtrieval Conferences (TREC): analysis of variance, cluster analyses, rank correlations, beadplots, miltidimensional scaling, and item response analysis. None of this work has yielded any substantial new insights. Prospects that additional work along these lines will yield more interesting results vary but are in general, not promising. Some suggestions are made for paying greater attention to richer descriptions of IR system behavior but within smaller, better controlled settings.
analysis of variance, clustering, information retreival, item response, multidimensional scaling, rank correlations, statistical analysis, TREC
, Over, P.
and Zhang, N.
Blind Men and Elephants: Six Approaches to TREC Data, Information Retrieval, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=151743
(Accessed June 3, 2023)