TREC-6 Interactive Track Report

Published

April 24, 1998

Author(s)

Paul D. Over

Abstract

This report is an introduction to the work of the TREK-6 Interactive Track with its goal of investigating interactive information retrieval by examining the process as well as the results.Twelve interactive information retrieval (IR) systems were run on a shared problem: a question-answer task, 6 statements of information need, and a collection of 210, 158 articles from the Financial Times of London 1991-1994. The track specification called for two levels of experimentation: cross-site system comparisons in terms of simple measures of end results and local experiments with their own hypotheses and attention to the search process. This report summarizes the cross-site experiment. It refers the reader to separate discussions of the experiments performed at each participating site their hypotheses, experimental systems, and results. The cross-site experiment can be seen as a case study in the application of experimental design principles and the use of a shared control IR system in addressing the problems of comparing experimental interactive IR systems across sites: isolating the effects of topics, human searches, and other site-specific factors within an affordable design. The cross-site results confirm the dominance of the topic effect, show the searcher effect is almost as often absent as present, and indicate that for several sites the 2-factor interactions are negligible. An analysis of variance found the system effect to be significant, but a multiple comparison test found no significant pairwise differences.

Citation

TREC-6 Interactive Track Report

Pub Type

Others

Download Paper

Local Download

Keywords

experimental design, information retrieval, TREC

Citation

Over, P. (1998), TREC-6 Interactive Track Report, TREC-6 Interactive Track Report, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=151357 (Accessed May 4, 2024)