Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Comparing Interactive Information Retrieval Systems Across Sites: The TREC-6 Interactive Track Matrix Experiment

Published

Author(s)

E S. Lagergren, Paul D. Over

Abstract

This is a case study in the design and analysis of a 9-site TREC-6 experiment aimed at comparing the performance of 12 interactive information retrieval (IR) systems on a shared problem: a question-answering task, 6 statements of information need, and a collection of 210,158 articles from the Financial Times of London 1991-1994. The study discusses the application of experimental design principles and the use of a shared control IR system in addressing the problems of comparing experimental interactive IR systems across sites: isolating the effects of topics, human searchers, and other site specific factors within an affordable design. The results confirm the dominance of the topic effect, show the searcher effect is almost as often absent as present, and indicate that for several sites the 2-factor interactions are negligible. An analysis of variance found the system effect to be significant pairwise differences.
Conference Dates
August 1, 1998
Conference Location
Melbourne, AS
Conference Title
ACM/SIGIR Conference on Research and Development in Information Retrieval

Keywords

experimental design, information retrieval, user study

Citation

Lagergren, E. and Over, P. (1998), Comparing Interactive Information Retrieval Systems Across Sites: The TREC-6 Interactive Track Matrix Experiment, ACM/SIGIR Conference on Research and Development in Information Retrieval, Melbourne, AS, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=151737 (Accessed April 23, 2024)
Created July 15, 1998, Updated February 19, 2017