One of the primary motivations for TREC was to standardize retrieval system evaluation. Prior to TREC, there was little explicit discussion of what constituted a minimally acceptable experimental design, and no hard evidence to support any position. TREC has succeeded in standardizing ad hoc retrieval evaluation, has validated the reliability of experiments based on test collections, and has empirically determined bounds on the sensitivity of test collection comparisons. A focus on evaluation in tracks where the result is not a ranked list of documents has extended the paradigm to new tasks.
Retrieval System Evaluation
TREC Chapter to be published: TREC: Experiment and Evaluation in Information Retrieval, 2005,
evaluation, information retrieval, TREC
and Voorhees, E.
Retrieval System Evaluation, TREC Chapter to be published: TREC: Experiment and Evaluation in Information Retrieval, 2005,
(Accessed June 8, 2023)