Soboroff, I.
(2010),
Test Collection Diagnosis and Treatment, Proceedings of the 2010 Workshop on Evaluating Information Access (EVIA 2010), Tokyo, -1, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=905766 (Accessed May 2, 2026)
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Test Collection Diagnosis and Treatment
Published
Author(s)
Abstract
Test collections are a mainstay of information retrieval research. Since the 1990s, large reusable test collections have been developed in the context of community evaluations such as TREC, NTCIR, CLEF, and INEX. Recently, advances in pooling practice as well as crowdsourcing technologies have placed test collection building back into the hands of the small research group or company. In all of these cases, practitioners should be aware of, and concerned about the quality of test collections. This paper surveys work in test collection quality measures, references case studies to illustrate their use, and provides guidelines on assessing the quality of test collections in practice.Proceedings Title
Proceedings of the 2010 Workshop on Evaluating Information Access (EVIA 2010)
Conference Dates
June 15-18, 2010
Conference Location
Tokyo
Pub Type
Conferences
Download Paper
Keywords
test collections, information retrieval
Citation
Issues
If you have any questions about this publication or are having problems accessing it, please contact [email protected].
Created June 15, 2010, Updated February 19, 2017