Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

TREC-COVID: Rationale and Structure of an Information Retrieval Shared Task for COVID-19

Published

Author(s)

Ellen M. Voorhees, Ian Soboroff, Tasmeer Alam, Kirk Roberts, William Hersh, Dina Demner-Fushman, Steven Bedrick, Kyle Lo, Lucy L. Wang

Abstract

TREC-COVID is an information retrieval (IR) shared task initiated to support clinicians and clinical research during the COVID-19 pandemic. IR for pandemics breaks many normal assumptions, which can be seen by examining nine important basic IR research questions related to pandemic situations. TREC-COVID differs from traditional IR shared task evaluations with special considerations for the expected users, IR modality considerations, topic development, participant requirements, assessment process, relevance criteria, evaluation metrics, iteration process, projected timeline, and the implications of data use as a post-task test collection. We describe how all these were addressed for the particular requirements of developing IR systems under a pandemic situation. Finally, initial participation numbers are also provided, which demonstrate the tremendous interest the IR community has in this effort.
Citation
Journal of the American Medical Informatics Association
Volume
27
Issue
9

Keywords

COVID, information retrieval, test collections, TREC

Citation

Voorhees, E. , Soboroff, I. , Alam, T. , Roberts, K. , Hersh, W. , Demner-Fushman, D. , Bedrick, S. , Lo, K. and Wang, L. (2020), TREC-COVID: Rationale and Structure of an Information Retrieval Shared Task for COVID-19, Journal of the American Medical Informatics Association, [online], https://doi.org/10.1093/jamia/ocaa091 (Accessed July 14, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created July 8, 2020, Updated February 14, 2023