Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Searching for Answers in a Pandemic: An Overview of TREC-COVID

Published

Author(s)

Ellen M. Voorhees, Ian Soboroff, Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Lucy L. Wang, William Hersh

Abstract

We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July, 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging topics at that state of the still-emerging pandemic. This paper provides a comprehensive overview of the structure and results of TREC-COVID. Specifically, the paper provides details on the background, task structure, topic structure, corpus, participation, pooling, assessment, judgments, results, top-performing systems, lessons learned, and benchmark datasets.
Citation
Journal of Biomedical Informatics
Volume
121

Keywords

COVID, information retrieval, test collections, TREC

Citation

Voorhees, E. , Soboroff, I. , Roberts, K. , Alam, T. , Bedrick, S. , Demner-Fushman, D. , Lo, K. , Wang, L. and Hersh, W. (2021), Searching for Answers in a Pandemic: An Overview of TREC-COVID, Journal of Biomedical Informatics, [online], https://doi.org/10.1016/j.jbi.2021.103865 (Accessed February 23, 2024)
Created September 1, 2021, Updated March 15, 2023