Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Ellen M. Voorhees (Assoc)

NIST Fellow

Ellen Voorhees is a Fellow at the US National Institute of Standards and Technology (NIST).  For most of her tenure at NIST she managed the Text REtrieval Conference (TREC) project, a project that develops the infrastructure required for large-scale evaluation of search engines and other information access technology.  Currently she is examining how best to bring the benefits of large-scale community evaluations to bear on the problems of trustworthy AI. Voorhees' general research focuses on developing and validating appropriate evaluation schemes to measure system effectiveness for diverse user tasks.

Voorhees received a B.Sc. in computer science from the Pennsylvania State University, and M.Sc. and Ph.D. degrees in computer science from Cornell University.  Prior to joining NIST she was a Senior Member of Technical Staff at Siemens Corporate Research in Princeton, NJ where her work on intelligent agents applied to information access resulted in three patents.

Voorhees is a fellow of the ACM, a member of the Association for the Advancement of Artificial Intelligence (AAAI) and the Association for Computational Linguistics (ACL), and has been elected as a fellow of the Washington Academy of Sciences. She has published numerous articles on information retrieval techniques and evaluation methodologies and serves on the review boards of several journals and conferences.

Awards

 

  • Fellow of the ACM
  • Inaugural member of the ACM SIGIR Academy
  • Fellow of the Washington Academy of Science
  • U.S. Department of Commerce Gold Medal Award, 2021
  • U.S. Department of Commerce Bronze Medal Award, 2006
  • Siemens Outstanding Achievement Award, 1990

Publications

Human Preferences as dueling Bandits

Author(s)
Xinyi Yan, Chengxi Luo, Charles Clarke, Nick Craswell, Ellen M. Voorhees, Pablo Castells
The dramatic improvements in core information retrieval tasks engendered by neural rankers create a need for novel evaluation methods. If every ranker returns

Too many Relevants: Whither Cranfield Test Collections?

Author(s)
Ellen M. Voorhees, Nick Craswell, Jimmy Lin
This paper presents the lessons regarding the construction and use of large Cranfield-style test collections learned from the TREC 2021 Deep Learning track. The

Searching for Answers in a Pandemic: An Overview of TREC-COVID

Author(s)
Ellen M. Voorhees, Ian Soboroff, Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Lucy L. Wang, William Hersh
We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19

On the Quality of the TREC_COVID IR Test Collections

Author(s)
Ellen M. Voorhees, Kirk Roberts
Shared text collections continue to be vital infrastructure for IR research. The COVID-19 pandemic offered an opportunity to create a test collection that
Created October 9, 2019, Updated March 30, 2023