Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

System Performance and Natural Language Expression of Information Needs

Published

Author(s)

Walter S. Liggett Jr, C E. Buckley

Abstract

Consider information retrieval systems that respond to a query (a natural language statement of a topic, an information need) with an ordered list of 1000 documents from the document collection. From the responses to queries that all express the same topic, one can discern how the words associated with a topic result in particular system behavior. From what is discerned from different topics, one can hypothesize abstract topic factors that govern system performance. An example of such a factor is the specificity of the topic's primary key word. We illustrate these ideas with system responses for a group of topics from NIST's Text REtrieval Conference (TREC). We begin by showing how one can analyze the queries and system responses for a particular topic, and how one can formulate topic factors through comparison of different topics. We then show how to choose from a group of topics, pairs likely to offer particularly informative comparisons. Finally, we consider prospects for using topic factors in generalizing from observed system responses for a group of topic to conclusions about system performance on topics as yet unseen.
Citation
Journal of Classification
Volume
1 No 1

Keywords

archetypal analysis, indicator, multidimensional scaling, natural language

Citation

Liggett, W. and Buckley, C. (2005), System Performance and Natural Language Expression of Information Needs, Journal of Classification (Accessed March 29, 2024)
Created January 5, 2005, Updated February 17, 2017