Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Evaluation Methodologies for Information Management Systems

Published

Author(s)

Emile L. Morse

Abstract

The projects developed under the auspices of the Defense Advanced Research Projects Agency (DARPA) Information Management (IM) program are innovative approaches to tackle the hard problems associated with delivering critical information in a timely fashion to decision makers. To the extent that each of the information management systems interfaces with users, these systems must undergo testing with actual humans. The DARPA IM Evaluation project has developed an evaluation methodology that can assist system developers in assessing the usability and utility of their systems. The key components of an evaluation plan are data, users, tasks and metrics. The evaluation project recruited six IM project Principal Investigators (PI's) and devoted a year's effort in developing a method for getting from exploring and implementing systems to actually planning and performing structured, hypothesis-based evaluations. Five of the projects participated in this effort while a sixth project was integrated into and evaluated with a larger effort. This report describes the component systems, evaluation factors, and our experiences in constructing test plans.
Citation
D-LIB Magazine

Keywords

DARPA, evaluation, experimental design, human subjects, information management systems

Citation

Morse, E. (2002), Evaluation Methodologies for Information Management Systems, D-LIB Magazine, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=51044 (Accessed June 16, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created June 11, 2002, Updated February 17, 2017