Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

An Overview of Issues in Testing Intrusion Detection Systems



Peter M. Mell, R Lippmann, Chung Tong Hu, J Haines, M Zissman


While intrusion detection systems are becoming ubiquitous defenses in today's networks, currently we have no comprehensive and scientifically rigorous methodology to test the effectiveness of these systems. This paper explores the types of performance measurements that are desired and that have been used in the past. We review many past evaluations that have been designed to assess these metrics. We also discuss the hurdles that have blocked successful measurements in this area and present suggestions for research directed toward improving our measurement capabilities.
NIST Interagency/Internal Report (NISTIR) - 7007
Report Number


IDS performance measurement methodology, intrusion detection system (IDS), quantitative testing of IDSs


Mell, P. , Lippmann, R. , , C. , Haines, J. and Zissman, M. (2003), An Overview of Issues in Testing Intrusion Detection Systems, NIST Interagency/Internal Report (NISTIR), National Institute of Standards and Technology, Gaithersburg, MD, [online], (Accessed June 22, 2024)


If you have any questions about this publication or are having problems accessing it, please contact

Created July 11, 2003, Updated November 10, 2018