Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Micro-Signatures: The Effectiveness of Known Bad N-Grams for Network Anomaly Detection

Published

Author(s)

Richard Harang, Peter M. Mell

Abstract

The field of intrusion detection is divided into signature detection and anomaly detection. The former involves identifying patterns associated with known attacks and the latter involves attempting to learn a 'normal' pattern of activity and then producing security alerts when behaviors outside of those norms is detected. The n-grams methodology has arguably been the most successful technique for anomaly detection (including for network packet inspection). In this work, we identify a new type of intrusion detection that neither uses typical signatures nor is anomaly based (though it is closely related to both). We generate n-grams from both malicious content and Snort signatures and use sets of these 'micro-signatures' to identify attacks. This micro-signature capability arises implicitly when the training sets for n-gram anomaly detection systems are scrubbed of malicious content and thus is not new. It was added explicitly by the seminal Anagram network anomaly approach, but was portrayed as a minor enhancement and its effect was not evaluated. In reproducing the Anagram results we find that for our data, the micro-signatures provide the vast majority of the detection capability. What appears on the surface to be an anomaly detection approach achieves most of its effectiveness from a (sometimes merely implicit) signature subsystem. We furthermore find that these micro- signatures enable highly effective standalone detection systems as well as hybrid micro- signature/anomaly systems that generalize to multiple attack classes. Our results thus shed new light into the functioning of n-gram anomaly detection systems, reveal the need to evaluate the micro-signature contribution within n-gram anomaly research, and open a new avenue of research into how to best use micro-signatures in future detection systems.
Proceedings Title
Foundations and Practice of Security: 9th International Symposium, FPS 2016
Conference Dates
October 24-25, 2016
Conference Location
Quebec City, -1
Conference Title
9th International Symposium on Foundations and Practice of Security

Keywords

intrusion detection, anomaly detection, n-grams, micro-signatures, security
Created February 5, 2017, Updated November 10, 2018