Fitting Nature's Basic Functions Part IV: The Variable Projection Algorithm

Published

April 1, 2003

Author(s)

Bert W. Rust

Abstract

A recurring theme in Internet network traffic analysis is that most observed data show characteristics of a long (heavy) right-tail distribution taking on nonnegative values. Thus, extreme value distribution such as generalized Pareto distribution (GPD) provides a natural setup for modeling the tail behavior of network data. On the other hand, due to the enormous diversity in network traffic at different time scales, at different nodes, different sources or protocols, it is necessary to introduce a more flexible framework using finite mixtures. We believe that the generalized Pareto mixture distribution (GPMD) is such a general model and it should form a theoretical basis for any anomaly detection algorithms for extreme events detection. We will demonstrate these points through applying to the popular RTT pingER data. For example, in terms of pingER RTT performance, users might have felt more frequent network slowdowns even if the median performance improved after a NIST network upgrade. Potential extensions to include temporal dependence and continuous time process are also discussed. Our GPMD methodology as a powerful tool for detecting perturbations at extreme events may prove useful in other areas of network security

Citation

Computing in Science & Engineering

Volume

5 No. 2

Pub Type

Journals

Download Paper

Local Download

Keywords

fossil fuel emissions, global temperature variations, nonlinear least squares, variable projection algorithm

Citation

Rust, B. (2003), Fitting Nature's Basic Functions Part IV: The Variable Projection Algorithm, Computing in Science & Engineering, [online], https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=150866 (Accessed July 26, 2024)