Combining statistical and spectral analysis techniques in network traffic anomaly detection

Abstract

Rapid increase in number of computer attacks prompts a need to detect network anomalies quickly and effectively. This area has been widely studied and solutions typically use data not freely available. A labeled available network traffic flow dataset, Kyoto2006+, has been recently created. Most existing works using Kyoto2006+ for network anomaly detection, apply various clustering approaches. Clustering approaches typically require thresholds for minimum size or distance, or the number of clusters. Results could be sensitive to the selection of such thresholds. This paper leverages existing spectral analysis and statistical analysis techniques for network anomaly detection. One well known spectral analysis technique is Haar Wavelet filtering analysis. It measures the amount and magnitude of abrupt changes in data. Another popular approach is a statistical analysis technique called Principal Component Analysis (PCA). PCA describes data in a new dimension to unlock otherwise hidden characteristics. Both approaches have strengths and limitations. In response, this paper proposes a Hybrid PCA-Haar Wavelet Analysis; a modified PCA which incorporates time shifting to account for changes over time is considered. In addition, the hybrid approach uses PCA to describe the data and Haar Wavelet filtering for analysis. Based on prototyping and measurement, an investigation of the Hybrid PCA-Haar Wavelet Analysis technique is performed using the Kyoto2006+ dataset. We present experimental results to demonstrate the accuracy and precision of the hybrid approach as compared to the two algorithms individually. Furthermore, tests to examine the impact of various parameters used in the algorithm are discussed.