A36 Analysis of long term adverse experience data using the Weibull model

Abstract

Machine learning models are widely used for anomaly detection in network traffic. Effective transformation of the raw traffic data into mathematical expressions and hyper-parameter adjustment are two important steps before training the machine learning classifier, which is used to predict whether the unknown traffic is normal or abnormal. In this paper, a novel model SVM-L is proposed for anomaly detection in network traffic. In particular, raw URLs are treated as natural language, and then transformed into mathematical vectors via statistical laws and natural language processing technique. They are used as the training data for the traffic classifier, the kernel Support Vector Machine (SVM). Based on the idea of the dual formulation of kernel SVM and Linear Discriminant Analysis (LDA), we propose an optimization model to adjust the hyper-parameter of the classifier. The corresponding problem is simply one-dimensional, and is easily solved by the golden section method. Numerical tests indicate that the proposed model achieves more than 99% accuracy on all tested datasets, and outperforms the state of the arts in terms of standard evaluation measurements.