Hybridisation of classifiers for anomaly detection in big data

Abstract

Recently, the widespread use of cloud technologies has led to the rapid increase in the scale and complexity of this infrastructure. The degradation and downtimes in the performance metrics of these large-scale systems are considered to be a major problem. The key issue in addressing these problems is to detect anomalies that can occur in hardware, software and state of the systems of cloud infrastructure. In this paper, for the detection of anomalies in performance metrics of cloud infrastructure, a semi-supervised classification method based on an ensemble of classifiers is proposed. In the proposed method, to build ensemble Naive Bayes, J48, SMO, multilayer perceptron, IBK and PART algorithms are used. To detect anomalous behaviour on the performance metrics the public data of the Google and Yahoo! companies, Python 2.7, MATLAB, Weka and Google Cloud SDK Shell applications are used. In the result of the experimental study of the model, 90% detection accuracy is obtained.