A Novel Change-Point Detection Approach for Monitoring High-Dimensional Traffics in Distributed Systems

Abstract

Change-point detection is the problem of finding abrupt changes in time-series. However, the meaningful changes are usually difficult to identify from the original massive traffics, due to high dimension and strong periodicity. In this paper, we propose a novel change-point detection approach, which simultaneously detects change points from all dimensions of the traffics with three steps. We first reduce the dimensions by the classical Principal Component Analysis (PCA), then we apply an extended time-series segmentation method to detect the nontrivial change times, finally we identify the responsible applications for the changes by F-test. We demonstrate through experiments on datasets collected from four distributed systems with 44 applications that the proposed approach can effectively detect the nontrivial change points from the multivariate and periodical traffics. Our approach is more appropriate for mining the nontrivial changes in traffic data comparing with other clustering methods, such as center-based Kmeans and density-based DBSCAN.