A WED Method for Evaluating the Performance of Change-Point Detection Algorithms

Abstract

Change point detection (CPD) is to find the abrupt changes in a time series. Various computational algorithms have been developed for CPD. To compare the different CPD models, many performance metrics have been introduced to evaluate the algorithms. Each of the previous evaluation methods measures the different aspect of the methods. In this paper, a new weighted error distance (WED) method is proposed to evaluate the overall performance of a CPD model across multiple time series of different lengths. A concept of normalized error distance was introduced to allow comparison of the distances between an estimated change point position and the target change point among models that work on multiple time series. In this study, the WED metrics was applied on synthetic datasets with different sample sizes and variances to evaluate the different CPD models, including: Kolmogorov-Smirnov (KS), SSA and T algorithms. The test results showed the value of this WED method that contributes to the methodology for evaluating the performance of CPD models.