Change-point Detection Problem Research Articles

A self-supervised contrastive change point detection method for industrial time series

Manufacturing process monitoring is crucial to ensure production quality. This paper formulates the detection problem of abnormal changes in the manufacturing process as the change point detection (CPD) problem for the industrial temporal data. The premise of known data property and sufficient data annotations in existing CPD methods limits their application in the complex manufacturing process. Therefore, a self-supervised and non-parametric CPD method based on temporal trend-seasonal feature decomposition and contrastive learning (CoCPD) is proposed. CoCPD aims to solve CPD problem in an online manner. By bringing the representations of time series segments with similar properties in the feature space closer, our model can sensitively distinguish the change points that do not conform to either historical data distribution or temporal continuity. The proposed CoCPD is validated by a real-world body-in-white production case and compared with 10 state-of-the-art CPD methods. Overall, CoCPD achieves promising results by Precision 70.6%, Recall 68.8%, and the mean absolute error (MAE) 8.27. With the ability to rival the best offline baselines, CoCPD outperforms online baseline methods with improvements in Precision, Recall and MAE by 14.90%, 11.93% and 43.93%, respectively. Experiment results demonstrate that CoCPD can detect abnormal changes timely and accurately.

Detecting Change Intervals with Isolation Distributional Kernel

Detecting abrupt changes in data distribution is one of the most significant tasks in streaming data analysis. Although many unsupervised Change-Point Detection (CPD) methods have been proposed recently to identify those changes, they still suffer from missing subtle changes, poor scalability, or/and sensitivity to outliers. To meet these challenges, we are the first to generalise the CPD problem as a special case of the Change-Interval Detection (CID) problem. Then we propose a CID method, named iCID, based on a recent Isolation Distributional Kernel (IDK). iCID identifies the change interval if there is a high dissimilarity score between two non-homogeneous temporal adjacent intervals. The data-dependent property and finite feature map of IDK enabled iCID to efficiently identify various types of change-points in data streams with the tolerance of outliers. Moreover, the proposed online and offline versions of iCID have the ability to optimise key parameter settings. The effectiveness and efficiency of iCID have been systematically verified on both synthetic and real-world datasets.

Interpretation of a change point detection problem via adaptive lasso penalty and its application

Interpretation of a change point detection problem via adaptive lasso penalty and its application

Change point detection of multimode processes considering both mode transitions and parameter changes

Multimode processes are common in modern industry and refer to processes that work in multiple operating modes. Motivated by the torque control process of a wind turbine, we determine that there exist two types of changes in multimode processes: (i) mode transitions and (ii) parameter changes. Detecting both types of changes is an important issue in practice, but existing methods mainly consider one type of change, and thus, do not work well. To address this issue, we propose a novel modeling framework for the offline change point detection problem of multimode processes, which simultaneously considers mode transitions and parameter changes. We characterize each mode with a parametric cost function and formulate the problem as an optimization model. In the model, two penalty terms penalize the number of change points, and a series of constraints specify the multimode characteristics. With certain assumptions, the asymptotic property ensures the accuracy of the model solution. To solve the model, we propose an iterative algorithm and develop a multimode-pruned exact linear time (multi-PELT) method for initialization. The simulation study and the real case study demonstrate the effectiveness of our method against the state-of-the-art methods in terms of the accuracy of change point detection, mode identification, and parameter estimation.

Frequency Detection and Change Point Estimation for Time Series of Complex Oscillation

We consider detecting the evolutionary oscillatory pattern of a signal when it is contaminated by non-stationary noises with complexly time-varying data generating mechanism. A high-dimensional dense progressive periodogram test is proposed to accurately detect all oscillatory frequencies. A further phase-adjusted local change point detection algorithm is applied in the frequency domain to detect the locations at which the oscillatory pattern changes. Our method is shown to be able to detect all oscillatory frequencies and the corresponding change points within an accurate range with a prescribed probability asymptotically. A Gaussian approximation scheme and an overlapping-block multiplier bootstrap methodology for sums of complex-valued high dimensional non-stationary time series without variance lower bounds are established, which could be of independent interest. This study is motivated by oscillatory frequency estimation and change point detection problems encountered in physiological time series analysis. An application to spindle detection and estimation in electroencephalogram recorded during sleep is used to illustrate the usefulness of the proposed methodology.

Bayesian Change Point Detection with Spike-and-Slab Priors

We study the use of spike-and-slab priors for consistent estimation of the number of change points and their locations. Leveraging recent results in the variable selection literature, we show that an estimator based on spike-and-slab priors achieves optimal localization rate in the multiple offline change point detection problem. Based on this estimator, we propose a Bayesian change point detection method, which is one of the fastest Bayesian methodologies. We demonstrate through empirical work the good performance of our approach vis-a-vis some state-of-the-art benchmarks. Interestingly, despite having a Gaussian noise assumption, our approach is more robust to misspecification of the error terms than the competing methods in numerical experiments. Supplementary materials for this article are available online.

ON THE COMPARATIVE EFFICIENCY OF CHANGE POINT DETECTION IN MULTIVARIATE TECHNOLOGICAL PROCESSES USING MULTIDIMENSIONAL DOUBLE CONTROL CHARTS

The problem of change point detection in multiparametric technological processes having a normal distribution and consisting in a shift from a given value of the sample mean and sample variance is investigated. Various types of control charts are considered, which make it possible to effectively detect simultaneous changes in the mean value and variance in multiparametric technological processes. By the method of statistical modeling, an analysis of the comparative effectiveness of control charts is carried out, practical recommendations are given.

Constructing outlier-free histograms with variable bin-width based on distance minimization

We propose a new method of constructing a variable bin width histogram that can accommodate the unbalanced distribution of the samples yet retaining, as a whole, the good aspect of both equal width (EW) and equal-area (EA) histograms that are being used popularly for data visualization and analysis. We formulate this as an optimal change point detection problem in which the bin boundaries are determined by minimizing the sum of the absolute error or the squared error in each bin. The former is based on Distance Minimization (DM) and new, and the latter is based on Variance Minimization (VM) and is considered the state-of-the-art. The constructed histograms can effectively be used to detect and visualize hidden outliers/anomalies by applying the interquartile range method in each bin. The final histograms are obtained by adjusting bin boundaries and heights accordingly after removing the detected outliers/anomalies. We further propose a method to annotate the constructed bins if the data for annotation is given for each sample as a set of nominal variables, using z-score with respect to their distribution within each bin. We applied our method to both real vinyl greenhouse datasets and two different sets of three synthetic datasets, and confirmed that both DM and VM methods work as intended, both can represent the sample distribution with a smaller number of bins than those by EW and EA methods, The use of interquartile range method can detect anomalies as well as outliers, and the terms selected for annotation are interpretable and reasonable. EW and EA methods have contrasting properties. DM and VM methods lie in between, but the former is closer to EA method and the latter to EW method. DM method runs substantially faster than VM method and performs slightly better than VM method in outlier detection and annotation tasks.

A Novel Combination Method of a Convolutional Neural Network and Energy Operators for the Detection of Change-Points in Electromyographic Signals

Currently, neural network algorithms based on time-domain features are used for change-point detection problems, and they have proven to be effective. However, due to the instability of human biosignals, establishing a training dataset with labels is difficult. For supervised learning methods, wherein parameters are updated on a small sample set through a feed-forward mechanism, it is difficult to ascertain the degree to which the performance of the trained neural network corresponds to the overfitting of the dataset upon which the network was trained. To this end, this paper attempted to directly replace the parameters in the convolutional neural network that need to be updated by training. A method based on the combination of the Teager–Kaiser energy operator (TKEO) and the convolutional neural network is proposed. We tested the proposed method on simulated EMG data with different signal-to-noise ratios and real data with labels, respectively. Compared with multiple detection methods, the proposed method had significant advantages in terms of reliability, accuracy, and computational speed. Furthermore, the proposed method does not require any prior knowledge about the signal, lending itself to be flexible and adaptable to any application. It may be a promising alternative to solving change-point detection problems.

Forecasting and change point test for nonlinear heteroscedastic time series based on support vector regression.

SVR-ARMA-GARCH models provide flexible model fitting and good predictive powers for nonlinear heteroscedastic time series datasets. In this study, we explore the change point detection problem in the SVR-ARMA-GARCH model using the residual-based CUSUM test. For this task, we propose an alternating recursive estimation (ARE) method to improve the estimation accuracy of residuals. Moreover, we suggest using a new testing method with a time-varying control limit that significantly improves the detection power of the CUSUM test. Our numerical analysis exhibits the merits of the proposed methods in SVR-ARMA-GARCH models. A real data example is also conducted using BDI data for illustration, which also confirms the validity of our methods.

Rough-Fuzzy CPD: a gradual change point detection algorithm

Changepoint detection is the problem of finding abrupt or gradual changes in time series data when the distribution of the time series changes significantly. There are many sophisticated statistical algorithms for solving changepoint detection problem, although there is not much work devoted towards gradual changepoints as compared to abrupt ones. Here we present a new approach to solve the changepoint detection problem using the fuzzy rough set theory which is able to detect such gradual changepoints. An expression for the rough-fuzzy estimate of changepoints is derived along with its mathematical properties concerning fast computation. In a statistical hypothesis testing framework, the asymptotic distribution of the proposed statistic on both single and multiple changepoints is derived under the null hypothesis enabling multiple changepoint detection. Extensive simulation studies have been performed to investigate how simple crude statistical measures of disparity can be subjected to improve their efficiency in the estimation of gradual changepoints. Also, the said rough-fuzzy estimate is robust to signal-to-noise ratio, a high degree of fuzziness in true changepoints, and also to hyperparameter values. Simulation studies reveal that the proposed method beats other methods of gradual changepoint detection (including MJPD, HSMUCE, fuzzy methods like FCP, FCMLCP etc) and also popular crisp methods like Binary Segmentation, PELT, and BOCD in detecting gradual changepoints. The applicability of the estimate is demonstrated using multiple real-life datasets including Covid-19. We have developed the python package roufcp for broader dissemination of the methods.

Multi-Source Change-Point Detection over Local Observation Models

In this work, we address the problem of change-point detection (CPD) on high-dimensional, multi-source, and heterogeneous sequential data with missing values. We present a new CPD methodology based on local latent variable models and adaptive factorizations that enhances the fusion of multi-source observations with different statistical data-type and face the problem of high dimensionality. Our motivation comes from behavioral change detection in healthcare measured by smartphone monitored data and Electronic Health Records. Due to the high dimension of the observations and the differences in the relevance of each source information, other works fail in obtaining reliable estimates of the change-points location. This leads to methods that are not sensitive enough when dealing with interspersed changes of different intensity within the same sequence or partial missing components. Through the definition of local observation models (LOMs), we transfer the local CP information to homogeneous latent spaces and propose several factorizations that weight the contribution of each source to the global CPD. With the presented methods we demonstrate a reduction in both the detection delay and the number of not-detected CPs, together with robustness against the presence of missing values on a synthetic dataset. We illustrate its application on real-world data from a smartphone-based monitored study and add explainability on the degree of each source contributing to the detection.

Seeded binary segmentation: a general methodology for fast and optimal changepoint detection

Summary We propose seeded binary segmentation for large-scale changepoint detection problems. We construct a deterministic set of background intervals, called seeded intervals, in which single changepoint candidates are searched for. The final selection of changepoints based on these candidates can be done in various ways, adapted to the problem at hand. The method is thus easy to adapt to many changepoint problems, ranging from univariate to high dimensional. Compared to recently popular random background intervals, seeded intervals lead to reproducibility and much faster computations. For the univariate Gaussian change in mean set-up, the methodology is shown to be asymptotically minimax optimal when paired with appropriate selection criteria. We demonstrate near-linear runtimes and competitive finite sample estimation performance. Furthermore, we illustrate the versatility of our method in high-dimensional settings.

Multivariate change point detection for heterogeneous series

The multivariate change point detection problem has been encountered across various fields. Most approaches to this problem assume the series is homogeneous, i.e., all the coordinates change concurrently. Hence, the specific subset of the coordinates containing the change points cannot be determined. In this work, we propose S-MCPD, which is capable of detecting the position of multivariate change points for heterogeneous series by identifying specific coordinates of those changed. Specifically, the problem is discussed in the context of variable selection and transformed into the form of sparse group lasso. In simulation studies, we compared S-MCPD with four existing methods, inspect, sbs, dc, and cpm. The results showed that the performance of S-MCPD was comparable to that of inspect and was superior to other methods in terms of evaluation metrics. In addition, S-MCPD can determine not only the positions of change points, but also the subset of coordinates containing the change points, while other existing methods are unable to achieve this. Moreover, S-MCPD does not depend on the constant variance assumption and works quite well even when the covariance changes, which makes our method more practical. These are the two important contributions of our work. Finally, we applied S-MCPD to two real-world datasets to show its effectiveness.

Distribution-free changepoint detection tests based on the breaking of records

The analysis of record-breaking events is of interest in fields such as climatology, hydrology or anthropology. In connection with the record occurrence, we propose three distribution-free statistics for the changepoint detection problem. They are CUSUM-type statistics based on the upper and/or lower record indicators observed in a series. Using a version of the functional central limit theorem, we show that the CUSUM-type statistics are asymptotically Kolmogorov distributed. The main results under the null hypothesis are based on series of independent and identically distributed random variables, but a statistic to deal with series with seasonal component and serial correlation is also proposed. A Monte Carlo study of size, power and changepoint estimate has been performed. Finally, the methods are illustrated by analyzing the time series of temperatures at Madrid, Spain. The R package RecordTest publicly available on CRAN implements the proposed methods.

Quality Change: Norm or Exception? Measurement, Analysis and Detection of Quality Change in Wikipedia

Wikipedia has been turned into an immensely popular crowd-sourced encyclopedia for information dissemination on numerous versatile topics in the form of subscription free content. It allows anyone to contribute so that the articles remain comprehensive and updated. For enrichment of content without compromising standards, the Wikipedia community enumerates a detailed set of guidelines, which should be followed. Based on these, articles are categorized into several quality classes by the Wikipedia editors with increasing adherence to guidelines. This quality assessment task by editors is laborious as well as demands platform expertise. As a first objective, in this paper, we study evolution of a Wikipedia article with respect to such quality scales. Our results show novel non-intuitive patterns emerging from this exploration. As a second objective we attempt to develop an automated data driven approach for the detection of the early signals influencing the quality change of articles. We posit this as a change point detection problem whereby we represent an article as a time series of consecutive revisions and encode every revision by a set of intuitive features. Finally, various change point detection algorithms are used to efficiently and accurately detect the future change points. We also perform various ablation studies to understand which group of features are most effective in identifying the change points. To the best of our knowledge, this is the first work that rigorously explores English Wikipedia article quality life cycle from the perspective of quality indicators and provides a novel unsupervised page level approach to detect quality switch, which can help in automatic content monitoring in Wikipedia thus contributing significantly to the CSCW community.

Optimal Nonparametric Multivariate Change Point Detection and Localization

We study the multivariate nonparametric change point detection problem, where the data are a sequence of independent <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -dimensional random vectors whose distributions are piecewise-constant with Lipschitz densities changing at unknown times, called change points. We quantify the size of the distributional change at any change point with the supremum norm of the difference between the corresponding densities. We are concerned with the localization task of estimating the positions of the change points. In our analysis, we allow for the model parameters to vary with the total number of time points, including the minimal spacing between consecutive change points and the magnitude of the smallest distributional change. We provide information-theoretic lower bounds on both the localization rate and the minimal signal-to-noise ratio required to guarantee consistent localization. We formulate a novel algorithm based on kernel density estimation that nearly achieves the minimax lower bound, save possibly for logarithm factors. We have provided extensive numerical evidence to support our theoretical findings.

Linear Time Dynamic Programming for Computing Breakpoints in the Regularization Path of Models Selected From a Finite Set

Many learning algorithms are formulated in terms of finding model parameters which minimize a data-fitting loss function plus a regularizer. When the regularizer involves the pseudo-norm, the resulting regularization path consists of a finite set of models. The fastest existing algorithm for computing the breakpoints in the regularization path is quadratic in the number of models, so it scales poorly to high-dimensional problems. We provide new formal proofs that a dynamic programming algorithm can be used to compute the breakpoints in linear time. Our empirical results include analysis of the proposed algorithm in the context of various learning problems (regression, changepoint detection, clustering, and matrix factorization). We use a detailed analysis of changepoint detection problems to demonstrate the improved accuracy and speed of our approach relative to grid search and a previous quadratic time algorithm.

A robust bootstrap change point test for high-dimensional location parameter

We consider the problem of change point detection for high-dimensional distributions in a location family when the dimension can be much larger than the sample size. In change point analysis, the widely used cumulative sum (CUSUM) statistics are sensitive to outliers and heavy-tailed distributions. In this paper, we propose a robust, tuning-free (i.e., fully data-dependent), and easy-to-implement change point test that enjoys strong theoretical guarantees. To achieve the robust purpose in a nonparametric setting, we formulate the change point detection in the multivariate U-statistics framework with anti-symmetric and nonlinear kernels. Specifically, the within-sample noise is canceled out by anti-symmetry of the kernel, while the signal distortion under certain nonlinear kernels can be controlled such that the between-sample change point signal is magnitude preserving. A (half) jackknife multiplier bootstrap (JMB) tailored to the change point detection setting is proposed to calibrate the distribution of our ℓ∞-norm aggregated test statistic. Subject to mild moment conditions on kernels, we derive the uniform rates of convergence for the JMB to approximate the sampling distribution of the test statistic, and analyze its size and power properties. Extensions to multiple change point testing and estimation are discussed with illustration from numerical studies.

Spatial rank-based high-dimensional change point detection via random integration

Detecting change points is an important task to identify an abrupt and significant change in the data generating process. Traditional change point detection methods are not applicable in high-dimensional situations due to many obstacles, such as the requirement of normality or the estimation of the covariance matrix. This paper presented a novel nonparametric method to overcome such issues by using the random integration with spatial ranks, which is tailored to high-dimensional change point detection problems. The proposed method is a unified framework that includes and extends many existing methods and can effectively handle high-dimensional non-normal data, whose asymptotic properties are established under mild conditions. In addition, we developed a computationally efficient algorithm to calculate the rejection thresholds and an effective post-signal diagnostic procedure to identify the potential directions. Finally, numerical studies together with real data examples demonstrated that the proposed method can identify the change point efficiently.