Change Point Detection In Time Series Research Articles

Authors’ reply to the Discussion of ‘Automatic change-point detection in time series via deep learning’ at the Discussion Meeting on ‘Probabilistic and statistical aspects of machine learning’

Authors’ reply to the Discussion of ‘Automatic change-point detection in time series via deep learning’ at the Discussion Meeting on ‘Probabilistic and statistical aspects of machine learning’

Automatic change-point detection in time series via deep learning

Abstract Detecting change points in data is challenging because of the range of possible types of change and types of behaviour of data when there is no change. Statistically efficient methods for detecting a change will depend on both of these features, and it can be difficult for a practitioner to develop an appropriate detection method for their application of interest. We show how to automatically generate new offline detection methods based on training a neural network. Our approach is motivated by many existing tests for the presence of a change point being representable by a simple neural network, and thus a neural network trained with sufficient data should have performance at least as good as these methods. We present theory that quantifies the error rate for such an approach, and how it depends on the amount of training data. Empirical results show that, even with limited training data, its performance is competitive with the standard cumulative sum (CUSUM) based classifier for detecting a change in mean when the noise is independent and Gaussian, and can substantially outperform it in the presence of auto-correlated or heavy-tailed noise. Our method also shows strong results in detecting and localizing changes in activity based on accelerometer data.

Research on time series change point detection and influencing factors under machine learning: based on PM2.5 concentration data in Hefei city

Research on time series change point detection and influencing factors under machine learning: based on PM2.5 concentration data in Hefei city

Genetic algorithm with a Bayesian approach for multiple change-point detection in time series of counting exceedances for specific thresholds

Although the applications of Non-Homogeneous Poisson Processes (NHPP) to model and study the threshold overshoots of interest in different time series of measurements have proven to provide good results, they needed to be complemented with an efficient and automatic diagnostic technique to establish the location of the change-points, which, when taken into account, make the estimated model fit poorly in regards of the information contained in the real one. Because of this, a new method is proposed to solve the segmentation uncertainty of the time series of measurements, where the generating distribution of exceedances of a specific threshold is the focus of investigation. One of the great contributions of the present algorithm is that all the days that trespassed are candidates to be a change-point, so all the possible configurations of overflow days under the heuristics of a genetic algorithm are the possible chromosomes, which will unite to produce new solutions. Also, such methods will be guarantee to non-local and the best possible one solution, reducing wasted machine time evaluating the least likely chromosomes to be a feasible solution. The analytical evaluation technique will be by means of the Minimum Description Length (MDL) as the objective function, which is the joint posterior distribution function of the parameters of the NHPP of each regime and the change-points that determines them and which account as well for the influence of the presence of said times. Thus, one of the practical implications of the present work comes in terms of overcoming the need of modeling the time series of measurements, where the distributions of exceedances of certain thresholds, or where the counting of certain events involving abrupt changes, is the main focus with applications in phenomena such as climate change, information security and epidemiology, to name a few.

Bayesian inference with the l1-ball prior: solving combinatorial problems with exact zeros

Abstract The l1-regularisation is very popular in high-dimensional statistics—it changes a combinatorial problem of choosing which subset of the parameter is zero, into a simple continuous optimisation. Using a continuous prior concentrated near zero, the Bayesian counterparts are successful in quantifying the uncertainty in the variable selection problems; nevertheless, the lack of exact zeros makes it difficult for broader problems such as change-point detection and rank selection. Inspired by the duality of the l1-regularisation as a constraint onto an l1-ball, we propose a new prior by projecting a continuous distribution onto the l1-ball. This creates a positive probability on the ball boundary, which contains both continuous elements and exact zeros. Unlike the spike-and-slab prior, this l1-ball projection is continuous and differentiable almost surely, making the posterior estimation amenable to the Hamiltonian Monte Carlo algorithm. We examine the properties, such as the volume change due to the projection, the connection to the combinatorial prior, the minimax concentration rate in the linear problem. We demonstrate the usefulness of exact zeros that simplify the combinatorial problems, such as the change-point detection in time series, the dimension selection of mixture models, and the low-rank plus-sparse change detection in medical images.

The min-Knapsack problem with compactness constraints and applications in statistics

In the min-Knapsack problem, one is given a set of items, each having a certain cost and weight. The objective is to select a subset with minimum cost, such that the sum of the weights is not smaller than a given constant. In this paper, we introduce an extension of the min-Knapsack problem with additional “compactness constraints” (mKPC), stating that selected items cannot lie too far apart. This extension has applications in statistics, including in algorithms for change-point detection in time series. We propose three solution methods for the mKPC. The first two methods use the same Mixed-Integer Programming (MIP) formulation but with two different approaches: passing the complete model with a quadratic number of constraints to a black-box MIP solver or dynamically separating the constraints using a branch-and-cut algorithm. Numerical experiments highlight the advantages of this dynamic separation. The third approach is a dynamic programming labelling algorithm. Finally, we focus on the particular case of the unit-cost mKPC (1c-mKPC), which has a specific interpretation in the context of the statistical applications mentioned above. We prove that the 1c-mKPC is solvable in polynomial time with a different ad-hoc dynamic programming algorithm. Experimental results show that this algorithm vastly outperforms both generic approaches for the mKPC and a simple greedy heuristic from the literature.

CPMAN: Change Point Detection Approach in Time Series Based on the Prediction of Multi-stage Attention Networks

Time series change point detection can identify the locations of abrupt points in many dynamic processes. It can help us to find anomaly data in an early stage. At the same time, detecting change points for long, periodic, and multiple input series data has received a lot of attention recently, and is universally applicable in many fields including power, environment, finance, and medicine. However, the performance of classical methods typically scales poorly for such time series. In this paper, we propose CPMAN, a novel prediction-based change point detection approach via multi-stage attention networks. Our model consists of two key modules. First, in the time series prediction module, we employ the multi-stage attention-based networks and integrate the multi-series fusion mechanism. This module can adaptively extract features from the relevant input series and capture the long-term temporal dependencies. Secondly, in the change point detection module, we use the wavelet analysis-based algorithm to detect change points efficiently and identify the change points and outliers. Extensive experiments are conducted on various real-world datasets and synthetic datasets, proving the superiority and effectiveness of CPMAN. Our approach outperforms the state-of-the-art methods by up to 12.1% on the F1 Score.

Detecting changes in mean in the presence of time‐varying autocovariance

There has been much attention in recent years to the problem of detecting mean changes in a piecewise constant time series. Often, methods assume that the noise can be taken to be independent, identically distributed (IID), which in practice may not be a reasonable assumption. There is comparatively little work studying the problem of mean changepoint detection in time series with nontrivial autocovariance structure. In this article, we propose a likelihood‐based method using wavelets to detect changes in mean in time series that exhibit time‐varying autocovariance. Our proposed technique is shown to work well for time series with a variety of error structures via a simulation study, and we demonstrate its effectiveness on two data examples arising in economics.

Change Point Analysis for Detecting Vaccine Safety Signals.

It is important to detect signals of abrupt changes in adverse event reporting in order to notice public safety concerns and take prompt action, especially for vaccines under national immunization programs. In this study, we assessed the applicability of change point analysis (CPA) for signal detection in vaccine safety surveillance. The performances of three CPA methods, namely Bayesian change point analysis, Taylor’s change point analysis (Taylor-CPA), and environmental time series change point detection (EnvCpt), were assessed via simulated data with assumptions for the baseline number of events and degrees of change. The analysis was validated using the Korea Adverse Event Reporting System (KAERS) database. In the simulation study, the Taylor-CPA method exhibited better results for the detection of a change point (accuracy of 96% to 100%, sensitivity of 7% to 100%, specificity of 98% to 100%, positive predictive value of 25% to 85%, negative predictive value of 96% to 100%, and balanced accuracy of 53% to 100%) than the other two CPA methods. When the CPA methods were applied to reports of syncope or dizziness following human papillomavirus (HPV) immunization in the KAERS database, Taylor-CPA and EnvCpt detected a change point (Q2/2013), which was consistent with actual public safety concerns. CPA can be applied as an efficient tool for the early detection of vaccine safety signals.

An Adaptive Intrusion Detection Scheme for Cloud Computing

To provide dynamic resource management, live virtual machine migration is used to move a virtual machine from one host to another. However, virtual machine migration poses challenges to cloud intrusion detection systems because movement of VMs from one host to another makes it difficult to create a consistent normal profile for anomaly detection. Hence, there is a need to provide an adaptive anomaly detection system capable of adapting to changes that occur in the cloud data during VM migration. To achieve this, the authors proposed a scheme for adaptive IDS for Cloud computing. The proposed adaptive scheme is comprised of four components: an ant colony optimization-based feature selection component, a statistical time series change point detection component, adaptive classification, and model update component, and a detection component. The proposed adaptive scheme was evaluated using simulated datasets collected from vSphere and performance comparison shows improved performance over existing techniques.

A Survey of Methods for Time Series Change Point Detection.

Change points are abrupt variations in time series data. Such abrupt changes may represent transitions that occur between states. Detection of change points is useful in modelling and prediction of time series and is found in application areas such as medical condition monitoring, climate change detection, speech and image analysis, and human activity analysis. This survey article enumerates, categorizes, and compares many of the methods that have been proposed to detect change points in time series. The methods examined include both supervised and unsupervised algorithms that have been introduced and evaluated. We introduce several criteria to compare the algorithms. Finally, we present some grand challenges for the community to consider.

A fuzzy/Bayesian approach for the time series change point detection problem

This paper addresses the change point detection problem in time series. A methodology based on the Metropolis-Hastings algorithm applied to time series modeled as a process with Beta distribution is discussed. In order to make this methodology useful in practice, a fuzzy cluster technique is applied to the initial time series at first, generating a new data set with Beta distribution. Bayesian procedures are considered for inference and the Metropolis-Hastings algorithm is used to sample from the posteriors. In the clustering process, a Kohonen neural network is used having as objective to find the best centers of the time series to be used in the fuzzyfication process. Finally, it will be presented a simulation results in the series of the electric energy consumption in Brazil, between January of 1976 and December of 2000, five months before the blackout occurred in 2001. Such result illustrates the efficiency of the proposed methodology for change point detection in time series.

Motion Entropy Feature and Its Applications to Event-Based Segmentation of Sports Video

An entropy-based criterion is proposed to characterize the pattern and intensity of object motion in a video sequence as a function of time. By applying a homoscedastic error model-based time series change point detection algorithm to this motion entropy curve, one is able to segment the corresponding video sequence into individual sections, each consisting of a semantically relevant event. The proposed method is tested on six hours of sports videos including basketball, soccer, and tennis. Excellent experimental results are observed.

An Algorithm Based on Singular Spectrum Analysis for Change-Point Detection

This paper is devoted to application of the singular-spectrum analysis to sequential detection of changes in time series. An algorithm of change-point detection in time series, based on sequential application of the singular-spectrum analysis is developed and studied. The algorithm is applied to different data sets and extensively studied numerically. For specific models, several numerical approximations to the error probabilities and the power function of the algorithm are obtained. Numerical comparisons with other methods are given.