Multiple Change-point Problem Research Articles

Good Practices and Common Pitfalls in Climate Time Series Changepoint Techniques: A Review

Abstract Climate changepoint (homogenization) methods abound today, with a myriad of techniques existing in both the climate and statistics literature. Unfortunately, the appropriate changepoint technique to use remains unclear to many. Further complicating issues, changepoint conclusions are not robust to perturbations in assumptions; for example, allowing for a trend or correlation in the series can drastically change changepoint conclusions. This paper is a review of the topic, with an emphasis on illuminating the models and techniques that allow the scientist to make reliable conclusions. Pitfalls to avoid are demonstrated via actual applications. The discourse begins by narrating the salient statistical features of most climate time series. Thereafter, single- and multiple-changepoint problems are considered. Several pitfalls are discussed en route and good practices are recommended. While most of our applications involve temperatures, a sea ice series is also considered. Significance Statement This paper reviews the methods used to identify and analyze the changepoints in climate data, with a focus on helping scientists make reliable conclusions. The paper discusses common mistakes and pitfalls to avoid in changepoint analysis and provides recommendations for best practices. The paper also provides examples of how these methods have been applied to temperature and sea ice data. The main goal of the paper is to provide guidance on how to effectively identify the changepoints in climate time series and homogenize the series.

Autocorrelation and Parameter Estimation in a Bayesian Change Point Model

A piecewise function can sometimes provide the best fit to a time series. The breaks in this function are called change points, which represent the point at which the statistical properties of the model change. Often, the exact placement of the change points is unknown, so an efficient algorithm is required to combat the combinatorial explosion in the number of potential solutions to the multiple change point problem. Bayesian solutions to the multiple change point problem can provide uncertainty estimates on both the number and location of change points in a dataset, but there has not yet been a systematic study to determine how the choice of hyperparameters or the presence of autocorrelation affects the inference made by the model. Here, we propose Bayesian model averaging as a way to address the uncertainty in the choice of hyperparameters and show how this approach highlights the most probable solution to the problem. Autocorrelation is addressed through a pre-whitening technique, which is shown to eliminate spurious change points that emerge due to a red noise process. However, pre-whitening a dataset tends to make true change points harder to detect. After an extensive simulation study, the model is applied to two climate applications: the Pacific Decadal Oscillation and a global surface temperature anomalies dataset.

Bayesian multiple changepoint detection with missing data and its application to the magnitude‐frequency distributions

AbstractThe detection of abrupt changes in an evolving pattern of time series in the presence of missing data still poses a challenge to real applications. We formulate the multiple changepoint problem into a latent Markov model on a countably infinite state space. For efficiency‐enhancing, we propose a partially collapsed Gibbs sampler for the inference of the joint posterior of the number of changepoints and their locations. Variants of Viterbi algorithms are suggested for obtaining the MAP estimates of random changepoints in the presence of missing data, which provides better performances in these varying‐dimensional problems. The method is generally applicable for multiple changepoint detection under a variety of missing data mechanism. The method is applied to a case study of the magnitude‐frequency distribution of the 2010 Darfield M7.1 earthquake sequence in New Zealand. We find out some unusual features of the seismic b‐value in the Darfield earthquake sequence. It is noted that two changepoints are detected and in contrast to the background seismic b‐value, relatively low b‐values in the early aftershock propagation period are identified. We suggest that this might be a forewarning of potentially devastatingly strong aftershocks. The advance in the method of b‐value changepoint detection will enhance our understanding of earthquake occurrence and potentially lead to improved risk forecasting.

Scalable Bayesian Multiple Changepoint Detection via Auxiliary Uniformisation

SummaryIn this paper, we perform a sparse filtering recursion for efficient changepoint detection for discrete‐time observations. We attach auxiliary event times to the chronologically ordered observations and formulate multiple changepoint problems of discrete‐time observations into continuous‐time observations. Ideally, both the computational and memory costs of the proposed auxiliary uniformisation forward‐filtering backward‐sampling algorithm can be quadratically scaled down to the number of changepoints instead of the number of observations, which would otherwise be prohibitive for a long sequence of observations. To avoid model bias, a time‐varying changepoint recurrence rate across different segments is assumed to characterise diverse scales of run lengths of the changepoints. We demonstrate the methods through simulation studies and real data analysis.

Empirical Likelihood Ratio Test for the Epidemic Change Model

Change point problem has been studied extensively since 1950s due to its broad applications in many elds such as nance, biology and so on. As a special case of the multiple change point problem, the epidemic change point problem has received a lot of attention especially in medical studies. In this paper, a nonparametric method based on the empirical likelihood is proposed to detect the epidemic changes of the mean after unknown change points. Under some mild conditions, the asymptotic null distribution of the empirical likelihood ratio test statistic is proved to be the extreme distribution. The consistency of the test is also proved. Simulations indicate that the test behaves comparable to the other available tests while it enjoys less constraint on the data distribution. The method is applied to the Standford heart transplant data and detects the change points successfully.

Detection of Multiple Change-Points in the Scale Parameter of a Gamma Distributed Sequence Based on Reversible Jump MCMC

In this paper, the multiple change-point problem in the scale parameter of a sequence of independent gamma distributed observations is discussed. A reversible jump Markov chain Monte Carlo (RJMCMC) algorithm is developed to compute the posterior probabilities of the number and positions of the multiple change-points. Four types of jumps are designed, and the acceptance probability of each type is given. The simulation studies show that the RJMCMC-based method is efficient in the detection of multiple change-points in the scale parameter in gamma distributed sequence, and performs better than a self-normalization based method. In addition, a real data example about successive rises and falls of Shanghai stock exchange composite index yield is used to illustrate the proposed methodology.

Consistent selection of the number of change-points via sample-splitting

In multiple change-point analysis, one of the major challenges is to estimate the number of change-points. Most existing approaches attempt to minimize a Schwarz information criterion which balances a term quantifying model fit with a penalization term accounting for model complexity that increases with the number of change-points and limits overfitting. However, different penalization terms are required to adapt to different contexts of multiple change-point problems and the optimal penalization magnitude usually varies from the model and error distribution. We propose a data-driven selection criterion that is applicable to most kinds of popular change-point detection methods, including binary segmentation and optimal partitioning algorithms. The key idea is to select the number of change-points that minimizes the squared prediction error, which measures the fit of a specified model for a new sample. We develop a cross-validation estimation scheme based on an order-preserved sample-splitting strategy, and establish its asymptotic selection consistency under some mild conditions. Effectiveness of the proposed selection criterion is demonstrated on a variety of numerical experiments and real-data examples.

A Bayesian multiple changepoint model for marked poisson processes with applications to deep earthquakes

A multiple changepoint model for marked Poisson process is formulated as a continuous time hidden Markov model, which is an extension of Chib’s multiple changepoint models (J Econ 86:221–241, 1998). The inference on the locations of changepoints and other model parameters is based on a two-block Gibbs sampling scheme. We suggest a continuous time version of forward-filtering backward-sampling algorithm for simulating the full trajectories of the latent Markov chain without utilizing the uniformization method. To retrieve the optimal posterior path of the latent Markov chain, i.e. the maximum a posteriori estimation of changepoint locations, a continuous-time version of Viterbi algorithm (CT-Viterbi) is proposed. The set of changepoint locations is obtainable either from the CT-Viterbi algorithm or the posterior samples of the latent Markov chain. The number of changepoints is determined according to a modified BIC criterion tailored particularly for the multiple changepoint problems of a marked Poisson process. We then perform a simulation study to demonstrate the methods. The methods are applied to investigate the temporal variabilities of seismicity rates and the magnitude-frequency distributions of medium size deep earthquakes in New Zealand.

Semiparametric method for detecting multiple change points model in financial time series

ABSTRACTThe problem of multiple change points has been discussed in these years on the background of financial shocks. In order to decrease the damage, it is worthy to find a more available model for the problem as precise as possible by the information from data set. This paper proposes the problem of detecting the change points by semiparametric test. The change points estimations are obtained by empirical likelihood method. Then some asymptotic results for multiple change points are obtained by loglikelihood ratio test and law of large numbers. Furthermore, the consistency of change points estimations is presented. Indeed, the method and steps to find the change points are derived. The simulation experiments prove that the semiparametric test is more efficient than nonparametric test. The diagnosis with simulation and the applications for multiple change points also illustrates the proposed model well.

A pruned recursive solution to the multiple change point problem

Long time series are often heterogeneous in nature. As such, the most appropriate model is one whose parameters are allowed to change through time. The exponential number of solutions to the multiple change point problem requires an efficient algorithm in order to be computationally feasible. Exact Bayesian solutions have at best quadratic complexity in the number of observations, which still can be too slow for very large data sets. Here, a pruned dynamic programming algorithm is proposed to fit a piecewise regression model with unknown break points to a data set. The algorithm removes unessential calculations, reducing the complexity of the most time consuming step of the algorithm from quadratic in the number of observations to quadratic in the average distance between change points. A distance measure is introduced that can be used to determine the divergence of the approximate joint posterior distribution from the exact posterior distribution. Analysis of two real data sets shows that this approximate algorithm produces a nearly identical representation of the joint posterior distribution on the locations of the change points, but with a significantly faster run time than its exact counterpart.

Online Steady-State Detection for Process Control Using Multiple Change-Point Models and Particle Filters

Steady-state detection is critical in process performance assessment, fault detection, and process automation and control. We proposed a robust online steady-state detection algorithm using multiple change-point model and particle filtering techniques. The steady-state detection problem is formulated as a multiple change-point problem using a segmented linear model. A particle filtering algorithm with stratified importance sampling and partial Gibbs move is developed to estimate this model. A generic timeliness improvement strategy is proposed to reduce the detection delay. Extensive numerical analysis shows that the proposed method is more accurate and robust than the other existing methods.

An exact approach to Bayesian sequential change point detection

Change point models seek to fit a piecewise regression model with unknown breakpoints to a data set whose parameters are suspected to change through time. However, the exponential number of possible solutions to a multiple change point problem requires an efficient algorithm if long time series are to be analyzed. A sequential Bayesian change point algorithm is introduced that provides uncertainty bounds on both the number and location of change points. The algorithm is able to quickly update itself in linear time as each new data point is recorded and uses the exact posterior distribution to infer whether or not a change point has been observed. Simulation studies illustrate how the algorithm performs under various parameter settings, including detection speeds and error rates, and allow for comparison with several existing multiple change point algorithms. The algorithm is then used to analyze two real data sets, including global surface temperature anomalies over the last 130 years.

Nonparametric maximum likelihood approach to multiple change-point problems

In multiple change-point problems, different data segments often follow different distributions, for which the changes may occur in the mean, scale or the entire distribution from one segment to another. Without the need to know the number of change-points in advance, we propose a nonparametric maximum likelihood approach to detecting multiple change-points. Our method does not impose any parametric assumption on the underlying distributions of the data sequence, which is thus suitable for detection of any changes in the distributions. The number of change-points is determined by the Bayesian information criterion and the locations of the change-points can be estimated via the dynamic programming algorithm and the use of the intrinsic order structure of the likelihood function. Under some mild conditions, we show that the new method provides consistent estimation with an optimal rate. We also suggest a prescreening procedure to exclude most of the irrelevant points prior to the implementation of the nonparametric likelihood method. Simulation studies show that the proposed method has satisfactory performance of identifying multiple change-points in terms of estimation accuracy and computation time.

Detection of multiple change-points in multivariate data

The statistical analysis of change-point detection and estimation has received much attention recently. A time point such that observations follow a certain statistical distribution up to that point and a different distribution – commonly of the same functional form but different parameters after that point – is called a change-point. Multiple change-point problems arise when we have more than one change-point. This paper develops a method for multivariate normally distributed data to detect change-points and estimate within-segment parameters using maximum likelihood estimation.

Estimating join points and modelling for multiple change point problem

In life time data analysis, it is often realistic to believe that early failures occur at one pace and afterwards at different rates. In this article, we propose a general modelling method for a continuous hazard function with multiple change points. We assume the multiple change point curve to be composed of phases that are complex functions of time. The complex functions can be represented by suitable known functions of time. We have tried four such functional combinations for different phases. We have illustrated our proposed methodology with an application to modelling lifetimes of printed circuit boards. To conclude, we have compared our models with previously developed single change point model and displayed that our models fit the data much better.

Multiple change-point detection via a screening and ranking algorithm

Let Y1, …, Yn be a sequence whose underlying mean is a step function with an unknown number of the steps and unknown change points. The detection of the change points, namely the positions where the mean changes, is an important problem in such fields as engineering, economics, climatology and bioscience. This problem has attracted a lot of attention in statistics, and a variety of solutions have been proposed and implemented. However, there is scant literature on the theoretical properties of those algorithms. Here, we investigate a recently developed algorithm called the Screening and Ranking Algorithm (SaRa). We characterize the theoretical properties of SaRa and show its superiority over other commonly used algorithms. In particular, we develop a false discovery rate approach to the multiple change-point problem and show a strong sure coverage property for the SaRa.

A Bayesian approach to detecting change points in climatic records

AbstractGiven distinct climatic periods in the various facets of the Earth's climate system, many attempts have been made to determine the exact timing of ‘change points’ or regime boundaries. However, identification of change points is not always a simple task. A time series containing N data points has approximately Nk distinct placements of k change points, rendering brute force enumeration futile as the length of the time series increases. Moreover, how certain are we that any one placement of change points is superior to the rest? This paper introduces a Bayesian Change Point algorithm which provides uncertainty estimates both in the number and location of change points through an efficient probabilistic solution to the multiple change point problem. To illustrate its versatility, the Bayesian Change Point algorithm is used to analyse both the NOAA/NCDC annual global surface temperature anomalies time series and the much longer δ18O record of the Plio‐Pleistocene. Copyright © 2012 Royal Meteorological Society

Implied distributions in multiple change point problems

A method for efficiently calculating exact marginal, conditional and joint distributions for change points defined by general finite state Hidden Markov Models is proposed. The distributions are not subject to any approximation or sampling error once parameters of the model have been estimated. It is shown that, in contrast to sampling methods, very little computation is needed. The method provides probabilities associated with change points within an interval, as well as at specific points.

Estimating change-points in biological sequences via the cross-entropy method

The genomes of complex organisms, including the human genome, are known to vary in GC content along their length. That is, they vary in the local proportion of the nucleotides G and C, as opposed to the nucleotides A and T. Changes in GC content are often abrupt, producing well-defined regions.We model DNA sequences as a multiple change-point process in which the sequence is separated into segments by an unknown number of change-points, with each segment supposed to have been generated by a different process. Multiple change-point problems are important in many biological applications, particularly in the analysis of DNA sequences. Multiple change-point problems also arise in segmentation of protein sequences according to hydrophobicity.We use the Cross-Entropy method to estimate the positions of the change-points. Parameters of the process for each segment are approximated with maximum likelihood estimates. Numerical experiments illustrate the effectiveness of the approach. We obtain estimates of the locations of change-points in artificially generated sequences and compare the accuracy of these estimates with those obtained via other methods such as IsoFinder (Oliver et al. in Nucl. Acids Res. 32:W283–W292, 2004) and Markov Chain Monte Carlo. Lastly, we provide examples with real data sets to illustrate the usefulness of our method.

Multiple change-point estimation with U-statistics

We consider a multiple change-point problem: a finite sequence of independent random variables consists of segments given by a known number of the so-called change-points such that the underlying distribution differs from segment to segment. The task is to estimate these change-points under no further assumptions on the within-segment distributions. In this completely nonparametric framework the proposed estimator is defined as the maximizing point of weighted multivariate U-statistic processes. Under mild moment conditions we prove almost sure convergence and the rate of convergence.