Process Change Point Research Articles

Modelling Nonstationary Gene Regulatory Processes

An important objective in systems biology is to infer gene regulatory networks from postgenomic data, and dynamic Bayesian networks have been widely applied as a popular tool to this end. The standard approach for nondiscretised data is restricted to a linear model and a homogeneous Markov chain. Recently, various generalisations based on changepoint processes and free allocation mixture models have been proposed. The former aim to relax the homogeneity assumption, whereas the latter are more flexible and, in principle, more adequate for modelling nonlinear processes. In our paper, we compare both paradigms and discuss theoretical shortcomings of the latter approach. We show that a model based on the changepoint process yields systematically better results than the free allocation model when inferring nonstationary gene regulatory processes from simulated gene expression time series. We further cross-compare the performance of both models on three biological systems: macrophages challenged with viral infection, circadian regulation in Arabidopsis thaliana, and morphogenesis in Drosophila melanogaster.

Identifying the time of polynomial drift in the mean of autocorrelated processes

AbstractControl charts are used to detect changes in a process. Once a change is detected, knowledge of the change point would simplify the search for and identification of the special ause. Consequently, having an estimate of the process change point following a control chart signal would be useful to process engineers. This paper addresses change point estimation for covariance‐stationary autocorrelated processes where the mean drifts deterministically with time. For example, the mean of a chemical process might drift linearly over time as a result of a constant pressure leak. The goal of this paper is to derive and evaluate an MLE for the time of polynomial drift in the mean of autocorrelated processes. It is assumed that the behavior in the process mean over time is adequately modeled by the kth‐order polynomial trend model. Further, it is assumed that the autocorrelation structure is adequately modeled by the general (stationary and invertible) mixed autoregressive‐moving‐average model. The estimator is intended to be applied to data obtained following a genuine control chart signal in efforts to help pinpoint the root cause of process change. Application of the estimator is demonstrated using a simulated data set. The performance of the estimator is evaluated through Monte Carlo simulation studies for the k=1 case and across several processes yielding various levels of positive autocorrelation. Results suggest that the proposed estimator provides process engineers with an accurate and useful estimate for the last sample obtained from the unchanged process. Copyright © 2009 John Wiley & Sons, Ltd.

Modelling non-stationary dynamic gene regulatory processes with the BGM model

Recently, a Bayesian network model for inferring non-stationary regulatory processes from gene expression time series has been proposed. The Bayesian Gaussian Mixture (BGM) Bayesian network model divides the data into disjunct compartments (data subsets) by a free allocation model, and infers network structures, which are kept fixed for all compartments. Fixing the network structure allows for some information sharing among compartments, and each compartment is modelled separately and independently with the Gaussian BGe scoring metric for Bayesian networks. The BGM model can equally be applied to both static (steady-state) and dynamic (time series) gene expression data. However, it is this flexibility that renders its application to time series data suboptimal. To improve the performance of the BGM model on time series data we propose a revised approach in which the free allocation of data points is replaced by a changepoint process so as to take the temporal structure into account. The practical inference follows the Bayesian paradigm and approximately samples the network, the number of compartments and the changepoint locations from the posterior distribution with Markov chain Monte Carlo (MCMC). Our empirical results show that the proposed modification leads to a more efficient inference tool for analysing gene expression time series.

Estimating change-points in biological sequences via the cross-entropy method

The genomes of complex organisms, including the human genome, are known to vary in GC content along their length. That is, they vary in the local proportion of the nucleotides G and C, as opposed to the nucleotides A and T. Changes in GC content are often abrupt, producing well-defined regions.We model DNA sequences as a multiple change-point process in which the sequence is separated into segments by an unknown number of change-points, with each segment supposed to have been generated by a different process. Multiple change-point problems are important in many biological applications, particularly in the analysis of DNA sequences. Multiple change-point problems also arise in segmentation of protein sequences according to hydrophobicity.We use the Cross-Entropy method to estimate the positions of the change-points. Parameters of the process for each segment are approximated with maximum likelihood estimates. Numerical experiments illustrate the effectiveness of the approach. We obtain estimates of the locations of change-points in artificially generated sequences and compare the accuracy of these estimates with those obtained via other methods such as IsoFinder (Oliver et al. in Nucl. Acids Res. 32:W283–W292, 2004) and Markov Chain Monte Carlo. Lastly, we provide examples with real data sets to illustrate the usefulness of our method.

Identifying the time of step change in the mean of autocorrelated processes

Control charts are used to detect changes in a process. Once a change is detected, knowledge of the change point would simplify the search for and identification of the special cause. Consequently, having an estimate of the process change point following a control chart signal would be useful to process analysts. Change-point methods for the uncorrelated process have been studied extensively in the literature; however, less attention has been given to change-point methods for autocorrelated processes. Autocorrelation is common in practice and is often modeled via the class of autoregressive moving average (ARMA) models. In this article, a maximum likelihood estimator for the time of step change in the mean of covariance-stationary processes that fall within the general ARMA framework is developed. The estimator is intended to be used as an “add-on” following a signal from a phase II control chart. Considering first-order pure and mixed ARMA processes, Monte Carlo simulation is used to evaluate the performance of the proposed change-point estimator across a range of step change magnitudes following a genuine signal from a control chart. Results indicate that the estimator provides process analysts with an accurate and useful estimate of the last sample obtained from the unchanged process. Additionally, results indicate that if a change-point estimator designed for the uncorrelated process is applied to an autocorrelated process, the performance of the estimator can suffer dramatically.

장기억 과정에서 빠른 베이지안 변화점검출

이 논문에서는 장기억 과정에서의 변화점을 빨리 검출하는 베이지안 추론방법에 대해 알아본다. 장기억 과정에서의 베이지안 추정은 장기억 모수값에 따라 전체 자료에 대한 부분차분을 계산해야 하기 때문에 수행시간이 많이 걸린다는 문제가 있다. 이 논문에서는 이러한 문제를 해결하고자 장기억 모수공간을 그룹화하여 순서형으로 범주화시킨 후 설명력이 가장 높은 범주의 대표값을 선택하게 하였다. 이 방법은 초기단계에서 범주의 대표값에 대해 한번씩만 부분차분을 계산하면 되기 때문에, 매번 계산해야 하는 추정하는 방법보다, 특히 시계열자료의 수가 많은 경우, 상대적으로 빠른 베인지안 추론이 가능하다. 또한 장기억 모수공간이 (0,0.5) 이기 때문에 모수공간을 적절하게 그룹화한다면 장기억 모수를 선택하는 것이 모수를 추정하는 것에 비해 큰 차이가 없다. 이 논문에서는 나일강 수위자료 실증분석을 통해 제안된 방법의 타당성을 확인해본다. In this paper, we introduce a fast approach for Bayesian detection of change points in long-memory processes. Since a heavy computation is needed to evaluate the likelihood function of long-memory processes, a method for simplifying the computational process is required to efficiently implement a Bayesian inference. Instead of estimating the parameter, we consider selecting a element from the set of possible parameters obtained by categorizing the parameter space. This approach simplifies the detection algorithm and reduces the computational time to detect change points. Since the parameter space is (0, 0.5), there is no big difference between the result of parameter estimation and selection under a proper fractionation of the parameter space. The analysis of Nile river data showed the validation of the proposed method.

Ratio test for variance change point in linear process with long memory

In this paper we consider the detection problem of variance change point in linear process with long memory. We propose the ratio test to detect the variance change point. The limiting distribution for test statistics under H0 is derived and the consistency of the test is also established. In comparison with the existing CUSUM of squares (SCUSUM) test, the ratio test does not need to estimate the long memory parameter in practical situation and therefore it can be used more conveniently.

A hybrid fuzzy-statistical clustering approach for estimating the time of changes in fixed and variable sampling control charts

Control charts are the most popular Statistical Process Control (SPC) tools used to monitor process changes. When a control chart produces an out-of-control signal, it means that the process has changed. However, control chart signals do not indicate the real time of the process changes, which is essential for identifying and removing assignable causes and ultimately improving the process. Identifying the real time of the process change is known as change-point estimation problem. Most of the traditional change-point methods are based on maximum likelihood estimators (MLE) which need strict statistical assumptions. In this paper, first, we introduce clustering as a potential tool for change-point estimation. Next, we discuss the challenges of employing clustering methods for change-point estimation. Afterwards, based on the concepts of fuzzy clustering and statistical methods, we develop a novel hybrid approach which is able to effectively estimate change-points in processes with either fixed or variable sample size. Using extensive simulation studies, we also show that the proposed approach performs considerably well in all considered conditions in comparison to powerful statistical methods and popular fuzzy clustering techniques. The proposed approach can be employed for processes with either normal or non-normal distributions. It is also applicable to both phase-I and phase-II. Finally, it can estimate the true values of both in- and out-of-control states’ parameters.

An ideal-observer model for optimal inference in the presence of different types of uncertainty

Event Abstract Back to Event An ideal-observer model for optimal inference in the presence of different types of uncertainty Whether getting a new job or a new president, life is full of ‘changepoints’ that cause the rules of the game to change abruptly. Making predictions in such circumstances can be challenging because changepoints can render much of the past irrelevant. For example, the outcome of this year’s presidential election suggests that policies from recent years will not necessarily be good predictors of policies in the coming years. Here we examine how to identify changepoints in a noisy environment and then use that information to adjust how to use past experiences to predict future events. We present an ideal-observer model of a dynamic probabilistic estimation task designed to quantify the extent to which human subjects use new pieces of information in the presence of possible changepoints. At each time-step in this task, a bar appears on screen at a random location, and subjects are asked to predict the location of the bar on the next time-step. The subjects receive a cash reward based on the accuracy of their prediction. The location of the bar is determined by a changepoint process, which is sampled from a Gaussian distribution whose mean and variance change abruptly at changepoints but otherwise remain constant. Because the time of occurrence of changepoints are unknown to the subjects, they must contend with two different kinds of uncertainty: ‘expected’ uncertainty corresponding to the variance of the Gaussian distribution and ‘unexpected’ uncertainty corresponding to probability that a changepoint might occur [A.J.Yu and P.Dayan Neuron 46:681-692,2005]. When the frequency of changepoints (known as the hazard rate), but not their exact times of occurrence, is given, Adams and MacKay [Technical report, Cambridge University, 2007] and Fearnhead and Liu [J.R.Statist.Soc.B 69(4):589-605,2007] have developed online Bayesian models capable of making optimal predictions. However, in general, the hazard rate is unspecified and must be inferred from the data. We solve this problem by developing an online Bayesian algorithm capable of making optimal predictions in the face of expected and unexpected uncertainty and an unknown - and potentially variable - hazard rate. In particular, we model the occurrence of changepoints as a Bernoulli process and infer a constant hazard rate by keeping track of changepoints experienced over time. A hierarchical extension of the model in which the hazard rate is itself generated by a changepoint process allows us to make inferences about variable hazard rates. We compare performance of the model and human subjects on the dynamic probabilistic estimation task. We focus on the weight given to each datum in determining the next estimate as the hazard rate changes. A key result is that this weighting differs substantially in a stable environment, in which changepoints are rare and the past is a good predictor of the future, versus a volatile environment, in which changepoints are common and past trends are typically not helpful. By quantifying these principles, the model establishes a benchmark both for comparing human performance on this kind of prediction task and for analyzing a host of changepoint data. Conference: Computational and systems neuroscience 2009, Salt Lake City, UT, United States, 26 Feb - 3 Mar, 2009. Presentation Type: Oral Presentation Topic: Oral Presentations Citation: (2009). An ideal-observer model for optimal inference in the presence of different types of uncertainty. Front. Syst. Neurosci. Conference Abstract: Computational and systems neuroscience 2009. doi: 10.3389/conf.neuro.06.2009.03.044 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 30 Jan 2009; Published Online: 30 Jan 2009. Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Google Google Scholar PubMed Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

A change point model for the location parameter of exponential family densities

Knowing when a process has changed would simplify the search for and identification of the special cause. Consequently, having an estimate of the process change point following a control chart signal would be useful to process engineers. In this research a maximum likelihood change point estimator is proposed for the natural location parameter of densities belonging to the exponential family. It is assumed that the behavior in the location parameter over discrete sampling intervals is adequately modeled by a linear predictor. The estimator is intended to be applied to data obtained following signals from univariate statistical process control charts in an effort to aid process engineers diagnose the root cause of process change.

Identifying Change-Points in Biological Sequences via Sequential Importance Sampling

The genomes of complex organisms, including the human genome, are highly structured. This structure takes the form of segmental patterns of variation in various properties and may be caused by the division of genomes into regions of distinct function, by the contingent evolutionary processes that gave rise to genomes, or by a combination of both. Whatever the cause, identifying the change-points between segments is potentially important, as a means of discovering the functional components of a genome, understanding the evolutionary processes involved, and fully describing genomic architecture. One property of genomes that is known to display a segmental pattern of variation is GC content. The GC content of a portion of DNA is the proportion of GC pairs that it contains. Sharp changes in GC content can be observed in human and other genomes. Such change-points may be the boundaries of functional elements or may play a structural role. We model genome sequences as a multiple change-point process, that is, a process in which sequential data are separated into segments by an unknown number of change-points, with each segment supposed to have been generated by a different process. We consider a Sequential Importance Sampling approach to change-point modeling using Monte Carlo simulation to find estimates of change-points as well as parameters of the process on each segment. Numerical experiments illustrate the effectiveness of the approach. We obtain estimates for the locations of change-points in artificially generated sequences and compare the accuracy of these estimates to those obtained via Markov chain Monte Carlo and a well-known method, IsoFinder. We also provide examples with real data sets to illustrate the usefulness of this method.

A clustering approach to identify the time of a step change in Shewhart control charts

AbstractControl charts are the most popular statistical process control tools used to monitor process changes. When a control chart indicates an out‐of‐control signal it means that the process has changed. However, control chart signals do not indicate the real time of process changes, which is essential for identifying and removing assignable causes and ultimately improving the process. Identifying the real time of the change is known as the change‐point estimation problem. Most of the traditional methods of estimating the process change point are developed based on the assumption that the process follows a normal distribution with known parameters, which is seldom true. In this paper, we propose clustering techniques to estimate Shewhart control chart change points. The proposed approach does not depend on the true values of the parameters and even the distribution of the process variables. Accordingly, it is applicable to both phase‐I and phase‐II of normal and non‐normal processes. At the end, we discuss the performance of the proposed method in comparison with the traditional procedures through extensive simulation studies. Copyright © 2008 John Wiley & Sons, Ltd.

Estimation of the Change Point in Monitoring the Process Mean and Variance

Knowing the time of a process change could lead to quicker identification of the special cause and less process down time, as well as help to reduce the probability of incorrectly identifying the special cause. In this article, we propose the maximum likelihood estimator (MLE) for the process change point when a control chart with the fixed sampling rate (FSR) scheme or the variable sampling rate (VSR) scheme is used in monitoring a process to detect changes in the process mean and/or variance of a normal quality variable. We investigate the performance of this estimator when it is used in various types of control charts.

Change point estimation for monotonically changing Poisson rates in SPC

Knowing when a process has changed would simplify the search for and identification of the special cause. Although several change point methods have been suggested, many of them rely on the assumption that the effect present in the process output follows some known form (e.g. sudden shift or linear trend). Since processes are often influenced by several input factors, sudden shifts and linear trends do not always adequately describe the true nature of the process behavior. In this paper, we propose a maximum likelihood estimator for the change point of a Poisson rate parameter without requiring exact a priori knowledge regarding the form of the effect present. Instead, we assume the form of effect present can be characterized as belonging to the set of monotonic effects. We compare the proposed change point estimator to the commonly used maximum likelihood estimator for the process change point derived under a sudden and persistent shift assumption. We do this for a number of monotonic effects and following a signal from a Poisson CUSUM control chart. We conclude that it is better to use the proposed change point estimator when the form of the effect present is only known to be monotonic. The results show that the proposed estimator provides process engineers with an accurate and useful estimate of the last observation obtained from the unchanged process regardless of the form of monotonic effect that may be present.

Phylogenetic Mapping of Recombination Hotspots in Human Immunodeficiency Virus via Spatially Smoothed Change-Point Processes

We present a Bayesian framework for inferring spatial preferences of recombination from multiple putative recombinant nucleotide sequences. Phylogenetic recombination detection has been an active area of research for the last 15 years. However, only recently attempts to summarize information from several instances of recombination have been made. We propose a hierarchical model that allows for simultaneous inference of recombination breakpoint locations and spatial variation in recombination frequency. The dual multiple change-point model for phylogenetic recombination detection resides at the lowest level of our hierarchy under the umbrella of a common prior on breakpoint locations. The hierarchical prior allows for information about spatial preferences of recombination to be shared among individual data sets. To overcome the sparseness of breakpoint data, dictated by the modest number of available recombinant sequences, we a priori impose a biologically relevant correlation structure on recombination location log odds via a Gaussian Markov random field hyperprior. To examine the capabilities of our model to recover spatial variation in recombination frequency, we simulate recombination from a predefined distribution of breakpoint locations. We then proceed with the analysis of 42 human immunodeficiency virus (HIV) intersubtype gag recombinants and identify a putative recombination hotspot.

A Modified Bayes Information Criterion with Applications to the Analysis of Comparative Genomic Hybridization Data

In the analysis of data generated by change-point processes, one critical challenge is to determine the number of change-points. The classic Bayes information criterion (BIC) statistic does not work well here because of irregularities in the likelihood function. By asymptotic approximation of the Bayes factor, we derive a modified BIC for the model of Brownian motion with changing drift. The modified BIC is similar to the classic BIC in the sense that the first term consists of the log likelihood, but it differs in the terms that penalize for model dimension. As an example of application, this new statistic is used to analyze array-based comparative genomic hybridization (array-CGH) data. Array-CGH measures the number of chromosome copies at each genome location of a cell sample, and is useful for finding the regions of genome deletion and amplification in tumor cells. The modified BIC performs well compared to existing methods in accurately choosing the number of regions of changed copy number. Unlike existing methods, it does not rely on tuning parameters or intensive computing. Thus it is impartial and easier to understand and to use.

Estimating the Change Point of the Process Fraction Non‐conforming with a Monotonic Change Disturbance in SPC

AbstractKnowing when a process has changed would simplify the search for and identification of the special cause. In this paper, we propose a maximum‐likelihood estimator for the change point of the process fraction non‐conforming without requiring knowledge of the exact change type a priori. Instead, we assume the type of change present belongs to a family of monotonic changes. We compare the proposed change‐point estimator to the maximum‐likelihood estimator for the process change point derived under a simple step change assumption. We do this for a number of monotonic change types and following a signal from a binomial cumulative sum (CUSUM) control chart. We conclude that it is better to use the proposed change point estimator when the type of change present is only known to be monotonic. The results show that the proposed estimator provides process engineers with an accurate and useful estimate of the time of the process change regardless of the type of monotonic change that may be present. Copyright © 2006 John Wiley & Sons, Ltd.

ESTIMATION OF THE CHANGE POINT OF A GAMMA PROCESS BY USING THE S CONTROL CHART AND MLE

ABSTRACT The process personnel always seeks the opportunity to improve the processes. One of the essential steps for process improvement is to quickly recognize the starting time or the change point of a process disturbance. Different from the traditional normally distributed assumption for a process, this study considers a process which follows a gamma distribution. This study proposes an effective approach to estimate the change point of a gamma process. The proposed approach combines the commonly used S control chart with the maximum likelihood (ML) technique. This study applies the Monte Carlo technique to determine the upper and lower control limits for a gamma process when the S control chart is used to monitor a gamma process. The fruitful results with the use of the proposed approach are also demonstrated.

Estimation of the Change Point of a Normal Process Mean with a Linear Trend Disturbance in SPC

Knowing when a process changed would simplify the search and identification of the special cause. In this paper, we compare the maximum likelihood estimator (MLE) of the process change point designed for linear trends to the MLE of the process change point designed for step changes when a linear trend disturbance is present. As expected, our conclusions show that the MLE of the process change point designed for linear trends outperforms the MLE designed for step changes when the change type is a linear trend. We also present an approach based on the likelihood function for estimating a confidence set for the process change point. We study the performance of this estimator when it is used with a Shewhart x control chart and make direct performance comparisons with the estimated confidence sets obtained from the MLE for step changes. As expected, results show that better confidence can be obtained using the MLE for linear trends when a linear trend disturbance is present.

Estimating the Change Point of a Poisson Rate Parameter with a Linear Trend Disturbance

AbstractKnowing when a process changed would simplify the search and identification of the special cause. In this paper, we compare the maximum likelihood estimator (MLE) of the process change point designed for linear trends to the MLE of the process change point designed for step changes when a linear trend disturbance is present. We conclude that the MLE of the process change point designed for linear trends outperforms the MLE designed for step changes when a linear trend disturbance is present. We also present an approach based on the likelihood function for estimating a confidence set for the process change point. We study the performance of this estimator when it is used with a cumulative sum (CUSUM) control chart and make direct performance comparisons with the estimated confidence sets obtained from the MLE for step changes. The results show that better confidence can be obtained using the MLE for linear trends when a linear trend disturbance is present. Copyright © 2005 John Wiley & Sons, Ltd.