Identification Of Change Points Research Articles

Evaluation of Jenks Natural Breaks Clustering Algorithm for Changepoint Identification in Streaming Sensor Data

Evaluation of Jenks Natural Breaks Clustering Algorithm for Changepoint Identification in Streaming Sensor Data

Robust variance estimators in application to segmentation of measurement data distorted by impulsive and non-Gaussian noise

The paper algorithmizes the problem of regime change point identification for data measured in a system exhibiting impulsive behaviors. This is a fundamental challenge for annotation of measurement data, relevant, e.g., for designing data-driven autonomous systems. The contribution consists in formulation of an offline robust methodology based on the classical approach for structural break detection. The problem of data segmentation for the changing scale is solved as a process variable sensitive to the occurrence of a critical event in a reorganizing physical system. The advantage is that this approach does not require the existence of a variance of the data distribution. Efficiency has been evaluated for simulated data from two distributions and for real-world data sets measured in financial, mechanical, and medical systems. Simulation studies show that in the most challenging case, the error mean absolute in estimating the regime change is 20 times lower for the robust approach compared to the classical approach.

A novel RUL prediction method for rolling bearings based on dynamic control chart and adaptive incremental filtering

Abstract Degradation of rolling bearings typically consists of two stages: a stable stage (Stage I) characterized by stable fluctuations in the health indicator (HI), and a degradation stage (Stage II) where early damage leads to HI degradation, eventually reaching the failure threshold. Therefore, to achieve online RUL prediction for bearings, three aspects should be studied: 1) Degradation modeling; 2) Inter stage change point identification; 3) Online degradation state updating. Firstly, a two-stage degradation model is constructed by simultaneously considering inherent randomness, individual differences, and measurement errors. Then, a dynamic statistical process control (SPC) method is proposed to identify the change point from Stage I to Stage II. The SPC is designed to dynamically control limits based on the bearing's condition monitoring (CM) data to prevent false alarms. An adaptive incremental filtering (AIF) is proposed to update the degradation states by simultaneously considering the state increment and the dynamics of the system noise and measurement noise. The effectiveness of the proposed method is validated on 16004 bearing test data and XJTU-SY bearing data. Results show that the proposed method can accuracy identify the change point and improve the accuracy of the prediction result during stage II.

Multivariate Time Series Change-Point Detection with a Novel Pearson-like Scaled Bregman Divergence.

Change-point detection is a challenging problem that has a number of applications across various real-world domains. The primary objective of CPD is to identify specific time points where the underlying system undergoes transitions between different states, each characterized by its distinct data distribution. Precise identification of change points in time series omics data can provide insights into the dynamic and temporal characteristics inherent to complex biological systems. Many change-point detection methods have traditionally focused on the direct estimation of data distributions. However, these approaches become unrealistic in high-dimensional data analysis. Density ratio methods have emerged as promising approaches for change-point detection since estimating density ratios is easier than directly estimating individual densities. Nevertheless, the divergence measures used in these methods may suffer from numerical instability during computation. Additionally, the most popular -relative Pearson divergence cannot measure the dissimilarity between two distributions of data but a mixture of distributions. To overcome the limitations of existing density ratio-based methods, we propose a novel approach called the Pearson-like scaled-Bregman divergence-based (PLsBD) density ratio estimation method for change-point detection. Our theoretical studies derive an analytical expression for the Pearson-like scaled Bregman divergence using a mixture measure. We integrate the PLsBD with a kernel regression model and apply a random sampling strategy to identify change points in both synthetic data and real-world high-dimensional genomics data of Drosophila. Our PLsBD method demonstrates superior performance compared to many other change-point detection methods.

Groundwater Chemical Trends Analyses in the Piedmont Po Plain (NW Italy): Comparison with Groundwater Level Variations (2000–2020)

The concentrations of chemicals in the groundwater chemical values in the Piedmont Po Plain (NW Italy) show significant temporal variability and need to be characterised due to the lack of regional-scale assessments. The aim of this study was to analyse the trends (period 2000–2020) in the main physicochemical parameters and main ions in 227 wells in the shallow aquifer and to identify the potential causes. The identification of change points (points of sudden change) and comparisons with groundwater level variations were also performed. Results highlight general increasing trends for Na, Cl and HCO3, decreasing trends for SO4 and NO3, stationary conditions for pH and heterogeneous behaviours for electrolytic conductivity, Ca and Mg. Change points occurred in at least 50% of the monitoring wells, mainly during the 2008–2011 period. The comparison between groundwater levels and chemistry highlights a direct proportionality. Superimposed processes that induce an absence of proportionality are shown. The comparison of results with those of previous studies conducted under similar conditions revealed similar variations.. In conclusion, the potential responsible factors (e.g., road-salt dissolution and agricultural practices) and the relevant role of groundwater level variation were identified.

Joint Modeling of Change-Point Identification and Dependent Dynamic Community Detection

Joint Modeling of Change-Point Identification and Dependent Dynamic Community Detection

Likelihood Scores for Sparse Signal and Change-Point Detection

We consider here the identification of change-points on large-scale data streams. The objective is to find the most efficient way of combining information across data stream so that detection is possible under the smallest detectable change magnitude. The challenge comes from the sparsity of change-points when only a small fraction of data streams undergo change at any point in time. The most successful approach to the sparsity issue so far has been the application of hard thresholding such that only local scores from data streams exhibiting significant changes are considered and added. However the identification of an optimal threshold is a difficult one. In particular it is unlikely that the same threshold is optimal for different levels of sparsity. We propose here a sparse likelihood score for identifying a sparse signal. The score is a likelihood ratio for testing between the null hypothesis of no change against an alternative hypothesis in which the change-points or signals are barely detectable. By the Neyman-Pearson Lemma this score has maximum detection power at the given alternative. The outcome is that we have a scoring of data streams that is successful in detecting at the boundary of the detectable region of signals and change-points. The likelihood score can be seen as a soft thresholding approach to sparse signal and change-point detection in which local scores that indicate small changes are down-weighted much more than local scores indicating large changes. We are able to show sharp optimality of the sparsity likelihood score in the sense of achieving successful detection at the minimum detectable order of change magnitude as well as the best constant with respect this order of change.

A semiparametric approach to the detection of change-points in volatility dynamics of financial data

Volatility is one of the most important features of financial time series data. There are often structural changes in volatility over time, and an accurate estimation of the volatility of financial time series requires careful identification of change-points. A common approach to modeling the volatility of time series data is based on the well-known GARCH model. Although the problem of change-point estimation of volatility dynamics derived from the GARCH model has been considered in the literature, these approaches rely on parametric assumptions of the conditional error distribution, which are often violated in financial time series. This may lead to inaccuracies in change-point detection, resulting in unreliable GARCH volatility estimates. This article introduces a novel change-point detection algorithm based on a semiparametric GARCH model. The proposed method retains the structural advantages of the GARCH process while incorporating the flexibility of nonparametric conditional error distribution. The approach utilizes a penalized likelihood derived from a semiparametric GARCH model and an efficient binary segmentation algorithm. The results show that in terms of change-point estimation and detection accuracy, the semiparametric method outperforms the commonly used Quasi-MLE (QMLE) and other variations of the GARCH models in wide-ranging scenarios.

Observed trends and coherent changes in daily rainfall extremes over Greater Mumbai, 1985–2020

Climate extremes are predicted to become more frequent and severe in the future, owing to rising greenhouse gas emissions in the atmosphere. Using appropriate trend indices, this study intends to analyze the effects of climate change on rainfall variability and extreme rainfall events in the millennium city of Mumbai. The analysis was carried out for the period 1985–2020 for 2 stations and 23 stations were selected (2006–2020) later from the dense rain gauge network of Municipal Corporation of Greater Mumbai (MCGM). To comprehend the trend and behaviour of rainfall and extreme events, three statistical models were used: Mann–Kendall test, Sen’s slope estimator, and simple linear regression; for abrupt change point detection, Pettitt’s test, standard normal homogeneity test, Buishand’s range test, and von Neumann ratio test were used. The results indicated that average annual rainfall in the study area is 2208.82 mm which is increasing at the rate of 5.18 mm/year (at 99% confidence level). The frequency of heavy (> 120 mm/day) and extreme heavy rainfall events (250 mm/day) increased over Santacruz after 1994 and in the case of Colaba, it increased after 2005. The examination of rainfall and temperature data series for abrupt change point identification suggests that several change points exist, with the highest change points occurring between 2001 and 2005. Extreme climate event scenarios shown here could have large-scale negative consequences for the ecosystem and ecological resources of the study area.

An optimization method for paleomagnetic Euler pole analysis

Owing to the axial symmetry of the Earth’s magnetic field, paleomagnetic data only directly record the latitudinal and azimuthal positions of crustal blocks in the past, and paleolongitude cannot be constrained. An ability to overcome this obstacle is thus of fundamental importance to paleogeographic reconstruction. Paleomagnetic Euler pole (PEP) analysis presents a unique means to recover such information, but prior implementations of the PEP method have incorporated subjective decisions into its execution, undercutting its fidelity and rigor. Here we present an optimization approach to PEP analysis that addresses some of these deficiencies—namely the objective identification of change-points and small-circle arcs that together approximate an apparent polar wander path. We elaborate on our novel methodology and conduct some experiments with synthetic data to demonstrate its performance. We furthermore present implementations of our methods both as adaptable, stand-alone scripts in Python and as a streamlined interactive workflow that can be operated through a web browser.

Transport of lysosomes decreases in the perinuclear region: Insights from changepoint analysis

Lysosomes are membrane-bound organelles that serve as the endpoint for endocytosis, phagocytosis, and autophagy, degrading the molecules, pathogens, and organelles localized within them. These cellular functions require intracellular transport. We use fluorescence microscopy to characterize the motion of lysosomes as a function of intracellular region, perinuclear or periphery, and lysosome diameter. Single-particle tracking data are complemented by changepoint identification and analysis of a mathematical model for state switching. We first classify lysosomal motion as motile or stationary. We then study how lysosome location and diameter affects the proportion of time spent in each state and quantify the speed during motile periods. We find that the proportion of time spent stationary is strongly region dependent, with significantly decreased motility in the perinuclear region. Increased lysosome diameter only slightly decreases speed. Overall, these results demonstrate the importance of decomposing particle trajectories into qualitatively different behaviors before conducting population-wide statistical analysis. Our results suggest that intracellular region is an important factor to consider in studies of intracellular transport.

Functional Complex Networks Based on Operational Architectonics: Application on EEG-based Brain–computer Interface for Imagined Speech

A new method for analyzing brain complex dynamics and states is presented. This method constructs functional brain graphs and is comprised of two pylons: (a) Operational architectonics (OA) concept of brain and mind functioning. (b) Network neuroscience. In particular, the algorithm utilizes OA framework for a non-parametric segmentation of EEG signals, which leads to the identification of change points, namely abrupt jumps in EEG amplitude, called Rapid Transition Processes (RTPs). Subsequently, the time coordinates of RTPs are used for the generation of undirected weighted complex networks fulfilling a scale-free topology criterion, from which various network metrics of brain connectivity are estimated. These metrics form feature vectors, which can be used in machine learning algorithms for classification and/or prediction. The method is tested in classification problems on an EEG-based BCI data set, acquired from individuals during imagery pronunciation tasks of various words/vowels. The classification results, based on a Naïve Bayes classifier, show that the overall accuracies were found to be above chance level in all tested cases. This method was also compared with other state-of-the-art computational approaches commonly used for functional network generation, exhibiting competitive performance. The method can be useful to neuroscientists wishing to enhance their repository of brain research algorithms.

THEORETICAL AND EXPERIMENTAL INVESTIGATION OF ESTIMATING CHANGE POINT IN MULTIVARIATE PROCESSES VIA SIMULTANEOUS COVARIANCE MATRIX AND MEAN VECTOR

The identification of change points in statistical process control (SPC) data is the critical criterion for multivariate techniques when output is out-of-control condition. Therefore, monitoring all independent variables is essential and demands targeted attention to avoid errors at the systems control stage. However, estimating change-point in multivariate control charts is the main problem when these correlated quality characteristics monitor together. Therefore, we proposed a combination of an ensemble learning-based model of artificial neural networks with support vector machines to monitor process mean vector and covariance matrix shifts simultaneously to estimate the change point in a multivariable system. The performance of the final model indicated an estimated changing point with one sample over 6,000 simulated cases with a probability of 98 percent, which is a significantly high accuracy rating. Finding suggests the outcome of the project confirms that the proposed model can provide a precise estimating the change point by monitoring the mean vector and the covariance matrix simultaneously and, helps to identify those variable(s) responsible for an out-of-control condition. For further validation of the model, the performance of the proposed model has been compared with previous reported which confirms a better performance of the proposed model. Finally, the model was applied to monitor the performance of the solar hydrogen production system and the model identify the variables which have negative effects on the performance of the system.

Lane-based queue length estimation at signalized intersections using single-section license plate recognition data

Due to full record of discharging vehicle headway, License Plate Recognition (LPR) is used as an ideal source in most existing queue length estimation methods through a double-section detection using shock-wave models or input-output models. However, the impacts of heavy vehicles and miss detection by LPR detectors are mostly ignored. Therefore, this paper proposes a lane-based queue length estimation method using single-section LPR detection, considering miss detection and heavy vehicles. The queue length estimation problem is transformed to a change-point identification problem for discharging headways time-series, using E-Divisive with Medians (EDM) method. The maximal queue length is identified as the change-point of the discharging headways with the maximal differences between queued and non-queued vehicles, considering the queuing homogeneity of a lane group and the miss detection rate of LPR. The proposed method is validated using simulation and empirical cases with promising performance and good robustness under various conditions.

Testing reliability of the spatial Hurst exponent method for detecting a change point

AbstractThe reliability of using abrupt changes in the spatial Hurst exponent for identifying temporal points of abrupt change in climate dynamics is explored. If a spatio-temporal dynamical system undergoes an abrupt change at a particular time, the time series of spatial Hurst exponent obtained from the data of any variable of the system should also show an abrupt change at that time. As expected, spatial Hurst exponents for each of the two variables of a model spatio-temporal system – a globally coupled map lattice based on the Burgers' chaotic map – showed abrupt change at the same time that a parameter of the system was changed. This method was applied for the identification of change points in climate dynamics using the NCEP/NCAR data on air temperature, pressure and relative humidity variables. Different abrupt change points in spatial Hurst exponents were detected for the data of these different variables. That suggests, for a dynamical system, change point detected using the two-dimensional detrended fluctuation analysis method on a single variable alone is insufficient to comment about the abrupt change in the system dynamics and should be based on multiple variables of the dynamical system.

Bayesian Circle Segmentation with Application to DNA Copy Number Alteration Detection

Several statistical approaches have been proposed to consider circumstances under which one universal distribution is not capable of fit ting into the whole domain. This paper studies Bayesian detection of mul tiple interior epidemic/square waves in the interval domain, featured by two identical statistical distributions at both ends. We introduce a simple dimension-matching parameter proposal to implement the sampling-based posterior inference for special cases where each segmented distribution on a circle has the same set of regulating parameters. Molecular biology research reveals that, cancer progression may involve DNA copy number alteration at genome regions and connection of two biologically inactive chromosome ends results in a circle holding multiple epidemic/square waves. A slight modification of a simple novel Bayesian change point identification algo rithm, random grafting-pruning Markov chain Monte Carlo (RGPMCMC), is proposed by adjusting the original change point birth/death symmetric transition probability with a differ-by-one change point number ratio. The algorithm performance is studied through simulations with connection to DNA copy number alteration detection, which promises potential applica tion to cancer diagnosis at the genome level.

A longitudinal analysis of arterial stiffness and wave reflection in preeclampsia: Identification of changepoints

PurposePreeclampsia (PrE) is a leading complication of pregnancy characterized by vascular dysfunction. Characterizing the longitudinal changes in vascular function prior to PrE onset is critical to the identification of optimal timepoints for vascular assessment and the development of effective early screening strategies. MethodsIn this prospective longitudinal study of women with singleton high-risk pregnancies, arterial stiffness and wave reflection parameters were assessed using applanation tonometry at 10–13 weeks' gestation and repeated every 4 weeks throughout pregnancy. Changepoints in carotid-femoral pulse wave velocity (cfPWV), carotid-radial PWV (crPWV), augmentation index (AIx), time to wave reflection (T1R), pulse pressure amplification (PPA), and subendocardial viability ratio (SEVR) were compared between women who did and did not subsequently develop PrE. ResultsA changepoint in cfPWV and crPWV was detected at 14–17 weeks' gestation. cfPWV then increased in women who went on to develop PrE but decreased in women who did not; a 1.2 m/s difference in cfPWV between the groups was observed at 22–25 weeks' gestation. Conversely, crPWV converged in the two groups from a baseline difference of 1.05 m/s (95% credible interval: 0.37, 1.72). Women who subsequently developed PrE demonstrated an increase in AIx at 18–21 weeks' gestation that was not seen in women who did not develop PrE until 30–33 weeks. No differences in T1R, PPA, or SEVR were observed between the groups. ConclusionsAltered vascular adaptations were detected using measures of arterial stiffness and wave reflection in the early second trimester of pregnant women who developed PrE compared to those who did not. These findings demonstrate the potential clinical utility of arterial stiffness and wave reflection parameters as an early screening tool for PrE, which can be used to inform clinical management of high-risk pregnancies.

Elasticity as a measure for online determination of remission points in ongoing epidemics.

The correct identification of change-points during ongoing outbreak investigations of infectious diseases is a matter of paramount importance in epidemiology, with major implications for the management of health care resources, public health and, as the COVID-19 pandemic has shown, social live. Onsets, peaks, and inflexion points are some of them. An onset is the moment when the epidemic starts. A "peak" indicates a moment at which the incorporated values, both before and after, are lower: a maximum. The inflexion points identify moments in which the rate of growth of the incorporation of new cases changes intensity. In this study, after interpreting the concept of elasticity of a random variable in an innovative way, we propose using it as a new simpler tool for anticipating epidemic remission change-points. In particular, we propose that the "remission point of change" will occur just at the instant when the speed in the accumulation of new cases is lower than the average speed of accumulation of cases up to that moment. This gives stability and robustness to the estimation in the event of possible remission variations. This descriptive measure, which is very easy to calculate and interpret, is revealed as informative and adequate, has the advantage of being distribution-free and can be estimated in real time, while the data is being collected. We use the 2014-2016 Western Africa Ebola virus epidemic to demonstrate this new approach. A couple of examples analyzing COVID-19 data are also included.

Bearing remaining useful life estimation using an adaptive data-driven model based on health state change point identification and K-means clustering

Advance prediction about bearing remaining useful life (RUL) is a major activity which aims at scheduling proper future actions to avoid catastrophic events. However, the reliability of bearing life prediction models is subject to processes, such as construction of a robust bearing degradation health index, monotonicity and trendability of health index, uncertainty in construction of a failure threshold etc. Therefore, to achieve reliable bearing RUL estimates, this study proposes a fundamental framework wherein several data driven models are trained adaptively corresponding to the different bearing health states. The core idea is to selectively identify effective bearings from the training set of bearings whose failure patterns match closely with the evolving failure pattern of a bearing under operation. In each bearing, the locations of all health state change points are identified and then the training bearings are clustered into groups having similar failure trajectories using a K-means approach and developed similarity index. The proposed approach utilizes only partial data from the test bearing for RUL prediction and eliminates the need to manually pre-define a failure threshold limit. The prediction estimates are updated with every incoming data point acquired on the test bearing until failure. A cumulative function is proposed to make the trend of the adopted health indicator (HI) into being monotonic and trendable, which is then used as an input to the data driven model. A confidence value (CV) parameter is proposed to map the inputs of the data driven model, such the CV varies in a fixed range. Both simulated data and run-to-failure experimental data (IEEE PHM 2012 bearing data) have been used to demonstrate the effectiveness of the proposed method. The test results from the proposed methodology have been benchmarked with other approaches, further validating its generic character and robustness.

A Research on Change Point and Trend Scrutiny: with Reference to Castor in India

Castor is treated as an important non-edible oil crop among Mahua, Karanja and Jatropha. India occupies a leading position in the production of castor followed by China and Brazil. Castor has been used in various purposes from thousands of years ago which has important characteristics. The main objectives of current work are i) Identification of change point and ii) Trend analysis with respect to area, yield and production of castor for the period from 1961 to 2016 in India. The non-parametric statistical methods such as Pettitt's, Standard Normal Homogeneity (SNH) and Buishand’s Range tests have conceded to detect the mutation point whereas the magnitude of trend is measured and analysed with the help of Sen’s slope estimator and their significance is tested by MannKendall test. The results revealed that the change is identified by mutation point at initially 1988. Interestingly the maximum growth is captured in the second sub-time series on the basis of time scale and area, yield and production. However, the production of castor cannot meet up its annual enormous demand for India.