Change Point Detection Algorithm Research Articles

A Combined Seasonal Mann–Kendall and Innovative Approach for the Trend Analysis of Streamflow Rate in Two Croatian Rivers

Climate change profoundly impacts hydrological systems, particularly in regions such as Croatia, which is renowned for its diverse geography and climatic variability. This study examined the effect of climate change on streamflow rates in two Croatian rivers: Bednja and Gornja Dobra. Using seasonal Mann–Kendall (MK) tests, overall streamflow trends were evaluated. Additionally, innovative polygon trend analysis (IPTA), innovative visualization for innovative trend analysis (IV-ITA), and Bayesian changepoint detection and time series decomposition (BEAST) algorithms were used to assess the trends’ magnitudes and transitions. The seasonal MK analysis identified significant decreasing trends, primarily during summer. The results of IPTA and IV-ITA revealed consistent decreasing trends throughout most months, with a notable increase in September, especially at high flow values. The rivers’ behavior differed between the first and second halves of the month. BEAST analysis detected abrupt changes, including earlier shifts (1951–1968) in the Bednja and more recent ones (2013–2015) in both the Bednja and, to a lesser extent, the Gornja Dobra rivers. This comprehensive approach enhances our understanding of long-term streamflow trends and short-term fluctuations induced by climate change.

Social and spatial disparities in individuals’ mobility response time to COVID-19: A big data analysis incorporating changepoint detection and accelerated failure time models

Although an increasing number of studies have investigated the lag time between the outbreak of COVID-19 and behavior change, few have accurately measured response times to the epidemic at the individual scale as well as their social and spatial heterogeneities. Using a large-scale, long time series dataset of individual-level mobile phone trajectories from Shenzhen, China, we compared six changepoint detection (CPD) algorithms in terms of their performance in detecting true changepoints (CPs) in time series data of individuals’ daily travel distances. We found that the kernel-based CPD method outperformed other algorithms. We thus adopted this method to calculate Shenzhen residents’ mobility response times to the outbreak of COVID-19 and further used an accelerated failure time (AFT) model to explore factors affecting response times. The results suggest that the average and median mobility response times to the outbreak in Shenzhen were 4.64 days and 4 days, respectively. Males and the elderly responded more slowly to the outbreak, while responses were faster among residents in neighborhoods with a higher percentage of highly educated, married, or employed individuals; with better regional accessibility to the city center, railway stations, or the airport; and with higher residential density. These findings can assist policymakers in determining the policy timeline and re-assessing the effectiveness and equity impact of mobility restriction policies and designing more responsive policies varying by social groups and built environment features, helping build socially-resilient neighborhoods in the post-COVID era.

Analysis of trends and abrupt changes in groundwater and meteorological droughts in the United Kingdom

In recent years, the escalating impact of climate change, coupled with growing concerns regarding groundwater drought, has underscored the critical importance of analyzing hydrological trends. This is a foundational step in advancing effective water resource planning and management. This study presents a regional investigation of groundwater and meteorological droughts in the UK, utilizing the Standardized Groundwater Index (SGI) and the Standardized Precipitation Index (SPI). The analysis encompasses drought duration and an evaluation of the correlation between the SGI and SPI. Subsequently, the seasonal Mann-Kendall (MK) test and the Bayesian Changepoint Detection and Time Series Decomposition (BEAST) algorithm were employed to assess overall trends for both SGI and SPI. Additionally, these methods were utilized to pinpoint trend and seasonal abrupt changes within the time series of both indices.The seasonal MK test revealed statistically significant trends in many SGI stations across the UK, spanning from Northern Scotland down to the South of England, encompassing both increasing and decreasing trends. Additionally, the MK test showed slightly positive trends at the low SPI time scale, though these were not statistically significant. In contrast, more consistent and significant increasing trends were observed throughout the entire UK, except for Northern Scotland, which exhibited negative trends. Despite indications of a progressively more humid UK, the BEAST analysis enabled the detection of decreasing abrupt changes in trends. These changes result in sudden transitions from wet to dry conditions, which can be challenging to identify using the MK test alone. However, BEAST revealed both positive and negative abrupt changes across the entire UK, as well as positive or negative variations in seasonality. This suggests a high degree of variability in seasonal patterns. In summary, this study provides a detailed assessment of groundwater and meteorological drought trends and abrupt changes, considering the impact of climate change on various regions of the UK.

Early warning signals observed in motor activity preceding mood state change in bipolar disorder.

Alterations in motor activity are well-established symptoms of bipolar disorder, and time series of motor activity can be considered complex dynamical systems. In such systems, early warning signals (EWS) occur in a critical transition period preceding a sudden shift (tipping point) in the system. EWS are statistical observations occurring due to a system's declining ability to maintain homeostasis when approaching a tipping point. The aim was to identify critical transition periods preceding bipolar mood state changes. Participants with a validated bipolar diagnosis were included to a one-year follow-up study, with repeated assessments of the participants' mood. Motor activity was recorded continuously by a wrist-worn actigraph. Participants assessed to have relapsed during follow-up were analyzed. Recognized EWS features were extracted from the motor activity data and analyzed by an unsupervised change point detection algorithm, capable of processing multi-dimensional data and developed to identify when the statistical property of a time series changes. Of 49 participants, four depressive and four hypomanic/manic relapses among six individuals occurred, recording actigraphy for 23.8 ± 0.2 h/day, for 39.8 ± 4.6 days. The algorithm detected change points in the time series and identified critical transition periods spanning 13.5 ± 7.2 days. For depressions 11.4 ± 1.8, and hypomania/mania 15.6 ± 10.2 days. The change point detection algorithm seems capable of recognizing impending mood episodes in continuous flowing data streams. Hence, we present an innovative method for forecasting approaching relapses to improve the clinical management of bipolar disorder.

A computational topology-based spatiotemporal analysis technique for honeybee aggregation

A primary challenge in understanding collective behavior is characterizing the spatiotemporal dynamics of the group. We employ topological data analysis to explore the structure of honeybee aggregations that form during trophallaxis, which is the direct exchange of food among nestmates. From the positions of individual bees, we build topological summaries called CROCKER matrices to track the morphology of the group as a function of scale and time. Each column of a CROCKER matrix records the number of topological features, such as the number of components or holes, that exist in the data for a range of analysis scales, at a given point in time. To detect important changes in the morphology of the group from this information, we first apply dimensionality reduction techniques to these matrices and then use classic clustering and change-point detection algorithms on the resulting scalar data. A test of this methodology on synthetic data from an agent-based model of honeybees and their trophallaxis behavior shows two distinct phases: a dispersed phase that occurs before food is introduced, followed by a food-exchange phase during which aggregations form. We then move to laboratory data, successfully detecting the same two phases across multiple experiments. Interestingly, our method reveals an additional phase change towards the end of the experiments, suggesting the possibility of another dispersed phase that follows the food-exchange phase.

A constant-per-iteration likelihood ratio test for online changepoint detection for exponential family models

Online changepoint detection algorithms that are based on (generalised) likelihood-ratio tests have been shown to have excellent statistical properties. However, a simple online implementation is computationally infeasible as, at time T, it involves considering O(T) possible locations for the change. Recently, the FOCuS algorithm has been introduced for detecting changes in mean in Gaussian data that decreases the per-iteration cost to O(logT)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$O(\\log T)$$\\end{document}. This is possible by using pruning ideas, which reduce the set of changepoint locations that need to be considered at time T to approximately logT\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\log T$$\\end{document}. We show that if one wishes to perform the likelihood ratio test for a different one-parameter exponential family model, then exactly the same pruning rule can be used, and again one need only consider approximately logT\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\log T$$\\end{document} locations at iteration T. Furthermore, we show how we can adaptively perform the maximisation step of the algorithm so that we need only maximise the test statistic over a small subset of these possible locations. Empirical results show that the resulting online algorithm, which can detect changes under a wide range of models, has a constant-per-iteration cost on average.

Analysis of SPI index trend variations in the United Kingdom - A cluster-based and bayesian ensemble algorithms approach

Study regionUnited Kingdom (UK). Study focusA regional investigation of the Standard Precipitation Index (SPI) trends and abrupt changes in the UK has been carried out. The K-means algorithm was employed to partition the study area into six homogeneous regions, each distinguished by specific SPI characteristics. Subsequently, the seasonal Mann-Kendall (MK) test and the Bayesian Changepoint Detection and Time Series Decomposition (BEAST) algorithm were used to evaluate the overall trends for each cluster and SPI time scale, as well as to identify abrupt changes in trend and seasonality along the SPI time series, respectively. New hydrological insights for the regionThe seasonal MK test revealed statistically significant increasing SPI trends for all clusters, except for the southeastern area of the UK, where decreasing, but not statistically significant, SPI trends were observed. Moreover, despite a scenario suggesting an increasingly humid UK, the BEAST analysis allowed the detection of decreasing abrupt changes in trends, resulting in sudden changes from wet to dry conditions, that cannot be identified using the MK test. Alongside these, the BEAST analysis has also revealed positive abrupt changes in trends across all UK, as well as positive or negative variations in seasonality, which are followed by longer or shorter wet or dry periods, respectively. Overall, the study approach provides a detailed picture of the SPI trends and abrupt changes, in light of the impact of climate change on the different areas of the UK.

Gamma-Ray Burst Detection with Poisson-FOCuS and Other Trigger Algorithms

We describe how a novel online change-point detection algorithm, called Poisson-FOCuS, can be used to optimally detect gamma-ray bursts within the computational constraints imposed by miniaturized satellites such as the upcoming HERMES-Pathfinder constellation. Poisson-FOCuS enables testing for gamma-ray burst onset at all intervals in a count time series, across all timescales and offsets, in real time and at a fraction of the computational cost of conventional strategies. We validate an implementation with automatic background assessment through exponential smoothing, using archival data from Fermi-GBM. Through simulations of lightcurves modeled after real short and long gamma-ray bursts, we demonstrate that the same implementation has higher detection power than algorithms designed to emulate the logic of Fermi-GBM and Compton-BATSE, reaching the performance of a brute-force benchmark with oracle information on the true background rate, when not hindered by automatic background assessment. Finally, using simulated data with different lengths and means, we show that Poisson-FOCuS can analyze data twice as fast as a similarly implemented benchmark emulator for the historic Fermi-GBM on-board trigger algorithms.

RUL prediction for two-phase degrading systems considering physical damage observations

This paper focuses on a specific type of two-phase degrading system commonly encountered in industrial practice. The first phase is moderate with a low degradation rate while the second is rapid with a high rate. Current studies usually rely solely on sensor measurements to divide phases and predict the remaining useful life (RUL), ignoring the utilization of actual physical damage observations, such as wear depth and crack length. These observations, available during system shutdown periods, directly reflect system states and phase changes. To this end, we propose a novel RUL prediction framework consisting of offline training and online prediction processes. In the offline training process, the physical damage observations and sensor measurements are utilized to estimate the parameters of a two-phase Wiener process and a bijective function matrix. In the online prediction process, real-time sensor measurements are transformed into virtual damage observations for RUL prediction. To enhance the accuracy of phase change point detection, a change point detection algorithm is proposed for both processes. The effectiveness is demonstrated using a simulation and a real case study.

Correction: A semi‑supervised interactive algorithm for change point detection

Correction: A semi‑supervised interactive algorithm for change point detection

A Log-Linear Nonparametric Online Changepoint Detection Algorithm Based on Functional Pruning

A Log-Linear Nonparametric Online Changepoint Detection Algorithm Based on Functional Pruning

Comparison of Supervised Learning and Changepoint Detection for Insect Detection in Lidar Data

Concerns about decreases in insect population and biodiversity, in addition to the need for monitoring insects in agriculture and disease control, have led to an increased need for automated, non-invasive monitoring techniques. To this end, entomological lidar systems have been developed and successfully used for detecting and classifying insects. However, the data produced by these lidar systems create several problems from a data analysis standpoint: the data can contain millions of observations, very few observations contain insects, and the background environment is non-stationary. This study compares the insect-detection performance of various supervised machine learning and unsupervised changepoint detection algorithms and provides commentary on the relative strengths of each method. We found that the supervised methods generally perform better than the changepoint detection methods, at the cost of needing labeled data. The supervised learning method with the highest Matthew’s Correlation Coefficient score on the testing set correctly identified 99.5% of the insect-containing images and 83.7% of the non-insect images; similarly, the best changepoint detection method correctly identified 83.2% of the insect-containing images and 84.2% of the non-insect images. Our results show that both types of methods can reduce the need for manual data analysis.

Increased troponin I blood level and in-hospital outcome across successive COVID-19 waves

Abstract Introduction Troponin I blood level (TnI) is often elevated in patients hospitalized due to COVID-19, and high TnI is associated with adverse outcomes, in this scenario. However, the case rate of high TnI in hospitalized patients and respective case fatality rates (CFR) across different COVID-19 waves still demand investigation, as emerging SARS-CoV-2 variants may affect the cardiac tissue diversely. This study investigated the case rate of high TnI and respective in-hospital outcome in patients hospitalized due to COVID-19 during successive pandemic waves. Methods In a retrospective observational cohort study, patients hospitalized due to COVID-19 from March 1st 2020 to July 31st 2022 in a reference hospital were analyzed. Inclusion criteria were: positive SARS-CoV-2 RT-PCR, hospitalization in a COVID-19 cohort ward or ICU, and at least one TnI assessed during hospitalization. Patients presenting acute coronary syndromes were excluded. TnI was assessed using electrochemiluminescence technique, and the highest value during hospital stay considered for analysis. TnI was positive (+) if above normality reference level and negative (-), otherwise. Outcome was in-hospital survival. Age on admission, total length-of-stay (TLOS), TnI(+) case rate, and CFR (the rate of non-survivors to COVID-19 admissions) were analyzed. COVID-19 waves were analyzed on 7-day moving average positive cases series (1), and respective onset and offset assessed by change point detection algorithm and visual inspection. In each wave period, dominants circulating variants from surveillance data (2,3) were matched. Discrete variables were described as percentage and analyzed by Chi-squared test. Continuous variables were described as mean ± SD, and compared by Student t or 1-factor ANOVA test as appropriate. Bicaudal 95% confidence interval (CI) was calculated as needed. Alpha error level was set at 0.05. Results From March 1st 2020 to July 31st 2022, six COVID-16 waves were identified, and 628 patients met the inclusion criteria. Overall, TnI(+) case rate was 72.0% (95% CI [68.4% - 75.4%]), and CFR was higher among TnI(+) (63.7%) than TnI(-) (32.4%; p < 0.001). As of March 1st 2020, there was a significant trend toward increasing TnI(+) case rate (p trend < 0.001) and decreasing CFR in both TnI(+) (p trend < 0.001) and TnI(-) (p trend < 0.01) across successive COVID-19 waves. Patients with TnI(+) were older (66.1 ± 14.7 years old) and had a longer TLOS (27.8 ± 24.5 days) than with TnI(-) (respectively, 59.0 ± 15.7 years old and 20.6 ± 17.0 days; p < 0.001 for both). TnI(+) case rate, TnI(+)-related and TnI(-)-related CFR, age, and TLOS across COVID-19 waves are in Table 1. Conclusion From March 1st 2020 to July 31st 2022, in patients hospitalized due to COVID-19, TnI(+) was associated with increased case fatality rate as compared to TnI(-), across six successive COVID-19 waves. Patients presenting TnI(+) were older and had longer total length-of-stay as compared to TnI(-).Table 1

Changepoint detection-assisted nonparametric clustering for unsupervised temporal sign segmentation

Temporal sign segmentation aims to temporarily divide continuous sign language into category-agnostic sign segments. One of the challenges to temporal sign segmentation is the paucity of frame-level annotations for sign language videos, which restricts the applicability of supervised and semi-supervised approaches. To address this challenge, we consider temporal sign segmentation as a clustering problem defined as the grouping of semantically similar frames of a given video, which allows us to choose unsupervised clustering techniques as promising alternatives. However, most unsupervised clustering techniques are parametric. In the context of temporal sign segmentation, they assume that the number of sign segments of a given continuous sign video is predefined, which is implausible. The primary contributions of this study are as follows: (1) We propose the first nonparametric clustering algorithm for unsupervised temporal sign segmentation. The main concept is to enhance the hierarchical graph-based clustering algorithm to be nonparametric by adopting cost and penalty functions of a changepoint detection algorithm to determine the optimal number of sign segments. Experimental results show that the performance of the proposed unsupervised method is comparable to that of the latest semi-supervised sign segmentation method in terms of several metrics. Moreover, the execution time of the proposed clustering method was less than 1 s, thereby ensuring its applicability. (2) We identify that the conventional metrics for temporal sign segmentation do not sufficiently address over-segmentation. To overcome the difficulty, we propose a new main metric to evaluate the performance of temporal sign segmentation, called the adjusted MF1B.

Change point detection in text data

The analysis of text data using artificial intelligence and statistical methods has become increasingly important in recent years. One application is the automatic assignment of documents. For this purpose, a classification model is trained on the basis of historical data. If the structure of the texts to be classified changes over time, the quality of the classification will decrease. Change point detection algorithms can counteract this. Such algorithms automatically detect changes in the structure of the texts and indicate that the trained classification model has to be adapted. However, the undesired influence of the length of the document needs to be handled when modeling the text data. We present a multinomial change-point model detecting changes in text structures. The results are supported by simulation studies.

Robust nonparametric multiple changepoint detection for multivariate variability

Two robust, nonparametric multiple changepoint detection algorithms are introduced: DWBS and MKWP. These algorithms can detect multiple changes in the variability of a sequence of independent multivariate observations, even when the number of changepoints is unknown. The algorithms DWBS and MKWP require minimal distributional assumptions and are robust to outlying observations and heavy tails. The DWBS algorithm uses a local search method based on depth-based ranks and wild binary segmentation. The MKWP algorithm estimates changepoints globally via maximizing a penalized version of the classical Kruskal–Wallis ANOVA test statistic. It is demonstrated that this objective function can be maximized via the well-known PELT algorithm. Under mild, nonparametric assumptions, both of these algorithms are shown to be consistent for the correct number of changepoints and the correct location(s) of the changepoint(s). A data driven thresholding method for multivariate data is introduced, based on the Schwartz information criteria. The robustness and accuracy of the new methods is demonstrated with a simulation study, where the algorithms are compared to several existing algorithms. These new methods can estimate the number of changepoints and their locations accurately when the data are heavy tailed or skewed and the sample size is large. Lastly, the proposed algorithms are applied to a four-dimensional sequence of European daily stock returns.

Constructing Sentiment Signal-Based Asset Allocation Method with Causality Information

AbstractThis study demonstrates whether financial text is useful for the tactical asset allocation method using stocks. This can be achieved using natural language processing to create polarity indexes in financial news. We perform clustering of the created polarity indexes using the change point detection algorithm. In addition, we construct a stock portfolio and rebalanced it at each change point using an optimization algorithm. Consequently, the proposed asset allocation method outperforms the comparative approach. This result suggests that the polarity index is useful for constructing the equity asset allocation method.

Event detection in a noisy time series data using static smoothening and gradient variation analytics

Data-driven prognostic industrial asset health management is essential to improve reliability and availability of industrial machineries. Industrial sensors capture multiple phenomena of underlying assets like environment impact, system degradation, process variation, instrument noise, control system response or user induced actions. Some of these phenomena have distinct signature and have impact on component life and remaining useful life. Capturing these events’ signature help to apply advanced AI algorithm to categorize various failure modes and early detection. For Ex: Some of transient events can contribute to thermal cycling of component and in turn reduces life of high temperature and low thermal mass components. Due to random and nonstandard nature of these events, it is extremely challenging to detect and extract these events. Existing change point detection algorithms have limitations to detect sudden variations, which is common due to process or control actions. The noisy signal adds additional challenge to differentiate between important event and noise. In this work of time series analysis, we propose a new approach for consistent estimation of numbers and locations of the change-points. With this tunable algorithm combined with event labelling and pattern search, we can detect events of our functional needs and use them as a feature for our prediction models. This methodology has opened exciting opportunity to further analyse these events with development of classification system and time to fail prediction models and also apply large language models for time series data.

A Time Efficient Factorial Hidden Markov Model-Based Approach for Non-Intrusive Load Monitoring

Assessment of energy consumption behaviour plays an important role in designing demand reduction programs by utility companies. Knowledge of appliance activities in a household aids in conducting the energy consumption behaviour assessment for a community load. Non-intrusive load monitoring (NILM) is a tool that can help in identifying the appliance activities. In this paper, a Modified Factorial Hidden Markov Model (MFHMM) based NILM framework is proposed, which models dependencies among appliance operating states and differential appliance operating states by considering differential in power consumption profiles over time. All the appliances are modelled as individual load models using the Hidden Markov Model (HMM). The appliance operating states are obtained with the application of an iterative k-means clustering algorithm. The aggregated power consumption profile is divided into segments using an optimization-based change-point detection (CPD) algorithm. The NILM problem is solved for each of the segments, and the obtained solution is corrected based on the voltage profile at the aggregated load point. The approach of segmentation and efficient identification of appliance operating states make the model less time complex. Simulations are carried out on publicly available datasets named AMPds, REDD, and UK-DALE. The efficacy of the proposed framework over existing frameworks is evident from the simulation results.

A semi-supervised interactive algorithm for change point detection

A semi-supervised interactive algorithm for change point detection