Online Change-point Detection Research Articles

Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing

As a key technology of enabling Artificial Intelligence (AI) applications in 5G era, Deep Neural Networks (DNNs) have quickly attracted widespread attention. However, it is challenging to run computation-intensive DNN-based tasks on mobile devices due to the limited computation resources. What's worse, traditional cloud-assisted DNN inference is heavily hindered by the significant wide-area network latency, leading to poor real-time performance as well as low quality of user experience. To address these challenges, in this paper, we propose Edgent, a framework that leverages edge computing for DNN collaborative inference through device-edge synergy. Edgent exploits two design knobs: (1) DNN partitioning that adaptively partitions computation between device and edge for purpose of coordinating the powerful cloud resource and the proximal edge resource for real-time DNN inference; (2) DNN right-sizing that further reduces computing latency via early exiting inference at an appropriate intermediate DNN layer. In addition, considering the potential network fluctuation in real-world deployment, Edgentis properly design to specialize for both static and dynamic network environment. Specifically, in a static environment where the bandwidth changes slowly, Edgent derives the best configurations with the assist of regression-based prediction models, while in a dynamic environment where the bandwidth varies dramatically, Edgent generates the best execution plan through the online change point detection algorithm that maps the current bandwidth state to the optimal configuration. We implement Edgent prototype based on the Raspberry Pi and the desktop PC and the extensive experimental evaluations demonstrate Edgent's effectiveness in enabling on-demand low-latency edge intelligence.

Scan B-statistic for kernel change-point detection

Detecting the emergence of an abrupt change-point is a classic problem in statistics and machine learning. Kernel-based nonparametric statistics have been used for this task, which enjoys fewer assumptions on the distributions than the parametric approach and can handle high-dimensional data. In this article, we focus on the scenario when the amount of background data is large and propose a computationally efficient kernel-based statistics for change-point detection, inspired by the recently developed B-statistics. A novel theoretical result of the article is the characterization of the tail probability of these statistics using the change-of-measure technique, which focuses on characterizing the tail of the detection statistics rather than obtaining its asymptotic distribution under the null distribution. Such approximations are crucial to controlling the false alarm rate, which corresponds to the average run length in online change-point detection. Our approximations are shown to be highly accurate. Thus, they provide a convenient way to find detection thresholds for online cases without the need to resort to the more expensive simulations. We show that our methods perform well on both synthetic data and real data.

Offline fitting Markov switching model

Markov regime switching models remain enormously popular in speech recognition, economics, finance, etc. Nonparametric segmentation in switching models without probability assignment of jump moments is in many papers by Brodsky and Darkhovsky. We model all regimes as long SCOT strings. Stochastic C Ontext Tree (abbreviated as SCOT) is m-Markov Chain (m-MC) with every state of a string independent of the symbols in its more remote past than the context of length determined by the preceding symbols of this state. A parallel super-fast fitting and asymptotically optimal inference in a sparse SCOT model including the nonparametric homogeneity test are described in our previous papers. Our segmentation method is a combination of preliminary online change point detection with its subsequent offline Maximal Likelihood update.

Online Change-Point Detection in Sparse Time Series With Application to Online Advertising

Online advertising delivers promotional marketing messages to consumers through online media. Advertisers often have the desire to optimize their advertising spending strategies in order to gain the highest return on investment and maximize their key performance indicator. To build accurate advertisement performance predictive models, it is crucial to detect the change-points in the historical data and apply appropriate strategies to address a data pattern shift problem. However, with sparse data, which is common in online advertising and some other applications, online change-point detection is very challenging. We present a novel collaborated online change-point detection method in this paper. Through efficiently leveraging and coordinating with auxiliary time series, we can quickly and accurately identify the change-points in sparse and noisy time series. Simulation studies as well as real data experiments have justified the proposed method’s effectiveness in detecting change-points in sparse time series. Therefore, it can be used to improve the accuracy of predictive models.

Unsupervised online change point detection in high-dimensional time series

A critical problem in time series analysis is change point detection, which identifies the times when the underlying distribution of a time series abruptly changes. However, several shortcomings limit the use of some existing techniques in real-world applications. First, several change point detection techniques are offline methods, where the whole time series needs to be stored before change point detection can be performed. These methods are not applicable to streaming time series. Second, most techniques assume that the time series is low-dimensional and hence have problems handling high-dimensional time series, where not all dimensions may cause the change. Finally, most methods require user-defined parameters that need to be chosen based on the observed data, which limits their applicability to new unseen data. To address these issues, we propose an Information Gain-based method that does not require prior distributional knowledge for detecting change points and handles high-dimensional time series. The advantages of our proposed method compared to the state-of-the-art algorithms are demonstrated from theoretical basis, as well as via experiments on four synthetic and three real-world human activity datasets.

Bayesian Online Changepoint Detection Of Physiological Transitions.

Transition dynamics between two states can help elucidate the behavior of sequential events in physiological signals. By detecting transitions between healthy and pathological states within individual patients, we can help clinicians focus attention on critical transitions, to either preemptively treat adverse events or to detect changes resulting from treatments. We introduce a novel application of singlepoint Bayesian online changepoint detection to predict clinical state transitions, and apply this framework to detecting pathological transitions in preterm infants with episodes of apnea and bradycardia. Bayesian analysis of sequential physiological events provides insights on how to objectively classify clinically important state transitions that can be triggered by external or intrinsic mechanisms.

Intelligent identification and mitigation of GNSS multipath errors using adaptive BOCPD

Reliable knowledge of the complete navigation states is a key requirement for the mobile mapping and land localization applications. And it is the system integrating strapdown inertial navigation system (SINS) and global navigation satellite system (GNSS) that can seamlessly provide the position, velocity and orientation. In the SINS/GNSS integration, GNSS serves as the accurate reference, however, it is susceptible to the multipath propagations in the dense urban environment where GNSS signals may be reflected by buildings or bridges. When multipath situations occur, the positioning quality of GNSS is dramatically degraded, which also has a negative effect on the stable performance of the integration. Therefore, in order to degrade the negative effect caused by GNSS multipath errors, this paper proposes an intelligent identification and mitigation of GNSS multipath errors method, which employs adaptive bayesian online change point detection (BOCPD) based on the high-precision characteristics of the SINS in the short sampling period. The real tests are conducted, which demonstrate that the proposed approach can effectively identify and mitigate GNSS multipath errors, and then improve the accuracy and reliability of the integration.

Examining nonstationarity in the recruitment dynamics of fishes using Bayesian change point analysis

Marine ecosystems can undergo regime shifts, which result in nonstationarity in the dynamics of the fish populations inhabiting them. The assumption of time-invariant parameters in stock–recruitment models can lead to severe errors when forecasting renewal ability of stocks that experience shifts in their recruitment dynamics. We present a novel method for fitting stock–recruitment models using the Bayesian online change point detection algorithm, which is able to cope with sudden changes in the model parameters. We validate our method using simulations and apply it to empirical data of four demersal fishes in the southern Gulf of St. Lawrence. We show that all of the stocks have experienced shifts in their recruitment dynamics that cannot be captured by a model that assumes time-invariant parameters. The detected shifts in the recruitment dynamics result in clearly different parameter distributions and recruitment predictions between the regimes. This study illustrates how stock–recruitment relationships can experience shifts, which, if not accounted for, can lead to false predictions about a stock’s recovery ability and resilience to fishing.

Optimal Parameter Exploration for Online Change-Point Detection in Activity Monitoring Using Genetic Algorithms.

In recent years, smart phones with inbuilt sensors have become popular devices to facilitate activity recognition. The sensors capture a large amount of data, containing meaningful events, in a short period of time. The change points in this data are used to specify transitions to distinct events and can be used in various scenarios such as identifying change in a patient’s vital signs in the medical domain or requesting activity labels for generating real-world labeled activity datasets. Our work focuses on change-point detection to identify a transition from one activity to another. Within this paper, we extend our previous work on multivariate exponentially weighted moving average (MEWMA) algorithm by using a genetic algorithm (GA) to identify the optimal set of parameters for online change-point detection. The proposed technique finds the maximum accuracy and F_measure by optimizing the different parameters of the MEWMA, which subsequently identifies the exact location of the change point from an existing activity to a new one. Optimal parameter selection facilitates an algorithm to detect accurate change points and minimize false alarms. Results have been evaluated based on two real datasets of accelerometer data collected from a set of different activities from two users, with a high degree of accuracy from 99.4% to 99.8% and F_measure of up to 66.7%.

Detecting regime shifts in fish stock dynamics

Environmental factors such as water temperature, salinity, and the abundance of zooplankton can have major effects on certain fish stocks’ ability to produce juveniles and, thus, stock renewal ability. This variability in stock productivity manifests itself as different productivity regimes. Here, we detect productivity regime shifts by analyzing recruit-per-spawner time series with Bayesian online change point detection algorithm. The algorithm infers the time since the last regime shift (change in mean or variance or both) as well as the parameters of the data-generating process for the current regime sequentially. We demonstrate the algorithm’s performance using simulated recruitment data from an individual-based model and further apply the algorithm to stock assessment estimates for four Atlantic cod (Gadus morhua) stocks obtained from RAM legacy database. Our analysis shows that the algorithm performs well when the variability between the regimes is high enough compared with the variability within the regimes. The algorithm found several productivity regimes for all four cod stocks, and the findings suggest that the stocks are currently in low productivity regimes, which have started during the 1990s and 2000s.

Image-based monitoring for early detection of fouling in crystallisation processes

Fouling or encrustation is a significant problem in continuous crystallisation processes where crystal deposits at surfaces impede heat transfer, increase flow resistance and reduce product quality. This paper proposes an automatic algorithm to detect early stages of fouling using images of vessel surfaces from commodity cameras. Statistical analysis of the pixel intensity variation offers the ability to distinguish appearance of crystals in the bulk solution and on the crystalliser walls. This information is used to develop a fouling metric indicator and determine separately induction times for appearance of first crystals at the surfaces and in the bulk. A method to detect process state changes using Bayesian online change point detection is also proposed, where the first change point is used to determine induction time either at the surface or in the bulk, based on real-time online measurements without using any predetermined threshold which usually varies between experiments and depends on data acquisition equipment. This approach can be used for in situ monitoring of early signs of encrustation to allow early warning for corrective actions to be taken when operating continuous crystallisation processes.

An online copy number variant detection method for short sequencing reads

The availability of the next generation sequencing (NGS) technology in today's biomedical research has provided new opportunities in scientific discovery of genetic information. The high-throughput NGS technology, especially DNA-seq, is particularly useful in profiling a genome for the analysis of DNA copy number variants (CNVs). The read count (RC) data resulting from NGS technology are massive and information rich. How to exploit the RC data for accurate CNV detection has become a computational and statistical challenge. We provide a statistical online change point method to help detect CNVs in the sequencing RC data in this paper. This method uses the idea of online searching for change point (or breakpoint) with a Markov chain assumption on the breakpoints loci and an iterative computing process via a Bayesian framework. We illustrate that an online change-point detection method is particularly suitable for identifying CNVs in the RC data. The algorithm is applied to the publicly available NCI-H2347 lung cancer cell line sequencing reads data for locating the breakpoints. Extensive simulation studies have been carried out and results show the good behavior of the proposed algorithm. The algorithm is implemented in R and the codes are available upon request.

Water quality monitoring with online change-point detection methods

We develop an approach for water quality time series monitoring and contamination event detection. The approach combines affine projection algorithms and an autoregressive (AR) model to predict water quality time series. Then, we apply online change-point detection methods to the estimated residuals to determine the presence, or not, of contamination events. Particularly, we compare the performance of four change-point detection methods, namely, sequential probability ratio test (SPRT), cumulative sum (CUSUM), binomial event discriminator (BED), and online Bayesian change-point detection (OBCPD), by using residuals obtained from four water quality time series, chlorine, conductivity, total organic carbon, and turbidity. Our fundamental criterion for the performance evaluation of the four change-point detection methods is given by the receiver operating characteristic (ROC) curve which is characterized by the true positive rate as a function of the false positive rate. We highlight with detailed experiments that OBCPD provides the best performance for large contamination events, and we also provide insight on the choice of change-point detection algorithms to consider for designing efficient contamination detection schemes.

Accommodating Repair Actions into Gas Turbine Prognostics


 
 
 Elements of gas turbine degradation, such as compressor fouling, are recoverable through maintenance actions like compressor washing. These actions increase the usable engine life and optimise the performance of the gas turbine. However, these maintenance actions are performed by a separate organization to those undertaking fleet management operations, leading to significant uncertainty in the maintenance state of the asset. The uncertainty surrounding maintenance actions impacts prognostic efficacy. In this paper, we adopt Bayesian on-line change point detection to detect the compressor washing events. Then, the event detection information is used as an input to a prognostic algorithm, advising an update to the estimation of remaining useful life. To illustrate the capability of the approach, we demonstrated our on-line Bayesian change detection algorithms on synthetic and real aircraft engine service data, in order to identify the compressor washing events for a gas turbine and thus provide demonstrably improved prognosis.
 
 

On-line changepoint detection and parameter estimation with application to genomic data

An efficient on-line changepoint detection algorithm for an important class of Bayesian product partition models has been recently proposed by Fearnhead and Liu (in J. R. Stat. Soc. B 69, 589---605, 2007). However a severe limitation of this algorithm is that it requires the knowledge of the static parameters of the model to infer the number of changepoints and their locations. We propose here an extension of this algorithm which allows us to estimate jointly on-line these static parameters using a recursive maximum likelihood estimation strategy. This particle filter type algorithm has a computational complexity which scales linearly both in the number of data and the number of particles. We demonstrate our methodology on a synthetic and two real-world datasets from RNA transcript analysis. On simulated data, it is shown that our approach outperforms standard techniques used in this context and hence has the potential to detect novel RNA transcripts.

A new method for change-point detection developed for on-line analysis of the heart beat variability during sleep

We present a novel scaling-dependent measure for times series analysis, the progressive detrended fluctuation analysis (PDFA). Since this method progressively includes and analyzes all data points of the time series, it is suitable for on-line change-point detection: Sudden changes in the statistics of the data points, in the type of correlation or in the statistical variance, or both, are reliably indicated and localized in time. This is first shown for numerous artificially generated data sets of Gaussian random numbers. Also time series with various non-stationarities, such as non-polynomial trends and “spiking”, are included as examples. Although generally applicable, our method was specifically developed as a tool for numerical sleep evaluation based on heart rate variability in the ECG-channel of polysomnographic whole night recordings. It is demonstrated that PDFA can detect specific sleep stage transitions, typically ascending transitions involving sympathetic activation as for example short episodes of wakefulness, and that the method is capable to discern between NREM sleep and REM sleep.

On-line change-point detection (for state space models) using multi-process Kalman filters

In this paper we show how to use an on-line algorithm based on a multi-process-Kalman-filter-extending ideas described in Whittaker and Frühwirth-Schnatter (J. Whittaker, Frühwirth-Schnatter, Appl. Stat. 43 (4) (1994)) — to detect sequential change-points in noisy time series. We focus on types of change-points typically arising in biomedical signals, i.e. jumps or drifts in nonstationary time series possibly corrupted by embedded outliers. The algorithm has been implemented in a program written in Matlab 5.0 and was tested using vital parameters recorded during surgical procedures performed at the University Hospital of the Technical University of Munich, Klinikum Rechts der Isar, Munich.