Change Point Detection Algorithm Research Articles

Wavelet Methods for the Detection of Anomalies and their Application to Network Traffic Analysis

AbstractHere we develop an integrated tool for the online detection of network anomalies. We consider statistical change point detection algorithms, for both local changes in the variance and for the detection of jumps, and propose modified versions of these algorithms based on moving window techniques. We investigate performances on simulated data and on network traffic data with several superimposed attacks. All detection methods are based on wavelet packet transforms. Copyright © 2006 John Wiley & Sons, Ltd.

Short-Term Traffic Flow Prediction with Regime Switching Models

Accurate short-term prediction of traffic parameters is a critical component for many intelligent transportation system applications. Traffic flow is subject to abrupt disturbances because of various unexpected events (e.g., accidents, weather-induced disruption) that may change the underlying dynamics and the stability of the data generation process. Short-term prediction models that do not account for these changes produce biased and less accurate predictions. This paper proposes a new adaptive approach to short-term prediction that explicitly accounts for occasional regime changes by using statistical change-point detection algorithms. In this context, the expectation maximization and the CUSUM (cumulative sum) algorithms are implemented to detect shifts in the mean level of the process in real time. Autoregressive integrated moving average models are used for developing the forecasting models while the process mean is monitored by the two detection algorithms. The intercept of the forecasting models is updated on the basis of the detected shifts in the mean level to adapt to any potential new regimes. The proposed approach is tested on real-world loop data sets. The results show significant improvements in prediction accuracy compared with traditional autoregressive integrated moving average models with fixed parameters.

Short-Term Traffic Flow Prediction with Regime Switching Models

Accurate short-term prediction of traffic parameters is a critical component for many intelligent transportation system applications. Traffic flow is subject to abrupt disturbances because of various unexpected events (e.g., accidents, weather-induced disruption) that may change the underlying dynamics and the stability of the data generation process. Short-term prediction models that do not account for these changes produce biased and less accurate predictions. This paper proposes a new adaptive approach to short-term prediction that explicitly accounts for occasional regime changes by using statistical change-point detection algorithms. In this context, the expectation maximization and the CUSUM (cumulative sum) algorithms are implemented to detect shifts in the mean level of the process in real time. Autoregressive integrated moving average models are used for developing the forecasting models while the process mean is monitored by the two detection algorithms. The intercept of the forecasting models is updated on the basis of the detected shifts in the mean level to adapt to any potential new regimes. The proposed approach is tested on real-world loop data sets. The results show significant improvements in prediction accuracy compared with traditional autoregressive integrated moving average models with fixed parameters.

Application of anomaly detection algorithms for detecting SYN flooding attacks

We investigate statistical anomaly detection algorithms for detecting SYN flooding, which is the most common type of Denial of Service (DoS) attack. The two algorithms considered are an adaptive threshold algorithm and a particular application of the cumulative sum (CUSUM) algorithm for change point detection. The performance is investigated in terms of the detection probability, the false alarm ratio, and the detection delay, using workloads of real traffic traces. Particular emphasis is on investigating the tradeoffs among these metrics and how they are affected by the parameters of the algorithm and the characteristics of the attacks. Such an investigation can provide guidelines to effectively tune the parameters of the detection algorithm to achieve specific performance requirements in terms of the above metrics.

Adaptive change point detection algorithm for heart rate in children

METHODS: Following institutional ethical approval, trend heart rate data was collected from 53 children. Clinically detected changes in heart rate, using standard auditory and visual monitoring, were recorded in synchrony with the trend data. A purpose-built graphical interface was used for post-hoc expert marking of episodes of heart rate increase; graded as definitely or likely significant/ insignificant or artifact using predefined criteria. Following data segmentation, an automated change-point detection algorithm, with adaptive Kalman filtering and a local CUSUM, was used to identify points of increasing heart rate. The relative performance of the algorithm and real-time clinicians was compared against the post-hoc expert review.

Change-point detection in neuronal spike train activity

Animals respond to changes in their environment based on the information encoded in neuronal spike activity. One key issue is to determine how quickly and reliably the system can detect that a behaviorally relevant change has taken place. What are the neural mechanisms and computational principles that allow fast, reliable detection of changes in spike activity? Here we present an optimal statistical signal-processing algorithm for change-point detection, known as the cumulative sum (CUSUM) algorithm. We then show that the performance of a simple neuron model with leaky-integrate-and-fire dynamics can approach theoretically optimal performance limits under certain conditions.

Echo-shadow combination in ultrasonic defectoscopy

The long-term impact of rapid urbanization on air temperature (Ta), relative humidity (RH), vapor pressure (VP) and human thermal comfort in the Cairo governorate of Egypt was analyzed. Land use change (LUC) between 1973 and 2017 were derived from Landsat satellite data. Next, non-parametric change point and trend detection algorithms were applied to Ta, RH and VP over 1950–2017 to estimate the impacts of urbanization on urban climate. Three historical thermal comfort indices: temperature humidity index (THI), effective temperature index (ETI) and relative strain index (RSI) were estimated from climate data collected between 1950 and 2017 to assess the impact of urbanization on human thermal comfort. The results reveal substantial LUC, rapid increasing impervious surface areas in low-lying areas of Cairo at 75.2 km2/decade since the 1990s. Rapid urbanization had resulted in a statistically significant change point in Ta after 1995 with a warming trend of 0.19 °C/decade, a negative trend in RH of 0.55%/decade and a rising trend in VP of 0.24 hPa/decade. Severe heat stress levels emerged and persisted every July–September since 1994. THI, ETI, and RSI show statistically significant change points at 1994 and a rising trend of 0.33 °C/decade, 0.29 °C/decade, and 0.06/decade, respectively. The highest thermal discomfort risk was found in urban areas of the old Cairo, but the risk is marginally smaller at new cities where there are vegetation covers. This study clearly demonstrates the impacts of rapid urbanization on the urban climate of hot-arid environment.