Detection Of Abrupt Changes Research Articles

Item Pool Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection

In standardized educational testing, test items are reused in multiple test administrations. To ensure the validity of test scores, the psychometric properties of items should remain unchanged over time. In this article, we consider the sequential monitoring of test items, in particular, the detection of abrupt changes to their psychometric properties, where a change can be caused by, for example, leakage of the item or change of the corresponding curriculum. We propose a statistical framework for the detection of abrupt changes in individual items. This framework consists of (1) a multistream Bayesian change point model describing sequential changes in items, (2) a compound risk function quantifying the risk in sequential decisions, and (3) sequential decision rules that control the compound risk. Throughout the sequential decision process, the proposed decision rule balances the trade-off between two sources of errors, the false detection of prechange items, and the nondetection of postchange items. An item-specific monitoring statistic is proposed based on an item response theory model that eliminates the confounding from the examinee population which changes over time. Sequential decision rules and their theoretical properties are developed under two settings: the oracle setting where the Bayesian change point model is completely known and a more realistic setting where some parameters of the model are unknown. Simulation studies are conducted under settings that mimic real operational tests.

Efficient and Robust Online Trajectory Prediction for Non-Cooperative Unmanned Aerial Vehicles

As an important type of dynamic data-driven application system, unmanned aerial vehicles (UAVs) are widely used for civilian, commercial, and military applications across the globe. An increasing research effort has been devoted to trajectory prediction for non-cooperative UAVs to facilitate their collision avoidance and trajectory planning. Existing methods for UAV trajectory prediction typically suffer from two major drawbacks: inadequate uncertainty quantification of the impact of external factors (e.g., wind) and inability to perform online detection of abrupt flying pattern changes. This paper proposes a Gaussian process regression (GPR)-based trajectory prediction framework for UAVs featuring three novel components: 1) GPR with uniform confidence bounds for simultaneous predictive uncertainty quantification, 2) online trajectory change-point detection, and 3) adaptive training data pruning. The paper also demonstrates the superiority of the proposed framework to competing trajectory prediction methods via numerical studies using both simulation and real-world datasets.

Investigating aerosol vertical distribution using CALIPSO time series over the Middle East and North Africa (MENA), Europe, and India: A BFAST-based gradual and abrupt change detection

Monitoring spatio-temporal changes of aerosols is necessary to better understand atmospheric processes. Here, the vertical distribution of aerosols and how it has changed from 2006 to 2017 is studied using time series data from the Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) instrument on board the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite. Aerosol centroid altitude, a variable representing the aerosol vertical extent, is estimated using the aerosol flags of the Vertical Feature Mask (VFM) product of CALIOP. The case study includes four overpasses, with various aerosol types and densities, covering the Middle East and North Africa (MENA), and India regions characterized by frequent dust storms and smoke emissions, respectively, and Europe, a pioneer in establishing air quality policies. The values of aerosol centroid altitude are examined to determine the effects of climate (space) and season (time) on aerosol vertical distribution. For this purpose, 10-year mean values of aerosol centroid altitude were calculated over climate regions extracted from the Food and Agriculture Organization (FAO) climate map as well as over each season, and the resultant values were then statistically analyzed. The results indicate that environmental heat, steam from climate or season, leads to aerosols increasing in altitude. The time series was imported in a version of the Breaks For Additive Season and Trend (BFAST) algorithm, so-called BFAST01, allowing the magnitude of gradual changes along with the time and type of the major abrupt change to be determined. The strongest positive trends in aerosol centroid altitude were recorded in the desert regions. Positive trends were also observed in the mid-latitudes steppes and the North Sea. India was another region with a relatively positive trend which could be mainly caused by the increasing anthropogenic activities and smoke emission. The strongest negative trend detected in Europe is a result of strict environmental regulation policies and efforts for air quality improvement. The results show that a small fraction of granules (12 out of 4036) mainly in the northern part of overpass A, over Europe, the mid part of overpasses B and C, over the Middle East, and southern part of overpass D, over India, experienced structural changes. The reversal changes type 8 (50% of the abrupt changes) and interruption type 6 (33%) are the dominant types, mostly occurred around 2014 and 2012, respectively. Changing environmental factors such as temperature, wind speed, and anthropogenic activities are the main drivers of these changes. The study demonstrates the applicability of BFAST01 and the usability of CALIOP VFM time series data to monitor the vertical structure of the atmosphere.

Abrupt Change Detection Method Based on Features of Lorenz Trajectories

This paper presents a definition of bifurcation-type abrupt changes based on the bifurcation features of Lorenz trajectories. These abrupt changes are the result of the transition behavior of dynamical system trajectories among different equilibrium regions. We demonstrate that these bifurcation-type jumps can better reflect the nature of abrupt change. In analyzing the features of Lorenz equation trajectories, a dynamical method for detecting bifurcation-type abrupt changes is presented. A numerical solution of the Lorenz equation is adopted, using a curve integral or vector product to construct a time series of positive and negative values. Changes in the sign of this time series accurately determine whether the trajectory is in the right or left equilibrium region, and the points at which the time series is equal to zero are the times at which the trajectory jumps between different equilibrium regions, that is, the occurrence times of bifurcation-type abrupt changes. This method is completely dependent on the dynamical characteristics of the system. A theoretical approach for detecting abrupt climate changes based on the dynamical characteristics of the atmospheric model is described. Compared with the original method of identifying abrupt climate changes, this method has dynamic significance and can detect abrupt changes in multi-dimensional time series. Although this method can be applied theoretically, applications to real atmospheric data first require the data to be smoothed.

Information Geometric Theory in the Prediction of Abrupt Changes in System Dynamics.

Detection and measurement of abrupt changes in a process can provide us with important tools for decision making in systems management. In particular, it can be utilised to predict the onset of a sudden event such as a rare, extreme event which causes the abrupt dynamical change in the system. Here, we investigate the prediction capability of information theory by focusing on how sensitive information-geometric theory (information length diagnostics) and entropy-based information theoretical method (information flow) are to abrupt changes. To this end, we utilise a non-autonomous Kramer equation by including a sudden perturbation to the system to mimic the onset of a sudden event and calculate time-dependent probability density functions (PDFs) and various statistical quantities with the help of numerical simulations. We show that information length diagnostics predict the onset of a sudden event better than the information flow. Furthermore, it is explicitly shown that the information flow like any other entropy-based measures has limitations in measuring perturbations which do not affect entropy.

A New Approach to Detection of Changes in Multidimensional Patterns - Part II

Abstract In the paper we develop an algorithm based on the Parzen kernel estimate for detection of sudden changes in 3-dimensional shapes which happen along the edge curves. Such problems commonly arise in various areas of computer vision, e.g., in edge detection, bioinformatics and processing of satellite imagery. In many engineering problems abrupt change detection may help in fault protection e.g. the jump detection in functions describing the static and dynamic properties of the objects in mechanical systems. We developed an algorithm for detecting abrupt changes which is nonparametric in nature and utilizes Parzen regression estimates of multivariate functions and their derivatives. In tests we apply this method, particularly but not exclusively, to the functions of two variables.

Effects of Intermittent Water Allocation on Vegetation Dynamics in Lake Baiyangdian, North China

The ecological effects of intermittent water supplement in Lake Baiyangdian were analyzed with an NDVI time series based on the data from 1 April 1998 to 31 July 2008. Four NDVI categories were extracted, and twelve stages were grouped based on the specific replenishment time of water allocation. STR (a seasonal-trend decomposition procedure based on regression) and its corresponding linear regression and abrupt change detection were used to highlight the temporal variations in vegetation under the effects of water allocation. ANOVA was used to identify the response quality of the vegetation in each stage. The results showed that the significant increasing trend of each NDVI category was detected by linear regression (p < 0.001). The 95% abrupt change intervals showed uncertainty and complexity in the immediate vegetation response to water division times. The most abrupt changes were detected in the growing seasons. The extremely significant differences in NDVI variation before and after water allocations were detected by ANOVA (p < 0.001, t-test). The longer the duration of water allocation was maintained, the higher the detection of vegetation coverage was relatively. The increased vegetation coverage benefited from the constant water supply. Our results provide evidence of vegetation responses to intermittent water divisions. The positive effects of water allocation on regional ecological restoration were confirmed. This study strengthened the water division effects of regional vegetation restoration and provided a practical strategy for an effective implementation of water allocation.

Cyber-Attack Detection in Water Distribution Systems Based on Blind Sources Separation Technique

Service quality and efficiency of urban systems have been dramatically boosted by various high technologies for real-time monitoring and remote control, and have also gained privileged space in water distribution. Monitored hydraulic and quality parameters are crucial data for developing planning, operation and security analyses in water networks, which makes them increasingly reliable. However, devices for monitoring and remote control also increase the possibilities for failure and cyber-attacks in the systems, which can severely impair the system operation and, in extreme cases, collapse the service. This paper proposes an automatic two-step methodology for cyber-attack detection in water distribution systems. The first step is based on signal-processing theory, and applies a fast Independent Component Analysis (fastICA) algorithm to hydraulic time series (e.g., pressure, flow, and tank level), which separates them into independent components. These components are then processed by a statistical control algorithm for automatic detection of abrupt changes, from which attacks may be disclosed. The methodology is applied to the case study provided by the Battle of Attack Detection Algorithms (BATADAL) and the results are compared with seven other approaches, showing excellent results, which makes this methodology a reliable early-warning cyber-attack detection approach.

Change-point detection: application of Cusum method to real life data

Several procedures have been developed for the detection of abrupt changes in time series. Among these procedures, it can be mentioned the Cumulative Sum (Cusum) type method. It is in such a perspective that Katchekpele et al. (2017) proposed a method using a Cusum type test to detect a change-point in the unconditional variance of the generalised autoregressive conditional heteroskedasticity(GARCH) models. The aim of this paper is to present an application of their technique. After briefly recalling how the test statistic was constructed, the change-point detection algorithm is given and it is shown how it is applied to some real life data.

Adaptive Commissioning Methodology for Arc-Flash Mitigation in Steel Making Users

This article proposes a new adaptive commissioning methodology for arc flash mitigation in steel manufacturer users considering the Tap-to-Tap varying current consumption. This process is achieved through adaptative protection settings computed to reduce the total clearing time of relays adding the restriction of the incident energy. The proposed methodology carries out the detection of abrupt changes in current consumption to detect Tap-to-Tap change in operation stages of electric arc furnaces. The application of this methodology is performed on a steel manufacturer user and shows the reduction of the incident energy values according to Tap-to-Tap operation cycle.

Determining the most accurate program for the Mann-Kendall method in detecting climate mutation

In the detection of abrupt changes in a time sequence using the Mann-Kendall method, two programs (PRGM1 and PRGM2) were found to provide two different mutation points for the same temperature sequence. The code of PRGM1 was programmed according to the step-by-step direction of the Mann-Kendall method described in the original report, and PRGM2 is the self-included program therein. To determine the reason for the different calculation results between the two programs for the same method and the same time series, and thereby verify the correctness of the programs, this study performed some analyses. First, the original reference, in which the basic principle of the Mann-Kendall method was put forward and developed, was reviewed to find the original mathematical formula of the Mann-Kendall method. Then, the mutation points calculated by PRGM1, PRGM2, and additional methods of detecting climate mutation were comparatively analyzed. The results show that the self-compiled program (i.e., PRGM1) and the self-included program (i.e., PRGM2) have different definitions of their main statistics when calculating the rank of a time series. Mutation points obtained by other methods were found to be consistent with those calculated by PRGM1 but different from those calculated by PRGM2. This proves that the definition of the main statistics for the rank of a sequence in PRGM1 is correct. Certain problems still exist in the definition of the main statistics for the rank of a sequence in PRGM2.

Study on the characteristics of water level during the flood season in the Poyang Lake, China

Characteristics analysis on the water level in the Poyang Lake, China during the flood season in 1956-2012 are presented, including the trend test, abrupt change detection and periodic identification. The Mann-Kendall test and Spearman’s rho test were conducted with results showing no significant trend in both the flood season averaged WLs and monthly averaged WLs in the Poyang Lake. The sequential Mann-Kendall test and Pettitt test were carried out and showed no significant abrupt change in trend of WLs as well. Besides, the Morlet continuous wavelet transform was applied to identify the periodic characteristics in trend of WLs. The results indicated that there were three types scale of periodic component, in which the periodic oscillation in 35 years was the first main period of the annual flood season averaged WLs with basically two oscillations, while the periodic oscillation in 11 years and 5 years were the second and third main periods, respectively.

Quantifying attribution of runoff change for major rivers in China

The recorded runoff of the major rivers in China has been changing due to environment change. It is of significance to quantitatively analyze attribution of runoff change for supporting water resources assessment and management in many practices. Based on abrupt change detection of annual runoff series from 1956—2018 measured at the 8 representative hydrometric stations on major rivers in China, impacts of climate change and human activities on runoff changes were investigated by using hydrological simulation approach. Results show that (1) The abrupt change of annual runoff series recorded at the Wujiadu station on the Huai River, Datong station on the Yangtze River and the Wuzhou staion on the Pearl River are not detectable. However, the abrupt changes have been found for the recorded annual runoff series of the major rivers in North China with the earliest and the latest abrupt change years occurring in 1965 for the Tieling station on the Liao River and in 2003 for the Haerbin station on the Songhua River, respectively. The relationships between runoff and precipitation for periods before and after abrupt change year were markedly changed probably due to human activities. (2) RCCC-WBM model not only performs well on monthly runoff simulation for humid South China, but also achieves good simulation result for major rivers in arid North China. The model has been proved being qualified to naturalize runoff in the human-disturbed periods for major rivers in China. (3) In general, human-induced runoff changes are higher than the climatic impacts for major rivers in the North China, while climate change is a dominant driver of runoff variation for the Hai River and major rivers of China in its south.

Trend analysis and abrupt change detection of streamflow variations in the lower Tigris River Basin, Iraq

ABSTRACT The Tigris River Basin (TRB) is one of the major freshwater resources for Iraq. However, water shortages of the TRB increasingly have become one of the serious problems facing Iraq. The study was carried out to investigate the temporal and spatial variations of the annual and seasonal streamflow and understand the key driving factors for streamflow changes. Therefore, various statistical methods for detecting trends and abrupt changes were applied to long-term streamflow, temperature and precipitation data over the past nine decades along the TRB within Iraq. The findings indicated that: (1) a significant decreasing trend in winter, spring, and summer flows was the major reason leading to a negative annual trend detected in all hydrometric stations; (2) the flow regime of the Tigris River has altered significantly at an annual scale since 1973 (about 40% of the stations) after the construction of large dams in the upstream catchments and (3) the impacts of climate change have been clearly observed in most of the stations since the 1990s, as supported by the increasing trends in all mean temperatures and decreasing trends in approximately 50% of precipitation (p = 0.05). As a conclusion, it can be stated that the reduction in streamflow can be linked mainly to the increased dam construction, reduced precipitation, and increased temperature. The results presented in this study provide a better understanding to help basin managers, hydrologists and policy-makers in making better anthropogenic and climate change mitigation plans in the TRB in Iraq.

Reliable leak detection in a heat exchanger of a sodium-cooled fast reactor

The heat exchanger (HE) should be permanently monitored in order to detect water or nitrogen leaking into a sodium circuit, which can affect the sodium-cooled fast reactor (SFR) performance or safety. The HE must be shut down in a few seconds after the beginning of the leak, even very small, and repaired. The solution developed in the paper is based on the reliable detection of abrupt changes in the spectral density of measured vibro-acoustic signals. The main difficulty lies in the fact that the small HE leaks have to be detected in the presence of high normal operating noise coming from different equipment (pumps, turbine, etc.). The records from experimental mock-ups dedicated to sodium-gas HE studies and installed at the Commissariat à l’energie atomique et aux energies alternatives (CEA) have been used to assess the proposed solution. The significance of this research with respect to the previous works is twofold. First, the proposed solution detects a small leak in the sodium-gas HE with the signal-to-noise ratio SNR⩾-30 dB provided that the detection delay is upper bounded by 60 s, the probability of missed detection is upper bounded by 10-5 and the probability of false alarm (per hour) is upper bounded by 10-5h-1, which is much better than the usually requested SNR of -17 dB. Second, the analytical formulas for calculation of the probabilities of missed detection and false alarm have been proposed that guarantees the required statistical properties of the proposed test.

The Network Link Outlier Factor (NLOF) for Fault Localization

We describe and experimentally evaluate the performance of our Network Link Outlier Factor (NLOF) for locating faults in communication networks. The NLOF is a unique outlier score assigned to each link in a network. It is computed using four distinct stages in a data analytics pipeline. The input to the pipeline are flow records (e.g., NetFlow) and network topology data (e.g., Link Layer Discovery Protocol (LLDP)). In the first stage, flow record throughput values are clustered in two sub-stages: using Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and then our novel domain-specific ThroughPut Cluster (TPCluster) technique. In the second stage, flow outlier scores are determined within each cluster using a measure of proximity to a selected performance exemplar. In the third stage, flows are associated with network links using topology data. Finally, in the fourth stage the flow outliers are used to compute the outlier factor or score for each network link. The network link outlier scores are used with a detection rule to locate faults. We present the results of a wide set of Mininet experiments that appraise the fault detection/localization performance of NLOF. We find that NLOF allows for the detection of errors on edge links with a simple detection rule and the detection of errors on core links with a rule that includes topology relationships. NLOF is also compared to an abrupt change detection technique; while both have roughly the same detection power, the precision of NLOF is 42% higher and NLOF required 40% less time to detect failures on average.

Kernel-Based Edge-Preserving Methods for Abrupt Change Detection

Abrupt change detection is critical to monitor the occurrence of abnormal events from sensor data for situational awareness of complex systems. However, various disturbances and noises applied to the data observations may pose significant challenges to the robustness of many abrupt change detection methods. Recent researches have shown that bilateral filter can acquire outstanding performance on removing noises from images while preserving edge information. In this letter, we propose two improved edge-preserving memory-based cumulative sum (MB-CUSUM) methods that are able to make the abrupt change detection method more robust against noises. Our experimental studies show that the proposed methods can achieve superior performance over state-of-the-art methods to detect abrupt changes, which demonstrates the effectiveness and feasibility of their practical use.

Rural Communities’ Perceptions of Climate Change and Its Impacts on Their Livelihood in Southeastern Niger

Climate change exacerbates ecosystem degradation in Sahel where rural communities are highly dependent on natural resources and ecosystem services for their livelihoods. In order cope with the adverse effects of climate change by developing climate adaptation strategies, there has been a need to understand how communities perceive the change or variability in their local climate and how this change could affect their livelihood. This paper aims to examined rural communities’ perceptions about climate change in the commune of Chetimari located in the region of Diffa, southeastern Niger. It investigated particularly: 1) how communities’ perceptions of climate change? 2) What are the impacts of climate change on the livelihood of communities? And 3) What do elements and options to detect abrupt change points in annual precipitations and both annual maximum and minimum temperature? Survey data were collected from October to November 2018 from 101 households (15% of the total households in the study area) in three villages. Multiple correspondence Analysis (MCA) and Factorial Correspondence Analysis (FCA) were performed with XLSTAT to analyzed data from survey. Meteorological data including Monthly precipitation from 1981 to 2017, and Monthly temperatures (maximum, minimum) from 1986 to 2017 of the Diffa meteorological station were analyzed using R to perform the Man Kendal trend test and the Pettit test for detection of abrupt changes in the series. Results showed that community perceptions on temperature increase (not for precipitation) trends as indicators of climate change are in agreement with meteorologically observed trends. The findings showed that people in this zone are aware of climate change that they see as increase in maximum and minimum temperature, decrease in rainfall, frequency of extreme events (drought and flood), dry spells, decrease in number of rainy days, strong winds etc…Results revealed the most significant impacts of climate change affecting the livelihood of rural communities in this area. These are higher risk of crop damage from drought, farmer and herder conflict frequency, drying up of wells, food shortage/insecurity, silting pools, decline in soil fertility and livestock production, silting pasture areas, frequency of livestock diseases, increased weed and invasive species, livestock mortality. The perception of the climate change and its impacts on the main socioeconomic activities vary from a village to another, according the sexes and age ranges.
 We conclude that communities are relatively aware about the climate change issue. For a better management of climate-induced risks in the study area we stress the need to improve the awareness of climate change within the rural community by improving the availability and the quality of relevant climate information.

Comparison of different methods for detecting change points in hydroclimatic time series

An abrupt change is an important manifestation of hydroclimatic variability. Accurate detection of change points is a critical issue in hydroclimatic and climate change studies. In the article, we evaluated the performances of 12 methods (including both parametric and non-parametric) for detecting change points in hydroclimatic time series by considering the influences of eight major factors. Different methods exhibited different efficiencies and eight of the methods performed better which are recommended for application. Furthermore, the mean values of series and locations of change points were found to have little influence on the detection of change points. However, for a time series with smaller variance but bigger skewness and larger difference in the mean values before and after the change point, the abrupt changes can be more easily and accurately detected. Detection of change points in shorter series would have larger uncertainty. Based on the Monte-Carlo experiments, the efficiency of each method was quantified and its capability was quantitatively clarified. Detection of abrupt changes in precipitation over Southwest China showed that the Indian monsoon had a dominant influence on precipitation in the regions south of 30°N and west of 110°E. Since 2007 the Indian monsoon has maintained a weakening pattern, causing a decrease in precipitation on the Yunnan-Guizhou Plateau, which is one of the main causes of frequently occurring droughts. Results of this study can be a useful reference for choosing a method to detect change points in hydroclimate time series, and be an important complement for the detection and attribution of hydroclimatic variability.

Four decades of water and sediment discharge records in Subarnarekha and Burhabalang basins: an approach towards trend analysis and abrupt change detection

The outcomes of trend analyses, abrupt change detection of discharge, sediment load data of Subarnarekha River for the last four decades are presented in this paper and possible causes of these changes are discussed. Bivariate linear regression, Mann–Kendall non-parametric test, Pettitt test have been implemented on the daily discharge and sediment concentration data provided by Water Resources Information System of India (India-WRIS). A statistically significant decreasing trend of discharge is found in the Subarnarekha upstream, whereas other stations in downstream show an insignificant increasing trend during the past four decades. Sediment load shows a decreasing trend in the entire study area where the change is significant in downstream regions since last four decades. An abrupt downward shift on the discharge is found only in case of Muri station from the year 2000, located in the upstream of Subarnarekha main river. Abrupt changes on the sediment load were found in the downstream after the period of construction of dams on Subarnarekha River. More precisely, sediment load had started declining progressively from 1998 in the Subarnarekha River and since 2010 in its adjacent river, Burhabalang.