Types Of Drift Research Articles

Revisiting streaming anomaly detection: benchmark and evaluation

Anomaly detection in streaming data is an important task for many real-world applications, such as network security, fraud detection, and system monitoring. However, streaming data often exhibit concept drift, which means that the data distribution changes over time. This poses a significant challenge for many anomaly detection algorithms, as they need to adapt to the evolving data to maintain high detection accuracy. Existing streaming anomaly detection algorithms lack a unified evaluation framework that validly assesses their performance and robustness under different types of concept drifts and anomalies. In this paper, we conduct a systematic technical review of the state-of-the-art methods for anomaly detection in streaming data. We propose a new data generator, called SCAR (Streaming data generator with Customizable Anomalies and concept dRifts), that can synthesize streaming data based on synthetic and real-world datasets from different domains. Furthermore, we adapt four static anomaly detection models to the streaming setting using a generic reconstruction strategy as baselines, and then compare them systematically with 9 existing streaming anomaly detection algorithms on 76 synthesized datasets that have various types of anomalies and concept drifts. The challenges and future research directions for anomaly detection in streaming data are also presented.

A perturbative treatment of the Yarkovsky-driven drifts in the 2:1 mean motion resonance

Aims. Our aim is to gain a qualitative understanding as well as to perform a quantitative analysis of the interplay between the Yarkovsky effect and the Jovian 2:1 mean motion resonance under the planar elliptic restricted three-body problem. Methods. We adopted the semi-analytical perturbation method valid for arbitrary eccentricity to obtain the resonance structures inside the Jovian 2:1 resonance. We averaged the Yarkovsky force so it could be applied to the integrable approximations for the 2:1 resonance and the ν5 secular resonance. The rates of Yarkovsky-driven drifts in the action space were derived from the quasi-integrable approximations perturbed by the averaged Yarkovsky force. Pseudo-proper elements of test particles inside the 2:1 resonance were computed using N-body simulations incorporated with the Yarkovsky effect to verify the semi-analytical results. Results. In the planar elliptic restricted model, we identified two main types of systematic drifts in the action space: (Type I) for orbits not trapped in the ν5 resonance, the footprints are parallel to the resonance curve of the stable center of the 2:1 resonance; (Type II) for orbits trapped in the ν5 resonance, the footprints are parallel to the resonance curve of the stable center of the ν5 resonance. Using the semi-analytical perturbation method, a vector field in the action space corresponding to the two types of systematic drifts was derived. The Type I drift with small eccentricities and small libration amplitudes of 2:1 resonance can be modeled by a harmonic oscillator with a slowly varying parameter, for which an analytical treatment using the adiabatic invariant theory was carried out.

Current-controlled sedimentation in a megatransform system: a case study of the Charlie-Gibbs Fracture Zone

The Charlie-Gibbs Fracture Zone (CGFZ) is the main transform system in the North Atlantic Ocean. It serves as a primary deep-water gateway for the Iceland-Scotland Overflow Water (ISOW) current. Using high-resolution seismo-acoustic profiling, three types of contourite drift were identified in the valleys of the inactive fractures and active transform of the CGFZ: (1) channel-related drifts, (2) confined drift, and (3) plastered drift. The ISOW current is the main agent controlling drift formation in the northern valley and eastern part of the southern valley, while the Denmark Strait Overflow Water or Low Deep Water is suggested for the western part of the southern valley. Examination of two sediment cores shows that the contourite drifts at least at their upper parts consist of alternated muddy and silty contourites and pelagic/hemipelagic sediments. The contourite deposition corresponds to the present interglacial interval. During glacial interval (MIS 3-2) and the last glacial/interglacial transition, pelagic/hemipelagic sedimentation prevailed corresponding with high IRD input. The position of the highly productive Subarctic Front over the study area induced the formation of diatom ooze beds at the beginning of MIS 1 and high-carbonate pelagites during some intervals of MIS 3.

Variance Feedback Drift Detection Method for Evolving Data Streams Mining

Learning from changing data streams is one of the important tasks of data mining. The phenomenon of the underlying distribution of data streams changing over time is called concept drift. In classification decision-making, the occurrence of concept drift will greatly affect the classification efficiency of the original classifier, that is, the old decision-making model is not suitable for the new data environment. Therefore, dealing with concept drift from changing data streams is crucial to guarantee classifier performance. Currently, most concept drift detection methods apply the same detection strategy to different data streams, with little attention to the uniqueness of each data stream. This limits the adaptability of drift detectors to different environments. In our research, we designed a unique solution to address this issue. First, we proposed a variance estimation strategy and a variance feedback strategy to characterize the data stream’s characteristics through variance. Based on this variance, we developed personalized drift detection schemes for different data streams, thereby enhancing the adaptability of drift detection in various environments. We conducted experiments on data streams with various types of drifts. The experimental results show that our algorithm achieves the best average ranking for accuracy on the synthetic dataset, with an overall ranking 1.12 to 1.5 higher than the next-best algorithm. In comparison with algorithms using the same tests, our method improves the ranking by 3 to 3.5 for the Hoeffding test and by 1.12 to 2.25 for the McDiarmid test. In addition, they achieve a good balance between detection delay and false positive rates. Finally, our algorithm ranks higher than existing drift detection methods across the four key metrics of accuracy, CPU time, false positives, and detection delay, meeting our expectations.

Adaptive bagging-based dynamic ensemble selection in nonstationary environments

The classification of nonstationary data streams presents a substantial challenge mainly due to the simultaneous presence of class imbalance and concept drift. Ensemble learning is a widely acknowledged approach to address these two problems. Nevertheless, most of the existing methods establish independent strategies for each, ignoring their relationship and interactions, which prevents the corresponding strategies from performing as expected. This paper presents a novel algorithm AB-DES (Adaptive Bagging-based Dynamic Ensemble Selection), which employs bagging technique and dynamic ensemble selection system. In AB-DES, instances are stored to balance subsequent data chunks instead of relying solely on resampling methods. Then, all base classifiers are updated using a new dual adaptive sampling bagging strategy, which can help to reduce the classification inaccuracies involving by class imbalance. In the prediction phase, DES system constructs different ensemble models for different instances. The inclusion of a sliding window-based drift detector makes the above two strategies adapt to concept drift. Experimental results show that AB-DES achieves higher classification accuracy and exhibits enhanced robustness compared to 8 state-of-the-art methods on 10 synthetic datasets and 5 real world datasets with different types of concept drift.

An experimental review of the ensemble-based data stream classification algorithms in non-stationary environments

Data streams are sequences of fast-growing and high-speed data points that typically suffer from the infinite length, large volume, and specifically unstable data distribution. Ensemble learning as a prevalent classification approach is widely used in data stream mining studies. Besides the impressive performance of ensemble learning algorithms in providing a collection of diverse and accurate classifiers, they are specifically efficient in handling non-stationary data streams. Due to the component-based nature and the chance of dynamic updates for the components of the ensemble, this category is appropriate for dynamically learning the changing concepts of the data. Several review research have been conducted on the challenges of non-stationary data streams so far. None of the available surveys are dedicated to examine the effect of ensemble learning models on concept drift handling. This paper aims to provide a thorough theoretical and experimental review of the most significant ensemble-based data stream classification approaches in confronting with various types of concept drifts. In this comprehensive experimental analysis, 21 state-of-the-art ensemble algorithms are tested on 30 synthetic datasets under various types and volumes of concept drifts. This experimental analysis involves analyzing the classification accuracy, kappa score and running time of the algorithms, along with performing various statistical tests. In addition to conducting experimental analysis, a comprehensive investigation is carried out on the most significant challenges posed by non-stationary data streams, which provides the reader with valuable insights.

A systematic review on detection and adaptation of concept drift in streaming data using machine learning techniques

AbstractLast decade demonstrate the massive growth in organizational data which keeps on increasing multi‐fold as millions of records get updated every second. Handling such vast and continuous data is challenging which further opens up many research areas. The continuously flowing data from various sources and in real‐time is termed as streaming data. While deriving valuable statistics from data streams, the variation that occurs in data distribution is called concept drift. These drifts play a significant role in a variety of disciplines, including data mining, machine learning, ubiquitous knowledge discovery, quantitative decision theory, and so forth. As a result, a substantial amount of research is carried out for studying methodologies and approaches for dealing with drifts. However, the available material is scattered and lacks guidelines for selecting an effective technique for a particular application. The primary novel objective of this survey is to present an understanding of concept drift challenges and allied studies. Further, it assists researchers from diverse domains to accommodate detection and adaptation algorithms for concept drifts in their applications. Overall, this study aims to contribute to deeper insights into the classification of various types of drifts and methods for detection and adaptation along with their key features and limitations. Furthermore, this study also highlights performance metrics used to evaluate the concept drift detection methods for streaming data. This paper presents the future research scope by highlighting gaps in the existing literature for the development of techniques to handle concept drifts.This article is categorized under: Algorithmic Development > Ensemble Methods Application Areas > Data Mining Software Tools Fundamental Concepts of Data and Knowledge > Big Data Mining

Multi-type concept drift detection under a dual-layer variable sliding window in frequent pattern mining with cloud computing

The detection of different types of concept drift has wide applications in the fields of cloud computing and security information detection. Concept drift detection can indeed assist in promptly identifying instances where model performance deteriorates or when there are changes in data distribution. This paper focuses on the problem of concept drift detection in order to conduct frequent pattern mining. To address the limitation of fixed sliding windows in adapting to evolving data streams, we propose a variable sliding window frequent pattern mining algorithm, which dynamically adjusts the window size to adapt to new concept drifts and detect them in a timely manner. Furthermore, considering the challenge of existing concept drift detection algorithms that struggle to adapt to different types of drifting data simultaneously, we introduce an additional dual-layer embedded variable sliding window. This approach helps differentiate types of concept drift and incorporates a decay model for drift adaptation. The proposed algorithm can effectively detect different types of concept drift in data streams, perform targeted drift adaptation, and exhibit efficiency in terms of time complexity and memory consumption. Additionally, the algorithm maintains stable performance, avoiding abrupt changes due to window size variations and ensuring overall robustness.

BenchMFC: A benchmark dataset for trustworthy malware family classification under concept drift

Concept drift poses a critical challenge in deploying machine learning models to mitigate practical malware threats. It refers to the phenomenon that the distribution of test data changes over time, gradually deviating from the original training data and degrading model performance. A promising direction for addressing concept drift is to detect drift samples and then retrain the model. However, this field currently lacks a unified, well-curated, and comprehensive benchmark, which often leads to unfair comparisons and inconclusive outcomes. To improve the evaluation and advance further, this paper presents a new Benchmark dataset for trustworthy Malware Family Classification (BenchMFC), which includes 223 K samples of 526 families that evolve over years. BenchMFC provides clear family, packer, and timestamp tags for each sample, it thus can support research on three types of malware concept drift: 1) unseen families, 2) packed families, and 3) evolved families. To collect unpacked family samples from large-scale candidates, we introduce a novel crowdsourcing malware annotation pipeline, which unifies packing detection and family annotation as a consensus inference problem to prevent costly packing detection. Moreover, we provide two case studies to illustrate the application of BenchMFC in 1) concept drift detection and 2) model retraining. The first case demonstrates the impact of three types of malware concept drift and compares nine notable concept drift detectors. The results show that existing detectors have their own advantages in dealing with different types of malware concept drift, and there is still room for improvement in malware concept drift detection. The second case explores how static feature-based machine learning operates on packed samples when retraining a model. The experiments illustrate that packers do preserve some kind of signals that appear to be “effective” for machine learning models, but the robustness of these signals requires further research. BenchMFC has been released to the community at https://github.com/crowdma/benchmfc.

Autonomic active learning strategy using cluster-based ensemble classifier for concept drifts in imbalanced data stream

Systems are becoming autonomous nowadays. In real-world scenarios, the involvement of experts in generating a new decision has to be paid a certain amount of cost. Thus, in these conditions, creating a system with the least expert involvement and costs is the main destination. As a solution, an autonomic active learning strategy with a cluster-based ensemble classifier for concept drifts in imbalanced data stream (AACE-DI) is proposed in this paper. Initially, a cluster-based initializing formula is considered to select the most informative instances for labeling a dynamically changing ensemble classifier. Next, the automatically adjusting uncertainty strategy is introduced to prioritize the most uncertain, representative, and minority class data from imbalanced data stream, which also ensures the least labeling cost. This automatic formula not only outperforms the system but also makes it autonomic. Experimental evaluations are organized using 6 real-world data streams and 16 synthetic data streams with different types of concept drift and imbalanced ratios. To measure performance, evaluators, such as the prequential area under curve, recall, and geometric mean, are considered. Moreover, the proposed AACE-DI method achieves expected outcomes with the least cost as compared with other state-of-the-art learning methods.

Stream learning under concept and feature drift: A literature survey

<p class="Abstract">Stream data learning is an emerging machine learning topic, and it has many challenges. One of its challenges is the dynamic behavior or changes in the environment which leads to drifts. Two types of drift occur, namely, concept drift and feature drift. This article provides a survey on stream data learning with focusing on the issues of feature drift and the methods developed for handling it. After presenting the fundamental concepts and definition in this field, it provides an overview of the various models and methods developed for detecting feature drift and maintaining the validity of the machine learning models when the drift occurs. Furthermore, the article provides the generators used for creating dataset with feature drift to provide benchmarking for approaches of detecting or handling feature drift. The article provides also taxonomy of feature selection methods in both static and dynamic environment. It concludes that reinforcement-based models are promising for this task, and it lists various open challenges and future works in this area.</p>

Benchmarking Change Detector Algorithms from Different Concept Drift Perspectives

The stream mining paradigm has become increasingly popular due to the vast number of algorithms and methodologies it provides to address the current challenges of Internet of Things (IoT) and modern machine learning systems. Change detection algorithms, which focus on identifying drifts in the data distribution during the operation of a machine learning solution, are a crucial aspect of this paradigm. However, selecting the best change detection method for different types of concept drift can be challenging. This work aimed to provide a benchmark for four drift detection algorithms (EDDM, DDM, HDDMW, and HDDMA) for abrupt, gradual, and incremental drift types. To shed light on the capacity and possible trade-offs involved in selecting a concept drift algorithm, we compare their detection capability, detection time, and detection delay. The experiments were carried out using synthetic datasets, where various attributes, such as stream size, the amount of drifts, and drift duration can be controlled and manipulated on our generator of synthetic stream. Our results show that HDDMW provides the best trade-off among all performance indicators, demonstrating superior consistency in detecting abrupt drifts, but has suboptimal time consumption and a limited ability to detect incremental drifts. However, it outperforms other algorithms in detection delay for both abrupt and gradual drifts with an efficient detection performance and detection time performance.

Data drift in medical machine learning: implications and potential remedies.

Data drift refers to differences between the data used in training a machine learning (ML) model and that applied to the model in real-world operation. Medical ML systems can be exposed to various forms of data drift, including differences between the data sampled for training and used in clinical operation, differences between medical practices or context of use between training and clinical use, and time-related changes in patient populations, disease patterns, and data acquisition, to name a few. In this article, we first review the terminology used in ML literature related to data drift, define distinct types of drift, and discuss in detail potential causes within the context of medical applications with an emphasis on medical imaging. We then review the recent literature regarding the effects of data drift on medical ML systems, which overwhelmingly show that data drift can be a major cause for performance deterioration. We then discuss methods for monitoring data drift and mitigating its effects with an emphasis on pre- and post-deployment techniques. Some of the potential methods for drift detection and issues around model retraining when drift is detected are included. Based on our review, we find that data drift is a major concern in medical ML deployment and that more research is needed so that ML models can identify drift early, incorporate effective mitigation strategies and resist performance decay.

Towards Reliable Online Just-in-Time Software Defect Prediction

Throughout its development period, a software project experiences different phases, comprises modules with different complexities and is touched by many different developers. Hence, it is natural that problems such as Just-in-Time Software Defect Prediction (JIT-SDP) are affected by changes in the defect generating process (concept drifts), potentially hindering predictive performance. JIT-SDP also suffers from delays in receiving the labels of training examples (verification latency), potentially exacerbating the challenges posed by concept drift and further hindering predictive performance. However, little is known about what types of concept drift affect JIT-SDP and how they affect JIT-SDP classifiers in view of verification latency. This work performs the first detailed analysis of that. Among others, it reveals that different types of concept drift together with verification latency significantly impair the stability of the predictive performance of existing JIT-SDP approaches, drastically affecting their reliability over time. Based on the findings, a new JIT-SDP approach is proposed, aimed at providing higher and more stable predictive performance (i.e., reliable) over time. Experiments based on ten GitHub open source projects show that our approach was capable of produce significantly more stable predictive performances in all investigated datasets while maintaining or improving the predictive performance obtained by state-of-art methods.

Abnormal Data Recovery of Structural Health Monitoring for Ancient City Wall Using Deep Learning Neural Network

ABSTRACT Continuous structural health monitoring is of great importance to preventive conservation for ancient architectural heritages. However, abnormal monitoring data may trigger false alarming of structural damages. SHM of ancient buildings also needs abnormal data recovering. Most of the existing studies used the neural network with single input dimension and forward prediction to recover abnormal data, which is difficult to accurately predict long data sequences. This study developed a novel abnormal data recovery framework. The main novelty of the proposed framework is that the input and output configurations of the GRU model are optimized. Meanwhile, to make full use of the forward and backward information of the abnormal data sequence, bidirectional prediction is used to improve the prediction accuracy. The framework is implemented in the abnormal monitoring data recovering for an ancient city wall built 600 years ago in Beijing. Three types of abnormal data, including outlier, drift, and missing, are considered in this study. The results reveal that the proposed framework has high accuracy in abnormal data recovering of strain and crack width. The recovered data has the same regular diurnal variation as the normal monitoring data. The linear correlation between the structural responses and wall temperature gets obviously improved after data recovering. The proposed framework shows great capacity of abnormal data recovery for structural static responses of ancient buildings, which are usually influenced by environmental temperature variation.

Handling concept drift in deep learning applications for process monitoring

As machine learning applications transcend from laboratories to real-world operations in manufacturing use cases such as intelligent maintenance and quality control, questions regarding their continuous reliability and robustness arise. Static datasets used to develop machine learning models can only capture a subset of possible real-world conditions. Instances of concept drift, such as changing environmental-, equipment- and operating conditions may, over time, significantly degrade the performance of machine learning models, compromising safety, acceptance, and economics if not properly addressed. It is thus necessary to (1) detect drifts and (2) efficiently adapt the model to dynamically changing conditions. In this work, we propose a framework to tackle the issue of drift detection by utilizing the underlying neural network's uncertainty. We demonstrate the framework's effectiveness in a case study involving process monitoring of a real-world CNC milling process. The dataset includes accelerometer data and multiple types of realistic concept drifts. In addition, we introduce a methodology for inducing controllable, synthetic concept drift through a simulated sensor rotation. Our experiments show that the introduced drifts lead to a strong performance degradation of the model, which highlights the relevance for practitioners. All drifts are detected using the proposed framework, enabling continuously reliable applications.

Online AutoML: an adaptive AutoML framework for online learning

Automated Machine Learning (AutoML) has been used successfully in settings where the learning task is assumed to be static. In many real-world scenarios, however, the data distribution will evolve over time, and it is yet to be shown whether AutoML techniques can effectively design online pipelines in dynamic environments. This study aims to automate pipeline design for online learning while continuously adapting to data drift. For this purpose, we design an adaptive Online Automated Machine Learning (OAML) system, searching the complete pipeline configuration space of online learners, including preprocessing algorithms and ensembling techniques. This system combines the inherent adaptation capabilities of online learners with the fast automated pipeline (re)optimization capabilities of AutoML. Focusing on optimization techniques that can adapt to evolving objectives, we evaluate asynchronous genetic programming and asynchronous successive halving to optimize these pipelines continually. We experiment on real and artificial data streams with varying types of concept drift to test the performance and adaptation capabilities of the proposed system. The results confirm the utility of OAML over popular online learning algorithms and underscore the benefits of continuous pipeline redesign in the presence of data drift.

EnHAT — Synergy of a tree-based Ensemble with Hoeffding Adaptive Tree for dynamic data streams mining

The goal of this paper is to improve the predictive accuracy of data streaming algorithms without increasing the processing time of the incoming data. We propose the EnHAT (Ensemble Combined with Hoeffding Adaptive Tree) algorithm, which combines the state-of-the-art Hoeffding Adaptive Tree (HAT) algorithm with an ensemble of J48 decision trees induced from sequential chunks of the data stream. The slack time of HAT adaptation to a new window of incoming records is utilized in parallel for building a decision-tree ensemble. In our experiments on 4 benchmark streaming datasets and 4 synthetic datasets with different types of concept drift, EnHAT has reached the highest predictive accuracy in 23 out of 26 cases compared to an ensemble of J48 trees, HAT, and a single J48 model induced from the last sliding window. Thus, we can conclude that the ensemble/HAT synergy yields better prediction results than each one of the two approaches on its own. The higher accuracy does not come at the expense of any additional computational effort beyond the model induction times of the combined algorithms.

A multi-components approach to monitoring process structure and customer behaviour concept drift

Concept drifts within business processes are viewed as variations in the business circumstances, such as structural and behavioural changes in the control-flow, which necessitate process refinement and model updating. Existing approaches, such as relation-based precedence rules, tuned to detect drifts in the process structure are often not well suited to detecting changes in customer behaviour. This paper proposes a concept drift detector employing multi-components originating from Discrete-time Markov chains to detect, localize and reason about concept drifts in both process structure and customer behaviour of the control-flow. The approach was compared with three commonly used methods using 52 artificial event logs representing various types of drift (sudden and gradual, structural and behavioural). Experimental results demonstrated desirable performance with average F1 scores of 0.871 and 0.893 under structural and behavioural drifts, respectively. The approach was also employed in a real-life hospital billing dataset. The main contribution of this paper is a concept drift detector that is able to detect and explain root causes of control-flow changes whether such variations occurred suddenly or gradually.

Contourite and Turbidite Features in the Middle Caspian Sea and Their Connection to Geohazards Derived from High-Resolution Seismic Data

High fluvial input combined with specific topographic and oceanographic settings in the Caspian Sea create favorable conditions for contourite deposition. For the first time in its middle portion, contourite deposits have been observed in high-resolution seismic profiles. Various types of contourite drifts and mixed depositional systems have been revealed on the lower slope and in the adjacent basin, some of which are accompanied by sediment wave fields. The deposition of contourites or turbidites and their lateral distribution is controlled by sea-floor topography and oceanographic processes, as well as the modern activity of gravity flows downslope on the western Caucasian slope and in the channel system on the Mangyshlak Sill. The contourite drifts and sediment wave fields form several contourite depositional systems, which seem to merge in the Caspian contourite depositional complex. This occurs near the foot of slopes of the Derbent Basin and is related to the counterclockwise circum-Caspian current in the Middle Caspian Sea. The fact that the Caspian Sea is the largest lake in the world makes this region a significant area for research into the “lake contourites” issue. The Caspian Sea is an important oil-producing area, and sedimentary processes related to the contourite and turbidite can be a source of potential geohazards in the construction and exploitation of underwater engineering structures