Concept Drift Research Articles

CREDIT CARD FRAUD DETECTION USING MACHINE LEARNING ALGORITHMS

The Credit card frauds represent facile and amiable targets, particularly as the proliferation of e-commerce and various online platforms has led to a commensurate expansion in online payment modalities, thereby augmenting the susceptibility to online fraudulent activities. In response to the escalating rates of fraudulent incidents, researchers have undertaken the utilization of diverse machine learning methodologies to ascertain and scrutinize frauds within the realm of online transactions. The primary objective of this scholarly endeavor is to formulate and advance an innovative fraud detection modus operandi tailored for Streaming Transaction Data. The overarching goal is to meticulously scrutinize the historical transaction particulars of patrons and distill discernible behavioral patterns. This involves the clustering of cardholders into disparate cohorts predicated upon their transactional magnitudes. Subsequently, a sliding window strategy is employed to amalgamate transactions conducted by cardholders across distinct groups, facilitating the extraction of their respective behavioral patterns. Consecutively, diverse classifiers trained on these distinct groups, and the classifier exhibiting superior rating scores is earmarked as one of the preeminent methods for prognosticating fraudulent activities. This is succeeded by the implementation of a feedback mechanism aimed at mitigating the challenges posed by the phenomenon of concept drift. The empirical investigation detailed in this paper is grounded in the analysis of a European credit card fraud dataset. It is imperative for credit card companies to adeptly discern instances of fraudulent credit card transactions to preclude customers from incurring charges for items they did not legitimately acquire. The resolution to such quandaries lies in the realm of Data Science, a discipline whose significance, when coupled with Machine Learning, is of paramount importance. This undertaking endeavors to elucidate the construction of a model utilizing machine learning techniques for Credit Card Fraud Detection. The Credit Card Fraud Detection Problem entails the modeling of historical credit card transactions, incorporating data from those that transpired as fraudulent. Subsequently, this model is employed to ascertain the veracity of new transactions, distinguishing between fraudulent and non-fraudulent activities. The primary aim is the meticulous detection of 100% of fraudulent transactions, while concurrently minimizing instances of erroneous classifications of non-fraudulent transactions. Credit Card Fraud Detection serves as a quintessential exemplar of classification challenges. In the course of this endeavor, significant emphasis has been placed on the analysis and pre-processing of datasets. Furthermore, a diverse array of anomaly detection algorithms, including the Local Outlier Factor and Isolation Forest algorithm, have been deployed on Principal Component Analysis (PCA) transformed Credit Card Transaction data. KEYWORDS: Card-Not-Present frauds, Card-Present-Frauds, Concept Drift,,Credit card fraud, applications of machine learning, data science.

Dynamic Industrial Optimization: A Framework Integrates Online Machine Learning for Processing Parameters Design

We develop the online process parameter design (OPPD) framework for efficiently handling streaming data collected from industrial automation equipment. This framework integrates online machine learning, concept drift detection and Bayesian optimization techniques. Initially, concept drift detection mitigates the impact of anomalous data on model updates. Data without concept drift are used for online model training and updating, enabling accurate predictions for the next processing cycle. Bayesian optimization is then employed for inverse optimization and process parameter design. Within OPPD, we introduce the online accelerated support vector regression (OASVR) algorithm for enhanced computational efficiency and model accuracy. OASVR simplifies support vector regression, boosting both speed and durability. Furthermore, we incorporate a dynamic window mechanism to regulate the training data volume for adapting to real-time demands posed by diverse online scenarios. Concept drift detection uses the EI-kMeans algorithm, and the Bayesian inverse design employs an upper confidence bound approach with an adaptive learning rate. Applied to single-crystal fabrication, the OPPD framework outperforms other models, with an RMSE of 0.12, meeting precision demands in production.

Online learning for streaming data classification in nonstationary environments

AbstractIn this article, we implement the classification of nonstationary streaming data. Due to the inability to obtain full data in the context of streaming data, we adopt a strategy based on clustering structure for data classification. Specifically, this strategy involves dynamically maintaining clustering structures to update the model, thereby updating the objective function for classification. Simultaneously, incoming samples are monitored in real‐time to identify the emergence of new classes or the presence of outliers. Moreover, this strategy can also deal with the concept drift problem, where the distribution of data changes with the inflow of data. Regarding the handling of novel instances, we introduce a buffer analysis mechanism to delay their processing, which in turn improves the prediction performance of the model. In the process of model updating, we also introduce a novel renewable strategy for the covariance matrix. Numerical simulations and experiments on datasets show that our method has significant advantages.

Drift Adaptive Online DDoS Attack Detection Framework for IoT System

Internet of Things (IoT) security is becoming important with the growing popularity of IoT devices and their wide applications. Recent network security reports revealed a sharp increase in the type, frequency, sophistication, and impact of distributed denial of service (DDoS) attacks on IoT systems, making DDoS one of the most challenging threats. DDoS is used to commit actual, effective, and profitable cybercrimes. The current machine learning-based IoT DDoS attack detection systems use batch learning techniques, and hence are unable to maintain their performance over time in a dynamic environment. The dynamicity of heterogeneous IoT data causes concept drift issues that result in performance degradation and automation difficulties in detecting DDoS. In this study, we propose an adaptive online DDoS attack detection framework that detects and adapts to concept drifts in streaming data using a number of features often used in DDoS attack detection. This paper also proposes a novel accuracy update weighted probability averaging ensemble (AUWPAE) approach to detect concept drift and optimize zero-day DDoS detection. We evaluated the proposed framework using IoTID20 and CICIoT2023 dataset containing benign and DDoS traffic data. The results show that the proposed adaptive online DDoS attack detection framework is able to detect DDoS attacks with an accuracy of 99.54% and 99.33% for the respective datasets.

Online semi-supervised active learning ensemble classification for evolving imbalanced data streams

Concept drift is a core challenge in classification tasks of data streams. Although many drift adaptation methods have been presented, most of them assume that labels of all data are available, which is impractical in many real-world applications. Additionally, the absence of label makes the imbalance ratio of an imbalanced data stream difficultly being obtained in time, providing the inaccurate guidance for resampling and causing poor generalization. To tackle the joint challenges, an online semi-supervised active learning method is proposed to classifier imbalanced data streams with concept drift. A newly-arrived data is first added to the sliding window, and then assigned a pseudo label in terms of its nearest cluster. Meanwhile, semi-supervised clustering algorithm offers its predicted label. Based on the above two predictive labels, cluster-based query strategy provides the criteria for the evaluation and selection of representative instances. More especially, the uncertainty and importance of instances are defined to synthetically evaluate its representativeness. After obtaining true labels of typical ones, ensemble classifier is updated by all instances in current sliding window. Experimental results on 13 synthetic and real data streams indicate that the proposed method outperforms six comparative methods on both G-mean and Recall under various labeling budgets.

Searching by Topological Complexity: Lightweight Neural Architecture Search for Coal and Gangue Classification

Searching by Topological Complexity: Lightweight Neural Architecture Search for Coal and Gangue Classification

Concept Drift–Based Intrusion Detection For Evolving Data Stream Classification In IDS: Approaches And Comparative Study

Abstract Static machine and deep learning algorithms are commonly used in intrusion detection systems (IDSs). However, their effectiveness is constrained by the evolving data distribution and the obsolescence of the static data sources used for model training. Consequently, static classifiers lose efficacy, necessitating expensive model retraining with time. The aim is to develop a dynamic and adaptable IDS that mitigates the limitations of static models, ensuring real-time threat detection and reducing the need for frequent, resource-intensive model retraining. This research proposes an approach that amalgamates the adaptive random forest (ARF) classifier with Hoeffding’s bounds and a moving average test for the early and accurate detection of network intrusions. The ARF can adapt in real time to shifting network conditions and evolving attack patterns, constantly refining its intrusion detection capabilities. Furthermore, the inclusion of Hoeffding’s bounds and the moving average test adds a dimension of statistical rigor to the system, facilitating the timely recognition of concept drift and distinguishing benign network variations from potential intrusions. The synergy of these techniques results in reduced false positives and false negatives, thereby enhancing the overall detection rate. The proposed method delivers outstanding results, with 99.95% accuracy and an impressive 99.96% recall rate on the latest CIC-IDS 2018 dataset, outperforming the results of existing approaches.

Entropy-based concept drift detection in information systems

As time passes, the data within information systems may continuously evolve, causing the target concept to drift. To ensure the effectiveness of data-driven decision making, it is crucial to detect drift in a timely manner and gather relevant information. In this paper, we introduce two methods that can directly detect concept drift in the provided information system, by considering a new perspective on uncertainty. First, using entropy under a single attribute constraint, we define the uncertainty of the target concept in an information system. By integrating the uncertainty of each attribute, the overall uncertainty of the target concept in the information system is obtained. Subsequently, two concept drift detection methods are proposed, namely EBTBM (Entropy-Based Threshold-Based Method) and EBSBM (Entropy-Based Sampling-Based Method). These methods utilize the defined uncertainty of the target concept as a statistical measure of the difference between two data blocks. Finally, extensive experiments on artificial and real-world data sets are conducted to validate the effectiveness of the proposed concept drift detection methods.

Online Learning for Imbalanced Data Streams with Concept Drift by Belief Theory and Chaotic Function

Online Learning for Imbalanced Data Streams with Concept Drift by Belief Theory and Chaotic Function

Adaptive regularized ensemble for evolving data stream classification

Extracting knowledge from data streams requires fast incremental algorithms that are able to handle unlimited processing and ever-changing data with finite memory. A strategy for this challenge is the use of ensembles owing to their ability to tackle concept drift and achieve highly accurate predictions. However, ensembles often require a lot of computational resources. In this study, we propose a novel ensemble-based classification algorithm for data streams, Adaptive Regularized Ensemble (ARE), with low demand for computational resources. The algorithm combines strategies that contribute to high prediction accuracy using only incorrectly classified instances into the training step, random-sized feature subspace for each ensemble element and classifier selection for final ensemble voting. After an extensive experimental study, ARE exhibited high predictive performance and outperformed state-of-the-art ensembles on data streams for real and synthetic datasets while requiring a low processing time and memory consumption.

Dynamic Submodular-Based Learning Strategy in Imbalanced Drifting Streams for Real-Time Safety Assessment in Nonstationary Environments.

The design of real-time safety assessment (RTSA) approaches in nonstationary environments is meaningful to reduce the possibility of significant losses. However, several challenging problems are needed to be well considered. The performance of existing approaches will be negatively affected in the settings of imbalanced drifting streams. In this case, the model design with the incremental update should also be explored. Furthermore, the query strategy should also be well-designed. This article investigates a dynamic submodular-based learning strategy to address such issues. Specifically, an efficient incremental update procedure is designed with the structure of the broad learning system (BLS), which is beneficial to the detection of concept drift. Furthermore, a novel dynamic submodular-based annotation with an activation interval strategy is proposed to select valuable samples in imbalanced drifting streams. The lower bound of annotation value is also proven theoretically with a novel drift adaption mechanism. Numerous experiments are conducted with the realistic data of JiaoLong deep-sea manned submersible. The experimental results show that the proposed approach can achieve better assessment accuracy than typical existing approaches.

Dynamic selection of machine learning models for time-series data

Adapting to changes in time-series data is a major challenge in machine learning. This problem is particularly acute in the case of limited computing power (e.g., edge devices) that does not enable the independent training of new models. While multiple studies attempt to adequately train a machine learning model on rapidly shifting data (i.e., concept drift), the challenge of dynamically selecting the most effective machine learning model for future time steps has been largely overlooked. In this study, we propose Adaptive machine learning for Dynamic ENvironments (ADEN ), a method for analyzing future trends in time-series data and selecting the most suitable ML model. Our approach models multiple aspects of the analyzed data and analyzes the behavior of multiple machine learning models on earlier time steps. By training ADEN on multiple datasets, we can deploy a zero-shot model that does not require additional training when applied to new datasets. Our evaluation, conducted on 46 time-series classification datasets, shows that ADEN not only outperforms all baselines in terms of average performance but is also capable of avoiding sudden drops in performance that characterize all other evaluated algorithms.

Anticipative Bayesian classification for data streams with verification latency

Most of the existing adaptive classification algorithms in non-stationary data streams require recent labelled data for their updates. Such recent labels are often missing. For stream classification under verification latency only few approaches exist. Most of them assume clustered data or homogeneous drift in all features, which limits their applicability. We address this by proposing Anticipative Bayesian stream Classifier (ABClass), an approach that is capable of integrating and automatically selecting from different components. In its Bayesian classification framework, ABClass combines density estimation techniques, extended to extrapolate drift patterns over time, with unsupervised parameter tuning and unsupervised model selection. ABClass allows for multivariate density estimation and extrapolation techniques. In this work, we assume conditional independence between features given the class label for modelling feature-specific drift patterns. ABClass is generative and can also be used for explaining and visualising concept drift patterns. It is generic, making it easy to include further types of drift models, both for the class-conditional feature distribution and for the class prior distribution. The experimental evaluation on several real-world data streams shows its competitiveness compared to other state-of-the-art approaches. ABClass is in most cases ten- to hundred-times faster than its competitors, both for model fitting and for prediction.

Explainable data stream mining: Why the new models are better

Explainable Machine Learning brings expandability, interpretability, and accountability to Data Mining Algorithms. Existing explanation frameworks focus on explaining the decision process of a single model in a static dataset. However, in data stream mining changes in data distribution over time, called concept drift, may require updating the learning models to reflect the current data environment. It is therefore important to go beyond static models and understand what has changed among the learning models before and after a concept drift. We propose a Data Stream Explanability framework (DSE) that works together with a typical data stream mining framework where support vector machine models are used. DSE aims to help non-expert users understand model dynamics in a concept drifting data stream. DSE visualizes differences between SVM models before and after concept drift, to produce explanations on why the new model fits the data better. A survey was carried out between expert and non-expert users on the effectiveness of the framework. Although results showed non-expert users on average responded with less understanding of the issue compared to expert users, the difference is not statistically significant. This indicates that DSE successfully brings the explanability of model change to non-expert users.

A comprehensive analysis of concept drift locality in data streams

Adapting to drifting data streams is a significant challenge in online learning. Concept drift must be detected for effective model adaptation to evolving data properties. Concept drift can impact the data distribution entirely or partially, which makes it difficult for drift detectors to accurately identify the concept drift. Despite the numerous concept drift detectors in the literature, standardized procedures and benchmarks for comprehensive evaluation considering the locality of the drift are lacking. We present a novel categorization of concept drift based on its locality and scale. A systematic approach leads to a test bed of 2760 data stream benchmarks, reflecting various difficulty levels following our proposed categorization. We conduct a comparative assessment of 9 state-of-the-art drift detectors across diverse difficulties, highlighting their strengths and weaknesses for future research. We examine how drift locality influences the classifier performance and propose strategies for different drift categories to minimize the recovery time. Lastly, we provide lessons learned and recommendations for future concept drift research. Our benchmark data streams and experiments are publicly available at https://github.com/gabrieljaguiar/locality-concept-drift.

GDJN: Global density joined network for abnormal event detection in surveillance videos

Detecting abnormal events in surveillance involves identifying unexpected behavior through video analysis. This involves recognizing patterns or deviations from normal behavior and taking actions to mitigate potential risks. However, the distribution of data can change over time, leading to concept drift, which can make it challenging to accurately detect abnormal events. To address this issue, a new approach using a global density network (GDN) has been proposed. The GDN allows for more efficient identification of object distributions in surveillance videos, leading to improved accuracy in abnormal event detection. The proposed method combines features extracted by a backbone network with a global density joined network (GDJN), which refines density features using dilated convolutional networks. A multistage long short-term memory (LSTM) network is then used to classify abnormal events. The experimental results are conducted on two datasets, UMN and UCSD Ped2. The achieved F1 scores were 93.42 and 94.46 respectively, with corresponding AUC values of 93.5 and 94.8.

Dealing with Drift of Adaptation Spaces in Learning-based Self-Adaptive Systems Using Lifelong Self-Adaptation

Recently, machine learning (ML) has become a popular approach to support self-adaptation. ML has been used to deal with several problems in self-adaptation, such as maintaining an up-to-date runtime model under uncertainty and scalable decision-making. Yet, exploiting ML comes with inherent challenges. In this article, we focus on a particularly important challenge for learning-based self-adaptive systems: drift in adaptation spaces. With adaptation space, we refer to the set of adaptation options a self-adaptive system can select from to adapt at a given time based on the estimated quality properties of the adaptation options. A drift of adaptation spaces originates from uncertainties, affecting the quality properties of the adaptation options. Such drift may imply that the quality of the system may deteriorate, eventually, no adaptation option may satisfy the initial set of adaptation goals, or adaptation options may emerge that allow enhancing the adaptation goals. In ML, such a shift corresponds to a novel class appearance, a type of concept drift in target data that common ML techniques have problems dealing with. To tackle this problem, we present a novel approach to self-adaptation that enhances learning-based self-adaptive systems with a lifelong ML layer. We refer to this approach as lifelong self-adaptation . The lifelong ML layer tracks the system and its environment, associates this knowledge with the current learning tasks, identifies new tasks based on differences, and updates the learning models of the self-adaptive system accordingly. A human stakeholder may be involved to support the learning process and adjust the learning and goal models. We present a general architecture for lifelong self-adaptation and apply it to the case of drift of adaptation spaces that affects the decision-making in self-adaptation. We validate the approach for a series of scenarios with a drift of adaptation spaces using the DeltaIoT exemplar.

A Novel Neural Ensemble Architecture for On-the-fly Classification of Evolving Text Streams

We study on-the-fly classification of evolving text streams in which the relation between the input data and target labels changes over time—i.e., “concept drift.” These variations decrease the model’s performance, as predictions become less accurate over time and they necessitate a more adaptable system. While most studies focus on concept drift detection and handling with ensemble approaches, the application of neural models in this area is relatively less studied. We introduce Adaptive Neural Ensemble Network ( AdaNEN ), a novel ensemble-based neural approach, capable of handling concept drift in data streams. With our novel architecture, we address some of the problems neural models face when exploited for online adaptive learning environments. Most current studies address concept drift detection and handling in numerical streams, and the evolving text stream classification remains relatively unexplored. We hypothesize that the lack of public and large-scale experimental data could be one reason. To this end, we propose a method based on an existing approach for generating evolving text streams by introducing various types of concept drifts to real-world text datasets. We provide an extensive evaluation of our proposed approach using 12 state-of-the-art baselines and 13 datasets. We first evaluate concept drift handling capability of AdaNEN and the baseline models on evolving numerical streams; this aims to demonstrate the concept drift handling capabilities of our method on a general spectrum and motivate its use in evolving text streams. The models are then evaluated in evolving text stream classification. Our experimental results show that AdaNEN consistently outperforms the existing approaches in terms of predictive performance with conservative efficiency.

Multi-type concept drift detection under a dual-layer variable sliding window in frequent pattern mining with cloud computing

The detection of different types of concept drift has wide applications in the fields of cloud computing and security information detection. Concept drift detection can indeed assist in promptly identifying instances where model performance deteriorates or when there are changes in data distribution. This paper focuses on the problem of concept drift detection in order to conduct frequent pattern mining. To address the limitation of fixed sliding windows in adapting to evolving data streams, we propose a variable sliding window frequent pattern mining algorithm, which dynamically adjusts the window size to adapt to new concept drifts and detect them in a timely manner. Furthermore, considering the challenge of existing concept drift detection algorithms that struggle to adapt to different types of drifting data simultaneously, we introduce an additional dual-layer embedded variable sliding window. This approach helps differentiate types of concept drift and incorporates a decay model for drift adaptation. The proposed algorithm can effectively detect different types of concept drift in data streams, perform targeted drift adaptation, and exhibit efficiency in terms of time complexity and memory consumption. Additionally, the algorithm maintains stable performance, avoiding abrupt changes due to window size variations and ensuring overall robustness.

Concept Drift Adaptation by Exploiting Drift Type

Concept drift is a phenomenon where the distribution of data streams changes over time. When this happens, model predictions become less accurate. Hence, models built in the past need to be re-learned for the current data. Two design questions need to be addressed in designing a strategy to re-learn models: which type of concept drift has occurred, and how to utilize the drift type to improve re-learning performance. Existing drift detection methods are often good at determining when drift has occurred. However, few retrieve information about how the drift came to be present in the stream. Hence, determining the impact of the type of drift on adaptation is difficult. Filling this gap, we designed a framework based on a lazy strategy called Type-Driven Lazy Drift Adaptor (Type-LDA). Type-LDA first retrieves information about both how and when a drift has occurred, then it uses this information to re-learn the new model. To identify the type of drift, a drift type identifier is pre-trained on synthetic data of known drift types. Furthermore, a drift point locator locates the optimal point of drift via a sharing loss. Hence, Type-LDA can select the optimal point, according to the drift type, to re-learn the new model. Experiments validate Type-LDA on both synthetic data and real-world data, and the results show that accurately identifying drift type can improve adaptation accuracy.