Concept Drift Detection Method Research Articles

Temporal Attention for Few-Shot Concept Drift Detection in Streaming Data

Concept drift describes unforeseeable changes in the underlying distribution of streaming data over time. Concept drift is a phenomenon in which the statistical properties of a target domain change over time in an arbitrary way. These changes might be caused by changes in hidden variables that cannot be measured directly. With the onset of the big data era, domains such as social networks, meteorology, and finance are generating copious amounts of streaming data. Embedded within these data, the issue of concept drift can affect the attributes of streaming data in various ways, leading to a decline in the accuracy and performance of models. There is a pressing need for new models to re-adapt to the changes in streaming data. Traditional concept drift detection algorithms struggle to effectively capture and utilize the key feature points of concept drift within complex time series, thereby failing to maintain the accuracy and efficiency of the models. In light of these challenges, this study introduces a novel concept drift detection method that incorporates a temporal attention mechanism within a prototypical network. By integrating a temporal attention mechanism during the feature extraction process, our approach enhances the capability to process complex time series data, preserves temporal locality, strengthens the learning of key features, and reduces the amount of labeled data required. This method significantly improves the detection accuracy and efficiency of small sample streaming data by better capturing the local features of the data. Experiments conducted across multiple datasets demonstrate that this method exhibits comprehensive leading performance in terms of accuracy and F1-score, with excellent recall and precision, thereby validating its effectiveness in enhancing concept drift detection in streaming data.

Concept drift detection methods based on different weighting strategies

Concept drift detection methods based on different weighting strategies

CD-BTMSE: A Concept Drift detection model based on Bidirectional Temporal Convolutional Network and Multi-Stacking Ensemble learning

The existence of concept drift phenomenon seriously affects the quality of data, it is an urgent need to investigate accurate concept drift detection methods to improve the data quality. Concept drift detection is widely used in the application scenarios of anomaly detection, intrusion detection, fake reviews detection, etc. In recent years, although scholars have proposed many concept drift detection methods, these methods can only cope with a single type of concept drift and are unable to effectively deal with time-series data. In this paper, we propose a Concept Drift detection model based on Bidirectional Temporal convolutional network and Multi-Stacking Ensemble learning (namely CD-BTMSE). CD-BTMSE choose six suitable base learners to solve the problems of overfitting, weak generalization ability and poor robustness of existing ensemble learning-based concept drift detection models, it also innovatively utilizes the Bidirectional Temporal Convolutional Network (BiTCN) model to improve the detection accuracy of concept drift through considering the temporal characteristics of the data as well as the bidirectional semantics in the detection process; At the same time, it utilizes the multi-stacking ensemble learning model to solve the problem of low accuracy of concept drift detection caused by the relatively high generalization error rate and poor generalization ability of the existing ensemble learning-based methods. In addition, the negative log-likelihood loss function in BiTCN model is replaced by a Focal Loss to solve the problem of category inhomogeneity, and the ReLU activation function is replaced by LeakyReLU to solve the neuron ”death” and gradient problems. Extensive experiments on six public datasets show that the proposed CD-BTMSE model outperforms seven state-of-the-art concept drift detection models in terms of accuracy, precision, recall and F1-measure, and it also has better stable performance.

A systematic review on detection and adaptation of concept drift in streaming data using machine learning techniques

AbstractLast decade demonstrate the massive growth in organizational data which keeps on increasing multi‐fold as millions of records get updated every second. Handling such vast and continuous data is challenging which further opens up many research areas. The continuously flowing data from various sources and in real‐time is termed as streaming data. While deriving valuable statistics from data streams, the variation that occurs in data distribution is called concept drift. These drifts play a significant role in a variety of disciplines, including data mining, machine learning, ubiquitous knowledge discovery, quantitative decision theory, and so forth. As a result, a substantial amount of research is carried out for studying methodologies and approaches for dealing with drifts. However, the available material is scattered and lacks guidelines for selecting an effective technique for a particular application. The primary novel objective of this survey is to present an understanding of concept drift challenges and allied studies. Further, it assists researchers from diverse domains to accommodate detection and adaptation algorithms for concept drifts in their applications. Overall, this study aims to contribute to deeper insights into the classification of various types of drifts and methods for detection and adaptation along with their key features and limitations. Furthermore, this study also highlights performance metrics used to evaluate the concept drift detection methods for streaming data. This paper presents the future research scope by highlighting gaps in the existing literature for the development of techniques to handle concept drifts.This article is categorized under: Algorithmic Development > Ensemble Methods Application Areas > Data Mining Software Tools Fundamental Concepts of Data and Knowledge > Big Data Mining

Entropy-based concept drift detection in information systems

As time passes, the data within information systems may continuously evolve, causing the target concept to drift. To ensure the effectiveness of data-driven decision making, it is crucial to detect drift in a timely manner and gather relevant information. In this paper, we introduce two methods that can directly detect concept drift in the provided information system, by considering a new perspective on uncertainty. First, using entropy under a single attribute constraint, we define the uncertainty of the target concept in an information system. By integrating the uncertainty of each attribute, the overall uncertainty of the target concept in the information system is obtained. Subsequently, two concept drift detection methods are proposed, namely EBTBM (Entropy-Based Threshold-Based Method) and EBSBM (Entropy-Based Sampling-Based Method). These methods utilize the defined uncertainty of the target concept as a statistical measure of the difference between two data blocks. Finally, extensive experiments on artificial and real-world data sets are conducted to validate the effectiveness of the proposed concept drift detection methods.

Dynamic Operation Optimization of Complex Industries Based on a Data-Driven Strategy

As industrial practices continue to evolve, complex process industries often exhibit characteristics such as multivariate correlation, dynamism, and nonlinearity, making traditional mechanism modeling inadequate in terms of addressing the intricacies of complex industrial problems. In recent years, with advancements in control theory and industrial practices, there has been a substantial increase in the volume of industrial data. Data-driven dynamic operation optimization techniques have emerged as effective solutions for handling complex industrial processes. By responding to dynamic environmental changes and utilizing advanced optimization algorithms, it is possible to achieve dynamic operational optimization in industrial processes, thereby reducing costs and emissions, improving efficiency, and increasing productivity. This correlates nicely with the goals set forth by conventional process operation optimization theories. Nowadays, this dynamic, data-driven strategy has shown significant potential in complex process industries characterized by multivariate correlations and nonlinear behavior. This paper approaches the subject from a data-driven perspective by establishing dynamic optimization models for complex industries and reviewing the state-of-the-art time series forecasting models to cope with changing objective functions over time. Meanwhile, aiming at the problem of concept drift in time series, this paper summarizes new concept drift detection methods and introduces model update methods to solve this challenge. In addressing the problem of solving dynamic multi-objective optimization problems, the paper reviews recent developments in dynamic change detection and response methods while summarizing commonly used as well as the latest performance measures for dynamic multi-objective optimization problems. In conclusion, a discussion of the research progress and challenges in the relevant domains is undertaken, followed by the proposal of potential directions for future research. This review will help to deeply understand the importance and application prospects of data-driven dynamic operation optimization in complex industrial fields.

Integrated detection and localization of concept drifts in process mining with batch and stream trace clustering support

Process mining can help organizations by extracting knowledge from event logs. However, process mining techniques often assume business processes are stationary, while actual business processes are constantly subject to change because of the complexity of organizations and their external environment. Thus, addressing process changes over time – known as concept drifts – allows for a better understanding of process behavior and can provide a competitive edge for organizations, especially in an online data stream scenario. Current approaches to handling process concept drift focus primarily on detecting and locating concept drifts, often through an integrated, albeit offline, approach. However, part of these integrated approaches rely on complex data structures related to tree-based process models, usually discovered through algorithms whose results are influenced by specific heuristic rules. Moreover, most of the proposed approaches have not been tested on public true concept drift-labeled event logs commonly used as benchmark, making comparative analysis difficult. In this article, we propose an online approach to detect and localize concept drifts in an integrated way using batch and stream trace clustering support. In our approach, cluster models provide input information for both concept drift detection and localization methods. Each cluster abstracts a behavior profile underlying the process and reveals descriptive information about the discovered concept drifts. Experiments with benchmark synthetic event logs with different control-flow changes, as well as with real-world event logs, showed that our approach, when relying on the same clustering model, is competitive in relation to baselines concept drift detection method. In addition, the experiment showed our approach is able to correctly locate the concept drifts detected and allows the analysis of such concept drifts through different process behavior profiles.

Build a Lightweight Dataset and Concept Drift Detection Method for Evolving Time Series Data Streams

Build a Lightweight Dataset and Concept Drift Detection Method for Evolving Time Series Data Streams

Efficient concept drift handling for batch android malware detection models

The rapidly evolving nature of Android apps poses a significant challenge to static batch machine learning algorithms employed in malware detection systems, as they quickly become obsolete. Despite this challenge, the existing literature pays limited attention to addressing this issue, with many advanced Android malware detection approaches, such as Drebin, DroidDet and MaMaDroid, relying on static models. In this work, we show how retraining techniques are able to maintain detector capabilities over time. Particularly, we analyze the effect of two aspects in the efficiency and performance of the detectors: (1) the frequency with which the models are retrained, and (2) the data used for retraining. In the first experiment, we compare periodic retraining with a more advanced concept drift detection method that triggers retraining only when necessary. In the second experiment, we analyze sampling methods to reduce the amount of data used to retrain models. Specifically, we compare fixed sized windows of recent data and state-of-the-art active learning methods that select those apps that help keep the training dataset small but diverse. Our experiments show that concept drift detection and sample selection mechanisms result in very efficient retraining strategies which can be successfully used to maintain the performance of the static Android malware state-of-the-art detectors in changing environments.

An incremental photovoltaic power prediction method considering concept drift and privacy protection

Concept drift (CD) and regional data sharing were considered potential factors critical to affecting data-driven model accuracy. In this paper, an incremental photovoltaic (PV) power prediction method is proposed for addressing the CD of streaming data and the privacy protection of regionally shared data. In the proposed scheme, a novel CD detection method is developed using Energy distance as a virtual drift criterion to measure the data distribution change and prediction error change as a real drift criterion to determine the drift time. In order to incorporate richer data features, this paper jointly trains a PV prediction model based on a broad learning system (BLS) through regional data sharing. Further, an incremental BLS model under the distributed federated learning (FL) framework is designed to maintain the privacy of each PV station's data. Finally, an incremental online model update strategy is proposed to learn new features continuously and without forgetting under the FL framework after CD occurs. The test results on the public dataset from DKASC, Alice Springs PV system, present that the proposed method can effectively detect the CD of streaming data and realize the fast online model update. Compared with the state-of-the-art techniques, the proposed method can significantly improve prediction accuracy while preserving privacy.

Unsupervised concept drift detection method based on robust random cut forest

Unsupervised concept drift detection method based on robust random cut forest

A new concept drift detection method based on the ranking of features in a data stream

The article presents a new concept drift detection method based on analyzing the importance of features of instances in the data stream. The data stream contains information about distribution patterns that reflect different concepts that may be hidden in the data stream. The presented drift detector concept uses information about the fluctuation of the most informative feature inside chunks of the data stream and compares it with the change of the same feature in neighbor chunks. In the case of data streams, the meaning of features can change over time. These changes affect the quality of the classification but can also be a significant indicator of ongoing concept drift. After detecting the drift, the classifier should be trained with the new dataset. But this issue is not addressed in this article.In this work, we propose a new concept drift detector in the data stream for the first time. This goal is achieved by observing the changing importance of features in different parts of the data stream. The proposed approach uses the feature significance measure as a drift detector. The obtained results indicate that the method can be introduced in practice. Because these are only preliminary results, in this paper, we focused on presenting the advantages of our strategy without comparison with other methods.

CODE: A Moving-Window-Based Framework for Detecting Concept Drift in Software Defect Prediction

Concept drift (CD) refers to data distributions that may vary after a minimum stable period. CD negatively influences models’ performance of software defect prediction (SDP) trained on past datasets when applied to the new datasets. Based on previous studies of SDP, it is confirmed that the accuracy of prediction models is negatively affected due to changes in data distributions. Moreover, cross-version (CV) defect data are naturally asymmetric due to the nature of their class imbalance. In this paper, a moving window-based concept-drift detection (CODE) framework is proposed to detect CD in chronologically asymmetric defective datasets and to investigate the feasibility of alleviating CD from the data. The proposed CODE framework consists of four steps, in which the first pre-processes the defect datasets and forms CV chronological data, the second constructs the CV defect models, the third calculates the test statistics, and the fourth provides a hypothesis-test-based CD detection method. In prior studies of SDP, it is observed that in an effort to make the data more symmetric, class-rebalancing techniques are utilized, and this improves the prediction performance of the models. The ability of the CODE framework is demonstrated by conducting experiments on 36 versions of 10 software projects. Some of the key findings are: (1) Up to 50% of the chronological-defect datasets are drift-prone while applying the most popular classifiers used from the SDP literature. (2) The class-rebalancing techniques had a positive impact on the prediction performance for CVDP by correctly classifying the CV defective modules and detected CD by up to 31% on the resampled datasets.

A multi-level weighted concept drift detection method

The concept drift detection method is an online learner. Its main task is to determine the position of drifts in the data stream, so as to reset the classifier after detecting the drift to improve the learning performance, which is very important in practical applications such as user interest prediction or financial transaction fraud detection. Aiming at the inability of existing drift detection methods to balance the detection delay, false positives, false negatives, and space–time efficiency, a new level transition threshold parameter is proposed, and a multi-level weighted mechanism including "Stable Level-Warning Level-Drift Level" is innovatively introduced in the concept drift detection. The instances in the window are weighted in levels, and the double sliding window is also applied. Based on this, a multi-level weighted drift detection method (MWDDM) is proposed. In particular, two variants which are MWDDM_H and MWDDM_M are proposed based on Hoeffding inequality and Mcdiarmid inequality, respectively. Experiments on artificial datasets show that MWDDM_H and MWDDM_M can detect abrupt and gradual concept drift faster than other comparison algorithms while maintaining a low false positive ratio and false negative ratio. Experiments on real-world datasets show that MWDDM has the highest classification accuracy in most cases while maintaining good space time efficiency.

Learn to adapt: Robust drift detection in security domain

Deploying robust machine learning models has to account for concept drifts arising due to the dynamically changing and non-stationary nature of data. Addressing drifts is particularly imperative in the security domain due to the ever-evolving threat landscape and lack of sufficiently labeled training data at the deployment time leading to performance degradation.Recently proposed concept drift detection methods in literature tackle this problem by identifying the changes in feature/data distributions and periodically retraining the models to learn new concepts. While these types of strategies should absolutely be conducted when possible, they are not robust towards attacker-induced drifts and suffer from a delay in detecting new attacks. We aim to address these shortcomings in this work. We propose a robust drift detector that not only identifies drifted samples but also discovers new classes as they arrive in an on-line fashion.We evaluate the proposed method with two security-relevant data sets—network intrusion data set released in 2018 and APT Command and Control dataset combined with web categorization data. Our evaluation shows that our drifting detection method is not only highly accurate but also robust towards adversarial drifts and discovers new classes from drifted samples.

Concept drift estimation with graphical models

This paper deals with the issue of concept-drift in machine learning in the context of high dimensional problems. In contrast to previous concept drift detection methods, this application does not depend on the machine learning model in use for a specific target variable, but rather, it attempts to assess the concept drift as an independent characteristic of the evolution of a dataset. This major achievement enables data to be tested for the presence of drift, independently of the specific problem at hand. This is extremely useful when the same dataset is utilized for different classifications simultaneously, as it is often the case in a business environment. Moreover, unlike previous approaches, this method does not require the re-testing of each new model; a strategy which could prove expensive in computational terms. The fundamental intention of this work is to make use of graphical models to elicit the visible structure of data and represent it as a network. Specifically, we investigate how a graphical model evolves by looking at the creation of new links, and the disappearance of existing ones, in different time periods. We perform this task in four steps. We compute the adjacency matrix of a graph in each period, we apply a function that maps each possible state of the adjacency matrix over time into a transition matrix. We use the information in the transition matrix to produce a metric to estimate the presence of a drift in the data. Eventually, we evaluate this method with both three real-world datasets and a synthetic one.

From concept drift to model degradation: An overview on performance-aware drift detectors

The dynamicity of real-world systems poses a significant challenge to deployed predictive machine learning (ML) models. Changes in the system on which the ML model has been trained may lead to performance degradation during the system’s life cycle. Recent advances that study non-stationary environments have mainly focused on identifying and addressing such changes caused by a phenomenon called concept drift. Different terms have been used in the literature to refer to the same type of concept drift and the same term for various types. This lack of unified terminology is set out to create confusion on distinguishing between different concept drift variants. In this paper, we start by grouping concept drift types by their mathematical definitions and survey the different terms used in the literature to build a consolidated taxonomy of the field. We also review and classify performance-based concept drift detection methods proposed in the last decade. These methods utilize the predictive model’s performance degradation to signal substantial changes in the systems. The classification is outlined in a hierarchical diagram to provide an orderly navigation between the methods. We present a comprehensive analysis of the main attributes and strategies for tracking and evaluating the model’s performance in the predictive system. The paper concludes by discussing open research challenges and possible research directions.

Concept drift type identification based on multi-sliding windows

Concept drift is a common and important issue in streaming data analysis and mining. Thus far, many concept drift detection methods have been proposed but may not be able to identify the type of concept drift, which will result in some difficulties, such as extracting the wrong key information, inadequate model learning and poor detection efficiency. To solve these problems, a concept drift type identification method is proposed based on multi-sliding windows (CDT_MSW). This method consists of three processes. During the first detection process, the drift position is detected by sliding the basic window forward. Then, in the growth process, the drift length is detected using the growth of the adjoint window, and the drift category is identified according to the drift length. Finally, during tracking process, the drift subcategory can be accurately identified according to the different tracking flow ratio curves generated during window tracking. Experimental results show that the proposed method can effectively identify the type of concept drift, accurately analyze the key information during online learning and improve the efficiency and generalization performance of streaming data analysis and mining.

Real-Time Concept Drift Detection and Its Application to ECG Data

Prediction of cardiac disease is one the most crucial topics in the sector of medical info evaluation. The stochastic nature and the variation concerning time in electrocardiogram (ECG) signals make it burdensome to investigate its characteristics. Being evolving in nature, it requires a dynamic predictive model. With the presence of concept drift, the model performance will get worse. Thus learning algorithms require an apt adaptive mechanism to accurately handle the drifting data streams. This paper proposes an inceptive approach, Corazon Concept Drift Detection Method (Corazon CDDM), to detect drifts and adapt to them in real-time in electrocardiogram signals. The proposed methodology results in achieving competitive results compared to the methods proposed in the literature for all types of datasets like synthetic, real-world & time-series datasets.

Bhattacharyya distance based concept drift detection method for evolving data stream

The majority of online learners assume that the data distribution to be learned is established in advance. There are many real-world problems where the distribution of the data changes as a function of time. Variations in data streams data distributions can sufficiently reduce the skill of the learning algorithm on new data, if the learning algorithm is not equipped to track such changes. Therefore, the algorithm should initiate required actions to make sure that the new information is learned properly. In this article, we propose Bhattacharyya Distance-based Concept Drift Detection Method (BDDM) which uses Bhattacharyya distance to identify gradual or abrupt variations in the distribution. Experiments executed in the MOA framework using three artificial data generators and ten real-world datasets suggest that BDDM improves the detections and accuracies in many scenarios.