Concept Drift Detection Research Articles

Unsupervised concept drift detection method based on robust random cut forest

Unsupervised concept drift detection method based on robust random cut forest

Model-centric transfer learning framework for concept drift detection

Concept drift refers to the inevitable phenomenon that influences the statistical features of the data stream. Detecting concept drift in data streams quickly and precisely remains challenging, and failure to detect it will render model trained on historical data ineffective. Current drift detection methods suffer from the following problems: detection delay, many false detected drifts, and rarely utilize the deep neural network directly in the field of concept drift detection, which is considerably competent at addressing the classification problems of data stream. Furthermore, the output of the model is usually taken as a metric to detect drift, while changes in the model parameters are often ignored which contain highly useful information. In this paper, we propose a model-centric framework for concept drift detection that uses deep neural network to detect concept drift by focusing on the change in the model itself, rather than the model output. In addition, transfer learning is developed to accelerate the drift detection process and reduce the computational complexity by freezing parts of the network. To further reduce false detected drifts, we propose long and short time windows method to determine the real drift from the potential detected drift. Experiments with real-world and artificial datasets have been undertaken to demonstrate the effectiveness of the proposed framework.

Concept Drift in Software Defect Prediction: A Method for Detecting and Handling the Drift

Software Defect Prediction (SDP) is crucial towards software quality assurance in software engineering. SDP analyzes the software metrics data for timely prediction of defect prone software modules. Prediction process is automated by constructing defect prediction classification models using machine learning techniques. These models are trained using metrics data from historical projects of similar types. Based on the learned experience, models are used to predict defect prone modules in currently tested software. These models perform well if the concept is stationary in a dynamic software development environment. But their performance degrades unexpectedly in the presence of change in concept (Concept Drift). Therefore, concept drift (CD) detection is an important activity for improving the overall accuracy of the prediction model. Previous studies on SDP have shown that CD may occur in software defect data and the used defect prediction model may require to be updated to deal with CD. This phenomenon of handling the CD is known as CD adaptation. It is observed that still efforts need to be done in this direction in the SDP domain. In this article, we have proposed a pair of paired learners (PoPL) approach for handling CD in SDP. We combined the drift detection capabilities of two independent paired learners and used the paired learner (PL) with the best performance in recent time for next prediction. We experimented on various publicly available software defect datasets garnered from public data repositories. Experimentation results showed that our proposed approach performed better than the existing similar works and the base PL model based on various performance measures.

Data Stream: Statistics, Challenges, Concept Drift Detector Methods, Applications and Datasets

Data Stream: Statistics, Challenges, Concept Drift Detector Methods, Applications and Datasets

An Experimental Evaluation of Process Concept Drift Detection

Process mining provides techniques to learn models from event data. These models can be descriptive (e.g., Petri nets) or predictive (e.g., neural networks). The learned models offer operational support to process owners by conformance checking, process enhancement, or predictive monitoring. However, processes are frequently subject to significant changes, making the learned models outdated and less valuable over time. To tackle this problem, Process Concept Drift (PCD) detection techniques are employed. By identifying when the process changes occur, one can replace learned models by relearning, updating, or discounting pre-drift knowledge. Various techniques to detect PCDs have been proposed. However, each technique's evaluation focuses on different evaluation goals out of accuracy, latency, versatility, scalability, parameter sensitivity, and robustness. Furthermore, the employed evaluation techniques and data sets differ. Since many techniques are not evaluated against more than one other technique, this lack of comparability raises one question: How do PCD detection techniques compare against each other? With this paper, we propose, implement, and apply a unified evaluation framework for PCD detection. We do this by collecting evaluation goals and evaluation techniques together with data sets. We derive a representative sample of techniques from a taxonomy for PCD detection. The implemented techniques and proposed evaluation framework are provided in a publicly available repository. We present the results of our experimental evaluation and observe that none of the implemented techniques works well across all evaluation goals. However, the results indicate future improvement points of algorithms and guide practitioners.

CatSight, a direct path to proper multi-variate time series change detection: perceiving a concept drift through common spatial pattern

Detecting changes in data streams, with the data flowing continuously, is an important problem which Industry 4.0 has to deal with. In industrial monitoring, the data distribution may vary after a change in the machine’s operating point; this situation is known as concept drift, and it is key to detecting this change. One drawback of conventional machine learning algorithms is that they are usually static, trained offline, and require monitoring at the input level. A change in the distribution of data, in the relationship between the input and the output data, would result in the deterioration of the predictive performance of the models due to the lack of an ability to generalize the model to new concepts. Drift detecting methods emerge as a solution to identify the concept drift in the data. This paper proposes a new approach for concept drift detection—a novel approach to deal with sudden or abrupt drift, the most common drift found in industrial processes-, called CatSight. Briefly, this method is composed of two steps: (i) Use of Common Spatial Patterns (a statistical approach to deal with data streaming, closely related to Principal Component Analysis) to maximize the difference between two different distributions of a multivariate temporal data, and (ii) Machine Learning conventional algorithms to detect whether a change in the data flow has been occurred or not. The performance of the CatSight method, has been evaluated on a real use case, training six state of the art Machine Learning (ML) classifiers; obtained results indicate how adequate the new approach is.

Historical Isolated Forest for detecting and adaptation concept drifts in nonstationary data streaming

Concept drift refers to sudden changes in the fundamental structure of the streaming data distribution over time. The core objective of concept drift research is to develop techniques and strategies for detect, understand, and adapt data streaming drifts. Data research has shown that if concept drift is not handled properly, machine learning in such an environment would provide subpar learning outcomes. In this paper, a historical Isolated Forest (HIF) is presented that depends on a decision tree, which split the data streaming into chunks and each chuck considers a region in the tree. HIF is employed to detect concept drifts and adapt this region with current changes. Which HIF stores previously generated models and employs the most similar model of each concept drift distribution as the current model until generate the best performance model. HIF doesn’t stop the main system model when retraining a new model, which HIF is divided into three primary parallel blocks: detection block, similarity block (online block), and retraining block (offline block). For several authentic data sets (three data sets), our suggested algorithm was verified and contrasted. the accuracy and execution speed were specifically assessed, and memory usage. The experimental results demonstrate that our modifications use fewer resources and have comparable or greater detection accuracy than the original IForestASD.

Batch classifier with adaptive update for backbone traffic classification

Network traffic analysis is an important method for ISPs to know the status of the network. However, the high speed and rapid evolution of backbone network traffic have brought new challenges to traffic analysis. Most existing traffic classifiers are based on full traffic, while processing all the large-scale backbone network traffic is time-intensive and resource-consuming. Moreover, the relationship between feature vectors and classification labels drifts with the rapid evolution of backbone network traffic, which leads to the degradation of the performance of the classification model. This paper presents a model that can implement real-time classification of sampled backbone network traffic and adaptively update the classification model when concept drift occurs. To accurately classify the backbone network traffic in real time, we propose the Multiple Counter Sketch (MC Sketch) to quickly extract feature vectors from the sampled traffic and design an Adaptive Batch Classifier based on Agglomerative Clustering (A-BCAC) to perform unsupervised batch clustering of feature vectors. Using the labels obtained from A-BCAC, we trained the classifier to classify the backbone traffic. In addition, we design an adaptive batch update model based on concept drift detection to solve the concept drift problem. The experimental results on sampled traffic collected on a 10Gbps link and the comparisons with state-of-the-art studies demonstrate the performance and efficiency of our method.

Ensemble Learning for Interpretable Concept Drift and Its Application to Drug Recommendation

During the COVID-19 epidemic, the online prescription pattern of Internet healthcare provides guarantee for the patients with chronic diseases and reduces the risk of cross-infection, but it also raises the burden of decision-making for doctors. Online drug recommendation system can effectively assist doctors by analysing the electronic medical records (EMR) of patients. Unlike commercial recommendations, the accuracy of drug recommendations should be very high due to their relevance to patient health. Besides, concept drift may occur in the drug treatment data streams, handling drift and location drift causes is critical to the accuracy and reliability of the recommended results. This paper proposes a multi-model fusion online drug recommendation system based on the association of drug and pathological features with online-nearline-offline architecture. The system transforms drug recommendation into pattern classification and adopts interpretable concept drift detection and adaptive ensemble classification algorithms. We apply the system to the Percutaneous Coronary Intervention (PCI) treatment process. The experiment results show our system performs nearly as good as doctors, the accuracy is close to 100%

AEDS-IoT: Adaptive clustering-based Event Detection Scheme for IoT data streams

The emergence of the Internet of Things is expected to produce a huge amount of data streams for sensing the phenomenon from the real world. These data patterns i.e. data distributions keep on evolving and are subject to change based on the changes in the external environment. Valuable and useful information should be extracted from this data stream in real-time- to find out the events of interest. Pattern extraction from stored data and its analytics is convenient, though stream analytics has major challenges associated with it like identifying the event patterns from limited data in the stream, concept drift detection, outlier detection, and handling, limited storage, and processing capacity etc. Traditional methods, in general, work well with historical data yet have limitations while dealing with real-time data streams. Hence, the present work proposes adaptive clustering-based Event Detection Scheme for IoT (AEDS-IoT) stream data, which can infer events of interest from data distribution patterns present in streaming data. Initially, the scheme uses an adaptive window to collect the streaming data and processes it to detect the events. Further, the event patterns are detected by adaptive clustering. After identifying event patterns, the user assigns labels based on spatial–temporal conditions. The proposed AEDS methodology updates the patterns generated from upcoming stream data received. The performance of the proposed scheme is compared against the state of the scheme using the Arhus dataset. To corroborate the scheme with experimental evidence, the scheme was applied to the data collected from the Ganga river(the life line of North India) using smart water sensors. The performance and efficacy of AEDS-IoT were also tested on these data to detect upcoming event patterns from the data stream.

Explainable concept drift in process mining

The execution of processes leaves trails of event data in information systems. These event data are analyzed to generate insights and improvements for the underlying process. However, companies do not execute these processes in a vacuum. The fast pace of technological development, constantly changing market environments, and fast consumer responses expose companies to high levels of uncertainty. This uncertainty often manifests itself in significant changes in the executed processes. Such significant changes are called concept drifts. Transparency about concept drifts is crucial to respond quickly and adequately, limiting the potentially negative impact of such drifts. Three types of knowledge are of interest to a process owner: When did a drift occur, what happened, and why did it happen. This paper introduces a framework to extract concept drifts and their potential root causes from event data. We extract time series describing process measures, detect concept drifts, and test these drifts for correlation. This framework generalizes existing work such that object-centric event data with multiple case notions, non-linear relationships, and an arbitrary number of process measures are supported. We provide an extendable implementation and evaluate our framework concerning the sensitivity of the time series construction and scalability of cause–effect testing. Furthermore, we provide a case study uncovering an explainable concept drift.

A Semisupervised Concept Drift Adaptation via Prototype-Based Manifold Regularization Approach with Knowledge Transfer

Data stream mining deals with processing large amounts of data in nonstationary environments, where the relationship between the data and the labels often changes. Such dynamic relationships make it difficult to design a computationally efficient data stream processing algorithm that is also adaptable to the nonstationarity of the environment. To make the algorithm adaptable to the nonstationarity of the environment, concept drift detectors are attached to detect the changes in the environment by monitoring the error rates and adapting to the environment’s current state. Unfortunately, current approaches to adapt to environmental changes assume that the data stream is fully labeled. Assuming a fully labeled data stream is a flawed assumption as the labeling effort would be too impractical due to the rapid arrival and volume of the data. To address this issue, this study proposes to detect concept drift by anticipating a possible change in the true label in the high confidence prediction region. This study also proposes an ensemble-based concept drift adaptation approach that transfers reliable classifiers to the new concept. The significance of our proposed approach compared to the current baselines is that our approach does not use a performance measur as the drift signal or assume a change in data distribution when concept drift occurs. As a result, our proposed approach can detect concept drift when labeled data are scarce, even when the data distribution remains static. Based on the results, this proposed approach can detect concept drifts and fully supervised data stream mining approaches and performs well on mixed-severity concept drift datasets.

Concept drift detection with quadtree-based spatial mapping of streaming data

Online learning is a complex task, especially when the data stream changes its distribution over time. It’s challenging to monitor and detect these changes to maintain the performance of the learning algorithm. In this work, we present a novel detection method built from a different perspective of other preexisting detectors from literature. It analyzes the space occupied by the data, assuming that it would be immutable unless changes in this space occur among data of different classes. The data is mapped into a quadtree-based memory structure that provides knowledge about which class (label) is dominant in a given region of the feature space. Drifts are detected by checking whether data assigned to a given class occupy spaces considered relevant to the other class. The proposed method was evaluated on benchmark binary classification problems. The results show that our method can compete with well-known drift detectors from the literature on synthetic and real-world datasets.

A new concept drift detection method based on the ranking of features in a data stream

The article presents a new concept drift detection method based on analyzing the importance of features of instances in the data stream. The data stream contains information about distribution patterns that reflect different concepts that may be hidden in the data stream. The presented drift detector concept uses information about the fluctuation of the most informative feature inside chunks of the data stream and compares it with the change of the same feature in neighbor chunks. In the case of data streams, the meaning of features can change over time. These changes affect the quality of the classification but can also be a significant indicator of ongoing concept drift. After detecting the drift, the classifier should be trained with the new dataset. But this issue is not addressed in this article.In this work, we propose a new concept drift detector in the data stream for the first time. This goal is achieved by observing the changing importance of features in different parts of the data stream. The proposed approach uses the feature significance measure as a drift detector. The obtained results indicate that the method can be introduced in practice. Because these are only preliminary results, in this paper, we focused on presenting the advantages of our strategy without comparison with other methods.

Concept Drift Analysis and Malware Attack Detection System Using Secure Adaptive Windowing

Concept drift is a main security issue that has to be resolved since it presents a significant barrier to the deployment of machine learning (ML) models. Due to attackers’ (and/or benign equivalents’) dynamic behavior changes, testing data distribution frequently diverges from original training data over time, resulting in substantial model failures. Due to their dispersed and dynamic nature, distributed denial-of-service attacks pose a danger to cybersecurity, resulting in attacks with serious consequences for users and businesses. This paper proposes a novel design for concept drift analysis and detection of malware attacks like Distributed Denial of Service (DDOS) in the network. The goal of this architecture combination is to accurately represent data and create an effective cyber security prediction agent. The intrusion detection system and concept drift of the network has been analyzed using secure adaptive windowing with website data authentication protocol (SAW_WDA). The network has been analyzed by authentication protocol to avoid malware attacks. The data of network users will be collected and classified using multilayer perceptron gradient decision tree (MLPGDT) classifiers. Based on the classification output, the decision for the detection of attackers and authorized users will be identified. The experimental results show output based on intrusion detection and concept drift analysis systems in terms of throughput, end-end delay, network security, network concept drift, and results based on classification with regard to accuracy, memory, and precision and F-1 score.

Обнаружение дрейфа концепта при классификации мобильных приложений с использованием автокодировщиков

The study observes the task of concept drift detection in multiclass applications classification tasks on the example of collected data set of network traffic in the form of IP packets from Purpose of the study: development and software implementation of an algorithm for a concept change detection in tasks of multiclass mobile application traffic classification using ANNs of the autoencoder type (AC). Novelty of the study consists in drift detection of one or several mobile applications based on changes in the statistical characteristics of one or several attributes without usage of true class labels implying ANNs of the autoencoder type. Results: The study developed an algorithm for concepts drift of application detection based on the analysis of changes in the statistical characteristics of attributes or a noticeable decrease in the quality of the analyzed applications classification. As for fundamental model of concept drift detector of analyzed applications, the study used autoencoders. The research contains basic theoretical positions of the algorithm creation. The study shows that in case of trained AC only on high-quality prototypes, it will be able to reconstruct normal observations but not abnormal observations (unknown concepts). As a result, when the autoencoder detects a significant reconstruction error, it classifies the observation data as abnormal. Estimation of reconstruction errors of the analyzed applications and excess of threshold value assess the presence of drift. The Python software environment provides the implementation of the presented solution.

ECG Signal Processing using an Optimized Sliding Window Approach for Concept Drift Detection and Adaption

ECG Signal Processing using an Optimized Sliding Window Approach for Concept Drift Detection and Adaption

Drift Detection Method Using Distance Measures and Windowing Schemes for Sentiment Classification

Textual data streams have been extensively used in practical applications where consumers of online products have expressed their views regarding online products. Due to changes in data distribution, commonly referred to as concept drift, mining this data stream is a challenging problem for researchers. The majority of the existing drift detection techniques are based on classification errors, which have higher probabilities of false-positive or missed detections. To improve classification accuracy, there is a need to develop more intuitive detection techniques that can identify a great number of drifts in the data streams. This paper presents an adaptive unsupervised learning technique, an ensemble classifier based on drift detection for opinion mining and sentiment classification. To improve classification performance, this approach uses four different dissimilarity measures to determine the degree of concept drifts in the data stream. Whenever a drift is detected, the proposed method builds and adds a new classifier to the ensemble. To add a new classifier, the total number of classifiers in the ensemble is first checked if the limit is exceeded before the classifier with the least weight is removed from the ensemble. To this end, a weighting mechanism is used to calculate the weight of each classifier, which decides the contribution of each classifier in the final classification results. Several experiments were conducted on real-world datasets and the results were evaluated on the false positive rate, miss detection rate, and accuracy measures. The proposed method is also compared with the state-of-the-art methods, which include DDM, EDDM, and PageHinkley with support vector machine (SVM) and Naïve Bayes classifiers that are frequently used in concept drift detection studies. In all cases, the results show the efficiency of our proposed method.

A Comparative Study of Situation Awareness-Based Decision-Making Model Reinforcement Learning Adaptive Automation in Evolving Conditions

Situation-awareness-based decision-making (SABDM) models constructed using cognitive maps and goal-direct task analysis techniques have been successfully used in decision support systems in safety-critical and mission-critical environments such as air traffic control and electrical energy distribution. Reinforcement learning (RL) and other machine learning techniques are used to automate situational awareness mental model parameter adjustments, reducing the expert work on the initial configuration and long-term maintenance without affecting the mental model’s structure and maintaining the situation-awareness-based decision-making model’s cognitive and explainability characteristics. Real-world models should evolve to cope with changes in the environmental conditions. This study evaluates the application of reinforcement learning as an online adaptive technique to adjust the situational-awareness mental model’s parameters under evolving conditions. We conducted evaluation experiments using real-world public datasets to compare the performance of the SABDM model with that of the reinforcement learning adaptation technique (SABDM/RL) and other adaptive machine learning methods under distinct concept drift evolving conditions. We measured the techniques’ overall and dynamic performance to understand how well they adapt to evolving environmental conditions. The experiments show that SABDM/RL performs similarly to modern online adaptive machine learning classification methods with the support of concept drift detection techniques while maintaining the mental model strength of the situation awareness-based systems.

Brute force concept drift detection

We present a brute-force approach to detect concept drift behind time sequence data. This approach, named Select-Starţ searches for start points of concept drift to minimize error. In other words, Select-Start searches for the start points of new concepts from the input sequence. Unlike many related works, Select-Start does not require a pre-specified error threshold to detect drift. This paper compares Select-Start with previous representative methods and clarifies its characteristics. The experimental results show that Select-Start is accurate for concept drift problems where the threshold changes slowly. However, existing methods are better at analyzing concept drift problems where the model behind data changes rapidly.