Group concept drift detection method based on Bennett bound

Integrated detection and localization of concept drifts in process mining with batch and stream trace clustering support

Big Data presents challenges for predictive analytic algorithms due to the possibility of non-stationary populations. Concept drift detection algorithms can be used to detect changes in underlying distribution in order to retrain. Most concept drift detection methods are known to scale to a relatively low number of features (a few hundred). However, in many areas, datasets with thousands or even tens of thousands of features are becoming common. This paper studies the behavior of supervised concept drift detection algorithms (Drift Detection Method (DDM), Early Drift Detection Method (EDDM)) and unsupervised algorithm (Friedman and Rafsky's algorithm) on high-dimensional datatsets. Our goal was to find if these algorithms can scale, first by studying the growth of execution time with the dimension of the dataset, and second by studying their comparative accuracy on high-dimensional datasets. The algorithms were run on datasets consisting of up to 100,000 features. Results show a linear growth of the execution time with respect to the dimension in each algorithm. The performance of unsupervised algorithm degraded significantly on datasets close to 100,000 dimensions. Our results also show that the drift detection accuracy of the three algorithms did not degrade as the number of features increase.

Neural network models are widely adopted in real-world applications for processing streaming data. However, these applications often face challenges in terms of accuracy degradation, caused by changes in the data distribution of the stream data compared to the training data. Two underlying reasons contribute to these changes. The first, known as the concept drift problem, occurs when there is a change in the correlation between the input data and the prediction output, making the models trained on the training data no longer suitable for the new data. The second reason, known as the novelty problem, arises when real-world data contains unexpected data categories that were not present in the training data, resulting in incorrect predictions. The research community has divided into different groups and each developed various methods to detect either concept drift or novelty distribution changes. However, these methods only address one aspect of the problem and are unable to distinguish between them. This leads to an inappropriate allocation of model maintenance resources, including the high cost of model retraining and the acquisition of true label data. In this study, we aim to address this gap by proposing a novel concept drift detection method that is capable of distinguishing between known labeled concept drift and novelty. Our method is also more efficient than existing drift detection methods, making it suitable for applications on neural networks.

One of the most significant challenges in data-driven modeling of complex systems is dealing with concept drift, i.e., the unpredictable changes in the underlying data distribution over time. In this work, nine concept drift detection (CDD) methods are evaluated with respect to different types of concept drift, including abrupt, gradual, incremental, and real concept drift, for both supervised and unsupervised application scenarios. For the supervised case, the methods EDDM, FHDDMSadd, MDDM-E, and EFDT are compared against a classification without change detection using Naïve Bayes as the base classifier. In the unsupervised application scenario, CluStream, ClusTree, DenStream, StreamKM++, and D-Stream are evaluated. The experiments are conducted using the Massive Online Analysis (MOA) evaluation platform, and the performance of each method is measured in terms of classification accuracy, memory consumption, and computation time. This empirical research shows that classification accuracy can be improved by 20% by implementing a CDD method, highlighting the importance of CDD in SHM data streams. However, there is no single method that proves to be superior in all scenarios, and the choice depends on the characteristics of the considered data stream and application requirements. Selecting the appropriate CDD method from the approximately 340 different methods found in the literature is not a trivial task and can lead to suboptimal selection. To tackle this issue, an innovative approach is proposed to assist researchers and practitioners find the appropriate CCD method for their application.

The concept drift detection method is an online learner. Its main task is to determine the position of drifts in the data stream, so as to reset the classifier after detecting the drift to improve the learning performance, which is very important in practical applications such as user interest prediction or financial transaction fraud detection. Aiming at the inability of existing drift detection methods to balance the detection delay, false positives, false negatives, and space–time efficiency, a new level transition threshold parameter is proposed, and a multi-level weighted mechanism including "Stable Level-Warning Level-Drift Level" is innovatively introduced in the concept drift detection. The instances in the window are weighted in levels, and the double sliding window is also applied. Based on this, a multi-level weighted drift detection method (MWDDM) is proposed. In particular, two variants which are MWDDM_H and MWDDM_M are proposed based on Hoeffding inequality and Mcdiarmid inequality, respectively. Experiments on artificial datasets show that MWDDM_H and MWDDM_M can detect abrupt and gradual concept drift faster than other comparison algorithms while maintaining a low false positive ratio and false negative ratio. Experiments on real-world datasets show that MWDDM has the highest classification accuracy in most cases while maintaining good space time efficiency.

CD-BTMSE: A Concept Drift detection model based on Bidirectional Temporal Convolutional Network and Multi-Stacking Ensemble learning

Machine learning, as a significant component of the Industrial Internet of Things (IIoT), has been widely applied in many fields. The continuously generated data from the various sensors are collected and stored, this is also known as a data stream. However, the non-stationary phenomenon in data stream, concept drift, is important to be detected immediately for the operation of the IoT system. Therefore, the detection method for concept drift is needed to alert the requirement to maintain or replace some components in advance, so as to avoid or mitigate the risk of malfunction of the IoT system. The majority of existing literature focuses on concept drift detection on binary classification. To fill this gap, here we propose an algorithm to detect multi-class. Moreover to improve the performance of detection, we also introduce an algorithm which integrates the existing error rate which is widely used with our newly proposed False Positive rate. The new method is called Drift Detection Method with False Positive rate for multi-label classification (DDM-FP-M). The DDM-FP-M firstly defines the false positive rate calculation method in multi-label classification, then integrates it with Drift Detection method with False positive rate (DDM-FP). The performance of the proposed method is evaluated through Intel Lab data and is found to outperform the Drift Detection method(DDM) over 50% cases.

Concept drift detection methods are crucial components of many online learning approaches. Accurate drift detections allow prompt reaction to drifts and help to maintain high performance of online models over time. Although many methods have been proposed, no attention has been given to data streams with imbalanced class distributions, which commonly exist in real-world applications, such as fault diagnosis of control systems and intrusion detection in computer networks. This paper studies the concept drift problem for online class imbalance learning. We look into the impact of concept drift on single-class performance of online models based on three types of classifiers, under seven different scenarios with the presence of class imbalance. The analysis reveals that detecting drift in imbalanced data streams is a more difficult task than in balanced ones. Minority-class recall suffers from a significant drop after the drift involving the minority class. Overall accuracy is not suitable for drift detection. Based on the findings, we propose a new detection method DDM-OCI derived from the existing method DDM. DDM-OCI monitors minority-class recall online to capture the drift. The results show a quick response of the online model working with DDM-OCI to the new concept.

Novel hybrid pair recommendations based on a large-scale comparative study of concept drift detection

Data streams typically evolve over time, manifesting concept drift, and are applied to Deep Neural Networks (DNNs). However, DNNs are predominantly not designed to adapt, resulting in decreasing accuracy. Ideally, a DNN would detect this drift and adapt rapidly. In the streaming machine learning field, adapting models to concept drift has been the subject of research for some time. However, DNN adaptation is lagging behind as DNNs typically require a greater number of instances and increased adaptation time compared to other model types, resulting in a lack of methods that adapt the classifying DNN in a streaming environment. The focus of this study is detection and adaptation to concept drift in high dimensional data with an experimental study on images where novel sub-classes are arising. In the concept drift field, less attention has been focused upon high dimensional data. Our system, termed DeepStreamEnsemble provides existing DNNs with concept drift detection and DNN adaptation capabilities. It is the first system to utilise DNN activations and streaming classifiers for concept drift detection and adaptation. We compare with eleven other state-of-the-art methods, with DeepStreamEnsemble outperforming other leading concept drift detection and adaptation methods by between 5% and 20% on accuracy.

Concept drift describes unforeseeable changes in the underlying distribution of streaming data over time. Concept drift is a phenomenon in which the statistical properties of a target domain change over time in an arbitrary way. These changes might be caused by changes in hidden variables that cannot be measured directly. With the onset of the big data era, domains such as social networks, meteorology, and finance are generating copious amounts of streaming data. Embedded within these data, the issue of concept drift can affect the attributes of streaming data in various ways, leading to a decline in the accuracy and performance of models. There is a pressing need for new models to re-adapt to the changes in streaming data. Traditional concept drift detection algorithms struggle to effectively capture and utilize the key feature points of concept drift within complex time series, thereby failing to maintain the accuracy and efficiency of the models. In light of these challenges, this study introduces a novel concept drift detection method that incorporates a temporal attention mechanism within a prototypical network. By integrating a temporal attention mechanism during the feature extraction process, our approach enhances the capability to process complex time series data, preserves temporal locality, strengthens the learning of key features, and reduces the amount of labeled data required. This method significantly improves the detection accuracy and efficiency of small sample streaming data by better capturing the local features of the data. Experiments conducted across multiple datasets demonstrate that this method exhibits comprehensive leading performance in terms of accuracy and F1-score, with excellent recall and precision, thereby validating its effectiveness in enhancing concept drift detection in streaming data.

Diversity measure as a new drift detection method in data streaming

A drift detection method for industrial images based on a defect segmentation model

In data stream analyses, detecting the concept drift accurately is important to maintain the classification performance. Most drift detection methods assume that the class labels become available immediately after a data sample arrives. However, this assumption is overly optimistic, as labeling costs are high and much time is needed to obtain the label of data samples. Therefore, it is un-realistic to attempt to acquire all of the labels when processing the data streams. In this paper, we propose a concept drift detection method under the assumption that there is limited access to labels. The proposed method detects concept drift on unlabeled data streams based on the class label information which is predicted by the classifier trained with the limited number of labeled data samples. Experimental results on synthetic and real streaming data show that the proposed method is competent to detect the concept drift using only a small amount of labeled data.

Concept drift in data stream poses many challenges and difficulties in mining this tradition-distinct database. In this paper, we focus on detecting concept drift in evolving data stream. We propose a novel method to detect concept drift using entropy over an adaptive sliding window. In the method, the sliding window is not fixed but dynamically determined. Another distinct property is that the method integrates an algorithm to find the exact timestamp for retraining the classifier whenever a concept drift is detected. In the experiments, we evaluate our method on publicly available data streams containing various types of concept drifts, and then compare it with four well known concept drift detection methods. The experimental results show that compared with four benchmarks, the proposed method is better than or comparable with other methods for most cases.