Concept Drift Detection Research Articles

Ensemble Method to classify multi class with concept drift

Real world scenarios is expected to have more number of classes, evolution of new class, change in concept, different distribution of data etc., This paper focus on the joint problem of multi class imbalance problem and different types of concept drift detection. The proposed work Enhanced Concept drift based Resampling ensemble (ECDRE) algorithm addresses the combined issues of Multi Class Imbalance problem with different types of Concept Drifts using the Ensemble Relearning Model. In this framework, EMUOB and EMOOB algorithm helps to re-balance the multi class distribution of the latest block. Window based strategy is used to classify the multi class imbalance dataset with different types of drifts. A group of classifiers form an ensemble and members belonging to the ensembles are updated frequently, where weights are calculated to emphasis misclassification cost. Meanwhile, ECDRE is swiftly adaptable to new conditions. Observed studies proof that ECDRE algorithm is more effective for learning non-stationary environment which suffers from multi class imbalance with different types of concept drift. CDRE algorithm was implemented to deal with multi-class problem and concept drift, but it is not much efficient to handle all types of concept drift. Even in situations of Data Streams which suffer from drifts such as real, virtual and recurring types, ECDRE works well and shows overall better results of above 80% accuracy.

Concept learning using one-class classifiers for implicit drift detection in evolving data streams

Data stream mining has become an important research area over the past decade due to the increasing amount of data available today. Sources from various domains generate a near-limitless volume of data in temporal order. Such data are referred to as data streams, and are generally nonstationary as the characteristics of data evolves over time. This phenomenon is called concept drift, and is an issue of great importance in the literature, since it makes models obsolete by decreasing their predictive performance. In the presence of concept drift, it is necessary to adapt to change in data to build more robust and effective classifiers. Drift detectors are designed to run jointly with classification models, updating them when a significant change in data distribution is observed. In this paper, we present an implicit (unsupervised) algorithm called One-Class Drift Detector (OCDD), which uses a one-class learner with a sliding window to detect concept drift. We perform a comprehensive evaluation on mostly recent 17 prevalent concept drift detection methods and an adaptive classifier using 13 datasets. The results show that OCDD outperforms the other methods by producing models with better predictive performance on both real-world and synthetic datasets.

Analysis of concept drift in fake reviews detection

Online reviews have a substantial impact on decision making in various areas of society, predominantly in the arena of buying and selling of goods. As such, the truthfulness of internet reviews is critical for both consumers and vendors. Fake reviews not only mislead innocent clients and influence customers' choice, leading to inaccurate descriptions and sales. This raises the need for efficient fake review detection models and tools that can address these issues. Analysing a text data stream of fake reviews in concept drift appears to reduce the effectiveness of the detection models. Despite several efforts to develop algorithms for detecting fake reviews, one crucial aspect that has not been addressed is finding a real correlation between the concept drift score and the classification of performance over-time in the real-world data stream. Consequently, we have introduced a comprehensive analysis to investigate the concept drift problem within fake review detection. There are two methods to achieve this goal: benchmarking concept drift detection method and content-based classification methods. We conducted our experiment using four real-world datasets from Yelp.com. The results demonstrated that there is a strong negative correlation between concept drift and the performance of fake review detection/prediction models, which indicates the difficulty of building more efficient models.

Concept drift detection on stream data for revising DBSCAN

AbstractData stream mining of IoT data can support operator to immediately isolate causes of equipment alarms. The challenge, however, is to keep their classifiers high purity (the data ratio with same proper class in a cluster) with concept drifting ascribed to differences between alarm models and entities. We propose to continuously update data class according to their distribution changes. Through evaluation, no purity deterioration was verified for oscillation condition data with a drifting rate of 1%. The result suggested that the method improves operator decision making.

Bayesian Nonparametric Unsupervised Concept Drift Detection for Data Stream Mining

Online data stream mining is of great significance in practice because of its ubiquity in many real-world scenarios, especially in the big data era. Traditional data mining algorithms cannot be directly applied to data streams due to (1) the possible change of underlying data distribution over time (i.e.,concept drift) and (2) delayed, short, or even no labels for streaming data in practice. A new research area, namedunsupervised concept drift detection, has emerged to tackle this difficulty mainly based on two-sample hypothesis tests, such as the Kolmogorov–Smirnov test. However, it is surprising that none of the existing methods in this area exploit the Bayesian nonparametric hypothesis test, which has clear interpretability and straightforward prior knowledge encoding ability and no strict or unrealistic requirement of prefixing the form for the underlying data distribution. In this article, we present a Bayesian nonparametric unsupervised concept drift detection method based on the Polya tree hypothesis test. The basic idea is to decompose the underlying data distribution into a multi-resolution representation that transforms the whole distribution hypothesis test into recursive and simple binomial tests. Also, an incremental mechanism is especially designed to improve its efficiency in the stream setting. The method effectively detect drifts, and it also locates where a drift happens and the posteriors of hypotheses. The experiments on synthetic data verify the desired properties of the proposed method, and the experiments on real-world data show the better performance of the method for data stream mining compared with its frequentist counterpart in the literature.

Using spectral entropy and bernoulli map to handle concept drift

Data stream mining is a relevant task to extract information from large amounts of data that continuously evolve over time. In this context, learning algorithms may combine a classifier and a drift detector to identify changes in the distribution of the predictions error in order to rapidly adapt or replace the predictive model. Several proposals have been presented in the literature for the detection of concept changes based on the error rate of the predictive models. In general, the error rate distribution grounds most of the approaches based on sequential analysis and statistical process control, or by monitoring distributions using sliding windows, which assume the prediction errors are generated independently. However, empirical studies have shown that the error rate can be influenced by temporal dependence. In addition, new approaches considering dynamical system tools have been proposed for concept drift detection in unsupervised scenarios containing temporal dependencies. Motivated by these approaches, this article proposes the Spectral Entropy Drift Detector (SEDD), which is based on Spectral Entropy, Bernoulli Map and on the surrogate stability concept. Experimental results using abrupt and gradual concept drift versions of different dataset generators as well as real-world data streams, run in the Massive Online Analysis (MOA) framework, suggest that SEDD was competitive with the state-of-the-art methods, especially when considering accuracy and false alarms.

Unsupervised concept drift detection based on multi-scale slide windows

In the past few decades, research related to concept drift learning has been increasing, and many concept drift learning algorithms have also been developed and applied to actual data stream processing. In general, concept drift research involves the development of methodologies and techniques for drift detection, understanding and adaptation. This paper focuses on concept drift detection, and proposes an unsupervised concept drift detection algorithm based on multi-scale slide windows, where the total average distance is obtained through k-means clustering and multi-scale windows and is used as a detection index for concept drift, and then uses the statistical process control system to determine the range of index thresholds. Proved by experiments of detecting the gradual and abrupt concept drift with five datasets of different dimensions, including Sin, Circle, Gaussian, Radar and Motion Sense datasets, the algorithm has a good concept drift detection effect.

Bi-directional online transfer learning: a framework

Transfer learning uses knowledge learnt in source domains to aid predictions in a target domain. When source and target domains are online, they are susceptible to concept drift, which may alter the mapping of knowledge between them. Drifts in online environments can make additional information available in each domain, necessitating continuing knowledge transfer both from source to target and vice versa. To address this, we introduce the Bi-directional Online Transfer Learning (BOTL) framework, which uses knowledge learnt in each online domain to aid predictions in others. We introduce two variants of BOTL that incorporate model culling to minimise negative transfer in frameworks with high volumes of model transfer. We consider the theoretical loss of BOTL, which indicates that BOTL achieves a loss no worse than the underlying concept drift detection algorithm. We evaluate BOTL using two existing concept drift detection algorithms: RePro and ADWIN. Additionally, we present a concept drift detection algorithm, Adaptive Windowing with Proactive drift detection (AWPro), which reduces the computation and communication demands of BOTL. Empirical results are presented using two data stream generators: the drifting hyperplane emulator and the smart home heating simulator, and real-world data predicting Time To Collision (TTC) from vehicle telemetry. The evaluation shows BOTL and its variants outperform the concept drift detection strategies and the existing state-of-the-art online transfer learning technique.

An Approach For Concept Drift Detection in a Graph Stream Using Discriminative Subgraphs

The emergence of mining complex networks like social media, sensor networks, and the world-wide-web has attracted considerable research interest. In a streaming scenario, the concept to be learned can change over time. However, while there has been some research done for detecting concept drift in traditional data streams, little work has been done on addressing concept drift in data represented as a graph . We propose a novel unsupervised concept-drift detection method on graph streams called Discriminative Subgraph-based Drift Detector (DSDD). The methodology starts by discovering discriminative subgraphs for each graph in the stream. We then compute the entropy of the window based on the distribution of discriminative subgraphs with respect to the graphs and then use the direct density-ratio estimation approach for detecting concept drift in the series of entropy values obtained by moving one step forward in the sliding window. The effectiveness of the proposed method is demonstrated through experiments using artificial and real-world datasets and its performance is evaluated by comparing against related baseline methods. Similarly, the usefulness of the proposed concept drift detection approach is studied by incorporating it in a popular graph stream classification algorithm and studying the impact of drift detection in classification accuracy.

State of health prediction for lithium‐ion batteries with a novel online sequential extreme learning machine method

State of health (SOH) prediction is always a research hotspot in the field of lithium-ion batteries (LIBs). Machine learning (ML) methods have received widespread attention for their high prediction accuracy. However, the existing studies only focus on extracting features from simple constant current charge-discharge curves or using features that require pre-processing, while the actual discharge current is random and can affect battery aging. Besides, the online sequential extreme learning machine (OSELM) currently used in ML lacks a more efficient online learning and update mechanism in terms of prediction. Therefore, this paper firstly extracts effective features from the random discharge data and conducts a mechanism analysis to verify its rationality. Then, we propose a drift detection based on the Bernstein inequality (BI-DD) algorithm and use it to guide the OSELM to save learning time. The experimental results show the OSELM based on the BI-DD can perform good learning for SOH prediction in a shorter time. The learning time can be reduced by up to 88.87% and the mean absolute error (MAE) does not exceed 1%, which is a promising SOH prediction method. Highlights Extract aging features from random discharge data and the rationality of the extracted features is analyzed according to the mechanism. A drift detection algorithm based on Bernstein inequality (BI-DD) is proposed. An OSELM based on concept drift detection is proposed and for SOH online learning and prediction.

Real-Time Construction Simulation Coupling a Concrete Temperature Field Interval Prediction Model with Optimized Hybrid-Kernel RVM for Arch Dams

Joint grouting simulation is an important aspect of arch dam construction simulation. However, the current construction simulation model simplifies the temperature factors in joint grouting simulation, which leads to the difference between the simulation results and the actual construction schedule. Furthermore, the majority of existing temperature prediction research is based on deterministic point predictions, which cannot quantify the uncertainties of the prediction values. Thus, this study presents a real-time construction simulation method coupling a concrete temperature field interval prediction model to address these problems. First, a real-time construction simulation model is established. Secondly, this paper proposes a concrete temperature interval prediction method based on the hybrid-kernel relevance vector machine (HK-RVM) with the improved grasshopper optimization algorithm (IGOA). The hybrid-kernel method is adopted to ensure the prediction accuracy and generalization ability of the model. Additionally, the improved grasshopper optimization algorithm (IGOA), which utilizes the tent chaotic map and cosine adaptive method to improve the algorithm performance, is developed for the parameter optimization of HK-RVM. Thirdly, concept drift detection based on variable window technology is proposed to update the prediction model. Finally, an arch dam project in China is used as a case study, by which the superiority and applicability of the proposed method are proven.

Concept Drift Detection: Dealing With Missing Values via Fuzzy Distance Estimations

In data streams, the data distribution of arriving observations at different time points may change—a phenomenon called concept drift. While detecting concept drift is a relatively mature area of study, solutions to the uncertainty introduced by observations with missing values have only been studied in isolation. No one has yet explored whether or how these solutions might impact drift detection performance. We, however, believe that data imputation methods may actually increase uncertainty in the data rather than reducing it. We also conjecture that imputation can introduce bias into the process of estimating distribution changes during drift detection, which can make it more difficult to train a learning model. Our idea is to focus on estimating the distance between observations rather than estimating the missing values, and to define membership functions that allocate observations to histogram bins according to the estimation errors. Our solution comprises a novel masked distance learning (MDL) algorithm to reduce the cumulative errors caused by iteratively estimating each missing value in an observation and a fuzzy-weighted frequency (FWF) method for identifying discrepancies in the data distribution. The concept drift detection algorithm proposed in this article is a singular and unified algorithm that can handle missing values, but not an imputation algorithm combined with a concept drift detection algorithm. Experiments on both synthetic and real-world datasets demonstrate the advantages of this method and show its robustness in detecting drift in data with missing values. The results show that compared to the best-performing algorithm that handles imputation and drift detection separately, MDL-FWF reduced the average drift detection difference from 10.75% to 5.83%. This is a nearly 46% improvement. These findings reveal that missing values exert a profound impact on concept drift detection, but using fuzzy set theory to model observations can produce more reliable results than imputation.

Comparing Concept Drift Detection with Process Mining Software

Organisations have seen a rise in the volume of data corresponding to business processes being recorded. Handling process data is a meaningful way to extract relevant information from business processes with impact on the company's values. Nonetheless, business processes are subject to changes during their executions, adding complexity to their analysis. This paper aims at evaluating currently available process mining tools and software that handle concept drifts, i.e. changes over time of the statistical properties of the events occurring in a process. We provide an in-depth analysis of these tools, comparing their differences, advantages, and disadvantages by testing against a log taken from a Process Control System. Thus, by highlighting the trade-off between the software, the paper gives the stakeholders the best options regarding their case use.

Novel hybrid pair recommendations based on a large-scale comparative study of concept drift detection

During the classification of streaming data, changes in the underlying distribution make formerly learned models insecure and imprecise, which is known as the concept drift phenomenon. Online learning derives information from a vast volume of stream data, which are usually affected by these changes in unforeseen ways and are currently generated primarily by the Internet of Things, social media applications, and the stock market. There is abundant literature focused on addressing concept drift using detectors, which essentially attempt to forecast the position of the change to improve the overall accuracy by altering the base learner. This paper presents novel hybrid pairs (classifier and detector) collected from a large-scale comparison of 15 drift detectors; drift detection method (DDM), early drift detection method (EDDM), EWMA for concept drift detection (ECDD), adaptive sliding window (ADWIN), geometrical moving average (GMA), drift detection methods based on Hoeffding’s bound (HDDMA and HDDMW), Fisher exact test drift detector (FTDD), fast Hoeffding drift detection method (FHDDM), Page–Hinkley test (PH), reactive drift detection method (RDDM), SEED, statistical test of equal proportions (STEPD), SeqDrift2, and Wilcoxon rank-sum test drift detector (WSTD) and six classifiers; Naïve Bayes (NB), Hoeffding tree (HT), Hoeffding option tree (HOT), Perceptron (P), decision stump (DS), and k-nearest neighbour (KNN), to determine and recommend the best pair in accordance with the properties of the dataset. The objective of this study is to assess the contribution of a detector to a classifier and obtain the most efficient matched pairs. Through these pairwise comparison experiments, the accuracy rates and evaluation times of the pairs, as well as their false positives, true negatives, false negatives, true positives, drift detection delay, and the MCC. Additionally, the Nemenyi test is employed to compare the pairs against other methods to identify the method(s) for which there is a statistical difference. The results of the experiments indicate that the most efficient pairs—which differed for each dataset type and size—primarily include the HDDMA, RDDM, WSTD, and FHDDM detectors.

Mining frequent itemsets from streaming transaction data using genetic algorithms

This paper presents a study of mining frequent itemsets from streaming data in the presence of concept drift. Streaming data, being volatile in nature, is particularly challenging to mine. An approach using genetic algorithms is presented, and various relationships between concept drift, sliding window size, and genetic algorithm constraints are explored. Concept drift is identified by changes in frequent itemsets. The novelty of this work lies in determining concept drift using frequent itemsets for mining streaming data, using the genetic algorithm framework. Formulas have been presented for calculating minimum support counts in streaming data using sliding windows. Testing highlighted that the ratio of the window size to transactions per drift was a key to good performance. Getting good results when the sliding window size was too small was a challenge since normal fluctuations in the data could appear to be a concept drift. Window size must be managed in conjunction with support and confidence values in order to achieve reasonable results. This method of detecting concept drift performed well when larger window sizes were used.

An overview of unsupervised drift detection methods

AbstractPractical applications involving big data, such as weather monitoring, identification of customer preferences, Internet log analysis, and sensors warnings require challenging data analysis, since these are examples of problems whose data are generated in streams and usually demand real‐time analytics. Patterns in such data stream problems may change quickly. Consequently, machine learning models that operate in this context must be updated over time. This phenomenon is called concept drift in machine learning and data mining literature. Several different directions have been pursued to learn from data stream and to deal with concept drift. However, most drift detection methods consider that an instance's class label is available right after its prediction, since these methods work by monitoring the prediction results of a base classifier or an ensemble of classifiers. Nevertheless, this constraint is unrealistic in several practical problems. To cope with this constraint, some works are focused on proposing efficient unsupervised or semi‐supervised concept drift detectors. While interesting and recent overview papers dedicated to supervised drift detectors have been published, the scenario is not the same in terms of unsupervised methods. Therefore, this work presents a comprehensive overview of approaches that tackle concept drift in classification problems in an unsupervised manner. Additional contribution includes a proposed taxonomy of state‐of‐the‐art approaches for concept drift detection based on unsupervised strategies.This article is categorized under: Technologies > Classification Technologies > Machine Learning

CDDM: Concept Drift Detection Model for Data Stream

<p>Data stream is the huge amount of data generated in various fields, including financial processes, social media activities, Internet of Things applications, and many others. Such data cannot be processed through traditional data mining algorithms due to several constraints, including limited memory, data speed, and dynamic environment. Concept Drift is known as the main constraint of data stream mining, mainly in the classification task. It refers to the change in the data stream underlining distribution over time. Thus, it results in accuracy deterioration of classification models and wrong predictions. Spam emails, consumer behavior changes, and adversary activates, are examples of Concept Drift. In this paper, a Concept Drift detection model is introduced, Concept Drift Detection Model (CDDM). It monitors the accuracy of the classification model over a sliding window, assuming the decline in accuracy indicates a drift occurrence. A modification over CDDM is a weighted version of the CDDM as W-CDDM.</p><p>Both models have evaluated against two real datasets and four artificial datasets. The experimental results of abrupt drift show that CDDM, W-CDDM outperforms the other models in the dataset of 100K and 1M instances, respectively. Regarding gradual drift, the W-CDDM overtook the rest in terms of accuracy, run time, and detection delays in the dataset of 100 K instances. While in the dataset of 1M instances, CDDM has got the highest accuracy using the NB classifier. Moreover, W-CDDM achieves the highest accuracy on real datasets.</p>

Efficient mining of IoT based data streams for advanced computer vision systems

With emergence of Internet of Things (IoT) and subsequent technologies, smart devices are being increasingly used in various domains such as smart homes, smart parking, intelligent transportation etc. Vast amount of image and video data has been produced by IoT based systems in the form of continuous and possibly infinite image and video streams. This demands the development of advanced predictive vision systems which exploits stream mining concepts for intelligent processing of visual data streams. Among other challenges faced by visual data streams, a major challenge is concept drift, which is caused by overtime change in data distribution. In the presence of skewed data, the detection of concept drift becomes more challenging. When analyzing the data generated from smart devices and other performance critical wireless sensors, concept drift affects data integrity and accuracy of prediction results. EWMA for Concept Drift Detection (ECDD) has been proposed in the literature for detecting data streams. However, ECDD has a high prediction error rate which makes it less useful for performance critical data streams generated by imaging and video data streams. In this paper, Vision based Drift Detection Method (VisDDM) is proposed, which systematically handles abrupt and gradual concept drift in data streams. Experiments have been performed using synthetic and real world datasets from different application domains. Our proposed VisDDM algorithm is able to handle abrupt and gradual drift types and outperformed the existing drift detection methods in terms of accuracy and mean evaluation time.

GEP‐based classifiers with drift‐detection

AbstractIn the paper, we propose two gene expression programming (GEP)‐based ensemble classifiers with different drift detection mechanisms. In the related work section, we briefly review GEP as a classification tool, incremental classifiers, and concept drift detectors. Next, the structure of our two‐level GEP ensemble with metagenes is described. Further on, two integrated classifiers with drift detection algorithm and Wilcoxon rank sum test drift detector are proposed. The approach is validated in the computational experiment in which several real‐life and artificial datasets with concept drift have been used. Experiment confirmed that the proposed approach can be competitive to existing solutions. In the conclusion section, we briefly outline directions for future research.

A Novel Drift Detection Algorithm Based on Features’ Importance Analysis in a Data Streams Environment

AbstractThe training set consists of many features that influence the classifier in different degrees. Choosing the most important features and rejecting those that do not carry relevant information is of great importance to the operating of the learned model. In the case of data streams, the importance of the features may additionally change over time. Such changes affect the performance of the classifier but can also be an important indicator of occurring concept-drift. In this work, we propose a new algorithm for data streams classification, called Random Forest with Features Importance (RFFI), which uses the measure of features importance as a drift detector. The RFFT algorithm implements solutions inspired by the Random Forest algorithm to the data stream scenarios. The proposed algorithm combines the ability of ensemble methods for handling slow changes in a data stream with a new method for detecting concept drift occurrence. The work contains an experimental analysis of the proposed algorithm, carried out on synthetic and real data.