Concept Drift Research Articles

SABeDM: a sliding adaptive beta distribution model for concept drift detection in a dynamic environment

SABeDM: a sliding adaptive beta distribution model for concept drift detection in a dynamic environment

An adaptive XGBoost-based optimized sliding window for concept drift handling in non-stationary spatiotemporal data streams classifications

An adaptive XGBoost-based optimized sliding window for concept drift handling in non-stationary spatiotemporal data streams classifications

Temporal silhouette: validation of stream clustering robust to concept drift

Stream clustering is required in applications where data is generated continuously or periodically and must be processed considering its temporal nature. In the absence of a ground truth, internal validation is the only option to evaluate the quality of performances. Traditional internal validation is commonly used also in stream clustering, even in spite of the fact that it becomes inconsistent in the event of data evolution. Recent trends opt for incremental approaches, but these are closer to change detection rather than validation methods and limit themselves by imposing online validation on online analysis. In this work we study the impact of concept drift in the validation of stream clustering and propose the Temporal Silhouette index, therefore making internal validation conform to streaming data. We conduct tests with more than 200 datasets and contrast performances of four popular stream clustering algorithms with seven validation methods (three static internal, three incremental internal, one external) and the proposed index. Results show the suitability of the Temporal Silhouette index for stream clustering validation in the event of concept drift and different types of outliers. The demand for reliable unsupervised learning in applications that process data in streams is ever-increasing, and such reliability inevitably requires the use of validation. This fact highlights the significance of the novel approach proposed in this work.

Lightweight and Robust Malware Detection Using Dictionaries of API Calls

Malware in today’s business world has become a powerful tool used by cyber attackers. It has become more advanced, spreading quickly and causing significant harm. Modern malware is particularly dangerous because it can go undetected, making it difficult to investigate and stop in real time. For businesses, it is vital to ensure that the computer systems are free from malware. To effectively address this problem, the most responsive solution is to operate in real time at the system’s edge. Although machine learning and deep learning have given promising performance for malware detection, the significant challenge is the required processing power and resources for implementation at the system’s edge. Therefore, it is important to prioritize a lightweight approach at the system’s edge. Equally important, the robustness of the model against the concept drift at the system’s edge is crucial to detecting the evolved zero-day malware attacks. Application programming interface (API) calls emerge as the most promising candidate to provide such a solution. However, it is quite challenging to create API call features to achieve a lightweight implementation, high malware detection rate, robustness, and fast execution. This study seeks to investigate and analyze the reuse rate of API calls in both malware and goodware, shedding light on the limitations of API call dictionaries for each class using different datasets. By leveraging these dictionaries, a statistical classifier (STC) is introduced to detect malware samples. Furthermore, the study delves into the investigation of model drift in the STC model, employing entirely distinct datasets for training and testing purposes. The results show the outstanding performance of the STC model in accurately detecting malware, achieving a recall value of one, and exhibiting robustness against model drift. Furthermore, the proposed STC model shows comparable performance to deep learning algorithms, which makes it a strong competitor for performing real-time inference on edge devices.

Online Feature Screening for Data Streams With Concept Drift

Screening feature selection methods are often used as a preprocessing step for reducing the number of variables before training step. Traditional screening methods only focus on dealing with complete high dimensional datasets. Modern datasets not only have higher dimension and larger sample size, but also have properties such as streaming input, sparsity and concept drift. Therefore a considerable number of online feature selection methods were introduced to handle these kind of problems in recent years. Online screening methods are one of the categories of online feature selection methods. The methods that we proposed in this research are capable of handling all three situations mentioned above. Our research study focuses on classification datasets. Our experiments show proposed methods can generate the same feature importance as their offline version with faster speed and less storage consumption. Furthermore, the results show that online screening methods with integrated model adaptation have a higher true feature detection rate than without model adaptation on data streams with the concept drift property. Among the two large real datasets that potentially have the concept drift property, online screening methods with model adaptation show advantages in either saving computing time and space, reducing model complexity, or improving prediction accuracy.

Autonomic active learning strategy using cluster-based ensemble classifier for concept drifts in imbalanced data stream

Systems are becoming autonomous nowadays. In real-world scenarios, the involvement of experts in generating a new decision has to be paid a certain amount of cost. Thus, in these conditions, creating a system with the least expert involvement and costs is the main destination. As a solution, an autonomic active learning strategy with a cluster-based ensemble classifier for concept drifts in imbalanced data stream (AACE-DI) is proposed in this paper. Initially, a cluster-based initializing formula is considered to select the most informative instances for labeling a dynamically changing ensemble classifier. Next, the automatically adjusting uncertainty strategy is introduced to prioritize the most uncertain, representative, and minority class data from imbalanced data stream, which also ensures the least labeling cost. This automatic formula not only outperforms the system but also makes it autonomic. Experimental evaluations are organized using 6 real-world data streams and 16 synthetic data streams with different types of concept drift and imbalanced ratios. To measure performance, evaluators, such as the prequential area under curve, recall, and geometric mean, are considered. Moreover, the proposed AACE-DI method achieves expected outcomes with the least cost as compared with other state-of-the-art learning methods.

Dynamic budget allocation for sparsely labeled drifting data streams

Learning from non-stationary data streams is inherently challenging due to their evolving nature and concept drift. Furthermore, the assumption that all instances come labeled is often impractical in real-world applications. Many strategies have been proposed to tackle learning from sparsely labeled data streams. However, they typically rely on fixed labeling budgets, which can be a limitation in the context of drifting data streams. In this study, we introduce a novel active learning strategy that dynamically manages the labeling budget to optimize its utilization and adapt promptly to concept drift. Our approach continuously monitors the data stream for concept drift, and upon detecting such drift, it dynamically increases the maximum labeling budget for a predefined time window. This adjustment provides the classifier with more flexibility to adapt to the new concept. We conducted experiments using 7 synthetic data generators encompassing various drifting scenarios and 7 real-world data streams with different labeling budgets. Our results demonstrate that offering a flexible budget to the classifier can significantly enhance performance compared to merely increasing a fixed budget. Notably, our strategy outperformed state-of-the-art active learning strategies, all while maintaining a comparable or lower number of labeled instances. Experiments are available at https://github.com/gabrieljaguiar/DBAL.

Multi-Cat Monitoring System Based on Concept Drift Adaptive Machine Learning Architecture

In multi-cat households, monitoring individual cats’ various behaviors is essential for diagnosing their health and ensuring their well-being. This study focuses on the defecation and urination activities of cats, and introduces an adaptive cat identification architecture based on deep learning (DL) and machine learning (ML) methods. The architecture comprises an object detector and a classification module, with the primary focus on the design of the classification component. The DL object detection algorithm, YOLOv4, is used for the cat object detector, with the convolutional neural network, EfficientNetV2, serving as the backbone for our feature extractor in identity classification with several ML classifiers. Additionally, to address changes in cat composition and individual cat appearances in multi-cat households, we propose an adaptive concept drift approach involving retraining the classification module. To support our research, we compile a comprehensive cat body dataset comprising 8934 images of 36 cats. After a rigorous evaluation of different combinations of DL models and classifiers, we find that the support vector machine (SVM) classifier yields the best performance, achieving an impressive identification accuracy of 94.53%. This outstanding result underscores the effectiveness of the system in accurately identifying cats.

Adaptive Fast XGBoost for Multiclass Classification

The popularization of sensoring and connectivity technologies like 5G and IoT are boosting the generation of data streams. Such kinds of data are one of the last frontiers of data mining applications. However, data streams are massive and unbounded sequences of non-stationary data objects that are continuously generated at rapid rates. To deal with these challenges, the learning algorithms should analyze the data just once and update their classifiers to handle the concept drifts. The literature presents some algorithms to deal with the classification of multiclass data streams. However, most of them have high processing time. Therefore, this work proposes a XGBoost-based classifier called AFXGB-MC to fast classify non-stationary data streams with multiple classes. We compared it with the six state-of-the-art algorithms for multiclass classification found in the literature. The results pointed out that AFXGB-MC presents similar accuracy performance, but with faster processing time, being twice faster than the second fastest algorithm from the literature, and having fast drift recovery time.

Build a Lightweight Dataset and Concept Drift Detection Method for Evolving Time Series Data Streams

Build a Lightweight Dataset and Concept Drift Detection Method for Evolving Time Series Data Streams

Label-free timing analysis of SiPM-based modularized detectors with physics-constrained deep learning

Pulse timing is an important topic in nuclear instrumentation, with far-reaching applications from high energy physics to radiation imaging. While high-speed analog-to-digital converters become more and more developed and accessible, their potential uses and merits in nuclear detector signal processing are still uncertain, partially due to associated timing algorithms which are not fully understood and utilized. In this paper, we propose a novel method based on deep learning for timing analysis of modularized detectors without explicit needs of labeling event data. By taking advantage of the intrinsic time correlations, a label-free loss function with a specially designed regularizer is formed to supervise the training of neural networks (NNs) towards a meaningful and accurate mapping function. We mathematically demonstrate the existence of the optimal function desired by the method, and give a systematic algorithm for training and calibration of the model. The proposed method is validated on two experimental datasets based on silicon photomultipliers as main transducers. In the toy experiment, the NN model achieves the single-channel time resolution of 8.8 ps and exhibits robustness against concept drift in the dataset. In the electromagnetic calorimeter experiment, several NN models (fully-connected, convolutional neural network and long short-term memory) are tested to show their conformance to the underlying physical constraint and to judge their performance against traditional methods. In total, the proposed method works well in either ideal or noisy experimental condition and recovers the time information from waveform samples successfully and precisely.

Tourism demand forecasting under conceptual drift during COVID-19: an ensemble deep learning model

ABSTRACT To address the issue of tourism demand forecasting in the context of concept drift, a new ensemble deep learning model based on transformer is proposed, which includes three parts: data processing, base predictor pool construction, base predictor selection and combination. In the first part, the relevant data are collected and converted into the input form required by transformer. In the second part, a base predictor pool containing multiple predictors is constructed, where each predictor can capture a specific concept from historical data. In the final part, a predictor selection algorithm is proposed to select ‘effective predictors’ from the base predictor pool. These effective predictors are further integrated to generate the final forecasts. The proposed model is applied to the forecast of tourist volume of two attractions in China. The results show that the proposed model outperforms the benchmark models in the context of concept drift, benchmarked against eight models.

QuadCDD: A Quadruple-based Approach for Understanding Concept Drift in Data Streams

Concept drift is a prevalent phenomenon in data streams that necessitates detection and in-depth understanding, as it signifies that the statistical properties of a target variable, which the model aims to predict, change over time in unforeseen ways. Existing detection methods predominantly aim to identify the drift start time, which lack comprehensive understanding of data streams, leading to a loss of drift information. In this paper, we present a novel Quadruple-based Approach for Understanding Concept Drift in Data Streams (QuadCDD) framework that not only detects and predicts the concept drift start point but also offers a more detailed analysis of concept drift through the use of quadruples, encompassing drift start, drift end, drift severity, and drift type. Our framework employs quadruples to enable informed decision-making and adopt appropriate actions to handle various concept drifts, effectively maintaining high and stable performance in data streams with concept drift. Experimental results validate the effectiveness of our QuadCDD framework in accurately detecting and understanding concept drifts, as well as in preserving the stability and performance of models in the presence of these drifts.

A reliable adaptive prototype-based learning for evolving data streams with limited labels

Data stream mining presents notable challenges in the form of concept drift and evolution. Existing learning algorithms, typically designed within a supervised learning framework, require class labels for all data points. However, this is an impractical requirement given the rapid pace of data streams, which often results in label scarcity. Recognizing the realistic necessity of learning from data streams with limited labels, we propose an adaptive, data-driven, prototype-based semi-supervised learning framework specifically tailored to handle evolving data streams. Our method employs a prototype-based data representation, summarizing the continuous flow of streaming data using dynamic prototypes at varying levels of granularity. This technique enables improved data abstraction, capturing the underlying local data distributions more accurately. The model also incorporates reliability modeling and efficient emerging class discovery, dynamically updating the significance of prototypes over time and swiftly adapting to local concept drift. We further leverage these adaptive prototypes to intuitively detect concept evolution, i.e., identifying novel classes from a local density perspective. To minimize the need for manual labeling while optimizing performance, we incorporate active learning into our method. This method employs a dual-criteria approach for data point selection, considering both uncertainty and local density. These manually labeled data points, together with unlabeled data, serve to update the model efficiently and robustly. Empirical validation using several benchmark datasets demonstrates promising performance in comparison to existing state-of-the-art techniques.

Research on the innovation path of corporate financial management based on improved Adaboost regression algorithm and analysis of management effectiveness

Abstract The first part of this paper investigates the implementation process of the Adaboost regression algorithm, initializes the training set weights to obtain the weak learner, and builds the final strong learner. Then, we explore the treatment of corporate financial management concept drift by assigning variance weights to financial data samples according to their temporal proximity to solve the problem of financial concept drift. Then, an optimized AdaBoost-SVM model is constructed to initialize the sample weights according to time and sample category, calculate the classification error, and build a weak classifier and a decision classifier based on the samples. Finally, the company's financial management costs, personnel, and model prediction accuracy were analyzed. The model predicts financial distress with a 78% accuracy, financial insolvency with a 93% accuracy, and financial normalcy with a 90% accuracy. This confirms the feasibility of this paper's methodology for corporate financial management.

Efficient concept drift handling for batch android malware detection models

The rapidly evolving nature of Android apps poses a significant challenge to static batch machine learning algorithms employed in malware detection systems, as they quickly become obsolete. Despite this challenge, the existing literature pays limited attention to addressing this issue, with many advanced Android malware detection approaches, such as Drebin, DroidDet and MaMaDroid, relying on static models. In this work, we show how retraining techniques are able to maintain detector capabilities over time. Particularly, we analyze the effect of two aspects in the efficiency and performance of the detectors: (1) the frequency with which the models are retrained, and (2) the data used for retraining. In the first experiment, we compare periodic retraining with a more advanced concept drift detection method that triggers retraining only when necessary. In the second experiment, we analyze sampling methods to reduce the amount of data used to retrain models. Specifically, we compare fixed sized windows of recent data and state-of-the-art active learning methods that select those apps that help keep the training dataset small but diverse. Our experiments show that concept drift detection and sample selection mechanisms result in very efficient retraining strategies which can be successfully used to maintain the performance of the static Android malware state-of-the-art detectors in changing environments.

Cross-Version Software Defect Prediction Considering Concept Drift and Chronological Splitting

Concept drift (CD) refers to a phenomenon where the data distribution within datasets changes over time, and this can have adverse effects on the performance of prediction models in software engineering (SE), including those used for tasks like cost estimation and defect prediction. Detecting CD in SE datasets is difficult, but important, because it identifies the need for retraining prediction models and in turn improves their performance. If the concept drift is caused by symmetric changes in the data distribution, the model adaptation process might need to account for this symmetry to maintain accurate predictions. This paper explores the impact of CD within the context of cross-version defect prediction (CVDP), aiming to enhance the reliability of prediction performance and to make the data more symmetric. A concept drift detection (CDD) approach is further proposed to identify data distributions that change over software versions. The proposed CDD framework consists of three stages: (i) data pre-processing for CD detection; (ii) notification of CD by triggering one of the three flags (i.e., CD, warning, and control); and (iii) providing guidance on when to update an existing model. Several experiments on 30 versions of seven software projects reveal the value of the proposed CDD. Some of the key findings of the proposed work include: (i) An exponential increase in the error-rate across different software versions is associated with CD. (ii) A moving-window approach to train defect prediction models on chronologically ordered defect data results in better CD detection than using all historical data with a large effect size (δ≥0.427).

OODT: LOS Signal Identification for Acoustic Indoor Localization From Stream Perspective

Acoustic-based technologies have attracted more attention in indoor ranging-based localization and tracking applications. To achieve high-accuracy ranging measurements, the precise identification of line-of-sight(LOS) acoustic signals is essential. This paper proposes a novel Off-Online Dynamic Training method(OODT) to identify LOS acoustic signals from stream perspective based on a few training data in dynamic indoor environment. The dynamic online training method is proposed to identify the unlabeled acoustic feature stream by the Parent-Child model, utilizing the dynamic prior probability from time series information and a selection strategy based on the prediction risk. Then, the trustworthy pseudo-labeled streaming samples are accumulated into the Child-Models by online learning in real-time. To reduce the impact of discarding the untrustworthy LOS signals, the Off-Online retraining method is proposed to incorporate the spatial information into the Parent-Models, which uses the category distribution of all pseudo-labeled streaming samples as the prior probability for iterative retraining. Subsequently, a new round of the dynamic online training is conducted to update the pseudo-labels of feature stream. Experiments demonstrate that the dynamic online training method has a higher identification precision of LOS acoustic signals from stream perspective, reaching 98% and more than 93.06% in above-ground and underground experimental scenarios. Moreover, the adaption for the concept drift is also verified with identification precision of 97.98% and 98.23%, respectively. The Off-Online retraining method further optimize the performance of the Parent-Child models in a single scenario. In conclusion, the proposed OODT method can autonomously identify dynamic acoustic signals with strong robustness and scenario drift adaptability.

Sparse and semi-attention guided faults diagnosis approach for distributed online services

Despite the rapid advance of unsupervised reconstruction models in online service fault diagnosis, existing methods still lead to frequent false positive or false negative alarms. Tracing to the cause, popular reconstruction frameworks may fall into an “identical shortcut”, where both normal and anomalous cases can be well-recovered and hence faults are escaped from monitoring. Moreover, regularly “concept drift” deployed by online services always leads to sequences distribution dramatically change, which is easily wrongly-detected as anomalies. In this work, we present SS-Attention to avoid two contradictory false alarms simultaneously in a unified framework. Specifically, we first revisit the frameworks of MLP, CNN, and transformer, and confirm the important role of self-attention in preventing the model from learning shortcut, which we therefore select as our backbone. Second, we come up with a sparse attention layer to help model capture more long-term dependencies and distinguish local “concept drift” from anomalies. Third, we propose a semi-attention layer that urges model only learn from normal regions and further avoid well-recovering the faults. We evaluate our model on three public online service datasets and surpass the state-of-the-art alternatives by a sufficiently large margin, achieving an average 95.91% F1-Score with an approx. 34.48% economy of training time.

SETL: a transfer learning based dynamic ensemble classifier for concept drift detection in streaming data

SETL: a transfer learning based dynamic ensemble classifier for concept drift detection in streaming data