Evolving Data Streams Research Articles

Synchronization-based semi-supervised data streams classification with label evolution and extreme verification delay

The critical need for classifying streaming data arises from its widespread use in real-world industries, where analyzing continuous, dynamic, and evolving data streams accurately and promptly is essential for informed decision-making and gaining predictive insights. Existing research mainly focuses on abundant supervision, overlooking the scarcity and delayed availability of labels, which can vary in timing. Addressing this, the article introduces a new learning method that employs a synchronization-based core support extraction technique. This technique is designed to manage changing concepts and delayed partial labeling by extracting key data points that act as pseudo-labels. Thanks to the concept of synchronization, these extracted key data points accurately represent the inherent local cluster structure in an intuitive manner and better maintain the class structure. Consequently, these pseudo-labels are utilized for classifying future incoming data batches. Furthermore, the method incorporates a knowledge base to summarize and represent all incoming streaming data. Building upon this knowledge base, an ensemble model for classification and an efficient new class detector are proposed. Both operate in a local fashion to ensure robust learning, even in complex class distributions. Evaluations on benchmark datasets reveal a consistent performance lead, surpassing established algorithms by up to 10%, achieving state-of-the-art results.

Gradient boosted trees for evolving data streams

AbstractGradient Boosting is a widely-used machine learning technique that has proven highly effective in batch learning. However, its effectiveness in stream learning contexts lags behind bagging-based ensemble methods, which currently dominate the field. One reason for this discrepancy is the challenge of adapting the booster to new concept following a concept drift. Resetting the entire booster can lead to significant performance degradation as it struggles to learn the new concept. Resetting only some parts of the booster can be more effective, but identifying which parts to reset is difficult, given that each boosting step builds on the previous prediction. To overcome these difficulties, we propose Streaming Gradient Boosted Trees (Sgbt), which is trained using weighted squared loss elicited in XGBoost. Sgbt exploits trees with a replacement strategy to detect and recover from drifts, thus enabling the ensemble to adapt without sacrificing the predictive performance. Our empirical evaluation of Sgbt on a range of streaming datasets with challenging drift scenarios demonstrates that it outperforms current state-of-the-art methods for evolving data streams.

Adaptive regularized ensemble for evolving data stream classification

Extracting knowledge from data streams requires fast incremental algorithms that are able to handle unlimited processing and ever-changing data with finite memory. A strategy for this challenge is the use of ensembles owing to their ability to tackle concept drift and achieve highly accurate predictions. However, ensembles often require a lot of computational resources. In this study, we propose a novel ensemble-based classification algorithm for data streams, Adaptive Regularized Ensemble (ARE), with low demand for computational resources. The algorithm combines strategies that contribute to high prediction accuracy using only incorrectly classified instances into the training step, random-sized feature subspace for each ensemble element and classifier selection for final ensemble voting. After an extensive experimental study, ARE exhibited high predictive performance and outperformed state-of-the-art ensembles on data streams for real and synthetic datasets while requiring a low processing time and memory consumption.

Multi-type concept drift detection under a dual-layer variable sliding window in frequent pattern mining with cloud computing

The detection of different types of concept drift has wide applications in the fields of cloud computing and security information detection. Concept drift detection can indeed assist in promptly identifying instances where model performance deteriorates or when there are changes in data distribution. This paper focuses on the problem of concept drift detection in order to conduct frequent pattern mining. To address the limitation of fixed sliding windows in adapting to evolving data streams, we propose a variable sliding window frequent pattern mining algorithm, which dynamically adjusts the window size to adapt to new concept drifts and detect them in a timely manner. Furthermore, considering the challenge of existing concept drift detection algorithms that struggle to adapt to different types of drifting data simultaneously, we introduce an additional dual-layer embedded variable sliding window. This approach helps differentiate types of concept drift and incorporates a decay model for drift adaptation. The proposed algorithm can effectively detect different types of concept drift in data streams, perform targeted drift adaptation, and exhibit efficiency in terms of time complexity and memory consumption. Additionally, the algorithm maintains stable performance, avoiding abrupt changes due to window size variations and ensuring overall robustness.

Online Sparse Representation Clustering for Evolving Data Streams.

Data stream clustering can be performed to discover the patterns underlying continuously arriving sequences of data. A number of data stream clustering algorithms for finding clusters in arbitrary shapes and handling outliers, such as density-based clustering algorithms, have been proposed. However, these algorithms are often limited in their ability to construct and merge microclusters by measuring the Euclidean distances between high-dimensional data objects, e.g., transferring valuable knowledge from historical landmark windows to the current landmark window, and exploiting evolving subspace structures adaptively. We propose an online sparse representation clustering (OSRC) method to learn an affinity matrix for evaluating the relationships among high-dimensional data objects in evolving data streams. We first introduce a low-dimensional projection (LDP) into sparse representation to adaptively reduce the potential negative influence associated with the noise and redundancy contained in high-dimensional data. Then, we take advantage of the l2,1 -norm optimization technique to choose the appropriate number of representative data objects and form a specific dictionary for sparse representation. The specific dictionary is integrated into sparse representation to adaptively exploit the evolving subspace structures of the high-dimensional data objects. Moreover, the data object representatives from the current landmark window can transfer valuable knowledge to the next landmark window. The experimental results based on a synthetic dataset and six benchmark datasets validate the effectiveness of the proposed method compared to that of state-of-the-art methods for data stream clustering.

Efficient approximation and privacy preservation algorithms for real time online evolving data streams

Efficient approximation and privacy preservation algorithms for real time online evolving data streams

Smoclust: synthetic minority oversampling based on stream clustering for evolving data streams

Many real-world data stream applications not only suffer from concept drift but also class imbalance. Yet, very few existing studies investigated this joint challenge. Data difficulty factors, which have been shown to be key challenges in class imbalanced data streams, are not taken into account by existing approaches when learning class imbalanced data streams. In this work, we propose a drift adaptable oversampling strategy to synthesise minority class examples based on stream clustering. The motivation is that stream clustering methods continuously update themselves to reflect the characteristics of the current underlying concept, including data difficulty factors. This nature can potentially be used to compress past information without caching data in the memory explicitly. Based on the compressed information, synthetic examples can be created within the region that recently generated new minority class examples. Experiments with artificial and real-world data streams show that the proposed approach can handle concept drift involving different minority class decomposition better than existing approaches, especially when the data stream is severely class imbalanced and presenting high proportions of safe and borderline minority class examples.

Towards time-evolving analytics: Online learning for time-dependent evolving data streams

Traditional historical data analytics is at risk in a world where volatility, uncertainty, complexity, and ambiguity are the new normal. While Streaming Machine Learning (SML) and Time-series Analytics (TSA) attack some aspects of the problem, we still need a comprehensive solution. SML trains models using fewer data and in a continuous/adaptive way relaxing the assumption that data points are identically distributed. TSA considers temporal dependence among data points, but it assumes identical distribution. Every Data Scientist fights this battle with ad-hoc solutions. In this paper, we claim that, due to the temporal dependence on the data, the existing solutions do not represent robust solutions to efficiently and automatically keep models relevant even when changes occur, and real-time processing is a must. We propose a novel and solid scientific foundation for Time-Evolving Analytics from this perspective. Such a framework aims to develop the logical, methodological, and algorithmic foundations for fast, scalable, and resilient analytics.

METER: A Dynamic Concept Adaptation Framework for Online Anomaly Detection

Real-time analytics and decision-making require online anomaly detection (OAD) to handle drifts in data streams efficiently and effectively. Unfortunately, existing approaches are often constrained by their limited detection capacity and slow adaptation to evolving data streams, inhibiting their efficacy and efficiency in handling concept drift , which is a major challenge in evolving data streams. In this paper, we introduce METER, a novel dynamic concept adaptation framework that introduces a new paradigm for OAD. METER addresses concept drift by first training a base detection model on historical data to capture recurring central concepts , and then learning to dynamically adapt to new concepts in data streams upon detecting concept drift. Particularly, METER employs a novel dynamic concept adaptation technique that leverages a hypernetwork to dynamically generate the parameter shift of the base detection model, providing a more effective and efficient solution than conventional retraining or fine-tuning approaches. Further, METER incorporates a lightweight drift detection controller, underpinned by evidential deep learning, to support robust and interpretable concept drift detection. We conduct an extensive experimental evaluation, and the results show that METER significantly outperforms existing OAD approaches in various application scenarios.

A reliable adaptive prototype-based learning for evolving data streams with limited labels

Data stream mining presents notable challenges in the form of concept drift and evolution. Existing learning algorithms, typically designed within a supervised learning framework, require class labels for all data points. However, this is an impractical requirement given the rapid pace of data streams, which often results in label scarcity. Recognizing the realistic necessity of learning from data streams with limited labels, we propose an adaptive, data-driven, prototype-based semi-supervised learning framework specifically tailored to handle evolving data streams. Our method employs a prototype-based data representation, summarizing the continuous flow of streaming data using dynamic prototypes at varying levels of granularity. This technique enables improved data abstraction, capturing the underlying local data distributions more accurately. The model also incorporates reliability modeling and efficient emerging class discovery, dynamically updating the significance of prototypes over time and swiftly adapting to local concept drift. We further leverage these adaptive prototypes to intuitively detect concept evolution, i.e., identifying novel classes from a local density perspective. To minimize the need for manual labeling while optimizing performance, we incorporate active learning into our method. This method employs a dual-criteria approach for data point selection, considering both uncertainty and local density. These manually labeled data points, together with unlabeled data, serve to update the model efficiently and robustly. Empirical validation using several benchmark datasets demonstrates promising performance in comparison to existing state-of-the-art techniques.

Uncertainty Estimation With Neural Processes for Meta-Continual Learning.

The ability to evaluate uncertainties in evolving data streams has become equally, if not more, crucial than building a static predictor. For instance, during the pandemic, a model should consider possible uncertainties such as governmental policies, meteorological features, and vaccination schedules. Neural process families (NPFs) have recently shone a light on predicting such uncertainties by bridging Gaussian processes (GPs) and neural networks (NNs). Their abilities to output average predictions and the acceptable variances, i.e., uncertainties, made them suitable for predictions with insufficient data, such as meta-learning or few-shot learning. However, existing models have not addressed continual learning which imposes a stricter constraint on the data access. Regarding this, we introduce a member meta-continual learning with neural process (MCLNP) for uncertainty estimation. We enable two levels of uncertainty estimations: the local uncertainty on certain points and the global uncertainty p(z) that represents the function evolution in dynamic environments. To facilitate continual learning, we hypothesize that the previous knowledge can be applied to the current task, hence adopt a coreset as a memory buffer to alleviate catastrophic forgetting. The relationships between the degree of global uncertainties with the intratask diversity and model complexity are discussed. We have estimated prediction uncertainties with multiple evolving types including abrupt/gradual/recurrent shifts. The applications encompass meta-continual learning in the 1-D, 2-D datasets, and a novel spatial-temporal COVID dataset. The results show that our method outperforms the baselines on the likelihood and can rebound quickly even for heavily evolved data streams.

Balancing Performance and Energy Consumption of Bagging Ensembles for the Classification of Data Streams in Edge Computing

In recent years, the Edge Computing (EC) paradigm has emerged as an enabling factor for developing technologies like the Internet of Things (IoT) and 5G networks, bridging the gap between Cloud Computing services and end-users, supporting low latency, mobility, and location awareness to delay-sensitive applications. An increasing number of solutions in EC have employed machine learning (ML) methods to perform data classification and other information processing tasks on continuous and evolving data streams. Usually, such solutions have to cope with vast amounts of data that come as data streams while balancing energy consumption, latency, and the predictive performance of the algorithms. Ensemble methods achieve remarkable predictive performance when applied to evolving data streams due to several models and the possibility of selective resets. This work investigates a strategy that introduces short intervals to defer the processing of mini-batches. Well balanced, our strategy can improve the performance (i.e., delay, throughput) and reduce the energy consumption of bagging ensembles to classify data streams. The experimental evaluation involved six state-of-art ensemble algorithms (OzaBag, OzaBag Adaptive Size Hoeffding Tree, Online Bagging ADWIN, Leveraging Bagging, Adaptive RandomForest, and Streaming Random Patches) applying five widely used machine learning benchmark datasets with varied characteristics on three computer platforms. As a result, our strategy can significantly reduce energy consumption in 96% of the experimental scenarios evaluated. Despite the trade-offs, it is possible to balance them to avoid significant loss in predictive performance.

CPOCEDS-concept preserving online clustering for evolving data streams

CPOCEDS-concept preserving online clustering for evolving data streams

Semi-supervised federated learning on evolving data streams

Federated learning allows multiple clients to jointly train a model on their private data without revealing their local data to a centralized server. Thereby, federated learning has attracted increasing attention in recent years, and many algorithms have been proposed. However, existing federated learning algorithms often focus on static data, which tend to fail in the scenarios of data streams. Due to the varying distributions/concepts within and among the clients over time, the joint learning model must learn these different emerging concepts dynamically and simultaneously. The task becomes more challenging when the continuous arriving data are partially labeled for the participating clients. In this paper, we propose SFLEDS (Semi-supervised Federated Learning on Evolving Data Streams), a new federated learning prototype-based method to tackle the problems of label scarcity, concept drift, and privacy preservation in the federated semi-supervised evolving data stream environment. Extensive experiments show that SFLEDS outperforms both state-of-the-art semi-supervised and supervised algorithms. The source code for the proposed method is publicly available on github (https://github.com/mvisionai/FedLimited).

Short-term solar irradiance forecasting in streaming with deep learning

Solar energy is one of the most common and promising sources of renewable energy. In photovoltaic (PV) systems, operators can benefit from future solar irradiance predictions for efficient load balancing and grid stability. Therefore, short-term solar irradiance forecasting plays a crucial role in the transition to renewable energy. Modern PV grids collect large volumes of data that provide valuable information for forecasting models. Although the nature of these data presents an ideal setting for online learning methodologies, research to date has mainly focused on offline approaches. Hence, this work proposes a novel data streaming method for real-time solar irradiance forecasting on days with variable weather conditions and cloud coverage. Our method operates under an asynchronous dual-pipeline framework using deep learning models. For the experimental study, two datasets from a Canadian PV solar plant have been simulated as streams at different data frequencies. The experiments involve an exhaustive parameter grid search to evaluate four state-of-the-art deep learning architectures: multilayer perceptron (MLP), long-short term memory network (LSTM), convolutional network (CNN), and Transformer network. The obtained results demonstrate the suitability of deep learning models for this problem. In particular, MLP and CNN achieved the best accuracy, with a high capacity to adapt to the evolving data stream.

Towards lowering computational power in IoT systems: Clustering algorithm for high-dimensional data stream using entropy window reduction

In a world of connectivity empowered by the advancement of the Internet of Things (IoT), an infinite number of data streams have emerged. Thus, data stream clustering is crucial for extracting hidden knowledge and data mining. Various data stream clustering methods have lately been introduced. Yet, the majority of such algorithms are affected by the curse of high dimensionality. Lately, a fully online buffer-based clustering algorithm for handling evolving data streams (BOCEDS) was developed. Similarly to other existing density-based clustering methods, BOCEDS is not capable of handling high-dimensional data and has high computational power and high memory utilization. This paper introduces an Entropy Window Reduction (EWR) algorithm, which is an improved version of the BOCEDS technique. EWR is a fully online clustering technique for handling high-dimensional data streams using feature ranking and sorting. This process is accomplished by calculating the entropy of specific features with respect to the time window. The findings of the experiments are compared to the outcomes of BOCEDS, CEDAS, and MuDi-Stream algorithms. The outcomes indicate that the EWR algorithm outperformed the baseline clustering algorithms. The results are demonstrated using the KDDCup’99 dataset in terms of quality and complexity evaluation on the average of F-Measures, Jaccard Index, Fowlkes–Mallows index, Purity, and Rand Index as well as the memory usage and computational power with 88%, 66%, 81%, 100%, and 66%, respectively. The results also show low memory usage and computing power in comparison with the baseline algorithms.

Skewed Evolving Data Streams Classification with Actionable Knowledge Extraction using Data Approximation and Adaptive Classification Framework

Skewed evolving data stream (SEDS) classification is a challenging research problem for online streaming data applications. The fundamental challenges in streaming data classification are class imbalance and concept drift. However, recently, either independently or together, the two topics have received enough attention; the data redundancy while performing stream data mining and classification remains unexplored. Moreover, the existing solutions for the classification of SEDSs have focused on solving concept drift and/or class imbalance problems using the sliding window mechanism, which leads to higher computational complexity and data redundancy problems. To end this, we propose a novel Adaptive Data Stream Classification (ADSC) framework for solving the concept drift, class imbalance, and data redundancy problems with higher computational and classification efficiency. Data approximation, adaptive clustering, classification, and actionable knowledge extraction are the major phases of ADSC. For the purpose of approximating unique items in the data stream with data pre-processing during the data approximation phase, we develop the Flajolet Martin (FM) algorithm. The periodically approximated tuples are grouped into distinct classes using an adaptive clustering algorithm to address the problem of concept drift and class imbalance. In the classification phase, the supervised classifiers are employed to classify the unknown incoming data streams into either of the classes discovered by the adaptive clustering algorithm. We then extract the actionable knowledge using classified skewed evolved data stream information for the end user decision-making process. The ADSC framework is empirically assessed utilizing two streaming datasets regarding classification and computing efficiency factors. The experimental results shows the better efficiency of the proposed ADSC framework as compared with existing classification methods.

StreamDFP: A General Stream Mining Framework for Adaptive Disk Failure Prediction

We explore machine learning for accurately predicting imminent disk failures and hence providing proactive fault tolerance for modern large-scale storage systems. Current disk failure prediction approaches are mostly offline and assume that the disk logs required for training learning models are available a priori. However, disk logs are often continuously generated as an evolving data stream, in which the statistical patterns vary over time (also known as concept drift). Such a challenge motivates the need of online techniques that perform training and prediction on the incoming stream of disk logs in real time, while being adaptive to concept drift. We first measure and demonstrate the existence of concept drift on various disk models in production. Motivated by our study, we design <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">StreamDFP</small> , a general stream mining framework for disk failure prediction with concept-drift adaptation based on three key techniques, namely online labeling, concept-drift-aware training, and general prediction, with a primary objective of supporting various machine learning algorithms. We extend <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">StreamDFP</small> to support online transfer learning for minority disk models with concept-drift adaptation. Our evaluation shows that <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">StreamDFP</small> improves the prediction accuracy significantly compared to without concept-drift adaptation under various settings, and achieves reasonably high stream processing performance.

A novel semi-supervised classification approach for evolving data streams

Classification plays a crucial role in mining the evolving data streams. The concept drift and concept evolution are the major issues of data streams classification, which greatly affect the classification performance. Most existing works of concept drift and evolution are supervised in nature, where labeling the data is time and resource consuming. In this paper, for the evolving data streams, a semi-supervised classification approach using partially labeled data is proposed. Firstly, an ensemble model dynamically maintains a series of micro-clusters to capture the concept drift. The ensemble model processes the instances in an online fashion rather than chunk-based. Secondly, the concept evolution detection module is constructed to detect the outliers by the local density. The module examines the current buffer to capture the class emergence in data streams with complex class distribution. For improving the execution efficiency of emerging class detection without compromising performance, several constructive strategies are adopted, including removing part of the buffer and selectively executing sample generation. The extensive experiments are constructed about the popular data streams sets and the processed industry data streams, whose results indicate the practicality and effectiveness of the proposed approach for the classification of evolving data streams.

Evolving data stream clustering based on constant false clustering probability

Today’s world needs new methods to deal with and analyze the ever-increasingly generated data streams. Two of the most challenging aspects of data streams are (i) concept drift, i.e. evolution of data stream over time, which requires the ability to make timely decisions against the high speed of receiving new data; (ii) limited memory storage and the impracticality of using memory due to the large amount of data. Clustering is one of the common methods to process data streams. In this paper, we propose a novel, fully-online, density-based method for clustering evolving data streams. In recent years, a number of methods have been proposed, which also have the ability to cluster data streams. The main limitation of these methods is the use of parameters based on knowledge-expert. This work is among the first works to address this issue. Constant False Clustering Probability, CFCP, has tried to choose the algorithm’s parameters based on Statistics. The proposed method has also the ability to identify clusters with arbitrary shapes. It is robust to noise and offers high accuracy and efficiency in both low and high dimensions. In this method, we determine the value of the parameters by using statistical theories and do not require more information, taking advantage of expert-knowledge. The presented experimental results show that the method performs data clustering at high speeds without reducing the quality compared to the state-of-the-art algorithms.