Imbalanced Data Streams Research Articles

An overview of complex data stream ensemble classification

With the rapid development of information technology, data streams in various fields are showing the characteristics of rapid arrival, complex structure and timely processing. Complex types of data streams make the classification performance worse. However, ensemble classification has become one of the main methods of processing data streams. Ensemble classification performance is better than traditional single classifiers. This article introduces the ensemble classification algorithms of complex data streams for the first time. Then overview analyzes the advantages and disadvantages of these algorithms for steady-state, concept drift, imbalanced, multi-label and multi-instance data streams. At the same time, the application fields of data streams are also introduced which summarizes the ensemble algorithms processing text, graph and big data streams. Moreover, it comprehensively summarizes the verification technology, evaluation indicators and open source platforms of complex data streams mining algorithms. Finally, the challenges and future research directions of ensemble learning algorithms dealing with uncertain, multi-type, delayed, multi-type concept drift data streams are given.

Missing information in imbalanced data stream: fuzzy adaptive imputation approach

Missing information in imbalanced data stream: fuzzy adaptive imputation approach

An online weighted sequential extreme learning machine for class imbalanced data streams

When general online classification algorithms deal with imbalanced data streams, there are always some problems, such as over fitting phenomenon caused by insufficient simple learning and instability of training model. In this paper, we introduce online sequential extreme learning machine (OSELM) as the basic theory model, and combine with the cost-sensitive strategy, then propose a cost-sensitive learning based online sequential extreme learning machine algorithm (C-OSELM). Firstly, in order to solve the problem that minority classes are easily misclassified due to class imbalance, use cost-sensitive strategy, by assigning different penalty parameters to various samples, a weighting matrix is constructed to improve the misclassification cost, thereby effectively alleviating the excessive deviation of decision surface. On this basis, in order to solve the problem that the penalty parameter is too single and the algorithm is not universal, the cost adjustment function is introduced to optimize the weight parameters to select the appropriate weight. Finally, 16 class II imbalanced datasets are used for comparison and verification. The experimental results show that the classification performances of the proposed C-OSELM algorithm are better than other comparative algorithms.

Deep learning framework for handling concept drift and class imbalanced complex decision-making on streaming data

AbstractIn present times, data science become popular to support and improve decision-making process. Due to the accessibility of a wide application perspective of data streaming, class imbalance and concept drifting become crucial learning problems. The advent of deep learning (DL) models finds useful for the classification of concept drift in data streaming applications. This paper presents an effective class imbalance with concept drift detection (CIDD) using Adadelta optimizer-based deep neural networks (ADODNN), named CIDD-ADODNN model for the classification of highly imbalanced streaming data. The presented model involves four processes namely preprocessing, class imbalance handling, concept drift detection, and classification. The proposed model uses adaptive synthetic (ADASYN) technique for handling class imbalance data, which utilizes a weighted distribution for diverse minority class examples based on the level of difficulty in learning. Next, a drift detection technique called adaptive sliding window (ADWIN) is employed to detect the existence of the concept drift. Besides, ADODNN model is utilized for the classification processes. For increasing the classifier performance of the DNN model, ADO-based hyperparameter tuning process takes place to determine the optimal parameters of the DNN model. The performance of the presented model is evaluated using three streaming datasets namely intrusion detection (NSL KDDCup) dataset, Spam dataset, and Chess dataset. A detailed comparative results analysis takes place and the simulation results verified the superior performance of the presented model by obtaining a maximum accuracy of 0.9592, 0.9320, and 0.7646 on the applied KDDCup, Spam, and Chess dataset, respectively.

Classification of Imbalanced Data Stream: Techniques and Challenges

As the number of generated data increases every day, this has brought the importance of data mining and knowledge extraction. In traditional data mining, offline status can be used for knowledge extraction. Nevertheless, dealing with stream data mining is different due to continuously arriving data that can be processed at a single scan besides the appearance of concept drift. As the pre-processing stage is critical in knowledge extraction, imbalanced stream data gain significant popularity in the last few years among researchers. Many real-world applications suffer from class imbalance including medical, business, fraud detection and etc. Learning from the supervised model includes classes whether it is binary- or multi-classes. These classes are often imbalance where it is divided into the majority (negative) class and minority (positive) class, which can cause a bias toward the majority class that leads to skew in predictive performance models. Handles imbalance streaming data is mandatory for more accurate and reliable learning models. In this paper, we will present an overview of data stream mining and its tools. Besides, summarize the problem of class imbalance and its different approaches. In addition, researchers will present the popular evaluation metrics and challenges prone from imbalanced streaming data.

Online ensemble learning algorithm for imbalanced data stream

In many practical applications, due to the inability to collect complete training data sets at one time, the adaptability of the classifier is poor. Online ensemble learning can better solve this problem. However, most of the data streams are imbalanced. Imbalanced data stream will greatly affect the performance of online ensemble learning algorithm. To reduce the impact of imbalanced data stream, this paper proposes a cost sensitive online ensemble learning algorithm for imbalanced data stream. The algorithm uses a variety of equalization methods, mainly including the construction of initial base-classifier, dynamic calculation of misclassification cost, sampling method of samples in data stream and calculation of weight of base-classifier. Those methods can reduce the influence of imbalanced data stream and improve the classification performance under imbalanced data stream. The experimental results show that the performance of the proposed algorithm has the better classification performance for imbalanced data stream. Finally, the algorithm is applied to the network intrusion detection, and the simulation experiment on NSL-KDD data set can reduce the missing alarm rate and the false alarm rate. The experimental results show that the algorithm can improve the detection accuracy, especially the recognition rate of unknown intrusion behavior.

The impact of data difficulty factors on classification of imbalanced and concept drifting data streams

Class imbalance introduces additional challenges when learning classifiers from concept drifting data streams. Most existing work focuses on designing new algorithms for dealing with the global imbalance ratio and does not consider other data complexities. Independent research on static imbalanced data has highlighted the influential role of local data difficulty factors such as minority class decomposition and presence of unsafe types of examples. Despite often being present in real-world data, the interactions between concept drifts and local data difficulty factors have not been investigated in concept drifting data streams yet. We thoroughly study the impact of such interactions on drifting imbalanced streams. For this purpose, we put forward a new categorization of concept drifts for class imbalanced problems. Through comprehensive experiments with synthetic and real data streams, we study the influence of concept drifts, global class imbalance, local data difficulty factors, and their combinations, on predictions of representative online classifiers. Experimental results reveal the high influence of new considered factors and their local drifts, as well as differences in existing classifiers’ reactions to such factors. Combinations of multiple factors are the most challenging for classifiers. Although existing classifiers are partially capable of coping with global class imbalance, new approaches are needed to address challenges posed by imbalanced data streams.

Hellinger Distance Weighted Ensemble for imbalanced data stream classification

The imbalanced data classification remains a vital problem. The key is to find such methods that classify both the minority and majority class correctly. The paper presents the classifier ensemble for classifying binary, non-stationary and imbalanced data streams where the Hellinger Distance is used to prune the ensemble. The paper includes an experimental evaluation of the method based on the conducted experiments. The first one checks the impact of the base classifier type on the quality of the classification. In the second experiment, the Hellinger Distance Weighted Ensemble (HDWE) method is compared to selected state-of-the-art methods using a statistical test with two base classifiers. The method was profoundly tested based on many imbalanced data streams and obtained results proved the HDWE method's usefulness.

A comprehensive active learning method for multiclass imbalanced data streams with concept drift

A challenge to many real-world applications is multiclass imbalance with concept drift. In this paper, we propose a comprehensive active learning method for multiclass imbalanced streaming data with concept drift (CALMID). First, we design a comprehensive online active learning framework that includes an ensemble classifier, a drift detector, a label sliding window, sample sliding windows and an initialization training sample sequence. Next, a variable threshold uncertainty strategy based on an asymmetric margin threshold matrix is designed to comprehensively address the problem that a given class can simultaneously be a majority to a given subset of classes while also being a minority to others. Last but not least, we design a novel sample weight formula that comprehensively considers the class imbalance ratio of the sample’s category and the prediction difficulty. On 10 multiclass synthetic streams with different imbalance ratios and concept drifts, and on 5 real-world imbalanced streams with 7 to 55 classes and unknown drifts, the experimental results demonstrate that the proposed CALMID is more effective and efficient than several state-of-the-art learning algorithms.

Anomaly Detection Aided Budget Online Classification for Imbalanced Data Streams

Learning from imbalanced data streams differs from the traditional learning paradigm due to the issues of imbalanced classes. It has significant implications in a myriad of real-world applications, ranging from financial risk, network security, to medical diagnosis. Moreover, outliers usually appear in data streams. The issue of class imbalance or anomaly itself could negatively affect the performance of the underlying learning algorithms, and their combination makes the learning problem harder. In this work, we propose an anomaly detection aided budget online weighted learning method (BOW-LM) to identify positive and negative instances from imbalanced data streams. BOW-LM is based on the widely used Feedforward Networks with Random Weights. An agile lightweight anomaly detector is designed based on the nonlinear mapping of the network. To reduce computational complexity and to response promptly, BOW-LM employs a matrix correction technique to update the learning model by only <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) operations for each data chunk with LL hidden layer nodes. Empirical studies on both synthetic and real-world datasets demonstrate the effectiveness of BOW-LM.

The prior probability in the batch classification of imbalanced data streams

In the diversity of contemporary decision-making tasks, where the data is no longer static and changes over time, data stream processing has become an important issue in the field of pattern recognition. In addition, most of the real problems are not balanced, representing their classes in various improportions. Following paper proposes the Prior Imbalance Compensation method, modifying on-the-fly predictions made by the base classifier, aiming at mapping prior probability in the statistics of assigned classes. It is intended to be a less computationally complex competition for popular algorithms such as smote, solving this problem by oversampling the training set. The proposed method has been tested using computer experiments on the example of a set of various data streams, leading to promising results, suggesting its usefulness in solving this type of problems.

Reinforcement Online Active Learning Ensemble for Drifting Imbalanced Data Streams

Applications challenged by the joint problem of concept drift and class imbalance are attracting increasing research interest. This paper proposes a novel Reinforcement Online Active Learning Ensemble for Drifting Imbalanced data stream (ROALE-DI). The ensemble classifier has a long-term stable classifier and a dynamic classifier group which applies a reinforcement mechanism to increase the weight of the dynamic classifiers, which perform better on the minority class, and decreases the weight of the opposite. When the data stream is class imbalanced, the classifiers will lack the training samples of the minority class. To supply training samples, when creating a new classifier, the labeled instances buffer is used to provide instances of the minority class. Then, a hybrid labeling strategy that combines the uncertainty strategy and imbalance strategy is proposed to define whether to obtain the real label of an instance. An experimental evaluation compares the classification performance of the proposed method with semi-supervised and supervised algorithms on both real-world and synthetic data streams. The results show that the ROALE-DI achieves higher Area Under the ROC Curve (AUC) and accuracy values with even fewer real labels, and the labeling cost dynamically adjusts according to the concept drift and class imbalance ratio.

Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams

This work aims to connect two rarely combined research directions, i.e., non-stationary data stream classification and data analysis with skewed class distributions. We propose a novel framework employing stratified bagging for training base classifiers to integrate data preprocessing and dynamic ensemble selection methods for imbalanced data stream classification. The proposed approach has been evaluated based on computer experiments carried out on 135 artificially generated data streams with various imbalance ratios, label noise levels, and types of concept drift as well as on two selected real streams. Four preprocessing techniques and two dynamic selection methods, used on both bagging classifiers and base estimators levels, were considered. Experimentation results showed that, for highly imbalanced data streams, dynamic ensemble selection coupled with data preprocessing could outperform online and chunk-based state-of-art methods.

Application of Imbalanced Data Classification Quality Metrics as Weighting Methods of the Ensemble Data Stream Classification Algorithms.

In the era of a large number of tools and applications that constantly produce massive amounts of data, their processing and proper classification is becoming both increasingly hard and important. This task is hindered by changing the distribution of data over time, called the concept drift, and the emergence of a problem of disproportion between classes—such as in the detection of network attacks or fraud detection problems. In the following work, we propose methods to modify existing stream processing solutions—Accuracy Weighted Ensemble (AWE) and Accuracy Updated Ensemble (AUE), which have demonstrated their effectiveness in adapting to time-varying class distribution. The introduced changes are aimed at increasing their quality on binary classification of imbalanced data. The proposed modifications contain the inclusion of aggregate metrics, such as F1-score, G-mean and balanced accuracy score in calculation of the member classifiers weights, which affects their composition and final prediction. Moreover, the impact of data sampling on the algorithm’s effectiveness was also checked. Complex experiments were conducted to define the most promising modification type, as well as to compare proposed methods with existing solutions. Experimental evaluation shows an improvement in the quality of classification compared to the underlying algorithms and other solutions for processing imbalanced data streams.

A Multi-Tier Streaming Analytics Model of 0-Day Ransomware Detection Using Machine Learning

Desktop and portable platform-based information systems become the most tempting target of crypto and locker ransomware attacks during the last decades. Hence, researchers have developed anti-ransomware tools to assist the Windows platform at thwarting ransomware attacks, protecting the information, preserving the users’ privacy, and securing the inter-related information systems through the Internet. Furthermore, they utilized machine learning to devote useful anti-ransomware tools that detect sophisticated versions. However, such anti-ransomware tools remain sub-optimal in efficacy, partial to analyzing ransomware traits, inactive to learn significant and imbalanced data streams, limited to attributing the versions’ ancestor families, and indecisive about fusing the multi-descent versions. In this paper, we propose a hybrid machine learner model, which is a multi-tiered streaming analytics model that classifies various ransomware versions of 14 families by learning 24 static and dynamic traits. The proposed model classifies ransomware versions to their ancestor families numerally and fuses those of multi-descent families statistically. Thus, it classifies ransomware versions among 40K corpora of ransomware, malware, and good-ware versions through both semi-realistic and realistic environments. The supremacy of this ransomware streaming analytics model among competitive anti-ransomware technologies is proven experimentally and justified critically with the average of 97% classification accuracy, 2.4% mistake rate, and 0.34% miss rate under comparative and realistic test.

RETRACTED ARTICLE: Comprehensive analysis for class imbalance data with concept drift using ensemble based classification

In many information system applications, the environment is dynamic and tremendous amount of streaming data is generated. This scenario enforces additional computational demand on the algorithm to process incoming instances incrementally using restricted memory and time compared to static data mining. Moreover, when the streams of data are collected from different sources, it may exhibit concept drift, which means the variation in the distribution of data and it can have a high degree of class imbalance. The problem of class imbalance occurs when there is a much lower number of an example representing one class than those of the other class. Concept drift and imbalanced streaming data are commonly found in real-world applications such as fraud detection, intrusion detection, decision support system and disease prediction. In this paper, the different concept drift detectors and handling approaches are analysed when dealing with imbalance data. A comparative analysis of concept drift is performed on various data sets like SEA synthetic data stream and real world datasets. Massive Online Analysis (MOA) tool is used to make the comparative study about different learners in a concept drifting environment. The performance measure such as Accuracy, Precision, Recall, F1-score and Kappa statistic has been used to evaluate the performance of the various learners on SEA synthetic data stream and real world dataset. Ensemble classifiers and single learners are employed and tested on the data samples of SEA synthetic data stream, electrical and KDD intrusion data set. The ensemble classifiers provide better accuracy when compared to the single classifier and ensemble based methods has shown good performance compared to strong single learners when dealing with concept drift and class imbalance data.

Incremental learning imbalanced data streams with concept drift: The dynamic updated ensemble algorithm

Learning nonstationary data streams has been well studied in recent years. However, most of the researches assume that the class imbalance of data streams is relatively balanced. Only a few approaches tackle the joint issue of concept drift and class imbalance due to its complexity. Meanwhile, the existing chunk ensembles for classifying imbalanced nonstationary data streams always need to store previous data, which consumes plenty of memory usage. To overcome these issues, we propose a chunk-based incremental ensemble algorithm called Dynamic Updated Ensemble (DUE) for learning imbalanced data streams with concept drift. Compared to the existing techniques, its merits are five-fold: (1) it learns one chunk at a time without requiring access to previous data; (2) it emphasizes misclassified examples in the model update procedure; (3) it can timely react to multiple kinds of concept drifts; (4) it can adapt to the new condition when switching majority class to minority class; (5) it keeps a limited number of classifiers to ensure high efficiency. Experiments on synthetic and real datasets demonstrate the effectiveness of DUE in learning nonstationary imbalanced data streams.

PWIDB: A framework for learning to classify imbalanced data streams with incremental data re-balancing technique

Abstract The performance of classification algorithms with highly imbalanced streaming data depends upon efficient balancing strategy. Some techniques of balancing strategy have been applied using static batch data to resolve the class imbalance problem, which is difficult if applied for massive data streams. In this paper, a new Piece-Wise Incremental Data re-Balancing (PWIDB) framework is proposed. The PWIDB framework combines automated balancing techniques using Racing Algorithm (RA) and incremental rebalancing technique. RA is an active learning approach capable of classifying imbalanced data and can provide a way to select an appropriate re-balancing technique with imbalanced data. In this paper, we have extended the capability of RA for handling imbalanced data streams in the proposed PWIDB framework. The PWIDB accumulates previous knowledge with increments of re-balanced data and captures the concept of the imbalanced instances. The PWIDB is an incremental streaming batch framework, which is suitable for learning with streaming imbalanced data. We compared the performance of PWIDB with a well-known FLORA technique. Experimental results show that the PWIDB framework exhibits an improved and stable performance compared to FLORA and accumulative re-balancing techniques.

GEP-based classifier with drift detection for mining imbalanced data streams

Abstract Mining data streams require to cope with time, data size and possible concept drift constraints. Even more challenging is the case where, apart from the above, one has to deal with imbalanced data. Mining non stationary and imbalanced data streams is a relatively new area of research. In this paper, we propose the Gene Expression Programming (GEP) classifier with drift detection and data reuse for mining imbalanced data streams. GEP is used to evolve a complex expression tree returning predictions. Drift detector role is to signal the occurrence of drift which triggers inducing a new learner. Data reuse mechanism allows for improving the balance between minority and majority instances in a subset of data used for evolving the learner. The proposed approach is validated experimentally. The experiment results confirm that our classifier produces high-quality predictions.

Adaptive Chunk-Based Dynamic Weighted Majority for Imbalanced Data Streams With Concept Drift.

One of the most challenging problems in the field of online learning is concept drift, which deeply influences the classification stability of streaming data. If the data stream is imbalanced, it is even more difficult to detect concept drifts and make an online learner adapt to them. Ensemble algorithms have been found effective for the classification of streaming data with concept drift, whereby an individual classifier is built for each incoming data chunk and its associated weight is adjusted to manage the drift. However, it is difficult to adjust the weights to achieve a balance between the stability and adaptability of the ensemble classifiers. In addition, when the data stream is imbalanced, the use of a size-fixed chunk to build a single classifier can create further problems; the data chunk may contain too few or even no minority class samples (i.e., only majority class samples). A classifier built on such a chunk is unstable in the ensemble. In this article, we propose a chunk-based incremental learning method called adaptive chunk-based dynamic weighted majority (ACDWM) to deal with imbalanced streaming data containing concept drift. ACDWM utilizes an ensemble framework by dynamically weighting the individual classifiers according to their classification performance on the current data chunk. The chunk size is adaptively selected by statistical hypothesis tests to access whether the classifier built on the current data chunk is sufficiently stable. ACDWM has four advantages compared with the existing methods as follows: 1) it can maintain stability when processing nondrifted streams and rapidly adapt to the new concept; 2) it is entirely incremental, i.e., no previous data need to be stored; 3) it stores a limited number of classifiers to ensure high efficiency; and 4) it adaptively selects the chunk size in the concept drift environment. Experiments on both synthetic and real data sets containing concept drift show that ACDWM outperforms both state-of-the-art chunk-based and online methods.