Data Stream Classification Research Articles

Semi-supervised classification on data streams with recurring concept drift and concept evolution

Mining non-stationary stream is a challenging task due to its unique property of infinite length and dynamic characteristics let alone the issues of concept drift, concept evolution and limited labeled data. Although more attention has been attracted on the issues of concept drift and evolution in data streams, however, most of existing methods are supervised in nature, which probably result in a worse classification performance and lower efficiency in the case with scarcity of labeled data. Thus, in this paper, we proposed a semi-supervised framework with recurring concept drift and novel class detection called ESCR, which aims to detect recurring concept drift and concept evolution in data streams with partially labeled data. It is firstly built on an ensemble model consisted of several clustering-based classifiers. In terms of this framework, we adopt Jensen–Shannon divergence based change detection technique on classifier confidence score instead of classification error rate to detect recurring concept drifts. Meanwhile, we take concept evolution into consideration by monitoring the outliers with strong cohesion. Moreover, we further improve the execution efficiency of our framework by exploiting the recursive function and dynamic programming. Finally, extensive experiments conducted on both benchmark and synthetic data sets demonstrate the effectiveness and efficiency of our proposed semi-supervised framework in the handling of data streams with recurring concept drifts and concept evolution, as compared to several well-known semi-supervised data stream classification methods.

Anomaly Detection Aided Budget Online Classification for Imbalanced Data Streams

Learning from imbalanced data streams differs from the traditional learning paradigm due to the issues of imbalanced classes. It has significant implications in a myriad of real-world applications, ranging from financial risk, network security, to medical diagnosis. Moreover, outliers usually appear in data streams. The issue of class imbalance or anomaly itself could negatively affect the performance of the underlying learning algorithms, and their combination makes the learning problem harder. In this work, we propose an anomaly detection aided budget online weighted learning method (BOW-LM) to identify positive and negative instances from imbalanced data streams. BOW-LM is based on the widely used Feedforward Networks with Random Weights. An agile lightweight anomaly detector is designed based on the nonlinear mapping of the network. To reduce computational complexity and to response promptly, BOW-LM employs a matrix correction technique to update the learning model by only <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (L <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) operations for each data chunk with LL hidden layer nodes. Empirical studies on both synthetic and real-world datasets demonstrate the effectiveness of BOW-LM.

A clustering and ensemble based classifier for data stream classification

In the era of data mining, the research industry has great attention to data stream mining as well as it has a great impact on a wide range of applications like networking, telecommunication, education, banking, weather forecasting, a stock market, and so on. Because of these data stream mining having more attention from researchers. The handling of concept drifting data streams is one of the major issues and challenges in the data stream mining field. In the presence of the concept drift, the performance of the learning algorithm always degrades. In this paper, a hybrid method has been proposed which are the combination of an ensemble, and grid and density-based clustering methods. The proposed method is tested on both synthetic as well as real data. The proposed method works well in the presence of concept drift and performance is measured in terms of time, accuracy, and memory. As compared with the state-of-art algorithms, the proposed method performed well and gave better accuracy using synthetic datasets like 88.29%, 71.34%, and 75.39% for Hyperplane, RBF, and LED respectively and for real datasets 86.17%, 86.28%, 95.15%, and 99.83% for Adult, Census-Income, KDDCup99%–10%, and Covertype respectively.

Cost-Sensitive Classification for Evolving Data Streams with Concept Drift and Class Imbalance.

Class imbalance and concept drift are two primary principles that exist concurrently in data stream classification. Although the two issues have drawn enough attention separately, the joint treatment largely remains unexplored. Moreover, the class imbalance issue is further complicated if data streams with concept drift. A novel Cost-Sensitive based Data Stream (CSDS) classification is introduced to overcome the two issues simultaneously. The CSDS considers cost information during the procedures of data preprocessing and classification. During the data preprocessing, a cost-sensitive learning strategy is introduced into the ReliefF algorithm for alleviating the class imbalance at the data level. In the classification process, a cost-sensitive weighting schema is devised to enhance the overall performance of the ensemble. Besides, a change detection mechanism is embedded in our algorithm, which guarantees that an ensemble can capture and react to drift promptly. Experimental results validate that our method can obtain better classification results under different imbalanced concept drifting data stream scenarios.

Adaptive Bagging Methods for Classification of Data Streams with Concept Drift

Adaptive Bagging Methods for Classification of Data Streams with Concept Drift

Diverse Instance-Weighting Ensemble Based on Region Drift Disagreement for Concept Drift Adaptation.

Concept drift refers to changes in the distribution of underlying data and is an inherent property of evolving data streams. Ensemble learning, with dynamic classifiers, has proved to be an efficient method of handling concept drift. However, the best way to create and maintain ensemble diversity with evolving streams is still a challenging problem. In contrast to estimating diversity via inputs, outputs, or classifier parameters, we propose a diversity measurement based on whether the ensemble members agree on the probability of a regional distribution change. In our method, estimations over regional distribution changes are used as instance weights. Constructing different region sets through different schemes will lead to different drift estimation results, thereby creating diversity. The classifiers that disagree the most are selected to maximize diversity. Accordingly, an instance-based ensemble learning algorithm, called the diverse instance-weighting ensemble (DiwE), is developed to address concept drift for data stream classification problems. Evaluations of various synthetic and real-world data stream benchmarks show the effectiveness and advantages of the proposed algorithm.

Improved Ensemble Classification for Evolving Data Streams

A major challenge for evolving data stream classification is feature evolution where features of stream instances are dynamically changing as they progress. Existing classification methods considered feature evolution either for fixed-size data or of limited degree with presumed dependence to history, making them unable to work effectively on evolving data streams of unbounded size and arbitrary feature evolution. Particularly, for evolving data streams containing instances of multiple labels, classification coping with feature evolution faces significant challenges. In this article, we present efficient ensemble methods for classifying evolving data streams of both single label and multiple labels through effective model coupling. For single-label classification, we present an improved unsupervised classification algorithm that applies multi-cluster feature selection (MCFS), which was originally proposed for static data classification, in the DXMiner framework to handle each window of instances in a dynamic stream. Our method generates an optimal feature subset and achieves a high classification accuracy. We further improve the time complexity of the feature selection process in MCFS by applying the Ball-tree searching technique. For multi-label classification, we propose an effective fixed-size ensemble classifier based on multi-label KNN, which works only for static multi-label data classification, by incorporating a weight adaptation strategy among the classifiers in the ensemble to dynamically update the model and cope with arbitrary feature evolution of stream instances as the stream progresses. Extensive experiment results on real-life data streams show that our algorithms outperform the existing results for single-label and multi-label classification in classification accuracy and efficiency.

The prior probability in the batch classification of imbalanced data streams

In the diversity of contemporary decision-making tasks, where the data is no longer static and changes over time, data stream processing has become an important issue in the field of pattern recognition. In addition, most of the real problems are not balanced, representing their classes in various improportions. Following paper proposes the Prior Imbalance Compensation method, modifying on-the-fly predictions made by the base classifier, aiming at mapping prior probability in the statistics of assigned classes. It is intended to be a less computationally complex competition for popular algorithms such as smote, solving this problem by oversampling the training set. The proposed method has been tested using computer experiments on the example of a set of various data streams, leading to promising results, suggesting its usefulness in solving this type of problems.

A Photonic Recurrent Neuron for Time-Series Classification

Neuromorphic photonics has turned into a key research area for enabling neuromorphic computing at much higher data-rates compared to their electronic counterparts, improving significantly the (multiply-and-accumulate) MAC/sec. At the same time, time-series classification problems comprise a large class of artificial intelligence (AI) applications where speed and latency can have a decisive role in their hardware deployment roadmap, highlighting the need for ultra-fast hardware implementations of simplified recurrent neural networks (RNN) that can be extended in more advanced long-short-term-memory (LSTM) and gated recurrent unit (GRU) machines. Herein, we experimentally demonstrate a novel photonic recurrent neuron (PRN) to classify successfully a time-series vector with 100-psec optical pulses and up to 10 Gb/s data speeds, reporting on the fastest all-optical real-time classifier. Experimental classification of 3-bit optical binary data streams is presented, revealing an average accuracy of >91% and confirming the potential of PRNs to boost speed and latency performance in time-series AI applications.

Heterogeneous ensemble selection for evolving data streams

Ensemble learning has been widely applied to both batch data classification and streaming data classification. For the latter setting, most existing ensemble systems are homogenous, which means they are generated from only one type of learning model. In contrast, by combining several types of different learning models, a heterogeneous ensemble system can achieve greater diversity among its members, which helps to improve its performance. Although heterogeneous ensemble systems have achieved many successes in the batch classification setting, it is not trivial to extend them directly to the data stream setting. In this study, we propose a novel HEterogeneous Ensemble Selection (HEES) method, which dynamically selects an appropriate subset of base classifiers to predict data under the stream setting. We are inspired by the observation that a well-chosen subset of good base classifiers may outperform the whole ensemble system. Here, we define a good candidate as one that expresses not only high predictive performance but also high confidence in its prediction. Our selection process is thus divided into two sub-processes: accurate-candidate selection and confident-candidate selection. We define an accurate candidate in the stream context as a base classifier with high accuracy over the current concept, while a confident candidate as one with a confidence score higher than a certain threshold. In the first sub-process, we employ the prequential accuracy to estimate the performance of a base classifier at a specific time, while in the latter sub-process, we propose a new measure to quantify the predictive confidence and provide a method to learn the threshold incrementally. The final ensemble is formed by taking the intersection of the sets of confident classifiers and accurate classifiers. Experiments on a wide range of data streams show that the proposed method achieves competitive performance with lower running time in comparison to the state-of-the-art online ensemble methods.

A Novel Algorithm for Distributed Data Stream Using Big Data Classification Model

In order to solve the problem of real-time detection of power grid equipment anomalies, this paper proposes a data flow classification model based on distributed processing. In order to realize distributed processing of power grid data flow, a local node mining method and a global mining mode based on uneven data flow classification are designed. A data stream classification model based on distributed processing is constructed, then the corresponding data sequence is selected and formatted abstractly, and the local node mining method and global mining mode under this model are designed. In the local node miner, the block-to-block mining strategy is implemented by acquiring the current data blocks. At the same time, the expression and real-time maintenance of local mining patterns are completed by combining the clustering algorithm, thus improving the transmission rate of information between each node and ensuring the timeliness of the overall classification algorithm.

Literature review and analysis on big data stream classification techniques

Rapid growth in technology and information lead the human to witness the improved growth in velocity, volume of data, and variety. The data in the business organizations demonstrate the development of big data applications. Because of the improving demand of applications, analysis of sophisticated streaming big data tends to become a significant area in data mining. One of the significant aspects of the research is employing deep learning approaches for effective extraction of complex data representations. Accordingly, this survey provides the detailed review of big data classification methodologies, like deep learning based techniques, Convolutional Neural Network (CNN) based techniques, K-Nearest Neighbor (KNN) based techniques, Neural Network (NN) based techniques, fuzzy based techniques, and Support vector based techniques, and so on. Moreover, a detailed study is made by concerning the parameters, like evaluation metrics, implementation tool, employed framework, datasets utilized, adopted classification methods, and accuracy range obtained by various techniques. Eventually, the research gaps and issues of various big data classification schemes are presented.

Preprocessed dynamic classifier ensemble selection for highly imbalanced drifted data streams

This work aims to connect two rarely combined research directions, i.e., non-stationary data stream classification and data analysis with skewed class distributions. We propose a novel framework employing stratified bagging for training base classifiers to integrate data preprocessing and dynamic ensemble selection methods for imbalanced data stream classification. The proposed approach has been evaluated based on computer experiments carried out on 135 artificially generated data streams with various imbalance ratios, label noise levels, and types of concept drift as well as on two selected real streams. Four preprocessing techniques and two dynamic selection methods, used on both bagging classifiers and base estimators levels, were considered. Experimentation results showed that, for highly imbalanced data streams, dynamic ensemble selection coupled with data preprocessing could outperform online and chunk-based state-of-art methods.

A VFDT algorithm optimization and application thereof in data stream classification

Data stream mining is a new technology for dynamically extracting feature patterns from data streams. Its core technology is data stream mining algorithms. Data stream mining algorithms are divided into clustering and classification algorithms. In data stream classification algorithms, VFDT (The Concept-adapting Very Fast Decision Tree algorithm is an effective classification decision tree algorithm. The algorithm dynamically constructs a decision tree based on the improvement of the Hoefdding tree. With the inflow of data, new branches are continuously added and outdated Branch. However, the algorithm has the problem of concept drift, and some nodes may no longer meet the Hoefdding bound, which affects the classification mining effect. This paper proposes an optimization algorithm (VFDT), which reduces the impact of concept drift and network noise by adding sliding window technology and fuzzy technology to the VFDT classification mining algorithm. Experimental results show that the algorithm can effectively improve the accuracy of stream data classification mining. Reduce classification errors.

A computing model for trend analysis in stock data stream classification

&lt;p&gt;For several decades, many statistical and scientific efforts took place for the better analysis or prediction of stock trading. But still it is open to offer new avenues for the scientists to rethink and discover new inferences by adopting latest technological scenarios. In this regard, this paper is trying to apply classification techniques on stock data stream through feature extraction for the trend analysis. The proposed work is involving k-means for clustering samples into two clusters (the stocks in trend as one cluster and another on as stocks not in trend). The trend analysis is done based on density estimation of the stocks with respect to sectors. A well-known data representation method that is histogram is used to represent the sector which is in trend. This work has been implemented and experimented by considering live NSE (india) data using python and its related tools.&lt;/p&gt;

A computing model for trend analysis in stock data stream classification

A computing model for trend analysis in stock data stream classification

Lessons learned from data stream classification applied to credit scoring

The financial credibility of a person is a factor used to determine whether a loan should be approved or not, and this is quantified by a ‘credit score,’ which is calculated using a variety of factors, including past performance on debt obligations, profiling, amongst others. Machine learning has been widely applied to automate the development of effective credit scoring models over the years. Yet, studies show that the development of robust credit scoring models may take longer than a year, and thus, if the behavior of customers changes over time, the model will be outdated even before its deployment. In this paper, we made 3 anonymized real-world credit scoring datasets available alongside the results obtained. In each of these datasets, we verify whether the credit scoring task should be thought as an ephemeral scenario since many of the variables may drift over time, and thus, data stream mining techniques should be used since they were tailored for incremental learning and to detect and adapt to changes in the data distribution. Therefore, we compare both traditional batch machine learning algorithms with data stream algorithms in different validation schemes using both Kolmogorov–Smirnov and Population Stability Index metrics. Furthermore, we also provide insights on the importance of features according to their Information Value, Mean Decrease Impurity, and Mean Positional Gain metrics, such that the last depicts changes in the importance of features over time. For 2 of the 3 tested datasets, the results obtained by data stream learners are comparable to predictive models currently in use, thus showing the efficiency of data stream classification for the credit scoring task.

A Unified Framework for Sparse Online Learning

The amount of data in our society has been exploding in the era of big data. This article aims to address several open challenges in big data stream classification. Many existing studies in data mining literature follow the batch learning setting, which suffers from low efficiency and poor scalability. To tackle these challenges, we investigate a unified online learning framework for the big data stream classification task. Different from the existing online data stream classification techniques, we propose a unified Sparse Online Classification (SOC) framework. Based on SOC, we derive a second-order online learning algorithm and a cost-sensitive sparse online learning algorithm, which could successfully handle online anomaly detection tasks with the extremely unbalanced class distribution. As the performance evaluation, we analyze the theoretical bounds of the proposed algorithms and conduct an extensive set of experiments. The encouraging experimental results demonstrate the efficacy of the proposed algorithms over the state-of-the-art techniques on multiple data stream classification tasks.

Passive concept drift handling via variations of learning vector quantization

Concept drift is a change of the underlying data distribution which occurs especially with streaming data. Besides other challenges in the field of streaming data classification, concept drift has to be addressed to obtain reliable predictions. Robust Soft Learning Vector Quantization as well as Generalized Learning Vector Quantization has already shown good performance in traditional settings and is modified in this work to handle streaming data. Further, momentum-based stochastic gradient descent techniques are applied to tackle concept drift passively due to increased learning capabilities. The proposed work is tested against common benchmark algorithms and streaming data in the field and achieved promising results.

Early classification of spatio-temporal events using partial information.

This paper investigates event extraction and early event classification in contiguous spatio-temporal data streams, where events need to be classified using partial information, i.e. while the event is ongoing. The framework incorporates an event extraction algorithm and an early event classification algorithm. We apply this framework to synthetic and real problems and demonstrate its reliability and broad applicability. The algorithms and data are available in the R package eventstream, and other code in the supplementary material.