Data Stream Classification Algorithms Research Articles

An online ensemble classification algorithm for multi-class imbalanced data stream

An online ensemble classification algorithm for multi-class imbalanced data stream

An experimental review of the ensemble-based data stream classification algorithms in non-stationary environments

Data streams are sequences of fast-growing and high-speed data points that typically suffer from the infinite length, large volume, and specifically unstable data distribution. Ensemble learning as a prevalent classification approach is widely used in data stream mining studies. Besides the impressive performance of ensemble learning algorithms in providing a collection of diverse and accurate classifiers, they are specifically efficient in handling non-stationary data streams. Due to the component-based nature and the chance of dynamic updates for the components of the ensemble, this category is appropriate for dynamically learning the changing concepts of the data. Several review research have been conducted on the challenges of non-stationary data streams so far. None of the available surveys are dedicated to examine the effect of ensemble learning models on concept drift handling. This paper aims to provide a thorough theoretical and experimental review of the most significant ensemble-based data stream classification approaches in confronting with various types of concept drifts. In this comprehensive experimental analysis, 21 state-of-the-art ensemble algorithms are tested on 30 synthetic datasets under various types and volumes of concept drifts. This experimental analysis involves analyzing the classification accuracy, kappa score and running time of the algorithms, along with performing various statistical tests. In addition to conducting experimental analysis, a comprehensive investigation is carried out on the most significant challenges posed by non-stationary data streams, which provides the reader with valuable insights.

Adaptive regularized ensemble for evolving data stream classification

Extracting knowledge from data streams requires fast incremental algorithms that are able to handle unlimited processing and ever-changing data with finite memory. A strategy for this challenge is the use of ensembles owing to their ability to tackle concept drift and achieve highly accurate predictions. However, ensembles often require a lot of computational resources. In this study, we propose a novel ensemble-based classification algorithm for data streams, Adaptive Regularized Ensemble (ARE), with low demand for computational resources. The algorithm combines strategies that contribute to high prediction accuracy using only incorrectly classified instances into the training step, random-sized feature subspace for each ensemble element and classifier selection for final ensemble voting. After an extensive experimental study, ARE exhibited high predictive performance and outperformed state-of-the-art ensembles on data streams for real and synthetic datasets while requiring a low processing time and memory consumption.

Model update for intrusion detection: Analyzing the performance of delayed labeling and active learning strategies

Intrusion Detection Systems (IDS) help protect computer networks by identifying and detecting attempts to obtain unauthorized access to data via computer networks by inspecting packets separately or in the context of flows. Intrusion detection is a classification task performed on continuously generated packets with a non-stationary distribution (data stream). As such, the decision models used for Intrusion Detection must be constantly updated to account for changes in what constitutes normal traffic of a network and correctly identify attacks. This work evaluates two approaches concerning label availability to update the intrusion detection classifier. First, we analyze the impact of delayed labeling of samples on the classifiers' performance, and second, we evaluate the impact of using active learning strategies on the classifiers. Our experimental evaluation uses two datasets (CIC-IDS2017 and CSE-CIC-IDS2018) to compare different data stream classification algorithms under different evaluation measures. Based on comparison results, we studied different active learning techniques to estimate the impact of delayed labeling on packet-based IDS. We found that the performance of the classifiers is inversely proportional to the label delivery rate. Besides, the active learning strategies helped keep the performance compatible with the baselines, even with a small set of labeled instances.

Clustering-based Active Learning Classification towards Data Stream

Many practical applications, such as social media and monitoring system, will constantly generate streaming data, which has problems of instability, lack of labels and multiclass imbalance. In order to solve these problems, a cluster-based active learning method is proposed to achieve data stream classification. Firstly, a label query strategy combining marginal threshold matrix is proposed, which selects difficult to classify or potential concept drift samples for marking, to solve the problem of high cost label and unbalanced data. Secondly, dynamic maintenance of a group of micro clusters, by adjusting the weight of micro clusters in the model, correctly reflects the current data distribution, and finally, uses the buffer to store new micro clusters to participate in the update of the model, to adapt to the new data environment. Experimental results on three real data sets and three synthetic data sets show that compared with the classical data stream classification algorithm, it is less affected by concept drift and has higher classification accuracy than the online semi-supervised learning algorithm ADSM. The average accuracy of the six datasets increased by 5.56%, 2.32%, 1.77%, 1.83%, 3.78%, and 2.04%, respectively. The model processes data streams online and improves classification performance with less memory consumption.

Employing chunk size adaptation to overcome concept drift

Modern analytical systems must process streaming data and correctly respond to data distribution changes. The phenomenon of changes in data distributions is called concept drift, and it may harm the quality of the used models. Additionally, the possibility of concept drift appearance causes that the used algorithms must be ready for the continuous adaptation of the model to the changing data distributions. This work focuses on non-stationary data stream classification, where a classifier ensemble is used. To keep the ensemble model up to date, the new base classifiers are trained on the incoming data blocks and added to the ensemble while, at the same time, outdated models are removed from the ensemble. One of the problems with this type of model is the fast reaction to changes in data distributions. We propose the new Chunk Adaptive Restoration framework that can be adapted to any block-based data stream classification algorithm. The proposed algorithm adjusts the data chunk size in the case of concept drift detection to minimize the impact of the change on the predictive performance of the used model. The experimental research, backed up with the statistical tests, has proven that Chunk Adaptive Restoration significantly reduces the model’s restoration time.

Concept Drift Data Stream Classification Algorithm Based on McDiarmid Bound

Concept Drift Data Stream Classification Algorithm Based on McDiarmid Bound

Research on Mining of Applied Mathematics Educational Resources Based on Edge Computing and Data Stream Classification

Facing the massive data of higher education institutions, data mining technology is an intelligent information processing technology that can effectively discover knowledge from the massive data and can discover important information that people have previously ignored from the huge data information. This article is dedicated to the development of applied mathematics education resource mining technology based on edge computing and data stream classification. First of all, this article establishes a resource system architecture suitable for existing applied mathematics education through edge computing technology, which can effectively improve the efficiency of data mining. Secondly, the data stream classification algorithm is used for information extraction and classification integration of massive applied mathematical education data. This method provides potential and valuable information for decision-makers and education practitioners. Finally, the simulation and performance test of the system verify that it has the functions of mathematical information mining and data processing. This system will provide strong support for applied mathematics education reform.

A VFDT algorithm optimization and application thereof in data stream classification

Data stream mining is a new technology for dynamically extracting feature patterns from data streams. Its core technology is data stream mining algorithms. Data stream mining algorithms are divided into clustering and classification algorithms. In data stream classification algorithms, VFDT (The Concept-adapting Very Fast Decision Tree algorithm is an effective classification decision tree algorithm. The algorithm dynamically constructs a decision tree based on the improvement of the Hoefdding tree. With the inflow of data, new branches are continuously added and outdated Branch. However, the algorithm has the problem of concept drift, and some nodes may no longer meet the Hoefdding bound, which affects the classification mining effect. This paper proposes an optimization algorithm (VFDT), which reduces the impact of concept drift and network noise by adding sliding window technology and fuzzy technology to the VFDT classification mining algorithm. Experimental results show that the algorithm can effectively improve the accuracy of stream data classification mining. Reduce classification errors.

Application of Imbalanced Data Classification Quality Metrics as Weighting Methods of the Ensemble Data Stream Classification Algorithms.

In the era of a large number of tools and applications that constantly produce massive amounts of data, their processing and proper classification is becoming both increasingly hard and important. This task is hindered by changing the distribution of data over time, called the concept drift, and the emergence of a problem of disproportion between classes—such as in the detection of network attacks or fraud detection problems. In the following work, we propose methods to modify existing stream processing solutions—Accuracy Weighted Ensemble (AWE) and Accuracy Updated Ensemble (AUE), which have demonstrated their effectiveness in adapting to time-varying class distribution. The introduced changes are aimed at increasing their quality on binary classification of imbalanced data. The proposed modifications contain the inclusion of aggregate metrics, such as F1-score, G-mean and balanced accuracy score in calculation of the member classifiers weights, which affects their composition and final prediction. Moreover, the impact of data sampling on the algorithm’s effectiveness was also checked. Complex experiments were conducted to define the most promising modification type, as well as to compare proposed methods with existing solutions. Experimental evaluation shows an improvement in the quality of classification compared to the underlying algorithms and other solutions for processing imbalanced data streams.

A new approach for data stream classification: unsupervised feature representational online sequential extreme learning machine

The characteristics of the data stream have brought enormous challenges to classification algorithms. Concept drift is the most concerning characteristics, and developed classification algorithms must tackle the concept drift problem. Therefore, Extreme Learning Machines (ELM) based algorithms have been developed to respond to the characteristics of the data stream. However, due to randomly assigned input layer weights, ELM based algorithms have encountered problems such as producing inconsistent outputs, generating ill-conditioned matrix, and mapping the inputs to the worst representative space. To overcome these problems, this paper aims to build a stable and well-constructed classifier that responds to the requirements of the data stream by considering all characteristics. A novel data stream classification approach based online sequential ELM (OS-ELM) with unsupervised feature representation learning (UFROS-ELM) and ensemble UFROS-ELM approach based on majority learning is presented in this paper. UFROS-ELM is a modification of the OS-ELM with ELM-AE and concept drift mechanism. ELM-AE is utilized for computing the best discriminative input weights of the classifier. The classifier is then initialized by using the determined weights, first chunk, and OS-ELM algorithm. When a new data chunk arrives, the classifier firstly searches any concept drift occurrence. If it is detected, ELM-AE is utilized to reconstruct the classifier to adapt to the changes. Otherwise, the classifier is sequentially updated updates by processing the current chunk. The results are achieved on the well-known real and artificial data sets and compared with state-of-the-art data stream classification algorithms. The experimental studies demonstrate the achievements of the proposed approaches.

A Survey on Multi-Label Data Stream Classification

Nowadays, many real-world applications of our daily life generate massive volume of streaming data at a higher speed than ever before, to name a few, Web clicking data streams, sensor network data and credit transaction streams. Contrary to traditional data mining using static datasets, there are several challenges for data stream mining, for instance, finite memory, one-pass and timely reaction. In this survey, we provide a comprehensive review of existing multi-label streams mining algorithms and categorize these methods based on different perspectives, which mainly focus on the multi-label data stream classification. We first briefly summarize existing multi-label and data stream classification algorithms and discuss their merits and demerits. Secondly, we identify mining constraints on classification for multi-label streaming data, and present a comprehensive study in algorithms for multi-label data stream classification. Finally, several challenges and open issues in multi-label data stream classification are discussed, which are worthwhile to be pursued by the researchers in the future.

An online classification algorithm for large scale data streams: iGNGSVM

Stream Processing has recently become one of the current commercial trends to face huge amounts of data. However, normally these techniques need specific infrastructures and high resources in terms of memory and computing nodes. This paper shows how mini-batch techniques and topology extraction methods can help making gigabytes of data to be manageable for just one server using computationally costly Machine Learning techniques as Support Vector Machines. The algorithm iGNGSVM is proposed to improve the performance of Support Vector Machines in datasets where the data is continuously arriving. It is benchmarked against a mini-batch version of LibSVM, achieving good accuracy rates and performing faster than this.

Dynamic extreme learning machine for data stream classification

In our society, many fields have produced a large number of data streams. How to mining the interesting knowledge and patterns from continuous data stream becomes a problem which we have to solve. Different from conventional classification algorithms, data stream classification algorithms have to adjust their classification models with the change of data stream because of concept drift. However, conventional classification models will keep stable once models are trained. To solve the problem, a dynamic extreme learning machine for data stream classification (DELM) is proposed. DELM utilizes online learning mechanism to train ELM as basic classifier and trains a double hidden layer structure to improve the performance of ELM. When an alert about concept drift is set, more hidden layer nodes are added into ELM to improve the generalization ability of classifier. If the value measuring concept drift reaches the upper limit or the accuracy of ELM is in a low level, the current classifier will be deleted, and the algorithm will use new data to train a new classifier so as to learn new concept. The experimental results showed DELM could improve the accuracy of classification result, and can adapt to new concept in a short time.

Preserving output-privacy in data stream classification

Privacy-preservation has emerged to be a major concern in devising a data mining system. But, protecting the privacy of data mining input does not guarantee a privacy-preserved output. This paper focuses on preserving the privacy of data mining output and particularly the output of classification task. Further, instead of static datasets, we consider the classification of continuously arriving data streams: a rapidly growing research area. Due to the challenges of data stream classification such as vast volume, a mixture of labeled and unlabeled instances throughout the stream and timely classifier publication, enforcing privacy-preservation techniques becomes even more challenging. In order to achieve this goal, we propose a systematic method for preserving output-privacy in data stream classification that addresses several applications like loan approval, credit card fraud detection, disease outbreak or biological attack detection. Specifically, we propose an algorithm named Diverse and k-Anonymized HOeffding Tree (DAHOT) that is an amalgamation of popular data stream classification algorithm Hoeffding tree and a variant of k-anonymity and l-diversity principles. The empirical results on real and synthetic data streams verify the effectiveness of DAHOT as compared to its bedrock Hoeffding tree and two other techniques, one that learns sanitized decision trees from sampled data stream and other technique that uses ensemble-based classification. DAHOT guarantees to preserve the private patterns while classifying the data streams accurately.

HRNeuro-fuzzy: Adapting neuro-fuzzy classifier for recurring concept drift of evolving data streams using rough set theory and holoentropy

Data stream classification plays a vital role in data mining techniques which extracts the most important patterns from the real world database. Nowadays, many applications like sensor network, video surveillance and network traffic generate a huge amount of data streams. Due to the ambiguity in input data, imprecise input information and concept drift, some problems arise in classifying the data stream. To resolve these problems, we propose a HRNeuro fuzzy system in this paper based on rough set theory and holoentropy function. At first, the input database is given to the PCA algorithm to reduce the dimension of the data. An adaptive neuro fuzzy classifier is utilized where the designing of membership function and rule base are the two important aspects. Then, neuro-fuzzy system undergoes updating when the change of detection occurs between the data streams. Here, the updating behaviour of membership function and rules are performed using rough set theory and holoentropy function. The experimental results are evaluated for the datasets and the performance is analysed by some metrics and compared with the existing systems such as JIT adaptive K-NN and HRFuzzy system. From the result, it is concluded that our proposed fuzzy classifier attains the higher accuracy of 96% which proves the efficient performance of data stream classification algorithm.

Learning on High Frequency Stock Market Data Using Misclassified Instances in Ensemble

Learning on non-stationary distribution has been shown to be a very challenging problem in machine learning and data mining, because the joint probability distribution between the data and classes changes over time. Many real time problems suffer concept drift as they changes with time. For example, in stock market, the customer’s behavior may change depending on the season of the year and on the inflation. Concept drift can occurs in the stock market for a number of reasons for example, trader’s preference for stocks change over time, increases in a stock’s value may be followed by decreases. The objective of this paper is to develop an ensemble based classification algorithm for non-stationary data stream which would consider misclassified instances during learning process. In addition, we are presenting here an exhaustive comparison of proposed algorithms with state-of-the-art classification approaches using different evaluation measures like recall, f-measure and g-mean.

An improved Hoeffding-ID data-stream classification algorithm

Depending on the use of the Internet and network, data-stream classification has been applied in the intrusion detection field. Due to unlimited and difficult storage features, the routine classification algorithm (eg. C4.5, currently widely used classification algorithm with higher classification accuracy) tends to incorrect classification and memory leaks. In this paper, we propose an improved Hoeffding tree data-stream classification algorithm, Hoeffding-ID and apply it to the network data-stream process of the intrusion detection field. Experimental results shows that the Hoeffding-ID algorithm has relative high detection accuracy, low positives rate and memory usage not increasing with the data samples.

Structural XML Classification in Concept Drifting Data Streams

Classification of large, static collections of XML data has been intensively studied in the last several years. Recently however, the data processing paradigm is shifting from static to streaming data, where documents have to be processed online using limited memory and class definitions can change with time in an event called concept drift. As most existing XML classifiers are capable of processing only static data, there is a need to develop new approaches dedicated for streaming environments. In this paper, we propose a new classification algorithm for XML data streams called XSC. The algorithm uses incrementally mined frequent subtrees and a tree-subtree similarity measure to classify new documents in an associative manner. The proposed approach is experimentally evaluated against eight state-of-the-art stream classifiers on real and synthetic data. The results show that XSC performs significantly better than competitive algorithms in terms of accuracy and memory usage.

The Key Technologies for Classification of Distributed Data Streams

With advances in data collection and generation technologies, environments that produce data streams is more and more. In recent years, the network application is further universal and the applications of a single data stream transfer toward a multi-node distributed data streams, such as sensor network, network monitoring, web log analysis and the credit card transaction data of multiple sites. These data is not only real-time, continuous and large scale, but also distributed. How to manage and analyze large dynamic datasets is an important subject that researchers are faced with. In view of the situation, it presented the formalization description of homogeneous and heterogeneous distributed data stream in this paper, analyzed advantages and disadvantages of the centralized stream processing architecture and distributed streaming processing architecture, discussed the recent progress in distributed data stream classification algorithm, summed up the problems and challenges faced by the distributed data stream mining, and possible future research directions.