Stream Classification Algorithms Research Articles

Research on concept drift algorithm based on evolutionary computation

Concept drift (CD) in data streams can significantly impact the performance and stability of data stream classification algorithms, diminishing the generalization capabilities of integrated learning models. This paper addresses CD issues in dichotomous data streams by introducing a novel modeling approach that leverages evolutionary computation techniques. The method entails grouping base learners based on their performance within a sliding window and then evolving the base learning periods using evolutionary techniques. Furthermore, the concept of “gene flow” is introduced to enhance diversity among base learners and improve CD prediction performance. Experimental results on real and artificial datasets demonstrate the superior comprehensive performance of the proposed method. Specifically, the BCDECA algorithm outperforms other similar methods, excelling in accuracy, diversity, convergence rate, and robustness on a range of datasets.

An experimental review of the ensemble-based data stream classification algorithms in non-stationary environments

Data streams are sequences of fast-growing and high-speed data points that typically suffer from the infinite length, large volume, and specifically unstable data distribution. Ensemble learning as a prevalent classification approach is widely used in data stream mining studies. Besides the impressive performance of ensemble learning algorithms in providing a collection of diverse and accurate classifiers, they are specifically efficient in handling non-stationary data streams. Due to the component-based nature and the chance of dynamic updates for the components of the ensemble, this category is appropriate for dynamically learning the changing concepts of the data. Several review research have been conducted on the challenges of non-stationary data streams so far. None of the available surveys are dedicated to examine the effect of ensemble learning models on concept drift handling. This paper aims to provide a thorough theoretical and experimental review of the most significant ensemble-based data stream classification approaches in confronting with various types of concept drifts. In this comprehensive experimental analysis, 21 state-of-the-art ensemble algorithms are tested on 30 synthetic datasets under various types and volumes of concept drifts. This experimental analysis involves analyzing the classification accuracy, kappa score and running time of the algorithms, along with performing various statistical tests. In addition to conducting experimental analysis, a comprehensive investigation is carried out on the most significant challenges posed by non-stationary data streams, which provides the reader with valuable insights.

Model update for intrusion detection: Analyzing the performance of delayed labeling and active learning strategies

Intrusion Detection Systems (IDS) help protect computer networks by identifying and detecting attempts to obtain unauthorized access to data via computer networks by inspecting packets separately or in the context of flows. Intrusion detection is a classification task performed on continuously generated packets with a non-stationary distribution (data stream). As such, the decision models used for Intrusion Detection must be constantly updated to account for changes in what constitutes normal traffic of a network and correctly identify attacks. This work evaluates two approaches concerning label availability to update the intrusion detection classifier. First, we analyze the impact of delayed labeling of samples on the classifiers' performance, and second, we evaluate the impact of using active learning strategies on the classifiers. Our experimental evaluation uses two datasets (CIC-IDS2017 and CSE-CIC-IDS2018) to compare different data stream classification algorithms under different evaluation measures. Based on comparison results, we studied different active learning techniques to estimate the impact of delayed labeling on packet-based IDS. We found that the performance of the classifiers is inversely proportional to the label delivery rate. Besides, the active learning strategies helped keep the performance compatible with the baselines, even with a small set of labeled instances.

Clustering-based Active Learning Classification towards Data Stream

Many practical applications, such as social media and monitoring system, will constantly generate streaming data, which has problems of instability, lack of labels and multiclass imbalance. In order to solve these problems, a cluster-based active learning method is proposed to achieve data stream classification. Firstly, a label query strategy combining marginal threshold matrix is proposed, which selects difficult to classify or potential concept drift samples for marking, to solve the problem of high cost label and unbalanced data. Secondly, dynamic maintenance of a group of micro clusters, by adjusting the weight of micro clusters in the model, correctly reflects the current data distribution, and finally, uses the buffer to store new micro clusters to participate in the update of the model, to adapt to the new data environment. Experimental results on three real data sets and three synthetic data sets show that compared with the classical data stream classification algorithm, it is less affected by concept drift and has higher classification accuracy than the online semi-supervised learning algorithm ADSM. The average accuracy of the six datasets increased by 5.56%, 2.32%, 1.77%, 1.83%, 3.78%, and 2.04%, respectively. The model processes data streams online and improves classification performance with less memory consumption.

Employing chunk size adaptation to overcome concept drift

Modern analytical systems must process streaming data and correctly respond to data distribution changes. The phenomenon of changes in data distributions is called concept drift, and it may harm the quality of the used models. Additionally, the possibility of concept drift appearance causes that the used algorithms must be ready for the continuous adaptation of the model to the changing data distributions. This work focuses on non-stationary data stream classification, where a classifier ensemble is used. To keep the ensemble model up to date, the new base classifiers are trained on the incoming data blocks and added to the ensemble while, at the same time, outdated models are removed from the ensemble. One of the problems with this type of model is the fast reaction to changes in data distributions. We propose the new Chunk Adaptive Restoration framework that can be adapted to any block-based data stream classification algorithm. The proposed algorithm adjusts the data chunk size in the case of concept drift detection to minimize the impact of the change on the predictive performance of the used model. The experimental research, backed up with the statistical tests, has proven that Chunk Adaptive Restoration significantly reduces the model’s restoration time.

Stream Classification Algorithm Based on Decision Tree

Due to the rise of many fields such as e-commerce platforms, a large number of stream data has emerged. The incomplete labeling problem and concept drift problem of these data pose a huge challenge to the existing stream data classification methods. In this respect, a dynamic stream data classification algorithm is proposed for the stream data. For the incomplete labeling problem, this method introduces randomization and iterative strategy based on the very fast decision tree VFDT algorithm to design an iterative integration algorithm, and the algorithm uses the previous model classification result as the next model input and implements the voting mechanism for new data classification. At the same time, the window mechanism is used to store data and calculate the data distribution characteristics in the window, then, combined with the calculated result and the predicted amount of data to adjust the size of the sliding window. Experiments show the superiority of the algorithm in classification accuracy. The aim of the study is to compare different algorithms to evaluate whether classification model adapts to the current data environment.

Concept Drift Data Stream Classification Algorithm Based on McDiarmid Bound

Concept Drift Data Stream Classification Algorithm Based on McDiarmid Bound

Research on Mining of Applied Mathematics Educational Resources Based on Edge Computing and Data Stream Classification

Facing the massive data of higher education institutions, data mining technology is an intelligent information processing technology that can effectively discover knowledge from the massive data and can discover important information that people have previously ignored from the huge data information. This article is dedicated to the development of applied mathematics education resource mining technology based on edge computing and data stream classification. First of all, this article establishes a resource system architecture suitable for existing applied mathematics education through edge computing technology, which can effectively improve the efficiency of data mining. Secondly, the data stream classification algorithm is used for information extraction and classification integration of massive applied mathematical education data. This method provides potential and valuable information for decision-makers and education practitioners. Finally, the simulation and performance test of the system verify that it has the functions of mathematical information mining and data processing. This system will provide strong support for applied mathematics education reform.

A Benchmark of Data Stream Classification for Human Activity Recognition on Connected Objects

This paper evaluates data stream classifiers from the perspective of connected devices, focusing on the use case of Human Activity Recognition. We measure both the classification performance and resource consumption (runtime, memory, and power) of five usual stream classification algorithms, implemented in a consistent library, and applied to two real human activity datasets and three synthetic datasets. Regarding classification performance, the results show the overall superiority of the Hoeffding Tree, the Mondrian forest, and the Naïve Bayes classifiers over the Feedforward Neural Network and the Micro Cluster Nearest Neighbor classifiers on four datasets out of six, including the real ones. In addition, the Hoeffding Tree and—to some extent—the Micro Cluster Nearest Neighbor, are the only classifiers that can recover from a concept drift. Overall, the three leading classifiers still perform substantially worse than an offline classifier on the real datasets. Regarding resource consumption, the Hoeffding Tree and the Mondrian forest are the most memory intensive and have the longest runtime; however, no difference in power consumption is found between classifiers. We conclude that stream learning for Human Activity Recognition on connected objects is challenged by two factors which could lead to interesting future work: a high memory consumption and low F1 scores overall.

An Approach For Concept Drift Detection in a Graph Stream Using Discriminative Subgraphs

The emergence of mining complex networks like social media, sensor networks, and the world-wide-web has attracted considerable research interest. In a streaming scenario, the concept to be learned can change over time. However, while there has been some research done for detecting concept drift in traditional data streams, little work has been done on addressing concept drift in data represented as a graph . We propose a novel unsupervised concept-drift detection method on graph streams called Discriminative Subgraph-based Drift Detector (DSDD). The methodology starts by discovering discriminative subgraphs for each graph in the stream. We then compute the entropy of the window based on the distribution of discriminative subgraphs with respect to the graphs and then use the direct density-ratio estimation approach for detecting concept drift in the series of entropy values obtained by moving one step forward in the sliding window. The effectiveness of the proposed method is demonstrated through experiments using artificial and real-world datasets and its performance is evaluated by comparing against related baseline methods. Similarly, the usefulness of the proposed concept drift detection approach is studied by incorporating it in a popular graph stream classification algorithm and studying the impact of drift detection in classification accuracy.

A VFDT algorithm optimization and application thereof in data stream classification

Data stream mining is a new technology for dynamically extracting feature patterns from data streams. Its core technology is data stream mining algorithms. Data stream mining algorithms are divided into clustering and classification algorithms. In data stream classification algorithms, VFDT (The Concept-adapting Very Fast Decision Tree algorithm is an effective classification decision tree algorithm. The algorithm dynamically constructs a decision tree based on the improvement of the Hoefdding tree. With the inflow of data, new branches are continuously added and outdated Branch. However, the algorithm has the problem of concept drift, and some nodes may no longer meet the Hoefdding bound, which affects the classification mining effect. This paper proposes an optimization algorithm (VFDT), which reduces the impact of concept drift and network noise by adding sliding window technology and fuzzy technology to the VFDT classification mining algorithm. Experimental results show that the algorithm can effectively improve the accuracy of stream data classification mining. Reduce classification errors.

Application of Imbalanced Data Classification Quality Metrics as Weighting Methods of the Ensemble Data Stream Classification Algorithms.

In the era of a large number of tools and applications that constantly produce massive amounts of data, their processing and proper classification is becoming both increasingly hard and important. This task is hindered by changing the distribution of data over time, called the concept drift, and the emergence of a problem of disproportion between classes—such as in the detection of network attacks or fraud detection problems. In the following work, we propose methods to modify existing stream processing solutions—Accuracy Weighted Ensemble (AWE) and Accuracy Updated Ensemble (AUE), which have demonstrated their effectiveness in adapting to time-varying class distribution. The introduced changes are aimed at increasing their quality on binary classification of imbalanced data. The proposed modifications contain the inclusion of aggregate metrics, such as F1-score, G-mean and balanced accuracy score in calculation of the member classifiers weights, which affects their composition and final prediction. Moreover, the impact of data sampling on the algorithm’s effectiveness was also checked. Complex experiments were conducted to define the most promising modification type, as well as to compare proposed methods with existing solutions. Experimental evaluation shows an improvement in the quality of classification compared to the underlying algorithms and other solutions for processing imbalanced data streams.

A new approach for data stream classification: unsupervised feature representational online sequential extreme learning machine

The characteristics of the data stream have brought enormous challenges to classification algorithms. Concept drift is the most concerning characteristics, and developed classification algorithms must tackle the concept drift problem. Therefore, Extreme Learning Machines (ELM) based algorithms have been developed to respond to the characteristics of the data stream. However, due to randomly assigned input layer weights, ELM based algorithms have encountered problems such as producing inconsistent outputs, generating ill-conditioned matrix, and mapping the inputs to the worst representative space. To overcome these problems, this paper aims to build a stable and well-constructed classifier that responds to the requirements of the data stream by considering all characteristics. A novel data stream classification approach based online sequential ELM (OS-ELM) with unsupervised feature representation learning (UFROS-ELM) and ensemble UFROS-ELM approach based on majority learning is presented in this paper. UFROS-ELM is a modification of the OS-ELM with ELM-AE and concept drift mechanism. ELM-AE is utilized for computing the best discriminative input weights of the classifier. The classifier is then initialized by using the determined weights, first chunk, and OS-ELM algorithm. When a new data chunk arrives, the classifier firstly searches any concept drift occurrence. If it is detected, ELM-AE is utilized to reconstruct the classifier to adapt to the changes. Otherwise, the classifier is sequentially updated updates by processing the current chunk. The results are achieved on the well-known real and artificial data sets and compared with state-of-the-art data stream classification algorithms. The experimental studies demonstrate the achievements of the proposed approaches.

A Survey on Multi-Label Data Stream Classification

Nowadays, many real-world applications of our daily life generate massive volume of streaming data at a higher speed than ever before, to name a few, Web clicking data streams, sensor network data and credit transaction streams. Contrary to traditional data mining using static datasets, there are several challenges for data stream mining, for instance, finite memory, one-pass and timely reaction. In this survey, we provide a comprehensive review of existing multi-label streams mining algorithms and categorize these methods based on different perspectives, which mainly focus on the multi-label data stream classification. We first briefly summarize existing multi-label and data stream classification algorithms and discuss their merits and demerits. Secondly, we identify mining constraints on classification for multi-label streaming data, and present a comprehensive study in algorithms for multi-label data stream classification. Finally, several challenges and open issues in multi-label data stream classification are discussed, which are worthwhile to be pursued by the researchers in the future.

Entity-level stream classification: exploiting entity similarity to label the future observations referring to an entity

Stream classification algorithms traditionally treat arriving instances as independent. However, in many applications, the arriving examples may depend on the “entity” that generated them, e.g. in product reviews or in the interactions of users with an application server. In this study, we investigate the potential of this dependency by partitioning the original stream of instances/“observations” into entity-centric substreams and by incorporating entity-specific information into the learning model. We propose a k-nearest-neighbour-inspired stream classification approach, in which the label of an arriving observation is predicted by exploiting knowledge on the observations belonging to this entity and to entities similar to it. For the computation of entity similarity, we consider knowledge about the observations and knowledge about the entity, potentially from a domain/feature space different from that in which predictions are made. To distinguish between cases where this knowledge transfer is beneficial for stream classification and cases where the knowledge on the entities does not contribute to classifying the observations, we also propose a heuristic approach based on random sampling of substreams using k Random Entities (kRE). Our learning scenario is not fully supervised: after acquiring labels for the initial m observations of each entity, we assume that no additional labels arrive and attempt to predict the labels of near-future and far-future observations from that initial seed. We report on our findings from three datasets.

Application of Normalized Compression Distance and Lempel-Ziv Jaccard Distance in Micro-electrode Signal Stream Classification for the Surgical Treatment of Parkinson’s Disease

Abstract Parkinson’s Disease can be treated with the use of microelectrode recording and stimulation. This paper presents a data stream classifier that analyses raw data from micro-electrodes and decides whether the measurements were taken from the subthalamic nucleus (STN) or not. The novelty of the proposed approach is based on the fact that distances based on raw data are used. Two distances are investigated in this paper, i.e. Normalized Compression Distance (NCD) and Lempel-Ziv Jaccard Distance (LZJD). No new features needed to be extracted due to the fact that in the case of high-dimensional data the process is extremely time-consuming. The k-nearest neighbour (k-NN) was chosen as the classifier due to its simplicity, which is essential in data stream classification. Results obtained from classifiers based on k-NN: k-NN, k-NN were compared with Probabilistic Approximate Window (k-NN with PAW); k-NN with Probabilistic Approximate Window and Adaptive Windowing (k-NN with PAW and ADWIN); and Self Adjusting Memory k-NN (SAM k-NN), which use the proposed distances, with the performance of the same classifiers but using standard Euclidean distance. Prequential accuracy was chosen as the performance measure. The results of the experiments performed with the described approach are in most cases better, i.e. the performance measures for kNN classifiers that use NCD and LZJD distances are better by up to 8.5 per cent and 14 per cent, respectively. Moreover, the proposed approach performs better when compared with other stream classification algorithms, i.e. Hoeffding Tree, Naive Bayes, and Leveraging Bagging. In the discussed case, an improvement of classification rate of up to 17.9 per cent when using Lempel-Ziv Jaccard Distance instead of the Euclidean was noted.

Dynamic extreme learning machine for data stream classification

In our society, many fields have produced a large number of data streams. How to mining the interesting knowledge and patterns from continuous data stream becomes a problem which we have to solve. Different from conventional classification algorithms, data stream classification algorithms have to adjust their classification models with the change of data stream because of concept drift. However, conventional classification models will keep stable once models are trained. To solve the problem, a dynamic extreme learning machine for data stream classification (DELM) is proposed. DELM utilizes online learning mechanism to train ELM as basic classifier and trains a double hidden layer structure to improve the performance of ELM. When an alert about concept drift is set, more hidden layer nodes are added into ELM to improve the generalization ability of classifier. If the value measuring concept drift reaches the upper limit or the accuracy of ELM is in a low level, the current classifier will be deleted, and the algorithm will use new data to train a new classifier so as to learn new concept. The experimental results showed DELM could improve the accuracy of classification result, and can adapt to new concept in a short time.

Preserving output-privacy in data stream classification

Privacy-preservation has emerged to be a major concern in devising a data mining system. But, protecting the privacy of data mining input does not guarantee a privacy-preserved output. This paper focuses on preserving the privacy of data mining output and particularly the output of classification task. Further, instead of static datasets, we consider the classification of continuously arriving data streams: a rapidly growing research area. Due to the challenges of data stream classification such as vast volume, a mixture of labeled and unlabeled instances throughout the stream and timely classifier publication, enforcing privacy-preservation techniques becomes even more challenging. In order to achieve this goal, we propose a systematic method for preserving output-privacy in data stream classification that addresses several applications like loan approval, credit card fraud detection, disease outbreak or biological attack detection. Specifically, we propose an algorithm named Diverse and k-Anonymized HOeffding Tree (DAHOT) that is an amalgamation of popular data stream classification algorithm Hoeffding tree and a variant of k-anonymity and l-diversity principles. The empirical results on real and synthetic data streams verify the effectiveness of DAHOT as compared to its bedrock Hoeffding tree and two other techniques, one that learns sanitized decision trees from sampled data stream and other technique that uses ensemble-based classification. DAHOT guarantees to preserve the private patterns while classifying the data streams accurately.

HRNeuro-fuzzy: Adapting neuro-fuzzy classifier for recurring concept drift of evolving data streams using rough set theory and holoentropy

Data stream classification plays a vital role in data mining techniques which extracts the most important patterns from the real world database. Nowadays, many applications like sensor network, video surveillance and network traffic generate a huge amount of data streams. Due to the ambiguity in input data, imprecise input information and concept drift, some problems arise in classifying the data stream. To resolve these problems, we propose a HRNeuro fuzzy system in this paper based on rough set theory and holoentropy function. At first, the input database is given to the PCA algorithm to reduce the dimension of the data. An adaptive neuro fuzzy classifier is utilized where the designing of membership function and rule base are the two important aspects. Then, neuro-fuzzy system undergoes updating when the change of detection occurs between the data streams. Here, the updating behaviour of membership function and rules are performed using rough set theory and holoentropy function. The experimental results are evaluated for the datasets and the performance is analysed by some metrics and compared with the existing systems such as JIT adaptive K-NN and HRFuzzy system. From the result, it is concluded that our proposed fuzzy classifier attains the higher accuracy of 96% which proves the efficient performance of data stream classification algorithm.

Incremental and Decremental Max-Flow for Online Semi-Supervised Learning

Max-flow has been adopted for semi-supervised data modelling, yet existing algorithms were derived only for the learning from static data. This paper proposes an online max-flow algorithm for the semi-supervised learning from data streams. Consider a graph learned from labelled and unlabelled data, and the graph being updated dynamically for accommodating online data adding and retiring. In learning from the resulting non stationary graph, we augment and de-augment paths to update max-flow with a theoretical guarantee that the updated max-flow equals to that from batch retraining. For classification, we compute min-cut over current max-flow, so that minimized number of similar sample pairs are classified into distinct classes. Empirical evaluation on real-world data reveals that our algorithm outperforms state-of-the-art stream classification algorithms.