Multi-label Data Streams Research Articles

Hoeffding adaptive trees for multi-label classification on data streams

Data stream learning is a very relevant paradigm because of the increasing real-world scenarios generating data at high velocities and in unbounded sequences. Stream learning aims at developing models that can process instances as they arrive, so models constantly adapt to new concepts and the temporal evolution in the stream. In multi-label data stream environments where instances have the peculiarity of belonging simultaneously to more than one class, the problem becomes even more complex and poses unique challenges such as different concept drifts impacting different labels at simultaneous or distinct times, higher class imbalance, or new labels emerging in the stream. This paper proposes a novel approach to multi-label data stream classification called Multi-Label Hoeffding Adaptive Tree (MLHAT). MLHAT leverages the Hoeffding adaptive tree to address these challenges by considering possible relations and label co-occurrences in the partitioning process of the decision tree, dynamically adapting the learner in each leaf node of the tree, and implementing a concept drift detector that can quickly detect and replace tree branches that are no longer performing well. The proposed approach is compared with other 18 online multi-label classifiers on 41 datasets. The results, validated with statistical analysis, show that MLHAT outperforms other state-of-the-art approaches in 12 well-known multi-label metrics.

A Weighted Ensemble Classification Algorithm Based on Nearest Neighbors for Multi-Label Data Stream

With the rapid development of data stream, multi-label algorithms for mining dynamic data become more and more important. At the same time, when data distribution changes, concept drift will occur, which will make the existing classification models lose effectiveness. Ensemble methods have been used for multi-label classification, but few methods consider both the accuracy and diversity of base classifiers. To address the above-mentioned problem, a Weighted Ensemble classification algorithm based on Nearest Neighbors for Multi-Label data stream (WENNML) is proposed. WENNML uses data blocks to train Active candidate Ensemble Classifiers (AEC) and Passive candidate Ensemble Classifiers (PEC). The base classifiers of AEC and PEC are dynamically updated using geometric and diversity weighting methods. When the difference value between the number of current instances and the number of warning instances reaches the passive warning value, the algorithm selects the optimal base classifiers from AEC and PEC according to the subset accuracy and hamming score and puts them into the predictive ensemble classifiers. Experiments are carried out on 12 kinds of datasets with 9 comparison algorithms. The results show that WENNML achieves the best average rankings among the four evaluation metrics.

Research on the Multilabel Data Stream Classification Method Based on Fuzzy Complex Set‐Valued Measure Learning

In the process of modern industrial production equipment developing towards the direction of structure, automation, and intelligence, motor is still the main power output equipment. If the data flow classification occurs during the operation of the motor, it will lead to problems such as the reduction of its operation efficiency and the increase in system energy consumption. In serious cases, it will even cause motor damage, and the overall system equipment will be shut down for maintenance for a long time, resulting in serious economic losses. Therefore, the research on intelligent multilabel data stream classification technology of motor is of great significance to ensure the stability and reliability of efficient operation of production equipment. In order to improve the recognition efficiency and accuracy of ms‐1dcnn’s multilabel data stream classification method in the environment of variable motor conditions and strong noise interference, a multiscale feature fusion framework is constructed based on the residual network structure. The implementation principles of two kinds of attention mechanism algorithms, squeeze and excitation module and convolution attention module, are studied, respectively. The attention module suitable for one‐dimensional residual network is designed and embedded into the residual module to build a multiscale attention residual network model. Finally, the effectiveness and superiority of the proposed model are verified by using the experimental platform data.

Unsupervised concept drift detection for multi-label data streams

Many real-world applications adopt multi-label data streams as the need for algorithms to deal with rapidly changing data increases. Changes in data distribution, also known as concept drift, cause existing classification models to rapidly lose their effectiveness. To assist the classifiers, we propose a novel algorithm called Label Dependency Drift Detector (LD3), an unsupervised concept drift detector using label dependencies within the data for multi-label data streams. Our study exploits the dynamic temporal dependencies between labels using a label influence ranking method, which leverages a data fusion algorithm and uses the produced ranking to detect concept drift. LD3 is the first unsupervised concept drift detection algorithm in the multi-label classification problem area. In this study, we perform an extensive evaluation of LD3 by comparing it with 14 prevalent supervised concept drift detection algorithms that we adapt to the problem area using 15 datasets and a baseline classifier. The results show that LD3 provides between 16.9 and 56% better predictive performance than comparable detectors on both real-world and synthetic data streams.

Adaptive ensemble of self-adjusting nearest neighbor subspaces for multi-label drifting data streams

Multi-label data streams are sequences of multi-label instances arriving over time to a multi-label classifier. The properties of the stream may continuously change due to concept drift. Therefore, algorithms must constantly adapt to the new data distributions. In this paper we propose a novel ensemble method for multi-label drifting streams named Adaptive Ensemble of Self-Adjusting Nearest Neighbor Subspaces (AESAKNNS). It leverages a self-adjusting kNN as a base classifier with the advantages of ensembles to adapt to concept drift in the multi-label environment. To promote diverse knowledge within the ensemble, each base classifier is given a unique subset of features and samples to train on. These samples are distributed to classifiers in a probabilistic manner that follows a Poisson distribution as in online bagging. Accompanying these mechanisms, a collection of ADWIN detectors monitor each classifier for the occurrence of a concept drift on the subspace. Upon detection, the algorithm automatically trains additional classifiers in the background to attempt to capture new concepts on new subspaces of features. The dynamic classifier selection chooses the most accurate classifiers from the active and background ensembles to replace the current ensemble. Our experimental study compares the proposed approach with 30 other classifiers, including problem transformation, algorithm adaptation, kNNs, and ensembles on 30 diverse multi-label datasets and 12 performance metrics. Results, validated using non-parametric statistical analysis, support the better performance of the AESAKNNS and highlight the contribution of its components in improving the performance of the ensemble.

High-Dimensional Multi-Label Data Stream Classification With Concept Drifting Detection

Multi-label data streams such as Web texts and images have been popular on the Web. These data present the characteristics of multiple label, high dimensionality, high volume, high velocity and especial concept drift etc. Thus, multi-label data stream classification is a very challenging and significant task especially in the handling of high-dimensional data with concept drifts. However, this challenge has received little attention from the research community. Therefore, we propose the max-relevance and min-redundancy based algorithm adaptation approach for the efficient and effective classification on multi-label data streams with high-dimensional attributes and concept drifts .<xref ref-type="fn" rid="fn1">¹</xref><fn id="fn1"><label>1.</label> Source codes and data sets are available at below. https://github.com/peipeilihfut/MLStreamClassification </fn> In order to reduce the impact from the high-dimensional data with noisy attributes, we first refine the minimal-redundancy-maximal-relevance criterion based on mutual information to select qualified features in multi-label data streams. Secondly, we propose the data distribution based concept drifting detection approach to distinguish concept drifts hidden in data streams. Finally, we build an incremental ensemble classification model for efficiently classifying multi-label data streams. Extensive studies show that our approach can get optimal subsets of features while maintaining a good performance in the multi-label classification, as compared to several state-of-the-art multi-label feature selection algorithms using two efficient multi-label classification methods as base classifiers. Meanwhile, our approach is superior to three well-known multi-label data stream classification approaches in the effectiveness and efficiency.

Self-adjusting [formula omitted] nearest neighbors for continual learning from multi-label drifting data streams

Drifting data streams and multi-label data are both challenging problems. Multi-label instances may simultaneously be associated with many labels and classifiers must predict the complete set of labels. Learning from data streams requires algorithms able to learn from potentially unbounded data that is constantly changing. When multi-label data arrives as a stream, the challenges of both problems must be addressed, but additional challenges unique to the combined problem also arise. Each label may experience different concept drifts, simultaneously or distinctly, and parameter optimizations may be different for each label. In this paper we present a self-adapting algorithm for drifting, multi-label data streams, that can adapt to a variety of concepts drifts, is robust to data-level difficulties, and mitigates the necessity to tune multiple parameters. The window of retained instances self-adjusts in size to retain only the current concept, enabling efficient response to abrupt concept drift. The value k is self-adapting for each label, relieving the necessity to tune and allowing it to change, over time, for each label individually. A novel, label-based mechanism disables individual labels that contribute to error, while another punitive measure removes erroneous instances entirely, increasing robustness to noise, concept drift and label differences. Extensive experiments on 35 multi-label streams and generators demonstrate the superiority and advantages of the self-adapting mechanisms proposed compared to existing state-of-the-art methods.

A Survey on Multi-Label Data Stream Classification

Nowadays, many real-world applications of our daily life generate massive volume of streaming data at a higher speed than ever before, to name a few, Web clicking data streams, sensor network data and credit transaction streams. Contrary to traditional data mining using static datasets, there are several challenges for data stream mining, for instance, finite memory, one-pass and timely reaction. In this survey, we provide a comprehensive review of existing multi-label streams mining algorithms and categorize these methods based on different perspectives, which mainly focus on the multi-label data stream classification. We first briefly summarize existing multi-label and data stream classification algorithms and discuss their merits and demerits. Secondly, we identify mining constraints on classification for multi-label streaming data, and present a comprehensive study in algorithms for multi-label data stream classification. Finally, several challenges and open issues in multi-label data stream classification are discussed, which are worthwhile to be pursued by the researchers in the future.

Multi-Label Punitive kNN with Self-Adjusting Memory for Drifting Data Streams

In multi-label learning, data may simultaneously belong to more than one class. When multi-label data arrives as a stream, the challenges associated with multi-label learning are joined by those of data stream mining, including the need for algorithms that are fast and flexible, able to match both the speed and evolving nature of the stream. This article presents a punitive k nearest neighbors algorithm with a self-adjusting memory (MLSAMPkNN) for multi-label, drifting data streams. The memory adjusts in size to contain only the current concept and a novel punitive system identifies and penalizes errant data examples early, removing them from the window. By retaining and using only data that are both current and beneficial, MLSAMPkNN is able to adapt quickly and efficiently to changes within the data stream while still maintaining a low computational complexity. Additionally, the punitive removal mechanism offers increased robustness to various data-level difficulties present in data streams, such as class imbalance and noise. The experimental study compares the proposal to 24 algorithms using 30 real-world and 15 artificial multi-label data streams on six multi-label metrics, evaluation time, and memory consumption. The superior performance of the proposed method is validated through non-parametric statistical analysis, proving both high accuracy and low time complexity. MLSAMPkNN is a versatile classifier, capable of returning excellent performance in diverse stream scenarios.

Robust Online Multilabel Learning Under Dynamic Changes in Data Distribution With Labels.

In this paper, a robust online multilabel learning method dealing with dynamically changing multilabel data streams is proposed. The proposed method has three advantages: 1) higher accuracy due to a newly defined objective function based on labels ranking; 2) fast training and update based on a newly derived closed-form (rather than gradient descent based) solution for the new objective function; and 3) high robustness to a newly identified concept drift in multilabel data streams, namely, changes in data distribution with labels (CDDL). The high robustness benefits from two novel works: 1) a new sequential update rule that preserves the labels ranking information learned from all old (but discarded) samples while updating the model only based on new incoming samples and 2) a fixed threshold for label bipartition that is insensitive to any kind of changes in data distribution including CDDL. The proposed method has been evaluated over 13 benchmark datasets from various domains. As shown in the experimental results, the proposed work is highly robust to CDDL in both the sequential model update and multilabel thresholding. Furthermore, the proposed method improves the performance in different evaluation measures, including Hamming loss, F1-measure, Precision, and Recall while taking short training time on most evaluated datasets.

Multi-label classification via label correlation and first order feature dependance in a data stream

Many batch learning algorithms have been introduced for offline multi-label classification (MLC) over the years. However, the increasing data volume in many applications such as social networks, sensor networks, and traffic monitoring has posed many challenges to batch MLC learning. For example, it is often expensive to re-train the model with the newly arrived samples, or it is impractical to learn on the large volume of data at once. The research on incremental learning is therefore applicable to a large volume of data and especially for data stream. In this study, we develop a Bayesian-based method for learning from multi-label data streams by taking into consideration the correlation between pairs of labels and the relationship between label and feature. In our model, not only the label correlation is learned with each arrived sample with ground truth labels but also the number of predicted labels are adjusted based on Hoeffding inequality and the label cardinality. We also extend the model to handle missing values, a problem common in many real-world data. To handle concept drift, we propose a decay mechanism focusing on the age of the arrived samples to incrementally adapt to the change of data. The experimental results show that our method is highly competitive compared to several well-known benchmark algorithms under both the stationary and concept drift settings.

Scalable and efficient multi-label classification for evolving data streams

Many challenging real world problems involve multi-label data streams. Efficient methods exist for multi-label classification in non-streaming scenarios. However, learning in evolving streaming scenarios is more challenging, as classifiers must be able to deal with huge numbers of examples and to adapt to change using limited time and memory while being ready to predict at any point. This paper proposes a new experimental framework for learning and evaluating on multi-label data streams, and uses it to study the performance of various methods. From this study, we develop a multi-label Hoeffding tree with multi-label classifiers at the leaves. We show empirically that this method is well suited to this challenging task. Using our new framework, which allows us to generate realistic multi-label data streams with concept drift (as well as real data), we compare with a selection of baseline methods, as well as new learning methods from the literature, and show that our Hoeffding tree method achieves fast and more accurate performance.