Multi-label Data Research Articles

Partial Multi-Label Learning via Exploiting Instance and Label Correlations

The goal of partial multi-label learning is to induce a multi-label classifier from partial multi-label data where each instance is annotated with a number of candidate labels but only a subset of them are valid. Many of the existing studies either fail to fully utilize instance and label correlations to eliminate noisy labels or build an oversimplified multi-label classifier, both of which are unfavorable for the improvement of generalization performance. In this article, we put forward a novel model named Pml-ilc to learn a multi-label classifier from partial multi-label data. Specifically, Pml-ilc first encodes instances and labels into a compact semantic space and takes full advantage of instance and label correlations to eliminate noisy labels. Then, it induces a linear mapping from the feature space to the label space while exploiting label-specific features and instance correlations to facilitate the multi-label classifier learning process. Finally, the above two steps are combined into a joint optimization problem and an efficient alternating optimization procedure is developed to find a satisfactory solution. Extensive experiments show that Pml-ilc achieves superior performance on both real-world and synthetic partial multi-label datasets in terms of different evaluation metrics.

Semi-supervised feature selection with minimal redundancy based on group optimization strategy for multi-label data

Semi-supervised feature selection with minimal redundancy based on group optimization strategy for multi-label data

Hoeffding adaptive trees for multi-label classification on data streams

Data stream learning is a very relevant paradigm because of the increasing real-world scenarios generating data at high velocities and in unbounded sequences. Stream learning aims at developing models that can process instances as they arrive, so models constantly adapt to new concepts and the temporal evolution in the stream. In multi-label data stream environments where instances have the peculiarity of belonging simultaneously to more than one class, the problem becomes even more complex and poses unique challenges such as different concept drifts impacting different labels at simultaneous or distinct times, higher class imbalance, or new labels emerging in the stream. This paper proposes a novel approach to multi-label data stream classification called Multi-Label Hoeffding Adaptive Tree (MLHAT). MLHAT leverages the Hoeffding adaptive tree to address these challenges by considering possible relations and label co-occurrences in the partitioning process of the decision tree, dynamically adapting the learner in each leaf node of the tree, and implementing a concept drift detector that can quickly detect and replace tree branches that are no longer performing well. The proposed approach is compared with other 18 online multi-label classifiers on 41 datasets. The results, validated with statistical analysis, show that MLHAT outperforms other state-of-the-art approaches in 12 well-known multi-label metrics.

Research of multi-label text classification based on label attention and correlation networks.

Multi-Label Text Classification (MLTC) is a crucial task in natural language processing. Compared to single-label text classification, MLTC is more challenging due to its vast collection of labels which include extracting local semantic information, learning label correlations, and solving label data imbalance problems. This paper proposes a model of Label Attention and Correlation Networks (LACN) to address the challenges of classifying multi-label text and enhance classification performance. The proposed model employs the label attention mechanism for a more discriminative text representation and uses the correlation network based on label distribution to enhance the classification results. Also, a weight factor based on the number of samples and a modulation function based on prediction probability are combined to alleviate the label data imbalance effectively. Extensive experiments are conducted on the widely-used conventional datasets AAPD and RCV1-v2, and extreme datasets EUR-LEX and AmazonCat-13K. The results indicate that the proposed model can be used to deal with extreme multi-label data and achieve optimal or suboptimal results versus state-of-the-art methods. For the AAPD dataset, compared with the suboptimal method, it outperforms the second-best method by 2.05% ∼ 5.07% in precision@k and by 2.10% ∼ 3.24% in NDCG@k for k = 1, 3, 5. The superior outcomes demonstrate the effectiveness of LACN and its competitiveness in dealing with MLTC tasks.

Noise-resistant fuzzy multineighbourhood rough set-based feature selection with label enhancement and its application for multilabel classification

Noise pollution in process of feature selection greatly reduces the classification efficacy of multilabel data, and the estimation of label descriptions is insufficient because of the lack of label enhancement for label distribution learning. To overcome the limitations, this work presents a noise-resistant fuzzy multineighbourhood rough set-based feature selection methodology with label enhancement and its application for multilabel classification. First, under the fuzzy T-equivalence relation, to construct new adaptive fuzzy neighbourhood class, the multineighbourhood radius set on feature space is integrated with fuzzy neighbourhood radius via sample interval on label space; then, the adaptive fuzzy multineighbourhood granules are obtained. Second, by integrating the parameterized fuzzy decision of labels and adaptive fuzzy multineighbourhood granules, noise-resistant lower and upper approximations via fuzzy multineighbourhood are constructed. A noise-resistant multilabel fuzzy multineighbourhood rough set model and its noise-resistant approximate accuracy can be proposed. Third, the description degree of label is computed to design label proportion after label enhancement. Certain fuzzy multineighbourhood entropy measures are developed to obtain label enhancement-based mutual information. When the algebraic and information perspectives are fused, label enhancement-based mutual information with fuzzy multineighbourhood via approximate accuracy is presented to evaluate the final association relationship between the features and label sets. Finally, a multilabel feature selection strategy via label enhancement will be constructed to obtain the best feature subset. The experimental results applied to 14 multilabel datasets indicate that this algorithm is significant.

Oversampling multi-label data based on natural neighbor and label correlation

In multi-label learning, addressing the class imbalance issue is of paramount importance. Oversampling methods are preferred as they offer a more general solution independent of the model choice, i.e., they alleviate the imbalance of datasets by augmenting instances in the pre-processing step. Existing neighbor-based oversampling methods employ an empirical number of neighbors (k=5) to identify the local region for new instances creation. However, a single fixed k value cannot fit all labels, because every label usually has its own distinct distribution and complexity. Furthermore, the label assignment for synthetic instances usually depends on the statistics of individual labels within the corresponding neighborhood, ignoring the informative correlation among labels. To overcome these limitations, we propose an oversampling method called Multi-Label Oversampling with Natural neighbor and label Correlation (MLONC). Our approach offers three main advantages: (1) the adaptive number of neighbors for each label related to the data complexity is obtained via natural neighbor detection; (2) it encourages generating more instances proximate to the decision boundary of highly imbalanced labels, and diminishes the impact of outliers; (3) exploitation of label correlation in label assignment enhances the quality of the synthetic instances. Experimental results demonstrate the effectiveness of MLONC under various base classifiers.

Multi-label feature selection based on minimizing feature redundancy of mutual information

Multi-label feature selection is an indispensable technology in the preprocessing of multi-label high-dimensional data. Approaches utilizing information theory and sparse models hold promise in this domain, demonstrating strong performance. Although there have been extensive literatures using l1 and l2,1-norms to identify label-specific features and common features in the feature space, they all ignore the redundant information interference problem when different features are learned simultaneously. Considering that features and labels in multi-label data are rarely linearly correlated, the MFS-MFR approach is presented to generate a representation of the nonlinear correlation between features and labels using the mutual information estimator. Following that, MFS-MFR detects specific and common features in the feature-label mutual information space using two coefficient matrices constrained by the l1 and l2,1-norms, respectively. In particular, we define a non-zero correlation constraint that effectively minimizes the redundant correlation between the two matrices. Moreover, a manifold regularization term is devised to preserve the local information of the mutual information space. To solve the optimization model with nonlinear binary regular term, we employ a novel solution approach called S-FISTA. Extensive experiments across 15 multi-label benchmark datasets, comparing against 11 top-performing multi-label feature selection methods, demonstrate the superior performance of MFS-MFR.

Evolutionary computation for feature selection in classification: A comprehensive survey of solutions, applications and challenges

Feature selection (FS), as one of the most significant preprocessing techniques in the fields of machine learning and pattern recognition, has received great attention. In recent years, evolutionary computation has become a popular technique for handling FS problems due to its superior global search performance. In this paper, a comprehensive review of evolutionary computation research on the FS problems is presented. Firstly, a new taxonomy for the basic components of evolutionary feature selection algorithms (EFSs) is proposed, including encoding strategy, population initialization, population updating, local search, multi-FS hybrid and ensemble. Secondly, we summarize the latest research progress of EFSs on some new and complex scenarios, including large-scale high-dimensional data, multi-objective/metric scenario, multi-label data, distributed storage data, multi-task scenario, multi-modal scenario, interpretable FS and stable FS, etc. Moreover, this survey provides also an in-depth analysis of real-world applications of EFSs, such as hyperspectral band selection, bioinformatics gene selection, text classification and fault detection, etc. Finally, several opportunities for future work are pointed out.

A novel ensemble causal feature selection approach with mutual information and group fusion strategy for multi-label data

PurposeAs intelligent technology advances, practical applications often involve data with multiple labels. Therefore, multi-label feature selection methods have attracted much attention to extract valuable information. However, current methods tend to lack interpretability when evaluating the relationship between different types of variables without considering the potential causal relationship.Design/methodology/approachTo address the above problems, we propose an ensemble causal feature selection method based on mutual information and group fusion strategy (CMIFS) for multi-label data. First, the causal relationship between labels and features is analyzed by local causal structure learning, respectively, to obtain a causal feature set. Second, we eliminate false positive features from the obtained feature set using mutual information to improve the feature subset reliability. Eventually, we employ a group fusion strategy to fuse the obtained feature subsets from multiple data sub-space to enhance the stability of the results.FindingsExperimental comparisons are performed on six datasets to validate that our proposal can enhance the interpretation and robustness of the model compared with other methods in different metrics. Furthermore, the statistical analyses further validate the effectiveness of our approach.Originality/valueThe present study makes a noteworthy contribution to proposing a causal feature selection approach based on mutual information to obtain an approximate optimal feature subset for multi-label data. Additionally, our proposal adopts the group fusion strategy to guarantee the robustness of the obtained feature subset.

Multi-source partial multi-label learning via tensor decomposition and nonconvex regularization

Multi-source partial multi-label learning (MSPMLL) aims to learn a multi-label classifier from training examples with multi-source features and multiple labels. Most of the existing MSPMLL models tackle each source independently to derive classifiers. Those methods cannot efficiently reveal the intrinsic structure and relationships among instances across different sources, nor deal with inconsistent labels caused by noise. To address those challenges, this paper proposes a tensor decomposition and nonconvex regularization approach to multi-source partial multi-label learning, nominated as MSPML-TLR. In this model, a hypergraph based affinity graph is constructed from each of multi-source data in the feature space. A tensor representation of multi-source multi-label data is proposed based on the affinity graphs. A multi-linear projection is learned to map the feature affinity space to a latent ground-truth label space via tensor robust principal component analysis. Furthermore, a notion called the parameterized tensorial logarithmic rank is designed to approximate the rank of the multi-linear projection and the latent ground-truth label tensor. Moreover, a hypergraph-based Laplacian regularization is imposed on each frontal slice of the latent ground-truth label tensor to preserve the topological structure of instances across multi-source feature spaces. An efficient algorithm is designed to implement MSPML-TLR and the convergence of the algorithm is rigorously proven. Extensive experiments on various real-world benchmark datasets demonstrate that the proposed model outperforms state-of-the-art methods in multi-label classification tasks.

Multi-label feature selection based on nonlinear mapping

Feature selection is one of the important pre-processing methods for dimensionality reduction in multi-label learning tasks, which has attracted extensive attention in recent years. Most of the existing approaches transform feature data into the label space during the feature-label mapping process by assuming linear relationship between the feature and label spaces. However, the linearity assumption does not hold in most cases, especially for high-dimensional spaces. This work proposes a novel dimension reduction model for multi-label data by using nonlinear mapping (NMFS). The model introduces a point-to-point sigmoid function to describe the intrinsic relationship from data space to label space. The proposed method improves the generalization ability of feature selection by limiting the range of data transformation to the interval [0,1] which is consistent with the predicted values of the labels. The feature weight matrix is constrained by the l2,1-norm to ensure its sparsity, which forms the basis of feature selection. The variables in the NMFS model are iteratively updated using the gradient momentum optimization strategy, and a sparse weight-coefficient matrix is obtained for multi-label feature ordering. Experimental results on 14 multi-label data sets verify the effectiveness of the proposed method, and show the proposed method is superior to the most advanced multi-label feature selection methods.

Feature selection for multi-labeled data based on label enhancement technique and mutual information

In multi-label data, the importance of each label within the logical label vector varies for each sample, and there exist inherent correlations among the labels. However, the logical label vector fails to capture these nuances. Consequently, relying solely on this vector for feature selection in multi-label data results in underutilization of supervisory information. To address this issue, this paper introduces a novel label enhancement algorithm. This algorithm leverages neighborhood information derived from features to transform the logical label vector into a label distribution that effectively reflects label differences and correlations. Subsequently, we propose a feature selection algorithm tailored for multi-label data, which incorporates both the transformed label distribution and mutual information. This algorithm not only accounts for the mutual information between features and label distributions but also captures the mutual information among features themselves. Finally, we evaluate our proposed feature selection algorithm against five state-of-the-art multi-label feature selection algorithms on ten publicly available datasets. The experimental results reveal that our algorithm outperforms its competitors in six distinct evaluation metrics, achieving an average performance improvement of approximately 9%. This substantial enhancement underscores the efficacy of our algorithm in handling complex multi-label data.

Multi-label feature selection using self-information in divergence-based fuzzy neighborhood rough sets

Multi-label feature selection corresponds to pattern recognition and knowledge mining, and its application has been expanded to different scenarios. As an excellent processing platform for uncertain and ambiguous information, divergence-based fuzzy rough sets (Div-FRSs) have been proposed and applied to feature selection. However, there are three critical problems to be solved when applying Div-FRSs to multi-label learning. The first is how to effectively dispose the noise produced by features in multi-label data. The second is how to synthetically consider the relevance among all labels. The last is how to thoroughly mine the uncertainty brought by upper approximations neglected in Div-FRSs existing researches. To address these issues, this study presents a new divergence-based fuzzy neighborhood rough set model (Div-FNRSs) for multi-label learning using self-information. First, the divergence-based fuzzy neighborhood relation and class are gradually raised to manage the noise in multi-label data, and fuzzy decision is introduced to dispose all labels as a whole. Combining them together, a new model Div-FNRSs is constructed. Then, divergence-based fuzzy neighborhood self-information containing upper approximations and lower approximations is designed to depict distinguishing ability of features through three-level uncertainty measure establishment and granulation property exploration. Furthermore, feature significance for choosing the optimal features is given and it motivates a heuristic feature-selection algorithm DivFNSI-FS. Finally, data experiments are completed to validate DivFNSI-FS effectiveness with six state-of-the-art multi-label feature selection approaches on fourteen multi-label datasets. A conclusion can be drawn that DivFNSI-FS outperforms existing algorithms to obtain better performance on eight commonly-used evaluation indexes.

An efficient multi-disease prediction model using advanced optimization aided weighted convolutional neural network with dilated gated recurrent unit

In the medical field, prediction accuracy over multi-diseases is significant and it is helpful for improving the patient’s health. Most of the conventional machine learning techniques concentrates only on detecting single diseases. Only a few systems are developed for predicting more than one disease. The classification of multi-label data is a challenging issue. Patients have symptoms of various diseases while analyzing the medical data and hence it is necessary to implement tools for the earlier identification of problems. The patterns in the health data have been effectively identified through deep learning-based health risk prediction models. Thus, an efficient prediction model for predicting various types of diseases is implemented in this work. Initially, the required data regarding various types of diseases will be gathered from Kaggle database. The garnered healthcare data are pre-processed for quality enhancement. The pre-processing procedures include data cleaning, data transformation, and outlier detection are performed at first. The outlier detection is done using the “Density-Based Spatial Clustering of Applications with Noise (DBSCAN)” approach. The pre-processed data is then given to the Weighted Convolutional Neural Network Feature with Dilated Gated Recurrent Unit (WCNNF-DGRU) model. Here, the pre-processed data is provided to the CNN structure for feature extraction, in which the weights are optimized by means of the Enhanced Kookaburra Optimization Algorithm (EKOA). Then the features from the weighted CNN layer are provided to the Dilated GRU structure to determine the final prediction output. Experimental verification is carried out on the implemented WCDG in predicting multiple diseases by comparing it with other conventional prediction models and optimization algorithms.

Imbalanced and missing multi-label data learning with global and local structure

Label missing and class imbalance problems are two hot research topics in machine learning, and they have been impeding the improvement of model performance, especially in the multi-label learning. Although some existing methods have proven to be effective, they are suitable for only one case. How to effectively address above two issues simultaneously is a challenging problem. In this paper, we propose a novel model named Imbalanced and Missing multi-Label data learning with Global and Local structure (IMLGL) to address the aforementioned challenge. There are following three advantages. At the empirical risk level, we introduce the label correlation matrix C into the loss function and devise a dynamic weighting method to address the aforementioned challenge. At the data level, we analyze the structural characteristics of the data, and introduce local low-rank and global high-rank term to enhance the generalization performance of the model. At the label level, a smoothing term is also introduced for learning the constraint classifier coefficient matrix W. Our method utilizes alternative optimization technique and alternating minimization method for solving. Extensive experiments on six datasets demonstrate the competitiveness of our approach.

An oversampling algorithm of multi-label data based on cluster-specific samples and fuzzy rough set theory

Imbalanced class distributions are common in real-world scenarios, including datasets with multiple labels. One widely acknowledged approach to addressing imbalanced distributions is through oversampling, a technique that both balances the class distribution and improves the effectiveness of classification models. However, when generating synthetic data for multi-label datasets, complexities arise due to the presence of multiple-label sets, which require careful placement and labeling. We propose MLCSMOTE-FRST, an algorithm for synthetic data generation based on label-specific clustering and fuzzy rough set theory. Generation ratios and dependency samples are provided by clusters specific to each label, with a focus on the overall label distribution and the distribution within each cluster. The labels are supported by intra-cluster positive samples, determined using fuzzy rough set theory, which helps to capture the consensus label set. Experimental results on multi-label datasets using four classifiers demonstrate the effectiveness of the proposed method in terms of macro-F1 and micro-F1 scores.

A novel multi-label feature selection method based on knowledge consistency-independence index

Multi-label classification encounters the challenge of dealing with high dimensional data. In response to this challenge, numerous researchers have proposed various multi-label feature selection methods from different perspectives. However, existing methods overlook the consistency and independence of knowledge granules, and thus fail to extract valuable and distinctive information from the knowledge granules that is relevant to the label space. To address this issue, we propose a novel multi-label feature selection method based on the knowledge consistency-independence index (CIMLFS). Firstly, we introduce the concepts of knowledge consistency granularity and knowledge independence granularity to explore valuable and distinctive information from the knowledge granule families. Secondly, based upon these concepts, we define the consistency coefficient, independence coefficient, and consistency gain for features, ultimately considering the three perspectives to achieve the knowledge consistency-independence index. Furthermore, we present a multi-label feature selection method utilizing the index. Finally, to assess the effectiveness of CIMLFS, we conduct comparative experiments with eight representative multi-label feature selection methods on twelve benchmark multi-label data sets and using four evaluation metrics. The final experimental results indicate that CIMLFS ranks the first on three metrics and the second on one metric.

Enhancing aspect-based multi-labeling with ensemble learning for ethical logistics.

In the dynamic domain of logistics, effective communication is essential for streamlined operations. Our innovative solution, the Multi-Labeling Ensemble (MLEn), tackles the intricate task of extracting multi-labeled data, employing advanced techniques for accurate preprocessing of textual data through the NLTK toolkit. This approach is carefully tailored to the prevailing language used in logistics communication. MLEn utilizes innovative methods, including sentiment intensity analysis, Word2Vec, and Doc2Vec, ensuring comprehensive feature extraction. This proves particularly suitable for logistics in e-commerce, capturing nuanced communication essential for efficient operations. Ethical considerations are a cornerstone in logistics communication, and MLEn plays a pivotal role in detecting and categorizing inappropriate language, aligning inherently with ethical norms. Leveraging Tf-IDF and Vader for feature enhancement, MLEn adeptly discerns and labels ethically sensitive content in logistics communication. Across diverse datasets, including Emotions, MLEn consistently achieves impressive accuracy levels ranging from 92% to 97%, establishing its superiority in the logistics context. Particularly, our proposed method, DenseNet-EHO, outperforms BERT by 8% and surpasses other techniques by a 15-25% efficiency. A comprehensive analysis, considering metrics such as precision, recall, F1-score, Ranking Loss, Jaccard Similarity, AUC-ROC, sensitivity, and time complexity, underscores DenseNet-EHO's efficiency, aligning with the practical demands within the logistics track. Our research significantly contributes to enhancing precision, diversity, and computational efficiency in aspect-based sentiment analysis within logistics. By integrating cutting-edge preprocessing, sentiment intensity analysis, and vectorization, MLEn emerges as a robust framework for multi-label datasets, consistently outperforming conventional approaches and giving outstanding precision, accuracy, and efficiency in the logistics field.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

Numerous high-dimension multilabel data are generated, posing a challenge for multilabel learning. Building effective learning models with discriminative features is essential to improve the performance of multilabel learning. Multilabel feature selection can filter out the discriminative features according to their contribution to classification. However, ambiguity, uncertainty, and missing labels coexist in real-life multilabel data, which brings adverse effects to multilabel feature selection. The multi-scale fuzzy rough set gives an effective way to mine intrinsic knowledge hidden in uncertain data. This paper first extends the multi-scale learning to multilabel data with missing labels and proposes a feature selection method for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures called FSMML. The missing label space construction and feature evaluation metric are carefully investigated in the framework of multi-scale learning. A multilabel multi-scale learning strategy is formalized with the fuzzy granularity cognitive mechanism as the core, and the multi-scale fusion fuzzy label learning is given to reconstruct the missing label space. Then, a novel multilabel multi-scale fuzzy rough sets with missing labels is developed, and the significance of each scale is quantified. Moreover, some multi-scale fusion fuzzy uncertainty measures are defined by capturing the sample fuzzy similarity in the feature and reconstructed label spaces. Accordingly, the relevance between features and label set and the interactivity and redundancy between features in feature evaluation are discussed. Finally, FSMML chooses high-quality features to maximize relevance and interactivity and minimize redundancy. Extensive experiments demonstrate the effectiveness of FSMML on fifteen datasets with missing labels.

Label relaxation and shared information for multi-label feature selection

Due to the rapid growth of labels and high-dimensional data, multi-label feature selection has attracted increasing attention. However, two common issues are ignored by existing multi-label feature selection methods: 1) The pseudo-label matrix is constructed by using logical label matrix directly. 2) There is an imbalance of label density in multi-label data. To tackle the mentioned issues, we innovatively propose a new method named Label relaxation and Shared information for Multi-label Feature Selection (LSMFS). Specifically, LSMFS combines a logical label matrix with a non-negative label relaxation matrix to fit a pseudo-label matrix, which is used for learning the correlations of class labels. LSMFS uses the feature weight matrix to capture a shared information of different related labels, which mitigates the influence of low-density labels on feature selection. The above principles are transformed into the objective function of LSMFS, and an alternate iterative optimization algorithm is developed to solve the objective function. Experiments on multi-label datasets from different domains have demonstrated the effectiveness of the proposed method LSMFS. Code is released at https://github.com/HQUF/LSMFS.