Multi-label Learning Research Articles

Multi-label feature selection via similarity constraints with non-negative matrix factorization

Feature selection plays a key role in preprocessing, effectively addressing the curse of dimensionality in multi-label learning. While current approaches commonly utilize feature or label similarity to construct the weight matrix, typically through manifold learning regularization, there has been a dearth of progress in developing similarity-constrained regularization terms for multi-label feature selection. To address this gap, this paper conducts a comprehensive investigation and proposes a novel similarity constraint leveraging non-negative matrix factorization techniques. Subsequently, a new algorithm is introduced termed Multi-label Feature Selection via Similarity Constraints with Non-negative Matrix Factorization (SCNMF). Initially, the Gaussian similarity matrix among features is computed and factorized into a weight matrix using non-negative matrix factorization. Subsequently, the Cosine distance matrix among labels is computed to constrain the weight matrix. Finally, an objective function is formulated based on the aforementioned constraint mechanisms and iteratively optimized. Extensive experiments conducted across more than 10 multi-label datasets demonstrate the superior performance of our proposed approach. The source code is accessible at the URL: https://github.com/BIGOatMNNU/SCNMF.

Advancing Automatic Gastritis Diagnosis: An Interpretable Multilabel Deep Learning Framework for the Simultaneous Assessment of Multiple Indicators

The evaluation of morphological features such as inflammation, gastric atrophy, and intestinal metaplasia is crucial for diagnosing gastritis. However, artificial intelligence (AI) analysis for nontumor diseases like gastritis is limited. Previous deep learning models have omitted important morphological indicators and cannot simultaneously diagnose gastritis indicators or provide interpretable labels. To address this, an attention-based multi-instance multilabel learning network (AMMNet) was developed to simultaneously achieve the multilabel diagnosis of activity, atrophy, and intestinal metaplasia with only slide-level weak labels. To evaluate AMMNet's real-world performance, a diagnostic test was designed to observe improvements in junior pathologists' diagnostic accuracy and efficiency with and without AMMNet assistance. In this study of 1,096 patients from 7 independent medical centers, AMMNet performed well in assessing activity (area under the curve (AUC): 0.93), atrophy (AUC: 0.97), and intestinal metaplasia (AUC: 0.93). The false-negative rates (FNRs) of these indicators were only 0.04, 0.08, and 0.18, respectively, and junior pathologists had lower FNRs with model assistance (0.15 vs. 0.10). Furthermore, AMMNet reduced the time required per whole-slide image (WSI) from 5.46 minutes to only 2.85 minutes, enhancing diagnostic efficiency. In block-level clustering analysis, AMMNet effectively visualized task-related patches within WSIs, improving interpretability. These findings highlight AMMNet’s effectiveness in accurately evaluating gastritis morphological indicators on multicenter datasets. Using multi-instance multilabel learning strategies to support routine diagnostic pathology deserves further evaluation.

Feature selection for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures

Numerous high-dimension multilabel data are generated, posing a challenge for multilabel learning. Building effective learning models with discriminative features is essential to improve the performance of multilabel learning. Multilabel feature selection can filter out the discriminative features according to their contribution to classification. However, ambiguity, uncertainty, and missing labels coexist in real-life multilabel data, which brings adverse effects to multilabel feature selection. The multi-scale fuzzy rough set gives an effective way to mine intrinsic knowledge hidden in uncertain data. This paper first extends the multi-scale learning to multilabel data with missing labels and proposes a feature selection method for multilabel classification with missing labels via multi-scale fusion fuzzy uncertainty measures called FSMML. The missing label space construction and feature evaluation metric are carefully investigated in the framework of multi-scale learning. A multilabel multi-scale learning strategy is formalized with the fuzzy granularity cognitive mechanism as the core, and the multi-scale fusion fuzzy label learning is given to reconstruct the missing label space. Then, a novel multilabel multi-scale fuzzy rough sets with missing labels is developed, and the significance of each scale is quantified. Moreover, some multi-scale fusion fuzzy uncertainty measures are defined by capturing the sample fuzzy similarity in the feature and reconstructed label spaces. Accordingly, the relevance between features and label set and the interactivity and redundancy between features in feature evaluation are discussed. Finally, FSMML chooses high-quality features to maximize relevance and interactivity and minimize redundancy. Extensive experiments demonstrate the effectiveness of FSMML on fifteen datasets with missing labels.

Discriminative label correlation based robust structure learning for multi-label feature selection

Feature selection is a key technique to tackle the curse of dimensionality in multi-label learning. Lots of embedded multi-label feature selection methods have been developed. However, they face challenges in identifying and excluding redundant features. To address these issues, this paper proposes a multi-label feature selection method that combines robust structural learning and discriminative label regularization. The proposed method starts from the feature space rather than data space, motivated by the principle that redundant features have high similarity or strong correlation. To exclude redundant features, a regularization on the feature selection matrix is designed by combining ℓ2,1-norm penalty with inner products of feature weight vectors. This regularization can help to learn a robust structure in the feature selection matrix. Meanwhile, both of the similarity and dissimilarity of labels of instances are involved in exploring discriminative label correlations. Extensive experiments verified the effectiveness of the proposed model for feature selection.

Partial multi-label feature selection via low-rank and sparse factorization with manifold learning

Feature selection is a commonly utilized methodology in multi-label learning (MLL) for tackling the challenge of high-dimensional data. Accurate annotation of relevant labels is crucial for successful multi-label feature selection (MFS). Nevertheless, multi-label datasets frequently consist of ground-truth and noisy labels in real-world applications, giving rise to the partial multi-label learning (PML) problem. The inclusion of noisy labels complicates the task of conventional MFS methods in accurately identifying the optimal features subset in such datasets. To tackle this issue, we propose a novel partial multi-label feature selection method with low-rank sparse factorization and manifold learning, called PMFS-LRS. Specifically, we first decompose the candidate label matrix into two distinct components: a low-rank matrix referring to ground-truth labels and a sparse matrix referring to noisy labels. This decomposition allows PMFS-LRS to effectively distinguish noise labels from ground-truth labels, thereby mitigating the impact of noisy data. Then, the local label correlations are explored using a manifold learning framework to improve the label disambiguation performance. Finally, a l2,1-norm regularization is integrated into the objective function to facilitate effective feature selection. Comprehensive experiments conducted on both real-world and synthetic PML datasets demonstrate that PMFS-LRS is superior to several existing state-of-the-art MFS methods.

Collaborative learning of supervision and correlation for generalized zero-shot extreme multi-label learning

Collaborative learning of supervision and correlation for generalized zero-shot extreme multi-label learning

Multi-Label Noise Robust Collaborative Learning for Remote Sensing Image Classification.

The development of accurate methods for multi-label classification (MLC) of remote sensing (RS) images is one of the most important research topics in RS. The MLC methods based on convolutional neural networks (CNNs) have shown strong performance gains in RS. However, they usually require a high number of reliable training images annotated with multiple land-cover class labels. Collecting such data is time-consuming and costly. To address this problem, the publicly available thematic products, which can include noisy labels, can be used to annotate RS images with zero-labeling cost. However, multi-label noise (which can be associated with wrong and missing label annotations) can distort the learning process of the MLC methods. To address this problem, we propose a novel multi-label noise robust collaborative learning (RCML) method to alleviate the negative effects of multi-label noise during the training phase of a CNN model. RCML identifies, ranks, and excludes noisy multi-labels in RS images based on three main modules: 1) the discrepancy module; 2) the group lasso module; and 3) the swap module. The discrepancy module ensures that the two networks learn diverse features, while producing the same predictions. The task of the group lasso module is to detect the potentially noisy labels assigned to multi-labeled training images, while the swap module is devoted to exchange the ranking information between two networks. Unlike the existing methods that make assumptions about noise distribution, our proposed RCML does not make any prior assumption about the type of noise in the training set. The experiments conducted on two multi-label RS image archives confirm the robustness of the proposed RCML under extreme multi-label noise rates. Our code is publicly available at: https://www.noisy-labels-in-rs.org.

An artificial immune system algorithm for classification tasks. An electronic nose case study

This study explores the application of an electronic nose (e-nose), employing an Artificial Immune System (AIS) as a competitive alternative to the typical machine learning techniques. The proposed algorithm's main function is to modify its dimensions and adjust the values of its parameters during the training or mutation process, with no complex operations and minimal parameter tuning. This analytical tool has been applied to the qualitative analysis of chocolate bars, classifying varieties in single and multi-label scenarios. The classification results demonstrate promising results in accurately distinguishing individual chocolate types, obtaining an average accuracy of 97%. Subsequently, multi-label classification (several labels associated with a single chocolate) was executed based on shared ingredient content, achieving an average accuracy of 94%. The evaluation and validation of the proposed AIS were also verified; the obtained performance metrics were compared with those of typical classification techniques, k-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Artificial Neural Network (ANN). Satisfactory results show the competitive performance of AIS in single-label classification tasks compared to named techniques. Promisingly, a novel method for multi-label learning in e-noses is obtained to accurately identify different types of chocolates, outperforming methods like KNN and SVM. These results demonstrate the AIS potential as an effective classification method for e-nose applications.

Conglomerate Crop Recommendation by using Multi-label Learning via Ensemble Supervised Clustering Techniques

Existing crop recommender related to either binary classification or multiclass classification. This paper presents conglomerate crop recommendations which consist of a number of different and distinct crops that are grouped together. In this work we focus on transferring knowledge from single label output prediction to multiple label predicted output for a given input data instances. We proposed ESCT algorithm i.e. Ensemble Supervised Clustering Techniques in our research work. ESCT provides a combined approach of conventional clustering and enhanced supervised clustering methodology to optimize the conglomerate recommendation. We are focusing on K-mean clustering for conventional approach and ICCC i.e. Inter cluster correlation coefficient to achieve enhancement in supervised clustering. In conventional K-mean clustering there is a big challenge on how to optimize the k-value of clustering which directly affects the convergence of the clusters. To resolve this problem, we mainly apply function approximation on K-Value which provides us with better clustering and fast convergence. Existing methods for inter-clustering do not adequately address one of the key challenges i.e. exploiting correlations between labels and that is achieved by ICCC algorithm. This model provides learning and prediction of unknown observation by using Back propagation MLL algorithm which provides improved performance.

The gut microbiome associates with phenotypic manifestations of post-acute COVID-19 syndrome

The mechanisms underlying the many phenotypic manifestations of post-acute COVID-19 syndrome (PACS) are poorly understood. Herein, we characterized the gut microbiome in heterogeneous cohorts of subjects with PACS and developed a multi-label machine learning model for using the microbiome to predict specific symptoms. Our processed data covered 585 bacterial species and 500 microbial pathways, explaining 12.7% of the inter-individual variability in PACS. Three gut-microbiome-based enterotypes were identified in subjects with PACS and associated with different phenotypic manifestations. The trained model showed an accuracy of 0.89 in predicting individual symptoms of PACS in the test set and maintained a sensitivity of 86% and a specificity of 82% in predicting upcoming symptoms in an independent longitudinal cohort of subjects before they developed PACS. This study demonstrates that the gut microbiome is associated with phenotypic manifestations of PACS, which has potential clinical utility for the prediction and diagnosis of PACS.

An Ensemble Methods of Predicting the New Labels with Concept Drift from a High-Dimensional Data Stream

Multi-Label Learning (MLL) has arisen in data engineering to identify instances based on a specific feature associated with a collection of labels. Adaptive learning necessitates classifying features with New Labels (NLs) if a data stream contains newer perspectives. As a result, an MLL with Emerging Multiple NLs (MuEMNL) and managing High-Dimensional data streams (MuEMNLHD) approaches were developed that divides the NL sets into multiple NLs for efficient classification. However, it did not handle concept drift issues when huge amounts of data arrived at high speeds using limited resources. Hence, this article proposes an adaptive ensemble learning approach to cope with a huge amount of data streams and solve concept drift issues by constructing a MuEMNL-Ensemble Neural Network (ENN) rather than a random forest classifier. It defines the number of NNs in the ensemble, whether or not they use constructive pruning, how many hidden nodes each NN uses, and how many training samples are used to train each NN independently. Also, to solve the concept drifts, pairwise and non-pairwise diversity measures are analyzed while constructing ensemble NN for efficient training using the entire learning examples. Moreover, the tradeoff between the NN’s precision and diversity is maintained simultaneously. At last, the test outcomes reveal that the proposed approach attains a better performance contrasted with the existing MLL approaches.

A ranking-based problem transformation method for weakly supervised multi-label learning

Problem transformation is a simple yet effective framework for multi-label learning, where the original multi-label problem can be transformed into a series of single-label subproblems. However, the existing problem transformation methods have difficulties in handling label defect issues in real applications, e.g. multi-label learning with missing labels, partial multi-label learning and noisy multi-label learning. To deal with these issues, we propose a novel problem transformation method named EPR (i.e., Ensemble of Pairwise Ranking learners) applicable to various multi-label tasks. In EPR, the weakly supervised multi-label problem is converted into an ensemble of supervised single-label patterns due to pairwise label ranking, which successfully enhances label correlation exploration and improves the utilization of instances with defect labels. Moreover, an ensemble pruning mechanism is presented to heuristically balance the model performance and efficiency. Extensive experiments demonstrate the effectiveness of EPR against state-of-the-art algorithms in diverse multi-label learning scenarios.

Evolutionary simultaneous under and oversampling of instances for dealing with class-imbalance datasets in multilabel problems

Multilabel classification has recently attracted great attention from the data mining research community. Multilabel classification is concerned with learning where each instance can be associated with multiple classes (or labels). Class-imbalance problems appear in any classification task when the class distribution of the instances is very different. In multilabel classification, this problem is ubiquitous, as a large percentage of labels suffer from a class-imbalanced distribution. The adaptation of single-label methods to deal with the class-imbalance problem in multilabel learning is problematic as many of their basic concepts are not easily transferred. In this paper, we propose the use of evolutionary computation to simultaneously oversample the minority class and undersample the majority class for multilabel problems. Letting the algorithm autonomously select the instances to undersample and oversample allows us to extend these two successful paradigms to the multilabel task. An extensive comparison setup of 35 datasets shows the advantages of using this approach to deal with class-imbalance datasets for multilabel problems compared with previously published methods as well as the basic classification algorithms with the original datasets.

Multiple reference points-based multi-objective feature selection for multi-label learning

Multiple reference points-based multi-objective feature selection for multi-label learning

Technique of Feature Extraction Based on Interpretation Analysis for Multilabel Learning in Nonintrusive Load Monitoring With Multiappliance Circumstances

Technique of Feature Extraction Based on Interpretation Analysis for Multilabel Learning in Nonintrusive Load Monitoring With Multiappliance Circumstances

Feature selection for multi-label learning based on variable-degree multi-granulation decision-theoretic rough sets

Multi-label learning (MLL) suffers from the high-dimensional feature space teeming with irrelevant and redundant features. To tackle this, several multi-label feature selection (MLFS) algorithms have emerged as vital preprocessing steps. Nonetheless, existing MLFS methods have their shortcomings. Primarily, while they excel at harnessing label-feature relationships, they often struggle to leverage inter-feature information effectively. Secondly, numerous MLFS approaches overlook the uncertainty in the boundary domain, despite its critical role in identifying high-quality features. To address these issues, this paper introduces a novel MLFS algorithm, named VMFS. It innovatively integrates multi-granulation rough sets with three-way decision, leveraging multi-granularity decision-theoretic rough sets (MGDRS) with variable degrees for optimal performance. Initially, we construct coarse decision (RDC), fine decision (RDF), and uncertainty decision (RDU) functions for each object based on MGDRS with variable degrees. These decision functions then quantify the dependence of attribute subsets, considering both deterministic and uncertain aspects. Finally, we employ the dependency to assess attribute importance and rank them accordingly. Our proposed method has undergone rigorous evaluation on various standard multi-label datasets, demonstrating its superiority. Experimental results consistently show that VMFS significantly outperforms other algorithms on most datasets, underscoring its effectiveness and reliability in multi-label learning tasks.

Multi-label Feature Selection based on Label-specific features and Manifold Learning

Each instance in multi-label data is associated with multiple labels, and there are irrelevant or redundant features in its feature space, which leads to the performance degradation of multi-label learning algorithms. Multi-label feature selection selects representative features from the feature space to improve the accuracy of the model. Due to the high cost of labels and the difficulty of data collection, there will be some missing labels in the data set, which affects the accuracy of feature selection. To solve this problem, a multi-label feature selection algorithm based on label-specific features and manifold learning is proposed. The algorithm uses the linear relationship between the features and labels in known label samples to build a linear regression model for learning label-specific features. By using the nonlinear relation between instances and the nonlinear relation between features, we can precisely learn the label-specific features. We use the Laplacian feature mapping method to construct the instance manifold model and the feature manifold model, which are also used as the regular term constraint weight matrix. The final model can not only complete the missing labels, but also select sparse and representative features. The feature selection is carried out by analyzing the weight of the feature given by the final model. Experiments were conducted to verify the effectiveness of the proposed algorithm under different label deletion rates on four evaluation indexes.

Combining Graph Transformers Based Multi-Label Active Learning and Informative Data Augmentation for Chest Xray Classification

Informative sample selection in active learning (AL) helps a machine learning system attain optimum performance with minimum labeled samples, thus improving human-in-the-loop computer-aided diagnosis systems with limited labeled data. Data augmentation is highly effective for enlarging datasets with less labeled data. Combining informative sample selection and data augmentation should leverage their respective advantages and improve performance of AL systems. We propose a novel approach to combine informative sample selection and data augmentation for multi-label active learning. Conventional informative sample selection approaches have mostly focused on the single-label case which do not perform optimally in the multi-label setting. We improve upon state-of-the-art multi-label active learning techniques by representing disease labels as graph nodes, use graph attention transformers (GAT) to learn more effective inter-label relationships and identify most informative samples. We generate transformations of these informative samples which are also informative. Experiments on public chest xray datasets show improved results over state-of-the-art multi-label AL techniques in terms of classification performance, learning rates, and robustness. We also perform qualitative analysis to determine the realism of generated images.

GLDL: Graph Label Distribution Learning

Label Distribution Learning (LDL), as a more general learning setting than generic single-label and multi-label learning, has been commonly used in computer vision and many other applications. To date, existing LDL approaches are designed and applied to data without considering the interdependence between instances. In this paper, we propose a Graph Label Distribution Learning (GLDL) framework, which explicitly models three types of relationships: instance-instance, label-label, and instance-label, to learn the label distribution for networked data. A label-label network is learned to capture label-to-label correlation, through which GLDL can accurately learn label distributions for nodes. Dual graph convolution network (GCN) Co-training with heterogeneous message passing ensures two GCNs, one focusing on instance-instance relationship and the other one targeting label-label correlation, are jointly trained such that instance-instance relationship can help induce label-label correlation and vice versa. Our theoretical study derives the error bound of GLDL. For verification, four benchmark datasets with label distributions for nodes are created using common graph benchmarks. The experiments show that considering dependency helps learn better label distributions for networked data, compared to state-of-the-art LDL baseline. In addition, GLDL not only outperforms simple GCN and graph attention networks (GAT) using distribution loss but is also superior to its variant considering label-label relationship as a static network. GLDL and its benchmarks are the first research endeavors to address LDL for graphs. Code and benchmark data are released for public access.

Variational Hybrid-Attention Framework for Multi-Label Few-Shot Aspect Category Detection

Multi-label few-shot aspect category detection (FS-ACD) is a challenging sentiment analysis task, which aims to learn a multi-label learning paradigm with limited training data. The difficulty of this task is how to use limited data to generalize effective discriminative representations for different categories. Nowadays, all advanced FS-ACD works utilize the prototypical network to learn label prototypes to represent different aspects. However, such point-based estimation methods are inherently noise-susceptible and bias-vulnerable. To this end, this paper proposes a novel Variational Hybrid-Attention Framework (VHAF) for the FS-ACD task. Specifically, to alleviate the data noise, we adopt a hybrid-attention mechanism to generate more discriminative aspect-specific embeddings. Then, based on these embeddings, we introduce the variational distribution inference to obtain the aspect-specific distribution as a more robust aspect representation, which can eliminate the scarce data bias for better inference. Moreover, we further leverage an adaptive threshold estimation to help VHAF better identify multiple relevant aspects. Extensive experiments on three datasets demonstrate the effectiveness of our VHAF over other state-of-the-art methods. Code is available at https://github.com/chengzju/VHAF.