Label Co-occurrence Research Articles

A Label Information Aware Model for Multi-label Text Classification

Multi-label text classification (MLTC) refers to that each document is associated with more than one label at the same time, which attract much attention from researchers in both academia and industry. Existing methods have difficulties in determining label-related components from documents, which cannot effectively establish the association between textual features and label information. In fact, there are some label information, such as label semantic information and co-occurrence relations among labels, could be used to improve the performance on multi-label text classification. In this paper, we propose a label information aware model to utilize these information. Our model makes use of label semantic information to determine label-related components from textual features for obtaining the label-specific textual presentation for each sample, and then take advantages of co-occurrence relations among labels to construct interaction among label-specific textual presentation. The superiority of our model has been proved through comparing our method with several existing models on two datasets.

Hoeffding adaptive trees for multi-label classification on data streams

Data stream learning is a very relevant paradigm because of the increasing real-world scenarios generating data at high velocities and in unbounded sequences. Stream learning aims at developing models that can process instances as they arrive, so models constantly adapt to new concepts and the temporal evolution in the stream. In multi-label data stream environments where instances have the peculiarity of belonging simultaneously to more than one class, the problem becomes even more complex and poses unique challenges such as different concept drifts impacting different labels at simultaneous or distinct times, higher class imbalance, or new labels emerging in the stream. This paper proposes a novel approach to multi-label data stream classification called Multi-Label Hoeffding Adaptive Tree (MLHAT). MLHAT leverages the Hoeffding adaptive tree to address these challenges by considering possible relations and label co-occurrences in the partitioning process of the decision tree, dynamically adapting the learner in each leaf node of the tree, and implementing a concept drift detector that can quickly detect and replace tree branches that are no longer performing well. The proposed approach is compared with other 18 online multi-label classifiers on 41 datasets. The results, validated with statistical analysis, show that MLHAT outperforms other state-of-the-art approaches in 12 well-known multi-label metrics.

Breaking the gap between label correlation and instance similarity via new multi-label contrastive learning

Multi-label text classification (MLTC) is a fundamental yet challenging task in natural language processing. Existing MLTC models mostly learn text representations and label correlations, separately; while the instance-level correlation, which is crucial for the classification is ignored. To rectify this, we propose a new multi-label contrastive learning model, that captures instance-level correlations, for the MLTC task. Specifically, we first learn label representations by using Graph Convolutional Network (GCN) on label co-occurrence graphs. We next learn text representations by taking label correlations into consideration. Through an attention mechanism, instance-level correlation can be established. To better utilize label correlations, we propose a new contrastive learning model, whose learning is guided by a new learning objective, to further refine label representations. We finally implement a k-NN mechanism, that identifies k nearest neighbors of a given text for final prediction. Intensive experimental studies over benchmark multi-label datasets demonstrate the effectiveness of our approach.

CoocNet: a novel approach to multi-label text classification with improved label co-occurrence modeling

CoocNet: a novel approach to multi-label text classification with improved label co-occurrence modeling

Towards long-tailed, multi-label disease classification from chest X-ray: Overview of the CXR-LT challenge

Many real-world image recognition problems, such as diagnostic medical imaging exams, are “long-tailed” – there are a few common findings followed by many more relatively rare conditions. In chest radiography, diagnosis is both a long-tailed and multi-label problem, as patients often present with multiple findings simultaneously. While researchers have begun to study the problem of long-tailed learning in medical image recognition, few have studied the interaction of label imbalance and label co-occurrence posed by long-tailed, multi-label disease classification. To engage with the research community on this emerging topic, we conducted an open challenge, CXR-LT, on long-tailed, multi-label thorax disease classification from chest X-rays (CXRs). We publicly release a large-scale benchmark dataset of over 350,000 CXRs, each labeled with at least one of 26 clinical findings following a long-tailed distribution. We synthesize common themes of top-performing solutions, providing practical recommendations for long-tailed, multi-label medical image classification. Finally, we use these insights to propose a path forward involving vision-language foundation models for few- and zero-shot disease classification.

Enhancing Multi-Label Classification via Dynamic Label-Order Learning

Generative methods tackle Multi-Label Classification (MLC) by autoregressively generating label sequences. These methods excel at modeling label correlations and have achieved outstanding performance. However, a key challenge is determining the order of labels, as empirical findings indicate the significant impact of different orders on model learning and inference. Previous works adopt static label-ordering methods, assigning a unified label order for all samples based on label frequencies or co-occurrences. Nonetheless, such static methods neglect the unique semantics of each sample. More critically, these methods can cause the model to rigidly memorize training order, resulting in missing labels during inference. In light of these limitations, this paper proposes a dynamic label-order learning approach that adaptively learns a label order for each sample. Specifically, our approach adopts a difficulty-prioritized principle and iteratively constructs the label sequence based on the sample s semantics. To reduce the additional cost incurred by label-order learning, we use the same SEQ2SEQ model for label-order learning and MLC learning and introduce a unified loss function for joint optimization. Extensive experiments on public datasets reveal that our approach greatly outperforms previous methods. We will release our code at https: //github.com/KagamiBaka/DLOL.

Semantic interleaving global channel attention for multilabel remote sensing image classification

ABSTRACT Multilabel remote sensing image classification (MLRSIC) has received increasing research interest. Taking the co-occurrence relationship of multiple labels as additional information helps to improve the overall performance. However, current methods only focus on using it to constrain the final feature which is output from a convolutional neural network (CNN). On the one hand, these methods need to exploit the potential of label correlation in feature representation fully. On the other hand, they increase the label noise sensitivity of the system, resulting in poor robustness. In this paper, a novel method called ‘Semantic Interleaving Global chaNnel Attention’ (SIGNA) is proposed for MLRSIC. First, the label co-occurrence graph is obtained according to the statistical information of the training set and fed into a graph neural network (GNN) to generate optimal semantic feature representations of each label. Next, the semantic features are interleaved with visual features which are extracted by CNNs to guide the overall features of the input image transform from the original feature space to the semantic feature space with embedded label relations. Then, global attention triggered by semantic interleaving is used to emphasize visual features in important channels. Finally, to make SIGNA easier to use and more optimized, multihead SIGNA-based feature adaptive weighting networks are proposed as plug-in blocks to plug into any layers of a CNN. For remote sensing images, better classification performance can be achieved by inserting the plug-in blocks into the shallow layers of CNNs. We conducted extensive experimental comparisons on three data sets: UCM, AID and DFC15. Experimental results demonstrate that the proposed SIGNA achieves superior classification performance compared to state-of-the-art (SOTA) methods. Notes that the codes of this paper will be open to the community for reproducibility research.

Learning group-wise spatial attention and label dependencies for multi-task thoracic disease classification

This paper considers the multi-label thoracic abnormality classification with chest X-ray images. In clinical settings, Chest X-ray imaging is a general diagnostic tool applied to visualize numerous thoracic pathological changes. While deep learning techniques have been extensively tested in this field, certain challenges persist. The data in existing thoracic abnormality datasets is insufficient, and some diseases are extremely imbalanced. Meanwhile, the dependencies between different labels are often ignored. To tackle these issues head-on, this paper introduces two crucial modules: the group-wise spatial attention (GWSA) module and the label co-occurrence dependency (LCD) module, integrated with DenseNet121 backbone. Specifically, GWSA enhances the spatial features within distinct groups while keeping the between-group feature discrimination. LCD models the correlations between different thoracic abnormalities to refine the predicted probabilities. In conjunction with the DenseNet121 backbone, these two modules reach an average AUC score of 0.818 on Chest X-ray14 dataset, achieving state-of-the-art. Source code is available at https://github.com/YujiaKCL/Chest-Xray14-GWSA-LCD.

Hierarchical Knowledge Guided Learning for Real-World Retinal Disease Recognition.

In the real world, medical datasets often exhibit a long-tailed data distribution (i.e., a few classes occupy the majority of the data, while most classes have only a limited number of samples), which results in a challenging long-tailed learning scenario. Some recently published datasets in ophthalmology AI consist of more than 40 kinds of retinal diseases with complex abnormalities and variable morbidity. Nevertheless, more than 30 conditions are rarely seen in global patient cohorts. From a modeling perspective, most deep learning models trained on these datasets may lack the ability to generalize to rare diseases where only a few available samples are presented for training. In addition, there may be more than one disease for the presence of the retina, resulting in a challenging label co-occurrence scenario, also known as multi-label, which can cause problems when some re-sampling strategies are applied during training. To address the above two major challenges, this paper presents a novel method that enables the deep neural network to learn from a long-tailed fundus database for various retinal disease recognition. Firstly, we exploit the prior knowledge in ophthalmology to improve the feature representation using a hierarchy-aware pre-training. Secondly, we adopt an instance-wise class-balanced sampling strategy to address the label co-occurrence issue under the long-tailed medical dataset scenario. Thirdly, we introduce a novel hybrid knowledge distillation to train a less biased representation and classifier. We conducted extensive experiments on four databases, including two public datasets and two in-house databases with more than one million fundus images. The experimental results demonstrate the superiority of our proposed methods with recognition accuracy outperforming the state-of-the-art competitors, especially for these rare diseases.

Label-Aware Dual Graph Neural Networks for Multi-Label Fundus Image Classification.

Fundus disease is a complex and universal disease involving a variety of pathologies. Its early diagnosis using fundus images can effectively prevent further diseases and provide targeted treatment plans for patients. Recent deep learning models for classification of this disease are gradually emerging as a critical research field, which is attracting widespread attention. However, in practice, most of the existing methods only focus on local visual cues of a single image, and ignore the underlying explicit interaction similarity between subjects and correlation information among pathologies in fundus diseases. In this paper, we propose a novel label-aware dual graph neural networks for multi-label fundus image classification that consists of population-based graph representation learning and pathology-based graph representation learning modules. Specifically, we first construct a population-based graph by integrating image features and non-image information to learn patient's representations by incorporating associations between subjects. Then, we represent pathologies as a sparse graph where its nodes are associated with pathology-based feature vectors and the edges correspond to probability of the co-occurrence of labels to generate a set of classifier scores by the propagation of multi-layer graph information. Finally, our model can adaptively recalibrate multi-label outputs. Detailed experiments and analysis of our results show the effectiveness of our method compared with state-of-the-art multi-label fundus image classification methods.

Address the Unseen Relationships: Attribute Correlations in Text Attribute Person Search.

Text attribute person search aims to identify the particular pedestrian by textual attribute information. Compared to person re-identification tasks which requires imagery samples as its query, text attribute person search is more useful under the circumstance where only witness is available. Most existing text attribute person search methods focus on improving the matching correlation and alignments by learning better representations of person-attribute instance pairs, with few consideration of the latent correlations between attributes. In this work, we propose a graph convolutional network (GCN) and pseudo-label-based text attribute person search method. Concretely, the model directly constructs the attribute correlations by label co-occurrence probability, in which the nodes are represented by attribute embedding and edges are by the filtered correlation matrix of attribute labels. In order to obtain better representations, we combine the cross-attention module (CAM) and the GCN. Furthermore, to address the unseen attribute relationships, we update the edge information through the instances through testing set with high predicted probability thus to better adapt the attribute distribution. Extensive experiments illustrate that our model outperforms the existing state-of-the-art methods on publicly available person search benchmarks: Market-1501 and PETA.

Mining Semantic Information With Dual Relation Graph Network for Multi-Label Image Classification

The purpose of multi-label image classification is to assign multiple labels for multiple objects presented in one image. Recent research efforts exploit graph convolution network (GCN) to learn the label co-occurrence dependencies for enhancing the semantic representation. Although these methods have achieved promising results, they can not capture the intrinsic correlation between objects in images and do not consider the inter-channel relationship. In addition, the previous methods treat each single image independently and fail to explore the relationship between different images. To address the above challenges, we propose a novel Dual Relation Graph Network (DRGN) model, which adopts a double branch structure to excavate rich semantic information from intra-image and cross-image simultaneously. Specifically, we first develop an intra-image channel-relation mining (ICM) module to mine the inter-channel relationship in features while learning the importance of different channels. Secondly, we design a new GCN-based intra-image spatial-relation exploring (ISE) module to capture the correlation between objects in individual image. Notably, ISE module and ICM module can complement and promote each other from the spatial and channel dimensions of images to improve the correlation between objects in individual image. Thirdly, we propose a novel GCN-based cross-image semantic learning (CSL) module to learn the semantic relationship between different images in the mini-batch. Through graph reasoning, our CSL module can iteratively refine input image features by acquiring common semantic information from other images in the mini-batch. Extensive experiments on the MS-COCO 2014, PASCAL VOC 2007, and VG-500 datasets demonstrate that the proposed DRGN model outperforms current state-of-the-art methods.

A Dual-branch Learning Model with Gradient-balanced Loss for Long-tailed Multi-label Text Classification

Multi-label text classification has a wide range of applications in the real world. However, the data distribution in the real world is often imbalanced, which leads to serious long-tailed problems. For multi-label classification, due to the vast scale of datasets and existence of label co-occurrence, how to effectively improve the prediction accuracy of tail labels without degrading the overall precision becomes an important challenge. To address this issue, we propose A Dual-Branch Learning Model with Gradient-Balanced Loss (DBGB) based on the paradigm of existing pre-trained multi-label classification SOTA models. Our model consists of two main long-tailed module improvements. First, with the shared text representation, the dual-classifier is leveraged to process two kinds of label distributions; one is the original data distribution and the other is the under-sampling distribution for head labels to strengthen the prediction for tail labels. Second, the proposed gradient-balanced loss can adaptively suppress the negative gradient accumulation problem related to labels, especially tail labels. We perform extensive experiments on three multi-label text classification datasets. The results show that the proposed method achieves competitive performance on overall prediction results compared to the state-of-the-art methods in solving the multi-label classification, with significant improvement on tail-label accuracy.

Causal multi-label learning for image classification

In this paper, we investigate the problem of causal image classification with multi-label learning. As multi-label learning involves a diversity of supervision signals, it is considered a challenging issue to solve. Previous approaches have attempted to improve performance by identifying label-related image areas or exploiting the co-occurrence of labels. However, these methods are often characterized by complicated procedures, tedious computations, and a lack of intuitive interpretations. To overcome these limitations, we propose a novel approach that incorporates the concept of causal inference, which has been shown to be beneficial in other computer vision problems. Our method, called causal multi-label learning (CMLL), enables the selection of multiple objects from the original image through a multi-class attention module. These objects are then subjected to causal intervention to learn the causal relationships between different labels. Our proposed approach is both elegant and effective, with low computational cost and few parameters required for the multi-class causal intervention approach. Extensive tests and ablation studies demonstrate that the proposed method significantly improves prediction performance without a significant increase in training and inference times.

Graph embedding based multi-label Zero-shot Learning

Multi-label Zero-shot Learning (ZSL) is more reasonable and realistic than standard single-label ZSL because several objects can co-exist in a natural image in real scenarios. Intra-class feature entanglement is a significant factor influencing the alignment of visual and semantic features, resulting in the model’s inability to recognize unseen samples comprehensively and completely. We observe that existing multi-label ZSL methods place a greater emphasis on attention-based refinement and decoupling of visual features, while ignoring the relationship between label semantics. Relying on label correlations to solve multi-label ZSL tasks has not been deeply studied. In this paper, we make full use of the co-occurrence relationship between category labels and build a directed weighted semantic graph based on statistics and prior knowledge, in which node features represent category semantics and weighted edges represent conditional probabilities of label co-occurrence. To guide the targeted extraction of visual features, node features and edge set weights are simultaneously updated and refined, and embedded into the visual feature extraction network from a global and local perspective. The proposed method’s effectiveness was demonstrated by simulation results on two challenging multi-label ZSL benchmarks: NUS-WIDE and Open Images. In comparison to state-of-the-art models, our model achieves an absolute gain of 2.4% mAP on NUS-WIDE and 2.1% mAP on Open Images respectively.

BioSift: A Dataset for Filtering Biomedical Abstracts for Drug Repurposing and Clinical Meta-Analysis.

This work presents a new, original document classification dataset, BioSift, to expedite the initial selection and labeling of studies for drug repurposing. The dataset consists of 10,000 human-annotated abstracts from scientific articles in PubMed. Each abstract is labeled with up to eight attributes necessary to perform meta-analysis utilizing the popular patient-intervention-comparator-outcome (PICO) method: has human subjects, is clinical trial/cohort, has population size, has target disease, has study drug, has comparator group, has a quantitative outcome, and an "aggregate" label. Each abstract was annotated by 3 different annotators (i.e., biomedical students) and randomly sampled abstracts were reviewed by senior annotators to ensure quality. Data statistics such as reviewer agreement, label co-occurrence, and confidence are shown. Robust benchmark results illustrate neither PubMed advanced filters nor state-of-the-art document classification schemes (e.g., active learning, weak supervision, full supervision) can efficiently replace human annotation. In short, BioSift is a pivotal but challenging document classification task to expedite drug repurposing. The full annotated dataset is publicly available and enables research development of algorithms for document classification that enhance drug repurposing.

Partial multi-label learning via three-way decision-based tri-training

In real-world application scenarios, multi-label learning (MLL) datasets often contain some irrelevant noisy labels, which degrades the performance of traditional multi-label learning models. In order to deal with this problem, partial multi-label learning (PML) is proposed, in which each instance is associated with a candidate label set, which includes multiple relevant ground-truth labels and some irrelevant noisy labels. The common strategy to deal with this problem is disambiguating the candidate label set, but the co-occurrence of noisy labels and ground-truth labels makes the disambiguation technique susceptible to error. In this paper, a novel disambiguation-free PML approach named PML-TT is proposed. Specifically, by adapting the tri-training framework, mutual cooperation and iteration between classifiers are used to correct noisy labels and improve the performance of the learning model. Moreover, the three-way decision is adapted to solve the conflict problem of the base classifier and obtain more useful training samples. In addition, the precise supervisory information of the non-candidate labels is exploited to make the predictions of the base classifier more accurate. Finally, experimental results on both synthetic and real-world PML datasets show that the proposed PML-TT approach can effectively reduce the negative influence of noisy labels and learn a robust model.

Aligning Image Semantics and Label Concepts for Image Multi-Label Classification

Image multi-label classification task is mainly to correctly predict multiple object categories in the images. To capture the correlation between labels, graph convolution network based methods have to manually count the label co-occurrence probability from training data to construct a pre-defined graph as the input of graph network, which is inflexible and may degrade model generalizability. Moreover, most of the current methods cannot effectively align the learned salient object features with the label concepts, so that the predicted results of model may not be consistent with the image content. Therefore, how to learn the salient semantic features of images and capture the correlation between labels, and then effectively align them is one of the key to improve the performance of image multi-label classification task. To this end, we propose a novel image multi-label classification framework which aims to align I mage S emantics with L abel C oncepts ( ISLC ). Specifically, we propose a residual encoder to learn salient object features in the images, and exploit the self-attention layer in aligned decoder to automatically capture the correlation between labels. Then, we leverage the cross-attention layers in aligned decoder to align image semantic features with label concepts, so as to make the labels predicted by model more consistent with image content. Finally, the output features of the last layer of residual encoder and aligned decoder are fused to obtain the final output feature for classification. The proposed ISLC model achieves good performance on various prevalent multi-label image datasets such as MS-COCO 2014, PASCAL VOC 2007, VG-500, and NUS-WIDE with 87.2%, 96.9%, 39.4%, and 64.2%, respectively.

Semantic representation and dependency learning for multi-label image recognition

Recently many multi-label image recognition (MLR) works have made significant progress by introducing pre-trained object detection models to generate lots of proposals or utilizing statistical label co-occurrence to enhance the correlation among different categories. However, these works have some limitations: (1) the effectiveness of the network significantly depends on pre-trained object detection models that bring expensive and unaffordable computation; (2) the network performance degrades when there exist occasional co-occurrence objects in images, especially for the rare categories. To address these problems, we propose a novel and effective semantic representation and dependency learning (SRDL) framework to learn category-specific semantic representation for each category and capture semantic dependency among all categories. Specifically, we design a category-specific attentional regions (CAR) module to generate channel/spatial-wise attention matrices to guide the model to focus on semantic-aware regions. We also design an object erasing (OE) module to implicitly learn semantic dependency among categories by erasing semantic-aware regions to regularize the network training. Extensive experiments and comparisons on two popular MLR benchmark datasets (i.e., MS-COCO and Pascal VOC 2007) demonstrate the effectiveness of the proposed framework over current state-of-the-art algorithms.

Double Attention Based on Graph Attention Network for Image Multi-Label Classification

The task of image multi-label classification is to accurately recognize multiple objects in an input image. Most of the recent works need to leverage the label co-occurrence matrix counted from training data to construct the graph structure, which are inflexible and may degrade model generalizability. In addition, these methods fail to capture the semantic correlation between the channel feature maps to further improve model performance. To address these issues, we propose DA-GAT (a D ouble A ttention framework based on the G raph A ttention ne T work) to effectively learn the correlation between labels from training data. First, we devise a new channel attention mechanism to enhance the semantic correlation between channel feature maps, so as to implicitly capture the correlation between labels. Second, we propose a new label attention mechanism to avoid the adverse impact of a manually constructed label co-occurrence matrix. It only needs to leverage the label embedding as the input of network, then automatically constructs the label relation matrix to explicitly establish the correlation between labels. Finally, we effectively fuse the output of these two attention mechanisms to further improve model performance. Extensive experiments are conducted on three public multi-label classification benchmarks. Our DA-GAT model achieves mean average precision of 87.1%, 96.6%, and 64.3% on MS-COCO 2014, PASCAL VOC 2007, and NUS-WIDE, respectively, and obviously outperforms other existing state-of-the-art methods. In addition, visual analysis experiments demonstrate that each attention mechanism can capture the correlation between labels well and significantly promote the model performance.