Label Dependencies Research Articles

InterLabelGO+: unraveling label correlations in protein function prediction.

Accurate protein function prediction is crucial for understanding biological processes and advancing biomedical research. However, the rapid growth of protein sequences far outpaces the experimental characterization of their functions, necessitating the development of automated computational methods. We present InterLabelGO+, a hybrid approach that integrates a deep learning-based method with an alignment-based method for improved protein function prediction. InterLabelGO+ incorporates a novel loss function that addresses label dependency and imbalance and further enhances performance through dynamic weighting of the alignment-based component. A preliminary version of InterLabelGO+ achieved a strong performance in the CAFA5 challenge, ranking sixth out of 1625 participating teams. Comprehensive evaluations on large-scale protein function prediction tasks demonstrate InterLabelGO+'s ability to accurately predict Gene Ontology terms across various functional categories and evaluation metrics. The source code and datasets for InterLabelGO+ are freely available on GitHub at https://github.com/QuanEvans/InterLabelGO. A web-server is available at https://seq2fun.dcmb.med.umich.edu/InterLabelGO/. The software is implemented in Python and PyTorch, and is supported on Linux and macOS.

A novel interpretable semi-supervised graph learning model for intelligent fault diagnosis of hydraulic pumps

Although deep learning has gained popularity in the field of fault diagnosis, its limitations are also equally apparent, including: (1) heavy reliance on a substantial volume of labeled samples; (2) a lack of interpretability. To confront these issues, this article proposes a novel interpretable semi-supervised graph learning model for intelligent fault diagnosis of hydraulic pumps. A comparison between the raw data and the model's hidden layer representations is conducted to minimize feature loss. The model commences by preliminarily learning fault information that is intermixed with noise, leveraging a substantial corpus of unlabeled data. In response to the intricacy of downstream tasks, an interpretable feature reconstruction module is introduced. This module employs a nonlinear surrogate model to fit and elucidate the learned features, embedding the explanation scores into the features to reconstruct the samples, a process utilized for model fine-tuning. The feature reconstruction module capitalizes on the explanatory power of the surrogate model, guiding the model to concentrate more on features with significant impact. This method not only provides interpretability during model training but also expedites the convergence speed of the model. Finally, two hydraulic pump experiment cases are used to verify the effectiveness of the model, and the results show that our method has obvious advantages in reducing label dependence and increasing model reliability for decision making.

Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations.

Graph contrastive learning (GCL) is a promising direction toward alleviating the label dependence, poor generalization and weak robustness of graph neural networks, learning representations with invariance, and discriminability by solving pretasks. The pretasks are mainly built on mutual information estimation, which requires data augmentation to construct positive samples with similar semantics to learn invariant signals and negative samples with dissimilar semantics to empower representation discriminability. However, an appropriate data augmentation configuration depends heavily on lots of empirical trials such as choosing the compositions of data augmentation techniques and the corresponding hyperparameter settings. We propose an augmentation-free GCL method, invariant-discriminative GCL (iGCL), that does not intrinsically require negative samples. iGCL designs the invariant-discriminative loss (ID loss) to learn invariant and discriminative representations. On the one hand, ID loss learns invariant signals by directly minimizing the mean square error (MSE) between the target samples and positive samples in the representation space. On the other hand, ID loss ensures that the representations are discriminative by an orthonormal constraint forcing the different dimensions of representations to be independent of each other. This prevents representations from collapsing to a point or subspace. Our theoretical analysis explains the effectiveness of ID loss from the perspectives of the redundancy reduction criterion, canonical correlation analysis (CCA), and information bottleneck (IB) principle. The experimental results demonstrate that iGCL outperforms all baselines on five node classification benchmark datasets. iGCL also shows superior performance for different label ratios and is capable of resisting graph attacks, which indicates that iGCL has excellent generalization and robustness. The source code is available at https://github.com/lehaifeng/ T-GCN/tree/master/iGCL.

ADVANCED TECHNIQUES IN MULTI-LABEL TEXT CLASSIFICATION: INTEGRATION OF BETA ANT COLONY AND DEEP LEARNING APPROACHES

The rapid growth of unstructured textual data in various domains has necessitated the development of sophisticated techniques for multi-label text classification. Traditional methods often struggle with handling the complexity and interdependence of multiple labels, leading to suboptimal performance. This paper presents an advanced approach that integrates the Beta Ant Colony Optimization (BACO) algorithm with deep learning techniques for multi-label text classification. The BACO algorithm effectively explores the feature space and selects relevant features by leveraging pheromone trails, while the deep learning model captures intricate patterns and relationships within the textual data. The integration of these two methodologies aims to enhance the efficiency and accuracy of multi-label classification tasks, particularly in domains where label dependency is prominent. Empirical evaluations on benchmark datasets demonstrate that the proposed hybrid approach outperforms existing state-of-the-art techniques in terms of precision, recall, F1-score, and computational efficiency. The findings suggest that combining heuristic optimization algorithms with deep learning can significantly improve multi-label text classification performance, providing a robust solution for real-world applications.

GloEC: a hierarchical-aware global model for predicting enzyme function.

The annotation of enzyme function is a fundamental challenge in industrial biotechnology and pathologies. Numerous computational methods have been proposed to predict enzyme function by annotating enzyme labels with Enzyme Commission number. However, the existing methods face difficulties in modelling the hierarchical structure of enzyme label in a global view. Moreover, they haven't gone entirely to leverage the mutual interactions between different levels of enzyme label. In this paper, we formulate the hierarchy of enzyme label as a directed enzyme graph and propose a hierarchy-GCN (Graph Convolutional Network) encoder to globally model enzyme label dependency on the enzyme graph. Based on the enzyme hierarchy encoder, we develop an end-to-end hierarchical-aware global model named GloEC to predict enzyme function. GloEC learns hierarchical-aware enzyme label embeddings via the hierarchy-GCN encoder and conducts deductive fusion of label-aware enzyme features to predict enzyme labels. Meanwhile, our hierarchy-GCN encoder is designed to bidirectionally compute to investigate the enzyme label correlation information in both bottom-up and top-down manners, which has not been explored in enzyme function prediction. Comparative experiments on three benchmark datasets show that GloEC achieves better predictive performance as compared to the existing methods. The case studies also demonstrate that GloEC is capable of effectively predicting the function of isoenzyme. GloEC is available at: https://github.com/hyr0771/GloEC.

Few-shot relational triple extraction with hierarchical prototype optimization

Relational Triple Extraction (RTE) aims to extract relations and entities from unstructured text. Current RTE models using supervised learning require a large amount of labeled data, which presents a challenge for real-world applications. Therefore, the research work on Few-Shot Relational Triple Extraction (FS-RTE) has been proposed. However, the existing work cannot effectively construct accurate prototypes from a small number of samples, and it is difficult to model the dependencies between entities and relations, resulting in poor performance in relational triple extraction. In this paper, we propose a Hierarchical Prototype Optimized FS-RTE method (HPO). In particular, to mitigate prototype bias built on a small number of samples, HPO uses prompt learning to merge the information of relational labels into the text. Then, the entity-level prototypes are constructed using a span encoder to avoid label dependency between entity tokens. Finally, the hierarchical contrastive learning (HCL) method is introduced to improve the metric space between the prototypes of entities and relations, respectively. Experiments conducted on two public datasets show that HPO can significantly outperform previous state-of-the-art methods.

Hierarchy-Aware and Label Balanced Model for Hierarchical Text Classification

Hierarchical text classification, where labels can be modeled as a hierarchical structure, is a special multi-label text classification sub-task. Current methods mainly improve model performance by modeling label dependencies. The model HGCLR with excellent performance obtains hierarchy-aware text representations through contrastive learning, but the degree of hierarchical awareness is insufficient. To address this problem, a new multi-label negative supervision method is proposed to drive text representations of samples with more different labels farther away. In addition, to tackle the problem of label imbalance in hierarchical text classification, asymmetric loss is employed to compute the classification loss, so that the model focuses on learning from difficult samples and the contributions of positive and negative labels to the loss function tend to be balanced. In summary, based on HGCLR, by adding multi-label negative supervision and replacing the classification loss with asymmetric loss, we propose a hierarchy-aware and label balanced model (HALB) for hierarchical text classification. Experimental results demonstrate that HALB outperforms several classical models for hierarchical text classification. HALB achieves the best results on both Micro-F1 and Macro-F1 and obtains average improvements of 0.51% and 1.28% respectively on four datasets compared with the strongest baseline model HGCLR.

Span-based few-shot event detection via aligning external knowledge

Few-shot Event Detection (FSED) aims to identify novel event types in new domains with very limited annotated data. Previous PN-based (Prototypical Network) joint methods suffer from insufficient learning of token-wise label dependency and inaccurate prototypes. To solve these problems, we propose a span-based FSED model, called SpanFSED, which decomposes FSED into two subprocesses, including span extractor and event classifier. In span extraction, we convert sequential labels into a global boundary matrix that enables the span extractor to acquire precise boundary information irrespective of label dependency. In event classification, we align event types with an outside knowledge base like FrameNet and construct an enhanced support set, which injects more trigger information into the prototypical network of event prototypes. The superior performance of SpanFSED is demonstrated through extensive experiments on four event detection datasets, i.e., ACE2005, ERE, MAVEN and FewEvent. Access to our code and data is facilitated through the following link: .

A false emotion opinion target extraction model with two stage BERT and background information fusion

As social media has gradually become an indispensable part of people’s life, more and more users begin to express their opinions on social media. These opinions contain rich emotional information, as well as many abnormal false emotion. And the opinion target rich in false emotion will have great potential value in the field of movie box office prediction, public opinion guidance and so on. Therefore, we propose a false emotion opinion target extraction task and two stage BERT+CRF model based on the concept of false emotion. Firstly, two stage BERT network includes C-BERT and W-BERT. We use C-BERT to learn the users comment feature, and W-BERT to learn the background content features of microblog. Then, we design five feature interaction fusion methods to fuse the features learned by C-BERT and W-BERT. At last, we transform the false emotion opinion target extraction task into a sequence labeling task, and use the CRF method to learn the label dependency of the context and output the optimal sequence among all candidate sequences. Due to the lack of false emotion opinion target datasets, we collate a false emotion opinion target dataset Weibo23. Comparing with the advanced models used in related research, including some sequence labeling models and named entity recognition models, two stage BERT+CRF model achieves an F1 score of 0.8834, which is about 10%–20% higher.

CARAT: Contrastive Feature Reconstruction and Aggregation for Multi-Modal Multi-Label Emotion Recognition

Multi-modal multi-label emotion recognition (MMER) aims to identify relevant emotions from multiple modalities. The challenge of MMER is how to effectively capture discriminative features for multiple labels from heterogeneous data. Recent studies are mainly devoted to exploring various fusion strategies to integrate multi-modal information into a unified representation for all labels. However, such a learning scheme not only overlooks the specificity of each modality but also fails to capture individual discriminative features for different labels. Moreover, dependencies of labels and modalities cannot be effectively modeled. To address these issues, this paper presents ContrAstive feature Reconstruction and AggregaTion (CARAT) for the MMER task. Specifically, we devise a reconstruction-based fusion mechanism to better model fine-grained modality-to-label dependencies by contrastively learning modal-separated and label-specific features. To further exploit the modality complementarity, we introduce a shuffle-based aggregation strategy to enrich co-occurrence collaboration among labels. Experiments on two benchmark datasets CMU-MOSEI and M3ED demonstrate the effectiveness of CARAT over state-of-the-art methods. Code is available at https://github.com/chengzju/CARAT.

Balancing efficiency vs. effectiveness and providing missing label robustness in multi-label stream classification

Available works addressing multi-label classification in a data stream environment focus on proposing accurate prediction models; however, they struggle to balance effectiveness and efficiency. In this work, we present a neural network-based approach that tackles this issue and is suitable for high-dimensional multi-label classification. The proposed model uses a selective concept drift adaptation mechanism that makes it well-suited for a non-stationary environment. We adapt the model to an environment with missing labels using a simple imputation strategy and demonstrate that it outperforms a vast majority of the supervised models. To achieve these, a weighted binary relevance-based approach named ML-BELS is introduced. To capture label dependencies, instead of a chain of stacked classifiers, the proposed model employs independent weighted ensembles as binary classifiers, with the weights generated by the predictions of a BELS classifier. We present an extensive assessment of the proposed model using 11 prominent baselines, five synthetic, and 13 real-world datasets, all with different characteristics. The results demonstrate that the proposed approach ML-BELS is successful in balancing effectiveness and efficiency, and is robust to missing labels and concept drift.

Retinal OCT Image Classification Based on CNN-RNN Unified Neural Networks

<p>Computer-aided diagnosis of retinopathy is a hot research topic in the field of medical image classification, where optical coherence tomography (OCT) is an important basis for the diagnosis of ophthalmic diseases. Traditional approaches to multi-label image classification learn independent classifiers for each category and employ ranking or thresholding on the classification results. These techniques, although working well, fail to explicitly exploit the label dependencies in an image. In this paper, two publicly available retinal OCT image datasets are integrated and screened. Then, an end-to-end deep learning algorithmic framework based on CNN-RNN Unified Neural Networks was proposed to automatically and reliably classify six categories of retinal OCT images. Numerical results suggest that the proposed algorithm works well in terms of accuracy, precision, sensitivity and specificity, approaching or even partially surpassing the performance of clinical experts. It is valuable in promoting computer-aided diagnosis towards practical clinical applications and improving the efficiency of clinical diagnosis of retinal diseases.</p> <p> </p>

Robust multi-label feature learning-based dual space

AbstractMulti-label learning handles instances associated with multiple class labels. The original label space is a logical matrix with entries from the Boolean domain $$\in \left\{ 0,1 \right\} $$ ∈ 0 , 1 . Logical labels cannot show the relative importance of each semantic label to the instances. Most existing methods map the input features to the label space using linear projections considering the label dependencies using a logical label matrix. However, the discriminative features are learned using one-way projection from the feature representation of an instance into a logical label space. There is no manifold in the learning space of logical labels, which limits the potential of learned models. We propose a novel method in multi-label learning to learn the projection matrix from the feature space to the semantic label space and project it back to the original feature space using encoder–decoder deep learning architecture. The key intuition which guides our method is that the discriminative features are identified due to mapping the features back and forth using two linear projections. To the best of our knowledge, this is one of the first attempts to study the ability to reconstruct the original features from the label manifold in multi-label learning. We show that the learned projection matrix identifies a subset of discriminative features across multiple semantic labels. Extensive experiments on real-world datasets show the superiority of the proposed method.

Multi-label feature selection with global and local label correlation

In various application domains, high-dimensional multi-label data has become more prevalent, presenting two significant challenges: instances with high-dimensional features and a large number of labels. In the context of multi-label feature selection, the objective is to choose a subset of features from a given set that is highly pertinent for predicting multiple labels or categories associated with each instance. However, certain characteristics of multi-label classification, such as label dependencies and imbalanced label distribution, have often been overlooked although they hold valuable insights for designing effective multi-label feature selection algorithms. In this paper, we propose a feature selection model which exploits explicit global and local label correlations to select discriminative features across multiple labels. In addition, by representing the feature matrix and label matrix in a shared latent space, the model aims to capture the underlying correlations between features and labels. The shared representation can reveal common patterns or relationships that exist across multiple labels and features. An objective function involving L2,1-norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. The proposed method was evaluated on 14 real-world multi-label datasets using six evaluation metrics, through comprehensive experiments. The results indicate its effectiveness, surpassing that of several representative methods.

Learning group-wise spatial attention and label dependencies for multi-task thoracic disease classification

This paper considers the multi-label thoracic abnormality classification with chest X-ray images. In clinical settings, Chest X-ray imaging is a general diagnostic tool applied to visualize numerous thoracic pathological changes. While deep learning techniques have been extensively tested in this field, certain challenges persist. The data in existing thoracic abnormality datasets is insufficient, and some diseases are extremely imbalanced. Meanwhile, the dependencies between different labels are often ignored. To tackle these issues head-on, this paper introduces two crucial modules: the group-wise spatial attention (GWSA) module and the label co-occurrence dependency (LCD) module, integrated with DenseNet121 backbone. Specifically, GWSA enhances the spatial features within distinct groups while keeping the between-group feature discrimination. LCD models the correlations between different thoracic abnormalities to refine the predicted probabilities. In conjunction with the DenseNet121 backbone, these two modules reach an average AUC score of 0.818 on Chest X-ray14 dataset, achieving state-of-the-art. Source code is available at https://github.com/YujiaKCL/Chest-Xray14-GWSA-LCD.

Multi-Label Feature Selection Based on Improved Ant Colony Optimization Algorithm with Dynamic Redundancy and Label Dependence

Multi-Label Feature Selection Based on Improved Ant Colony Optimization Algorithm with Dynamic Redundancy and Label Dependence

ProFun-SOM: Protein Function Prediction for Specific Ontology Based on Multiple Sequence Alignment Reconstruction.

Protein function prediction is crucial for understanding species evolution, including viral mutations. Gene ontology (GO) is a standardized representation framework for describing protein functions with annotated terms. Each ontology is a specific functional category containing multiple child ontologies, and the relationships of parent and child ontologies create a directed acyclic graph. Protein functions are categorized using GO, which divides them into three main groups: cellular component ontology, molecular function ontology, and biological process ontology. Therefore, the GO annotation of protein is a hierarchical multilabel classification problem. This hierarchical relationship introduces complexities such as mixed ontology problem, leading to performance bottlenecks in existing computational methods due to label dependency and data sparsity. To overcome bottleneck issues brought by mixed ontology problem, we propose ProFun-SOM, an innovative multilabel classifier that utilizes multiple sequence alignments (MSAs) to accurately annotate gene ontologies. ProFun-SOM enhances the initial MSAs through a reconstruction process and integrates them into a deep learning architecture. It then predicts annotations within the cellular component, molecular function, biological process, and mixed ontologies. Our evaluation results on three datasets (CAFA3, SwissProt, and NetGO2) demonstrate that ProFun-SOM surpasses state-of-the-art methods. This study confirmed that utilizing MSAs of proteins can effectively overcome the two main bottlenecks issues, label dependency and data sparsity, thereby alleviating the root problem, mixed ontology. A freely accessible web server is available at http://bliulab.net/ ProFun-SOM/.

Graph Reasoning With Supervised Contrastive Learning for Legal Judgment Prediction.

Given the fact descriptions of legal cases, the legal judgment prediction (LJP) problem aims to determine three judgment tasks of law articles, charges, and the term of penalty. Most existing studies have considered task dependencies while neglecting the prior dependencies of labels among different tasks. Therefore, how to make better use of the information on the relation dependencies among tasks and labels becomes a crucial issue. To this end, we transform the text classification problem into a node classification framework based on graph reasoning and supervised contrastive learning (SCL) techniques, named GraSCL. Specifically, we first design a graph reasoning network to model the potential dependency structures and facilitate relational learning under various graph topologies. Then, we introduce the SCL method for the LJP task to further leverage the label relation on the graph. To accommodate the node classification settings, we extend the traditional SCL method to novel variants for SCL at the node level, which allows the GraSCL framework to be trained efficiently even with small batches. Furthermore, to recognize the importance of hard negative samples in contrastive learning, we introduce a simple yet effective technique called online hard negative mining (OHNM) to enhance our SCL approach. This technique complements our SCL method and enables us to control the number and complexity of negative samples, leading to further improvements in the model's performance. Finally, extensive experiments are conducted on two well-known benchmarks, demonstrating the effectiveness and rationality of our proposed SCL approach as compared to the state-of-the-art competitors.

A Robust Semi-Supervised Broad Learning System Guided by Ensemble-Based Self-Training.

Broad learning system (BLS) with semi-supervised learning relieves label dependence and expands application. Despite some efforts and progress, the semi-supervised BLS still needs improvement, especially in handling imbalanced data or concept drift scenarios for self-training-based methods. To this extent, this article proposes a robust semi-supervised BLS guided by ensemble-based self-training (ESTSS-BLS). Distinctive to self-training that assigns the pseudo-label via a single classifier and confidence, the advocated ensemble-based self-training determines the pseudo-label according to the turnout of multiple BLSs. In addition, label purity is proposed to ensure the correctness and credibility of the auxiliary training data, which is a comprehensive evaluation of the voting. During iterative learning, a small portion of labeled data first trains multiple BLSs in parallel. Then, the system recursively updates its data, structure, and meta-parameters using label purity and a data-driven dynamic nodes mechanism that dynamically guides the network's structural adjustments to solve the concept drift problem caused by a large amount of auxiliary training data. The experimental results demonstrate that ESTSS-BLS exhibits exceptional performance compared to existing methods, with the lowest-time consumption and the highest accuracy, precision, recall, F1 score, and AUC. Exhilaratingly, it achieves an accuracy of 87.84% with only 0.1% labeled data on MNIST, and with just 2% labeled data, it matches the performance of supervised learning using all training data on NORB. In addition, ESTSS-BLS also performs stably on medical or biological data, verifying its high adaptability.

Graph Convolutional Multi-Label Hashing for Cross-Modal Retrieval.

Cross-modal hashing encodes different modalities of multimodal data into low-dimensional Hamming space for fast cross-modal retrieval. In multi-label cross-modal retrieval, multimodal data are often annotated with multiple labels, and some labels, e.g.", ocean" and "cloud", often co-occur. However, existing cross-modal hashing methods overlook label dependency that is crucial for improving performance. To fulfill this gap, this article proposes graph convolutional multi-label hashing (GCMLH) for effective multi-label cross-modal retrieval. Specifically, GCMLH first generates word embedding of each label and develops label encoder to learn highly correlated label embedding via graph convolutional network (GCN). In addition, GCMLH develops feature encoder for each modality, and feature fusion module to generate highly semantic feature via GCN. GCMLH uses teacher-student learning scheme to transfer knowledge from the teacher modules, i.e., label encoder and feature fusion module, to the student module, i.e., feature encoder, such that learned hash code can well exploit multi-label dependency and multimodal semantic structure. Extensive empirical results on several benchmarks demonstrate the superiority of the proposed method over existing state-of-the-arts.