Label Embedding Research Articles

Leaf cultivar identification via prototype-enhanced learning

Leaf cultivar identification, as a typical task of ultra-fine-grained visual classification (UFGVC), is facing a huge challenge due to the high similarity among different varieties. In practice, an instance may be related to multiple varieties to varying degrees, especially in the UFGVC datasets. However, deep learning methods trained on one-hot labels fail to reflect patterns shared across categories and thus perform poorly on this task. As an analogy to natural language processing (NLP), by capturing the co-relation between labels, label embedding can select the most informative words and neglect irrelevant ones when predicting different labels. Based on this intuition, we propose a novel method named Prototype-enhanced Learning (PEL), which is predicated on the assumption that label embedding encoded with the inter-class relationships would force the image classification model to focus on discriminative patterns. In addition, a new prototype update module is put forward to learn inter-class relations by capturing label semantic overlap and iteratively update prototypes to generate continuously enhanced soft targets. Prototype-enhanced soft labels not only contain original one-hot label information, but also introduce rich inter-category semantic association information, thus providing more effective supervision for deep model training. Extensive experimental results on 7 public datasets show that our method can significantly improve the performance on the task of ultra-fine-grained visual classification. The code is available at https://github.com/YIYIZH/PEL.

Wasserstein task embedding for measuring task similarities

Measuring similarities between different tasks is critical in a broad spectrum of machine learning problems, including transfer, multi-task, continual, and meta-learning. Most current approaches to measuring task similarities are architecture-dependent: (1) relying on pre-trained models, or (2) training networks on tasks and using forward transfer as a proxy for task similarity. In this paper, we leverage the optimal transport theory and define a novel task embedding for supervised classification that is model-agnostic, training-free, and capable of handling (partially) disjoint label sets. In short, given a dataset with ground-truth labels, we perform a label embedding through multi-dimensional scaling and concatenate dataset samples with their corresponding label embeddings. Then, we define the distance between two datasets as the 2-Wasserstein distance between their updated samples. Lastly, we leverage the 2-Wasserstein embedding framework to embed tasks into a vector space in which the Euclidean distance between the embedded points approximates the proposed 2-Wasserstein distance between tasks. We show that the proposed embedding leads to a significantly faster comparison of tasks compared to related approaches like the Optimal Transport Dataset Distance (OTDD). Furthermore, we demonstrate the effectiveness of our embedding through various numerical experiments and show statistically significant correlations between our proposed distance and the forward and backward transfer among tasks on a wide variety of image recognition datasets.

Label-Guided relation prototype generation for Continual Relation Extraction

Continual relation extraction (CRE) aims to extract relations towards the continuous and iterative arrival of new data. To address the problem of catastrophic forgetting, some existing research endeavors have focused on exploring memory replay methods by storing typical historical learned instances or embedding all observed relations as prototypes by averaging the hidden representation of samples and replaying them in the subsequent training process. However, this prototype generation method overlooks the rich semantic information within the label namespace and are also constrained by memory size, resulting in inadequate descriptions of relation semantics by relation prototypes. To this end, we introduce an approach termed Label-Guided Relation Prototype Generation. Initially, we enhance the representations of label embeddings through a technique named label knowledge infusion. Following that, we utilize the multi-head attention mechanism to form relation prototypes, allowing them to capture diverse aspects of typical instances. The embeddings of relation labels are utilized at this stage, leveraging their contained semantics. Additionally, we propose a feature-based distillation loss function called multi-similarity distillation, to ensure the model retains prior knowledge after learning new tasks. The experimental results indicate that our method has achieved competitive performance compared to the state-of-the-art baseline models in CRE.

Node classification based on Attribute Fuse Edge Features and Label Adaptive Adjustment

Most of existing graph representation learning methods only extract information from nodes and ignore the hidden information of edges. Nodes carry weak structural information thus affecting the specificity of node embeddings.To solve these problems, this paper proposes a node classification algorithm based on Attribute Fuse Edge Features and Label Adaptive Adjustment (AFEF_LAA).Firstly, Intimate-Relationship-Attribute of node is designed based on edge embeddings.Rz-Cos rule is constructed to perform the similarity metric between nodes and their neighbors to select intimate neighbor nodes.After that Reverse-TransE is constructed to encode embedding vectors of the edges connected to intimate neighborhood nodes.Secondly, a multi-fusion method based on smoothed neighborhood information is constructed.Each node in the original graph is smoothed together with its neighbor nodes.The smoothed original graph is multi-fused with multiple twin graphs. Finally, a strategy of label adaptive adjustment is proposed to select the label embedding vectors for input to the next-generation trainer by comparing accuracy.This strategy can improve the quality of graph embeddings while effectively avoiding the overfitting problem when processing high-dimensional data.AFEF_LAA is compared with the state-of-the-art algorithms on six graph datasets.Experimental results show that AFEF_LAA can achieve higher node classification accuracy.

Description-Enhanced Label Embedding Contrastive Learning for Text Classification.

Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation ( [Formula: see text]) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel relation of relation learning network( [Formula: see text]-Net) for text classification, in which text classification and [Formula: see text] classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend [Formula: see text]-Net to a novel description-enhanced label embedding network(DELE) from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that [Formula: see text]-Net can effectively improve the classification performance and DELE can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

A novel multi-layer discriminative dictionary learning approach for image classification

Discriminative dictionary learning (DDL) has been confirmed to be effective for image classification. However, existing DDL approaches often fail to extract deep hierarchical information due to the single-layer dictionary learning framework. Moreover, they overlook the atoms-label information in the dictionary, leading to reduced feature distinctiveness and lower classification accuracy. To overcome the above problem, a novel DDL method, called the Multi-layer local constraint and label embedding dictionary learning (M-LCLEDL), is proposed. Specifically, the novel multi-layer DDL framework, which is formed by stacking the DL process one by one, is designed to learn the deep hierarchical and nonlinear features. The layer-by-layer stacking of the DL process in the multi-layer DDL framework allows for the elimination of redundant and interfering features. This step-by-step elimination process enhances the stability and robustness of the framework. Additionally, to leverage the label information carried by the labeled training samples, atoms with label embedding and locality structure are introduced. The proposed approach includes a fast iteration strategy for efficient optimization. Experimental results demonstrate that the approach is relatively insensitive to dictionary size, achieving promising performance and greater stability compared to most DDL-based image classification approaches.

A multi-task framework based on decomposition for multimodal named entity recognition

Given a text-image pair, Multimodal Named Entity Recognition (MNER) is the task of identifying and categorizing entities in the text. Most existing work performs named entity labeling directly using final token representations derived by fusing image and text representations. Although they achieve promising results, these work may fail to effectively exploit text and image modalities. This is because they neglect the difference in the role of the two modalities: text modality can detect the boundary of an entity, while image modality is introduced to disambiguate the category of the entity. Based on these findings, in this paper, we construct two auxiliary tasks based on the decomposition strategy and propose a multi-task framework for MNER. Specifically, we first decompose MNER into two auxiliary tasks: entity boundary detection task and entity category classification task. Here, the former treats only the text modality as input and outputs the boundary labels, since it can achieve satisfactory boundary results by itself. The latter uses two modalities to yield category labels where image modality is dedicated to disambiguating categories. These two auxiliary tasks allow the effective exploitation of text and image modalities and put them back into their respective roles. Then, we vectorize their results to improve entity recognition using label clues from auxiliary tasks. Finally, we fuse features from text and image modalities and label embeddings from auxiliary tasks to fulfill MNER. Experimental results on two widely used MNER datasets show that our framework can yield new SOTA performance.

Adaptive multi-label structure preserving network for cross-modal retrieval

To integrate the label structures, which describe semantic correlations among labels, into the learned common representations, many existing methods leverage the label embeddings learned according to label structures, to map the data of different modalities into a common representation space. However, these methods cannot fully discover the semantic correlation between labels. In this paper, we propose an Adaptive Multi-label Structure Preserving Network (AMLSPN) to dynamically learn the multi-label correlations and multi-label embeddings for learning common representations, which can preserve the label structures. Our method introduces a series of multi-label correlation matrices to capture the structures of multi-label nodes in the multi-label graph. Moreover, we present a novel hierarchical correlation loss to supervise the learning process of these multi-label correlation matrices. Additionally, we introduce a group GCN to enhance the training speed of our model. Extensive evaluations on three benchmark datasets demonstrate that our proposed AMLSPN outperforms the state-of-the-art methods.

GloEC: a hierarchical-aware global model for predicting enzyme function.

The annotation of enzyme function is a fundamental challenge in industrial biotechnology and pathologies. Numerous computational methods have been proposed to predict enzyme function by annotating enzyme labels with Enzyme Commission number. However, the existing methods face difficulties in modelling the hierarchical structure of enzyme label in a global view. Moreover, they haven't gone entirely to leverage the mutual interactions between different levels of enzyme label. In this paper, we formulate the hierarchy of enzyme label as a directed enzyme graph and propose a hierarchy-GCN (Graph Convolutional Network) encoder to globally model enzyme label dependency on the enzyme graph. Based on the enzyme hierarchy encoder, we develop an end-to-end hierarchical-aware global model named GloEC to predict enzyme function. GloEC learns hierarchical-aware enzyme label embeddings via the hierarchy-GCN encoder and conducts deductive fusion of label-aware enzyme features to predict enzyme labels. Meanwhile, our hierarchy-GCN encoder is designed to bidirectionally compute to investigate the enzyme label correlation information in both bottom-up and top-down manners, which has not been explored in enzyme function prediction. Comparative experiments on three benchmark datasets show that GloEC achieves better predictive performance as compared to the existing methods. The case studies also demonstrate that GloEC is capable of effectively predicting the function of isoenzyme. GloEC is available at: https://github.com/hyr0771/GloEC.

Multi-label classification of retinal diseases based on fundus images using Resnet and Transformer.

Retinal disorders are a major cause of irreversible vision loss, which can be mitigated through accurate and early diagnosis. Conventionally, fundus images are used as the gold diagnosis standard in detecting retinal diseases. In recent years, more and more researchers have employed deep learning methods for diagnosing ophthalmic diseases using fundus photography datasets. Among the studies, most of them focus on diagnosing a single disease in fundus images, making it still challenging for the diagnosis of multiple diseases. In this paper, we propose a framework that combines ResNet and Transformer for multi-label classification of retinal disease. This model employs ResNet to extract image features, utilizes Transformer to capture global information, and enhances the relationships between categories through learnable label embedding. On the publicly available Ocular Disease Intelligent Recognition (ODIR-5k) dataset, the proposed method achieves a mean average precision of 92.86%, an area under the curve (AUC) of 97.27%, and a recall of 90.62%, which outperforms other state-of-the-art approaches for the multi-label classification. The proposed method represents a significant advancement in the field of retinal disease diagnosis, offering a more accurate, efficient, and comprehensive model for the detection of multiple retinal conditions.

A fiber recognition framework based on multi-head attention mechanism

Convolutional neural networks have been extensively studied in textile fiber recognition. However, the down-sampling performed by the convolutional and pooling layers on the extracted features result in a significant loss of fine-grained fiber features. To address this issue, a multi-head attention framework for fiber recognition has been proposed. First, textile surface images are captured using optical magnifiers and smart devices such as smartphones. Second, fiber features and label embeddings are learned separately by multi-head self-attention. Finally, the label embeddings are used to query the presence of fiber types, and pool type-related features in the multi-head cross-attention module to classify fibers. The experimental results demonstrate that the proposed method performs exceptionally well on the textile surface image dataset, with a mean average precision accuracy improvement of 6% compared with the convolutional neural network-based fiber recognition method Cu-Net, and a 5% improvement compared with the fiber recognition method FiberCT combining convolutional neural networks and attention mechanisms. It is worth mentioning that the fiber images captured at 200× magnification in the collected dataset are most favorable for models to recognize fiber. The proposed method achieves a mean average precision fiber recognition accuracy of 80.2% on images at 200× magnification.

Graph relationship-driven label coded mapping and compensation for multi-label textile fiber recognition

Recently, neural network-based methods for multi-label textile fiber recognition have achieved considerable success. However, a significant limitation of most current approaches is their disregard for the valuable dependencies that exist among different fiber categories. The universal multi-label image recognition methods that consider label relationships often fall short when applied to the challenge posed by the mixture of fibers in textile images. And these relationship modeling manners have not yet been used in the field of multi-label textile fiber recognition. In this work, a graph relationship-driven method is proposed for the recognition of multi-label textile fibers. Based on the graph attention network, the proposed method introduces global relation graph, label coded mapping, and label compensation to equip the feature learning backbone with the relationship modeling ability. First, feature learning backbone extracts the semantic features from images, and global relation graph mines shallow representations of global label relationships from label embeddings. Then, label coded mapping combines these semantic features and global features to model joint relationships. The obtained joint representations constitute the label code, which is used to map the fiber class indices. Finally, label compensation further extracts the deep representations of global label relationships and utilizes them to compensate the fiber class indices to obtain final prediction scores. By employing experiments on a textile fiber dataset and a rearranged dataset, the effectiveness and superiority of the proposed method are validated. Further experiments on PASCAL VOC 2007 showcase the potential applications of the proposed method beyond textile fiber recognition.

Label-text bi-attention capsule networks model for multi-label text classification

Multi-label text classification (MLTC) is the process of establishing relationships between documents and their corresponding labels. Previous research has focused on mining textual information, treating labels as information-less vectors in classification. This ignores the semantic and dependency relationships of labels. In real-life scenarios, the neglect of label information contradicts the classification process, which presents significant challenges for MLTC tasks. Label embedding partially resolves label information loss. Efficiently exploring semantic and dependency relationships of labels and their text connections remains a new challenge. In this paper, we propose a Label-Text Bi-Attention Capsule Networks (LTBACN) model for in-depth exploration of the dependency relationships between labels and text. Specifically, we first incorporate label information into nodes through label embedding, construct a graph structure to represent the dependency relationships between labels, and use Graph Convolutional Networks (GCN) to propagate information between nodes to further mine the relationships between labels. Subsequently, we employ a label-text bi-attention mechanism to learn the feature relationships between labels and text. The label-to-text attention mechanism extracts label-relevant text representations, while the text-to-label attention mechanism extracts the most relevant label representations for the text. We then merge these two types of feature representations to obtain fused representations that incorporate label-text bi-directional information. Finally, the fused features are fed into a capsule network classifier to capture multi-level semantic information and match the corresponding labels. The experimental results demonstrate that LTBACN outperforms other methods in terms of classification effectiveness. Compared to state-of-the-art methods, LTBACN achieves a significant improvement of 0.41%–0.68% in Micro−F1 measure, 0.52%–3.26% in Macro−F1 measure, 0.32%–2.18% in P@k measure, and 0.01%–1.18% in nDCG@k measure on the AAPD and RCV1-v2 datasets.

BeNet: BERT Doc-Label Attention Network for Multi-Label Text Classification

Multi-label Text Classification (MLTC) holds significant importance and serves as a foundational aspect in Natural Language Processing (NLP), which aims at assigning multiple labels for a given document. Many real-world tasks can be viewed as MLTC, such as tag recommendation, information retrieval, etc. Nevertheless, researchers are faced with numerous challenging issues regarding the establishment of linkages between labels or the differentiation of comparable sub-labels. To address this issue, we provide a novel approach known as the BERT Doc-Label Attention Network (BeNet) in this paper, which consist of the BERTdoc layer, the label embeddings layer, the doc encoder layer, the doc-label attention layer and the prediction layer. We apply the powerful technique of BERT to pretrain documents to capture their deep semantic features and encode them via Bi-LSTM to obtain a two-directional contextual representation of uniform length. Then we create label embeddings and feed them together with encoded-pretrained-documents to the doc-label attention mechanism to obtain interactive information between documents and their corresponding labels, finally using MLP to make predictions. We carry out experiments on two real-world datasets, and the empirical results demonstrate that our proposed model outperforms all state-of-the-art MLTC benchmarks. Furthermore, we have undertaken a case study to effectively illustrate the practical implementation of our method.

Disentangled Partial Label Learning

Partial label learning (PLL) induces a multi-class classifier from training examples each associated with a set of candidate labels, among which only one is valid. The formation of real-world data typically arises from heterogeneous entanglement of series latent explanatory factors, which are considered intrinsic properties for discriminating between different patterns. Though learning disentangled representation is expected to facilitate label disambiguation for partial-label (PL) examples, few existing works were dedicated to addressing this issue. In this paper, we make the first attempt towards disentangled PLL and propose a novel approach named TERIAL, which makes predictions according to derived disentangled representation of instances and label embeddings. The TERIAL approach formulates the PL examples as an undirected bipartite graph where instances are only connected with their candidate labels, and employs a tailored neighborhood routing mechanism to yield disentangled representation of nodes in the graph. Specifically, the proposed routing mechanism progressively infers the explanatory factors that contribute to the edge between adjacent nodes and augments the representation of the central node with factor-aware embedding information propagated from specific neighbors simultaneously via iteratively analyzing the promising subspace clusters formed by the node and its neighbors. The estimated labeling confidence matrix is also introduced to accommodate unreliable links owing to the inherent ambiguity of PLL. Moreover, we theoretically prove that the neighborhood routing mechanism will converge to the point estimate that maximizes the marginal likelihood of observed PL training examples. Comprehensive experiments over various datasets demonstrate that our approach outperforms the state-of-the-art counterparts.

Aligner²: Enhancing Joint Multiple Intent Detection and Slot Filling via Adjustive and Forced Cross-Task Alignment

Multi-intent spoken language understanding (SLU) has garnered growing attention due to its ability to handle multiple intent utterances, which closely mirrors practical scenarios. Unlike traditional SLU, each intent in multi-intent SLU corresponds to its designated scope for slots, which occurs in certain fragments within the utterance. As a result, establishing precise scope alignment to mitigate noise impact emerges as a key challenge in multi-intent SLU. More seriously, they lack alignment between the predictions of the two sub-tasks due to task-independent decoding, resulting in a limitation on the overall performance. To address these challenges, we propose a novel framework termed Aligner² for multi-intent SLU, which contains an Adjustive Cross-task Aligner (ACA) and a Forced Cross-task Aligner (FCA). ACA utilizes the information conveyed by joint label embeddings to accurately align the scope of intent and corresponding slots, before the interaction of the two subtasks. FCA introduces reinforcement learning, to enforce the alignment of the task-specific hidden states after the interaction, which is explicitly guided by the prediction. Extensive experiments on two public multi-intent SLU datasets demonstrate the superiority of our Aligner² over state-of-the-art methods. More encouragingly, the proposed method Aligner² can be easily integrated into existing multi-intent SLU frameworks, to further boost performance.

Adaptive attention-aware fusion for human-in-the-loop behavioral health detection

Identifying behavioral health is paramount for law enforcement officers to provide appropriate follow-up community care. In the current practice, law enforcement offices manually identify these behavioral health cases to allow the designation of the relevant follow-up resources. In this work, we develop a tool to automatically detect behavioral health cases from police public narrative reports by identifying behavioral health indicator signals. We propose a novel adaptive attention-aware fusion model for detecting behavioral health signals in sensitive police reports. Our model leverages contextual and semantic information from the reports and relevant behavioral health cues as keywords from a pre-trained attention-weighted keyword-based model. Our model also employs label self-attention mechanisms to correlate label embeddings with the report and keyword representations. Furthermore, we propose a novel clustering-based uncertainty-enabled informative sampling query strategy to integrate humans-in-the-loop in the active learning framework to reduce required annotation from experts. This querying strategy selects the most informative and diverse samples for expert annotation. Our experimental results showed that the proposed model outperforms state-of-the-art classifiers on a dataset of 300 manually annotated ground truth police reports, achieving an accuracy of 87.58% and an F1-score of 85.67%. Applying our querying strategy to our proposed model increased the detection of behavioral health, achieving an accuracy of 92% and an F1-score of 91.1%. Also, our proposed model achieves an accuracy score of 93.75% and an F1-score of 93.61% on unseen samples. Lastly, our proposed model demonstrates its interpretability by extracting the keywords associated with each behavioral health category.

Recursive label attention network for nested named entity recognition

Nested named entity recognition (NER) has been mainly studied by recognizing inner entities first, then outer entities, or vice versa, depending on the nesting level. However, the previous models on nested NER do not deal with the data sparseness problems of deep nested entities (i.e., insufficiency of training data for deeper nested entities) and the error propagation problems between nested entities (i.e., accumulation of errors caused by inner entities or outer entities). To alleviate the data sparseness problems, we propose a recursive label attention network. In contrast to previous models, the proposed model explicitly reflects the nested level and effectively re-uses lower level label information through level-reflected label embeddings. To counteract error propagation in the recursive architecture, we propose an inner entity pretraining strategy in which the proposed model is sequentially trained from lower to higher levels. In experiments, we use task-specific metrics for nested NER, unlike the evaluation metrics used in previous studies. The experimental results show that the proposed model is particularly effective in cases where entities of the same type were nested. In addition, it showed high F1-scores even when named entities are deeply nested. This fact reveals that our proposed model is effective for nest NER.

Redefining Islam (IST) Radicalism and Extremism: The Current Context of Indonesia

Radicalism is regarded as an internal threat in many countries, including Indonesia. Rulers will occasionally utilize radical label embedding to silence political opponents, including in the run-up to general elections. Some academicians believe that the terms radicalism and extremism are used interchangeably. Even though they still have disagreements, they regard the two as distinct. The same term, but understood differently by different parties, particularly those opposed to each other, can generate and exacerbate conflicts. This study used a qualitative technique to clarify radicalism and distinguish it from extremism, particularly from the perspective of people who (the Indonesians) proclaim themselves radicals. Semi-structured in-depth interviews were used to collect research data, which was then analyzed using a narrative technique. The findings of the study provide a thorough picture of radicalism. The informants did not object to being labelled as radical, but only in the sense that they interpreted radicalism which differed from the concept of extremism. Alternative responses to radicalism are discussed.

Innovative Label Embedding for Food Safety Comment Classification: Fusion of Self-Semantic and Self-Knowledge Features

Food safety comment classification represents a specialized task within the realm of text classification. The objective is to efficiently identify a large volume of food safety comments, aiding relevant authorities in timely food analysis and safety alerts. Traditional methods typically employ one-hot encoding for label processing. However, in real-world situations, classified labels often convey valuable semantic information and guidance. This paper introduces an innovative approach to enhance the classification performance of food safety comments by embedding label information. Initially, we extracted generic sentiment pivot words from various classification labels as label description information. Subsequently, we employ a joint embedding approach to integrate this label description information into the text. This process will pool the expressions of the pivot word into the corresponding sentiment labels in the known domains after averaging to get the embedded expression. This aims to acquire highly detailed self-semantic feature vectors and self-knowledge feature vectors that are integrated with labeled descriptive information. Then, feed the semantic representation of comments and the word-embedded representation of labeled description information into a time-step-based multilayer Bi-LSTM and a step-based multilayer CNN, respectively. Ultimately, we concatenate these two feature vectors to facilitate matching, thereby fusing the self-semantic and self-knowledge features of labeled description information to train a classification model for food safety comments. Experimental results on the food safety comment dataset showcase a noteworthy improvement of 1.74% and 1.27% in Macro_Precision and Macro_F1 metrics, respectively, compared to BERT, BERT-RNN, and BERT-CNN. Through extensive ablation experiments and additional studies, our method effectively embeds labeling information, demonstrating a clear advantage over traditional methods in the task of classifying food safety comments. Doi: 10.28991/HIJ-2024-05-01-013 Full Text: PDF