Label Embedding Research Articles

Multi-label classification of legal text based on label embedding and capsule network

Multi-label classification of legal text based on label embedding and capsule network

A Label Dependence-Aware Sequence Generation Model for Multi-Level Implicit Discourse Relation Recognition

Implicit discourse relation recognition (IDRR) is a challenging but crucial task in discourse analysis. Most existing methods train multiple models to predict multi-level labels independently, while ignoring the dependence between hierarchically structured labels. In this paper, we consider multi-level IDRR as a conditional label sequence generation task and propose a Label Dependence-aware Sequence Generation Model (LDSGM) for it. Specifically, we first design a label attentive encoder to learn the global representation of an input instance and its level-specific contexts, where the label dependence is integrated to obtain better label embeddings. Then, we employ a label sequence decoder to output the predicted labels in a top-down manner, where the predicted higher-level labels are directly used to guide the label prediction at the current level. We further develop a mutual learning enhanced training method to exploit the label dependence in a bottom-up direction, which is captured by an auxiliary decoder introduced during training. Experimental results on the PDTB dataset show that our model achieves the state-of-the-art performance on multi-level IDRR. We release our code at https://github.com/nlpersECJTU/LDSGM.

Inferring Prototypes for Multi-Label Few-Shot Image Classification with Word Vector Guided Attention

Multi-label few-shot image classification (ML-FSIC) is the task of assigning descriptive labels to previously unseen images, based on a small number of training examples. A key feature of the multi-label setting is that images often have multiple labels, which typically refer to different regions of the image. When estimating prototypes, in a metric-based setting, it is thus important to determine which regions are relevant for which labels, but the limited amount of training data makes this highly challenging. As a solution, in this paper we propose to use word embeddings as a form of prior knowledge about the meaning of the labels. In particular, visual prototypes are obtained by aggregating the local feature maps of the support images, using an attention mechanism that relies on the label embeddings. As an important advantage, our model can infer prototypes for unseen labels without the need for fine-tuning any model parameters, which demonstrates its strong generalization abilities. Experiments on COCO and PASCAL VOC furthermore show that our model substantially improves the current state-of-the-art.

Cross-Modal Feature Representation Learning and Label Graph Mining in a Residual Multi-Attentional CNN-LSTM Network for Multi-Label Aerial Scene Classification

The results of aerial scene classification can provide valuable information for urban planning and land monitoring. In this specific field, there are always a number of object-level semantic classes in big remote-sensing pictures. Complex label-space makes it hard to detect all the targets and perceive corresponding semantics in the typical scene, thereby weakening the sensing ability. Even worse, the preparation of a labeled dataset for the training of deep networks is more difficult due to multiple labels. In order to mine object-level visual features and make good use of label dependency, we propose a novel framework in this article, namely a Cross-Modal Representation Learning and Label Graph Mining-based Residual Multi-Attentional CNN-LSTM framework (CM-GM framework). In this framework, a residual multi-attentional convolutional neural network is developed to extract object-level image features. Moreover, semantic labels are embedded by language model and then form a label graph which can be further mapped by advanced graph convolutional networks (GCN). With these cross-modal feature representations (image, graph and text), object-level visual features will be enhanced and aligned to GCN-based label embeddings. After that, aligned visual signals are fed into a bi-LSTM subnetwork according to the built label graph. The CM-GM framework is able to map both visual features and graph-based label representations into a correlated space appropriately, using label dependency efficiently, thus improving the LSTM predictor’s ability. Experimental results show that the proposed CM-GM framework is able to achieve higher accuracy on many multi-label benchmark datasets in remote sensing field.

An adaptive convolution with label embedding for text classification

An adaptive convolution with label embedding for text classification

A multi-scale semantic attention representation for multi-label image recognition with graph networks

Multi-label image recognition is a basic and challenging task in computer vision and multimedia fields. Graph Convolutional Networks (GCNs) are often used to learn the multi-label semantic features and multi-label dependency. Although the label semantic features in GCNs can learn the global image visual representation well, they are rarely used on the local image regions. Therefore, we try to use GCNs to learn global and local features at the same time, and make a balance between them. In this paper, we give a multi-scale semantic attention model MS-SGA-GCN including three main modules (i.e., MS, SGA and GCN) for multi-label image recognition. The Multi-Scale module (MS) utilizes feature maps of different sizes to obtain global features and have strong generalization capabilities. Semantic Guide Attention module (SGA) applies the label embeddings learned by GCNs to guide the generation of the cross-modality class-specific attention maps, which can discover the locations of semantically related regions for each label. Experiments show that our model on two datasets MS-COCO and PASCAL VOC2007 separately achieves the classification accuracy by 83.4% and 94.2%, which has a competitive advantage over other mainstream models.

Effective Transfer Learning with Label-Based Discriminative Feature Learning.

The performance of natural language processing with a transfer learning methodology has improved by applying pre-training language models to downstream tasks with a large number of general data. However, because the data used in pre-training are irrelevant to the downstream tasks, a problem occurs in that it learns general features rather than those features specific to the downstream tasks. In this paper, a novel learning method is proposed for embedding pre-trained models to learn specific features of such tasks. The proposed method learns the label features of downstream tasks through contrast learning using label embedding and sampled data pairs. To demonstrate the performance of the proposed method, we conducted experiments on sentence classification datasets and evaluated whether the features of the downstream tasks have been learned through a PCA and a clustering of the embeddings.

Volumetric segmentation of white matter tracts with label embedding

Convolutional neural networks have achieved state-of-the-art performance for white matter (WM) tract segmentation based on diffusion magnetic resonance imaging (dMRI). However, the segmentation can still be difficult for challenging WM tracts with thin bodies or complicated shapes; the segmentation is even more problematic in challenging scenarios with reduced data quality or domain shift between training and test data, which can be easily encountered in clinical settings. In this work, we seek to improve the segmentation of WM tracts, especially for challenging WM tracts in challenging scenarios. In particular, our method is based on volumetric WM tract segmentation, where voxels are directly labeled without performing tractography. To improve the segmentation, we exploit the characteristics of WM tracts that different tracts can cross or overlap and revise the network design accordingly. Specifically, because multiple tracts can co-exist in a voxel, we hypothesize that the different tract labels can be correlated. The tract labels at a single voxel are concatenated as a label vector, the length of which is the number of tract labels. Due to the tract correlation, this label vector can be projected into a lower-dimensional space—referred to as the embedded space—for each voxel, which allows the segmentation network to solve a simpler problem. By predicting the coordinate in the embedded space for the tracts at each voxel and subsequently mapping the coordinate to the label vector with a reconstruction module, the segmentation result can be achieved. To facilitate the learning of the embedded space, an auxiliary label reconstruction loss is integrated with the segmentation accuracy loss during network training, and network training and inference are end-to-end. Our method was validated on two dMRI datasets under various settings. The results show that the proposed method improves the accuracy of WM tract segmentation, and the improvement is more prominent for challenging tracts in challenging scenarios.

Label embedding semantic-guided hashing

Hashing technologies have been widely used for information retrieval tasks due to their efficient retrieval and storage capabilities. Generally, most of the current supervised learning only utilizes labels to construct a binary similarity matrix of instance pairs and ignores the rich semantic information contained in the labels. Indeed, the reason why supervised hashing is better than unsupervised hashing is that the labels itself has strong discriminative information. Therefore, how to effectively explore the label information is one of the ways to improve the performance of retrieval tasks. In addition, existing hashing methods have the problems of high time consumption and weak scalability when facing large-scale data. To remedy these problems, in this paper, we present a flexible two-step label embedding hashing method named Label Embedding Semantic-Guided Hashing (LESGH). In the first step, LESGH leverages an asymmetric discrete learning framework to learn discriminative compact hash codes only from label information, and adds the constraints of bit-balance and bit-decorrelation to boost the quality of the hash code generation. In the second step, LESGH learns the hash projection function through the generated hash codes in the first step. Moreover, an effective and fast iterative discrete optimization algorithm is presented to solve the discrete problem instead of using the relaxation-based scheme. In doing so, we can not only simplify the optimization process, but also easily scale to large-scale data. We conduct several experiments on three public datasets, i.e., WIKI, MIRFlickr and NUS-WIDE, demonstrate that LESGH can improve the retrieval performance over the compared state-of-the-art baselines.

TN-ZSTAD: Transferable Network for Zero-Shot Temporal Activity Detection.

An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos. Currently, the most effective methods of temporal activity detection are based on deep learning, and they typically perform very well with large scale annotated videos for training. However, these methods are limited in real applications due to the unavailable videos about certain activity classes and the time-consuming data annotation. To solve this challenging problem, we propose a novel task setting called zero-shot temporal activity detection (ZSTAD), where activities that have never been seen in training still need to be detected. We design an end-to-end deep transferable network TN-ZSTAD as the architecture for this solution. On the one hand, this network utilizes an activity graph transformer to predict a set of activity instances that appear in the video, rather than produces many activity proposals in advance. On the other hand, this network captures the common semantics of seen and unseen activities from their corresponding label embeddings, and it is optimized with an innovative loss function that considers the classification property on seen activities and the transfer property on unseen activities together. Experiments on the THUMOS'14, Charades, and ActivityNet datasets show promising performance in terms of detecting unseen activities.

Ingredient-Guided Region Discovery and Relationship Modeling for Food Category-Ingredient Prediction.

Recognizing the category and its ingredient composition from food images facilitates automatic nutrition estimation, which is crucial to various health relevant applications, such as nutrition intake management and healthy diet recommendation. Since food is composed of ingredients, discovering ingredient-relevant visual regions can help identify its corresponding category and ingredients. Furthermore, various ingredient relationships like co-occurrence and exclusion are also critical for this task. For that, we propose an ingredient-oriented multi-task food category-ingredient joint learning framework for simultaneous food recognition and ingredient prediction. This framework mainly involves learning an ingredient dictionary for ingredient-relevant visual region discovery and building an ingredient-based semantic-visual graph for ingredient relationship modeling. To obtain ingredient-relevant visual regions, we build an ingredient dictionary to capture multiple ingredient regions and obtain the corresponding assignment map, and then pool the region features belonging to the same ingredient to identify the ingredients more accurately and meanwhile improve the classification performance. For ingredient-relationship modeling, we utilize the visual ingredient representations as nodes and the semantic similarity between ingredient embeddings as edges to construct an ingredient graph, and then learn their relationships via the graph convolutional network to make label embeddings and visual features interact with each other to improve the performance. Finally, fused features from both ingredient-oriented region features and ingredient-relationship features are used in the following multi-task category-ingredient joint learning. Extensive evaluation on three popular benchmark datasets (ETH Food-101, Vireo Food-172 and ISIA Food-200) demonstrates the effectiveness of our method. Further visualization of ingredient assignment maps and attention maps also shows the superiority of our method.

Graph attention mechanism with global contextual information for multi-label image recognition

Recent works have shown that multi-label image recognition is still a challenging task in computer vision due to the complicatedness and diversity of multi-label images. However, the existing works ignore the co-occurrence correlation and global contextual information between image space and objects. We present a model to solve these problems. On the one hand, we devise the graph attention mechanism to compute the hidden representations of different categories in multi-label images. It can specify different weights to different neighbor objects and well model the label dependency. On the other hand, we iterate the global contextual information by the second-order covariance pooling to enhance nonlinear modeling capability and use basic residual network to extract features. The proposed model is thoroughly evaluated on PASCAL VOC 2007 and MS-COCO datasets. Compared with classical ML-GCN, the model can better combine the image features and label embedding. Meanwhile, experiments show that it outperforms the state-of-the-art methods such as residual multi-layer perceptron, EfficientNet, and vision transformer.

Deep-learning-based latent space encoding for spectral unmixing of geological materials

The relatively coarse spatial resolution of hyperspectral images causes the mixing of disparate materials’ spectral responses in the sensor's instantaneous field of view (IFOV), resulting in mixed pixels. The current study proposes a capsule-based generative encoding model, called a denoising unmixing encoder network (DUENet), to formulate an end-to-end trainable spectral unmixing model. The reconstruction and cross-entropy losses and input prior-based constraints achieve joint optimization of denoising, data imputation, and spectral unmixing. Unlike earlier approaches, interpolation-based convolution and dynamic time wrapping (DTW)-based convolutional units facilitate DUENet to even unmix noisy spectra. In addition to embedding label information to improve the physical significance of the latent space, DUENet dynamically learns the parameters of the interpolation kernels. Benchmark airborne hyperspectral datasets (the Nabesna and Cuprite datasets) and simulated datasets were employed to evaluate the performance of the proposed approach. It was observed that the proposed joint optimization of spectral unmixing and denoising significantly improves the results. The adopted feature characterization using capsules improves the generalizability and gives good results, even with a limited number of training samples. This study shows the need for interpretability-based evaluation measures to analyze the unmixing frameworks based on the concepts learned for each endmember. The experiments confirm that the proposed strategy significantly reduces the models’ sensitivity to network parameters.

Motif‐based embedding label propagation algorithm for community detection

Community detection can exhibit the aggregation behavior of complex networks. Network motifs are the fundamental building blocks which can reveal the higher-order structure of complex networks. Label propagation algorithm has the advantage of approximately linear time complexity, unfortunately, the randomness of label update is a major but unsolved issue. For these reasons, this paper proposes a novel community detection method, named motif-based embedding label propagation algorithm (MELPA). First, complex network topology is reconstructed by merging higher-order topology with lower-order connectivity features, where higher-order topology is captured by mining network motifs. Second, We design a label propagation characteristic model according to nodes influence, then a new label update rule is formulated based on reconstructed weighted network, the rule integrates frequency among neighbor labels, influence of nodes, propagation characteristics and closeness of nodes to update the node label, the purpose is to overcome the randomness of label selection and identify a better and more stable community structure. Finally, extensive experiments on synthetic networks and real-world complex networks are conducted to verify the effectiveness of MELPA, especially for the complex networks with unobvious community structure, MELPA will get unexpected results.

Co-attention network with label embedding for text classification

Most existing methods for text classification focus on extracting a highly discriminative text representation, which, however, is typically computationally inefficient. To alleviate this issue, label embedding frameworks are proposed to adopt the label-to-text attention that directly uses label information to construct the text representation for more efficient text classification. Although these label embedding methods have achieved promising results, there is still much space for exploring how to use the label information more effectively. In this paper, we seek to exploit the label information by further constructing the text-attended label representation with text-to-label attention. To this end, we propose a Co-attention Network with Label Embedding (CNLE) that jointly encodes the text and labels into their mutually attended representations. In this way, the model is able to attend to the relevant parts of both. Experiments show that our approach achieves competitive results compared with previous state-of-the-art methods on 7 multi-class classification benchmarks and 2 multi-label classification benchmarks.

Local Constraint and Label Embedding Multi-layer Dictionary Learning for Sperm Head Classification

Morphological classification of human sperm heads is a key technology for diagnosing male infertility. Due to its sparse representation and learning capability, dictionary learning has shown remarkable performance in human sperm head classification. To promote the discriminability of the classification model, a novel local constraint and label embedding multi-layer dictionary learning model called LCLM-MDL is proposed in this study. Based on the multi-layer dictionary learning framework, two dictionaries are built on the basis of Laplacian regularized constraint and label embedding term in each layer, and the two dictionaries are approximated to each other as much as possible, so as to well exploit the nonlinear structure and discriminability features of the morphology of human sperm heads. In addition, to promote the robustness of the model, the asymmetric Huber loss is adopted in the last layer of LCLM-MDL, which approximates the misclassification error by using the absolute error function. Finally, the experimental results on HuSHeM dataset demonstrate the validity of the LCLM-MDL.

Discrete online cross-modal hashing

With the prevalence of multimedia content on the Web which usually continuously comes in a stream fashion, online cross-modal hashing methods have attracted extensive interest in recent years. However, most online hashing methods adopt a relaxation strategy or real-valued auxiliary variable strategy to avoid complex optimization of hash codes, leading to large quantization errors. In this paper, based on Discrete Latent Factor model-based cross-modal Hashing (DLFH), we propose a novel cross-modal online hashing method, i.e., Discrete Online Cross-modal Hashing (DOCH). To generate uniform high-quality hash codes of different modal, DOCH not only directly exploits the similarity between newly coming data and old existing data in the Hamming space, but also utilizes the fine-grained semantic information by label embedding. Moreover, DOCH can discretely learn hash codes by an efficient optimization algorithm. Extensive experiments conducted on two real-world datasets demonstrate the superiority of DOCH.

Multi-label text classification via joint learning from label embedding and label correlation

For the multi-label text classification problems with many classes, many existing multi-label classification algorithms become infeasible or suffer an unaffordable cost. Some researches hence perform the Label Space Dimension Reduction(LSDR) to solve this problem, but a number of methods ignore the sequence information of texts and the label correlation in the original label space, and treat the label as a meaningless multi-hot vector. In this paper, we put forward a multi-label text classification algorithm LELC(joint learning from Label Embedding and Label Correlation) based on the multi-layer attention and label correlation to solve the issue of multi-label text classification with a large number of class labels. Specifically, we firstly extract features through Bidirectional Gated Recurrent Unit Network(Bi-GRU), multi-layer attention and linear layers. Bi-GRU will capture the content information and sequence information of the text at the same time, and the attention mechanism can help us select the valid features related to labels. Then, we use matrix factorization to perform LSDR, and consider label correlation of the original label space in this process, which allows us to implicitly encode the latent space and simplify the model learning. Finally, Deep Canonical Correlation Analysis(CCA) technology is exploited to couple features and the latent space in an end-to-end pattern, so that these two can influence each other to learn the mapping of feature space to latent space. Experiments on 11 real-world datasets show the comparability between our proposed model and the state-of-the-art methods.

Medical code prediction via capsule networks and ICD knowledge

BackgroundClinical notes record the health status, clinical manifestations and other detailed information of each patient. The International Classification of Diseases (ICD) codes are important labels for electronic health records. Automatic medical codes assignment to clinical notes through the deep learning model can not only improve work efficiency and accelerate the development of medical informatization but also facilitate the resolution of many issues related to medical insurance. Recently, neural network-based methods have been proposed for the automatic medical code assignment. However, in the medical field, clinical notes are usually long documents and contain many complex sentences, most of the current methods cannot effective in learning the representation of potential features from document text.MethodsIn this paper, we propose a hybrid capsule network model. Specifically, we use bi-directional LSTM (Bi-LSTM) with forwarding and backward directions to merge the information from both sides of the sequence. The label embedding framework embeds the text and labels together to leverage the label information. We then use a dynamic routing algorithm in the capsule network to extract valuable features for medical code prediction task.ResultsWe applied our model to the task of automatic medical codes assignment to clinical notes and conducted a series of experiments based on MIMIC-III data. The experimental results show that our method achieves a micro F1-score of 67.5% on MIMIC-III dataset, which outperforms the other state-of-the-art methods.ConclusionsThe proposed model employed the dynamic routing algorithm and label embedding framework can effectively capture the important features across sentences. Both Capsule networks and domain knowledge are helpful for medical code prediction task.

Categorical Exploratory Data Analysis: From Multiclass Classification and Response Manifold Analytics Perspectives of Baseball Pitching Dynamics.

All features of any data type are universally equipped with categorical nature revealed through histograms. A contingency table framed by two histograms affords directional and mutual associations based on rescaled conditional Shannon entropies for any feature-pair. The heatmap of the mutual association matrix of all features becomes a roadmap showing which features are highly associative with which features. We develop our data analysis paradigm called categorical exploratory data analysis (CEDA) with this heatmap as a foundation. CEDA is demonstrated to provide new resolutions for two topics: multiclass classification (MCC) with one single categorical response variable and response manifold analytics (RMA) with multiple response variables. We compute visible and explainable information contents with multiscale and heterogeneous deterministic and stochastic structures in both topics. MCC involves all feature-group specific mixing geometries of labeled high-dimensional point-clouds. Upon each identified feature-group, we devise an indirect distance measure, a robust label embedding tree (LET), and a series of tree-based binary competitions to discover and present asymmetric mixing geometries. Then, a chain of complementary feature-groups offers a collection of mixing geometric pattern-categories with multiple perspective views. RMA studies a system’s regulating principles via multiple dimensional manifolds jointly constituted by targeted multiple response features and selected major covariate features. This manifold is marked with categorical localities reflecting major effects. Diverse minor effects are checked and identified across all localities for heterogeneity. Both MCC and RMA information contents are computed for data’s information content with predictive inferences as by-products. We illustrate CEDA developments via Iris data and demonstrate its applications on data taken from the PITCHf/x database.