Label Space Dimension Reduction Research Articles

Multi-label text classification via joint learning from label embedding and label correlation

For the multi-label text classification problems with many classes, many existing multi-label classification algorithms become infeasible or suffer an unaffordable cost. Some researches hence perform the Label Space Dimension Reduction(LSDR) to solve this problem, but a number of methods ignore the sequence information of texts and the label correlation in the original label space, and treat the label as a meaningless multi-hot vector. In this paper, we put forward a multi-label text classification algorithm LELC(joint learning from Label Embedding and Label Correlation) based on the multi-layer attention and label correlation to solve the issue of multi-label text classification with a large number of class labels. Specifically, we firstly extract features through Bidirectional Gated Recurrent Unit Network(Bi-GRU), multi-layer attention and linear layers. Bi-GRU will capture the content information and sequence information of the text at the same time, and the attention mechanism can help us select the valid features related to labels. Then, we use matrix factorization to perform LSDR, and consider label correlation of the original label space in this process, which allows us to implicitly encode the latent space and simplify the model learning. Finally, Deep Canonical Correlation Analysis(CCA) technology is exploited to couple features and the latent space in an end-to-end pattern, so that these two can influence each other to learn the mapping of feature space to latent space. Experiments on 11 real-world datasets show the comparability between our proposed model and the state-of-the-art methods.

A novel multi-leak sensor deployment strategy in water distribution networks based on the LSDR-JMI method

Water leakage in water distribution networks (WDNs) can cause serious economic and social impacts. Therefore, it is urgent to discover the efficient sensor deployment methods to monitor the leakage of WDNs. While related research studies mainly consider the single-point leakage, this paper firstly presents a pressure sensor deployment strategy to deal with the problem of the multi-point leakage occurred in WDNs. The novel proposed label space dimension reduction joint mutual information method (LSDR-JMI) is based on a filter multi-label feature selection algorithm. The label space dimension reduction (LSDR) method is based on the dependence of the label matrix on the leak cases to obtain the optimal label subspace. Then the optimal subset of feature matrix is calculated according to the relevant information between the optimal label subspace and the feature matrix. This method is applied to two hydraulic networks simulated by EPANET to demonstrate its effectiveness and advantage.

Multi-Label Learning via Feature and Label Space Dimension Reduction

In multi-label learning, each object belongs to multiple class labels simultaneously. In the data explosion age, the size of data is often huge, i.e., large number of instances, features and class labels. The high dimension of both the feature and label spaces has posed great challenges to multi-label learning problems, e.g., high time and memory costs. In this paper, we propose a new framework for multi-label learning with a large number of class labels and features, i.e., M ulti- L abel L earning via F eature and L abel S pace D imension R eduction, namely MLL-FLSDR. Specifically, both the feature space and label space are reduced to low dimensional spaces respectively, in which the local structure of data points is utilized to constrain the geometrical structure on both the learned low dimensional spaces and guarantee the qualities of them. Then, an effective multi-label classifier is constructed from the low dimensional feature space to the latent label space. Last, the final prediction for new test data examples can be obtained by recovering from their prediction results in the latent label space with an encoding matrix learned in the previous stage. Extensive comparison experiments with the state-of-the-art approaches manifest the effectiveness of the proposed method MLL-FLSDR.

Boolean Matrix Decomposition for Label Space Dimension Reduction: Method, Framework and Applications

Recently, the large number of labels challenges the traditional multi-label classification methods in various application scenes. We introduce a framework of multi-label classification based on Boolean matrix decomposition for giving various multi-label classifiers the ability to predict labels in a high dimension label space. The BMD module in this framework satisfies two conditions of ‘the column use condition’ and exacts BMD, which can reduce the burden on training and predicting task of multi-label classifier to some extent, and the predicted result of multi-label classifier can be restored to original label space by simply Boolean multiplying with matrix. Experimental results on yeast datasets demonstrate that our framework can work particularly well on datasets with a large number of labels and obtain a better predicting accuracy. In summary, the methods discussed in this paper constitute important basic for utilizing more multi-label classifiers in high label dimension space, which is the main contribution of this paper.

CLUSTER ANALYSIS OF MEDICAL TEXT DOCUMENTS BY USING SEMI-CLUSTERING APPROACH BASED ON GRAPH REPRESENTATION

The development of Internet resulted in an increasing number of online text re-positories. In many cases, documents are assigned to more than one class and automatic multi-label classification needs to be used. When the number of labels exceeds the number of the documents, effective label space dimension reduction may signifi-cantly improve classification accuracy, what is a major priority in the medical field. In the paper, we propose document clustering for label selection. We use semi-clustering method, by considering graph representation, where documents are represented by vertices and edge weights are calculated according to their mutual similarity. Assigning documents to semi-clusters helps in reducing number of labels, further used in multilabel classification process. The performance of the method is examined by experiments conducted on real medical datasets.

Multi-view label embedding

Multi-label classification has been successfully applied to image annotation, information retrieval, text categorization, etc. When the number of classes increases significantly, the traditional multi-label learning models will become computationally impractical. Label space dimension reduction (LSDR) is then developed to alleviate the effect of the high dimensionality of labels. However, almost all the existing LSDR methods focus on single-view learning. In this paper, we develop a multi-view label embedding (MVLE) model by exploiting the multi-view correlations. The label space and feature space of each view are bridged by a latent space. To exploit the consensus among different views, multi-view latent spaces are correlated by Hilbert–Schmidt independence criterion(HSIC). For a test sample, it is firstly embedded to the latent space of each view and then projected to the label space. The prediction is conducted by combining the multi-view outputs. Experiments on benchmark databases show that MVLE outperforms the state-of-the-art LSDR algorithms in both multi-view settings and different multi-view learning strategies.

A label compression method for online multi-label classification

Many modern applications deal with multi-label data, such as functional categorizations of genes, image labeling and text categorization. Classification of such data with a large number of labels and latent dependencies among them is a challenging task, and it becomes even more challenging when the data is received online and in chunks. Many of the current multi-label classification methods require a lot of time and memory, which make them infeasible for practical real-world applications. In this paper, we propose a fast linear label space dimension reduction method that transforms the labels into a reduced encoded space and trains models on the obtained pseudo labels. Additionally, it provides an analytical method to update the decoding matrix which maps the labels into the original space and is used during the test phase. Experimental results show the effectiveness of this approach in terms of running times and the prediction performance over different measures.

End-to-End Feature-Aware Label Space Encoding for Multilabel Classification With Many Classes.

To make the problem of multilabel classification with many classes more tractable, in recent years, academia has seen efforts devoted to performing label space dimension reduction (LSDR). Specifically, LSDR encodes high-dimensional label vectors into low-dimensional code vectors lying in a latent space, so as to train predictive models at much lower costs. With respect to the prediction, it performs classification for any unseen instance by recovering a label vector from its predicted code vector via a decoding process. In this paper, we propose a novel method, namely End-to-End Feature-aware label space Encoding (E2FE), to perform LSDR. Instead of requiring an encoding function like most previous works, E2FE directly learns a code matrix formed by code vectors of the training instances in an end-to-end manner. Another distinct property of E2FE is its feature awareness attributable to the fact that the code matrix is learned by jointly maximizing the recoverability of the label space and the predictability of the latent space. Based on the learned code matrix, E2FE further trains predictive models to map instance features into code vectors, and also learns a linear decoding matrix for efficiently recovering the label vector of any unseen instance from its predicted code vector. Theoretical analyses show that both the code matrix and the linear decoding matrix in E2FE can be efficiently learned. Moreover, similar to previous works, E2FE can be specified to learn an encoding function. And it can also be extended with kernel tricks to handle nonlinear correlations between the feature space and the latent space. Comprehensive experiments conducted on diverse benchmark data sets with many classes show consistent performance gains of E2FE over the state-of-the-art methods.

A probabilistic multi-label classifier with missing and noisy labels handling capability

Multi-label classification with a large set of labels is a challenging task. Label-Space Dimension Reduction (LSDR) is the most popular approach that addresses this problem. LSDR methods project the high-dimensional label vectors onto a low-dimensional space that can be predicted from the feature space. Many LSDR methods assume that the training data provide complete label vector for all training samples while this assumption is usually violated particularly when label vectors are high dimensional. In this paper, we propose a probabilistic model that has an effective mechanism to handle missing and noisy labels. In the proposed Bayesian network model, a set of auxiliary random variables, called experts, are incorporated to provide robustness to missing and noisy labels. Variational inference is utilized to find the desired probabilities in this model. The proposed approximate inference is highly parallelizable and can be implemented efficiently. Experiments on real-world datasets show that our method outperforms state-of-the-art multi-label classifiers by a large margin.

Dependence maximization based label space dimension reduction for multi-label classification

High dimensionality of label space poses crucial challenge to efficient multi-label classification. Therefore, it is needed to reduce the dimensionality of label space. In this paper, we propose a new algorithm, called dependence maximization based label space reduction (DMLR), which maximizes the dependence between feature vectors and code vectors via Hilbert–Schmidt independence criterion while minimizing the encoding loss of labels. Two different kinds of instance kernel are discussed. The global kernel for DMLRG and the local kernel for DMLRL take global information and locality information into consideration respectively. Experimental results over six categorization problems validate the superiority of the proposed algorithm to state-of-art label space dimension reduction methods in improving performance at the cost of a very short time.