Self-representation Learning Research Articles

Tensor-based unsupervised feature selection for error-robust handling of unbalanced incomplete multi-view data

Recent advancements in multi-view unsupervised feature selection (MUFS) have been notable, yet two primary challenges persist. First, real-world datasets frequently consist of unbalanced incomplete multi-view data, a scenario not adequately addressed by current MUFS methodologies. Second, the inherent complexity and heterogeneity of multi-view data often introduce significant noise, an aspect largely neglected by existing approaches, compromising their noise robustness. To tackle these issues, this paper introduces a Tensor-Based Error Robust Unbalanced Incomplete Multi-view Unsupervised Feature Selection (TERUIMUFS) strategy. The proposed MUFS framework specifically caters to unbalanced incomplete multi-view data, incorporating self-representation learning with a tensor low-rank constraint and sample diversity learning. This approach not only mitigates errors in the self-representation process but also corrects errors in the self-representation tensor, significantly enhancing the model’s resilience to noise. Furthermore, graph learning serves as a pivotal link between MUFS and self-representation learning. An innovative iterative optimization algorithm is developed for TERUIMUFS, complete with a thorough analysis of its convergence and computational complexity. Experimental results demonstrate TERUIMUFS’s effectiveness and competitiveness in addressing unbalanced incomplete multi-view unsupervised feature selection (UIMUFS), marking a significant advancement in the field.

Coding self-representative and label-relaxed hashing for cross-modal retrieval

In cross-modal retrieval, most existing hashing-based methods merely considered the relationship between feature representations to reduce the heterogeneous gap for data from various modalities, whereas they neglected the correlation between feature representations and the corresponding labels. This leads to the loss of significant semantic information, and the degradation of the class discriminability of the model. To tackle these issues, this paper presents a novel cross-modal retrieval method called coding self-representative and label-relaxed hashing (CSLRH) for cross-modal retrieval. Specifically, we propose a self-representation learning term to enhance the class-specific feature representations and reduce the noise interference. Additionally, we introduce a label-relaxed regression to establish semantic relations between the hash codes and the label information, aiming to enhance the semantic discriminability. Moreover, we incorporate a non-linear regression to capture the correlation of non-linear features in hash codes for cross-modal retrieval. Experimental results on three widely-used datasets verify the effectiveness of our proposed method, which can generate more discriminative hash codes to improve the precisions of cross-modal retrieval.

Dual contrastive learning for multi-view clustering

Self-representation subspace clustering, based on the self-representation property of linear subspaces, has demonstrated superior performance in subspace clustering. However, many existing methods learn self-representation in the embedding space, which fails to accurately capture the clustering structure of the data. Additionally, in the data embedding process, only the shared features among clusters are removed, and the discriminative features that distinguish different clusters cannot be effectively selected. To address these limitations, we propose a contrastive learning-based self-representation subspace clustering method named Multi-view Contrastive Subspace Clustering (MCSC). This method integrates feature selection and self-representation learning into a unified framework. It learns a shared self-representation coefficient in both the embedding space and the original space using contrastive learning, which allows for a more accurate description of the linear relationship between samples and a better depiction of the inter-sample structure simultaneously. Moreover, we employ the ℓ1,2-norm for feature selection, which eliminates shared redundancies while preserving important features specific to each cluster, making the samples more distinguishable. Extensive experiments on six datasets verify the validity of our proposed model.

Dual separated attention-based graph neural network

Graph neural networks (GNNs) offer a viable solution to model the inter-dependencies among labeled and unlabeled samples in a semi-supervised manner. However, their performances can degrade dramatically when the number of labels is extremely limited, which is due to the limitations of typical graph convolutional network design in most existing GNNs, including over-smoothing, difficulty in extending the propagation step, and failure to preserve the distinctiveness of each node. To address the issues, we propose a dual separated attention-based graph neural network (DSA-GNN) to deal with label scarcity in semi-supervised node classification. Firstly, DSA-GNN decouples feature propagation from representation transformation to alleviate the problems of over-smoothing and overfitting. Secondly, DSA-GNN separates self-representation learning from neighbor-representation learning by two feature extractors with different learnable parameters. As a result, the commonality between connected nodes and the distinctiveness of each node can be both preserved. Thirdly, DSA-GNN incorporates an attention-based label propagation mechanism to refine the label prediction of each node, by aggregating label prediction among the neighborhood based on adaptive edge coefficients. The extensive experimental results on the real-world datasets demonstrate the superiority of DSA-GNN for semi-supervised node classification, especially when the observed labels are extremely limited.

Precise feature selection via non-convex regularized graph embedding and self-representation for unsupervised learning

In graph embedding learning with unsupervised feature selection fields, similarity matrices are usually obtained only from the initial noise-laden samples, and the probable association between various features is often ignored. To tackle the issues described above, this article proposes an effective unsupervised group feature selection approach via non-convex regularized graph embedding and self-representation (NLGMS). NLGMS integrates the similarity matrix and feature selection as a joint system on the basis of graph. It combines self-representation learning into the system to preserve the global structure and learns the nonconvex projection matrix to direct the procedure of feature selection. Specifically, first, NLGMS presents a unified framework for global structure learning, local structure learning, and feature selection process so the data’s intrinsic structure is excellently captured and learned adaptively. Second, nonconvex ℓ2,0-norm is enforced upon the projection matrix to perform the process of feature selection, ensuring that the optimal feature subset can be picked precisely. Nonconvex ℓ2,0-norm constraint considers the probable association between various features so that it can deliver optimal performance by considering the feature subset as an integral whole, which other convex regularized constraints often fail to achieve. Finally, the introduction to self-representation learning can facilitate the reconstruction of samples in low-rank subspaces, strengthening the model’s robustness. To tackle this challenging nonconvex model, an innovative alternative optimization technique is exploited. In addition, to validate the superiority of NLGMS, exhaustive experiments are performed on one synthetic dataset, six benchmark datasets, and one fish image dataset. The source code is available at: https://github.com/hrbai/NLGMS.

CommGNAS: Unsupervised Graph Neural Architecture Search for Community Detection

Graph neural architecture search (GNAS) has been successful in many supervised learning tasks, such as node classification, graph classification, and link prediction. GNAS uses a search algorithm to sample graph neural network (GNN) architectures from the search space and evaluates sampled GNN architectures based on estimation strategies to generate feedback for the search algorithm. In traditional GNAS, the typical estimation strategy requires using labeled graph data to generate feedback, which plays a fundamental and vital role in the search algorithm to sample a better GNN architecture during the search process. However, a large portion of real-world graph data is unlabeled. The estimation strategy in traditional GNAS cannot use unlabeled graph data to generate feedback for the search algorithm, so the traditional supervised GNAS fails to solve unsupervised problems, such as community detection tasks. To solve this challenge, this paper proposed CommGNAS, an effective node representation learning method with unsupervised graph neural architecture search for community detection. In CommGNAS, we design an unsupervised evaluation strategy with self-supervised and self-representation learning. It represents the first research work in literature to solve the problems of unsupervised graph neural architecture search for community detection. The experimental results show that CommGNAS can obtain the best performance in community detection tasks on real-world graphs against the state-of-the-art baseline methods.

Multi-view subspace clustering based on adaptive search

Multi-view clustering has been widely applied to image classification, information retrieval, medical pathology analysis, and other fields. So far, many multi-view subspace clustering algorithms based on self-representation learning have been developed. However, most of them use the original data as a dictionary to construct affinity graphs, and their clustering performance depends largely on the quality of the original data features. If the raw data is corrupted by noise, directly using these noisy data as a self-representing dictionary may pose a series of research challenges. Secondly, in the process of self-representation learning, the internal relationship between data points is actually continuously updated and changed, and fixed data points not only affect the selection of view feature diversity, but also may lead to clustering results that are unusually sensitive to the input data. Finally, the coefficient matrix obtained by self-representation learning may contain noise or data structures that are not relevant to multi-view clustering. To remedy these shortcomings, this work proposes a new multi-view clustering algorithm, namely, multi-view subspace clustering based on adaptive search. Specifically, first, we find its similar data matrix for each view’s original data and perform dictionary learning by using the similar data matrix as a dictionary. This not only facilitates identifying the continuously varied internal relationships between data points, but also alleviates the errors that occur when the raw data is directly selected as a dictionary. Second, we introduce robust principal component analysis (RPCA) and rank constraints into the construction of affinity matrices to obtain cleaner and more robust affinity matrices that explore the same clustering properties among different views. In addition, we develop an augmented Lagrange multiplier (ALM) based method to solve the objective function of the model. Finally, we conducted substantial experiments on six real multi-view datasets to demonstrate that the MSC-AS algorithm is more robust and effective than some other advanced clustering algorithms.

Elastic Deep Sparse Self-Representation Subspace Clustering Network

Subspace clustering model based on self-representation learning often use ℓ1,ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _1, \\ell _2$$\\end{document} or kernel norm to constrain self-representation matrix of the dataset. In theory, ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _1$$\\end{document} norm can constrain the independence of subspaces, but which may lead to under-connection because the sparsity of the self-representation matrix. ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _2$$\\end{document} and nuclear norm regularization can improve the connectivity between clusters, but which may lead to over-connection of the self-representation matrix. Because a single regularization term may cause subspaces to be over or insufficiently divided, this paper proposes an elastic deep sparse self-representation subspace clustering network (EDS-SC), which imposes sparse constraints on deep features, and introduces the elastic network regularization mixed ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _1$$\\end{document} and ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _2$$\\end{document} norm to constraint self-representation matrix. The network can extract deep sparse features and provide a balance between subspace independence and connectivity. Experiments on human faces, objects, and medical imaging datasets prove the effectiveness of EDS-SC network.

Multigraph approximate-representation learning for hyperspectral band selection

ABSTRACT Unsupervised hyperspectral image (HSI) band selection methods have been attracting ever-increasing attention. However, the local structural features captured by most of the existing methods suffer from a certain degree of risk of feature ambiguity. Moreover, these methods typically have difficulty preserving the latent manifold structure of the HSI in selected subsets of the bands. To address these problems, firstly, the graph approximate representation learning (GARL) model is proposed in this article by incorporating graph-regularized sparse coding into self-representation learning to preserve the local manifold structure information of HSI in the selected subset of bands. Meanwhile, to alleviate the ambiguity of the local graph structures, we design a local-global feature relationship expression model to construct the latent long-distance contextual connectivity (LDCC) graphs between the obtained local graph structures. Then, to preserve HSI’s local and non-local manifold structure information as much as possible in the selected subset of bands, a multigraph approximate representation learning (MGARL) model is proposed in this article by incorporating the obtained LDCC graph into the proposed GARL model. Next, we design a solution method to solve the proposed formulation. Finally, extensive experimental results on four datasets reveal the promising performance of MGARL over some state-of-the-art methods.

Multi-task subspace clustering

In recent years, subspace clustering and multi-task clustering have received extensive attention due to their wide practical applications. Traditional subspace clustering is limited to exploring the underlying subspaces within a single task, while traditional multi-task clustering usually ignores the data distributed in some low-dimensional subspaces. However, multiple subspace clustering tasks, in reality, may be related. To resolve this issue, we introduce a multi-task subspace clustering method, which unifies transfer learning, local structure learning, and self-representation learning into a single paradigm. In particular, we suggest learning a projection matrix to perform dimensionality reduction, and exploring correlations among multiple tasks simultaneously. In addition, we design an efficient algorithm to solve the objective. The experimental findings demonstrate that the approach suggested in this study exhibits superior performance compared to the existing state-of-the-art methods in the domains of subspace clustering and multi-task clustering. Our MATLAB codes are available at https://github.com/HeKind/MTSC.

Contrastive self-representation learning for data clustering

This paper is concerned with self-representation subspace learning. It is one of the most representative subspace techniques, which has attracted considerable attention for clustering due to its good performance. Among these methods, low-rank representation (LRR) has achieved impressive results for subspace clustering. However, it only considers the similarity between the data itself, while neglecting the differences with other samples. Besides, it cannot well deal with noise and portray cluster-to-cluster relationships well. To solve these problems, we propose a Contrastive Self-representation model for Clustering (CSC). CSC simultaneously takes into account the similarity/dissimilarity between positive/negative pairs when learning the self-representation coefficient matrix of data while the form of the loss function can reduce the effect of noise on the results. Moreover, We use the ℓ1,2-norm regularizer on the coefficient matrix to achieve its sparsity to better characterize the cluster structure. Thus, the learned self-representation coefficient matrix well encodes both the discriminative information and cluster structure. Extensive experiments on seven benchmark databases indicate the superiority of our proposed method.

Subspace Clustering via Adaptive Non-Negative Representation Learning and Its Application to Image Segmentation

Self-representation subspace clustering based on graphs has the merits of capability and efficiency. However, the graph built by the self-representation methods has two issues: (i) usually lacking conciseness and informativeness due to the negative representation coefficients. (ii) no guarantee of an overall optimal solution due to the separation of representation learning and graph construction. To alleviate these issues, we propose a novel subspace clustering via learning non-negative representation with an adaptive graph. Specifically, we explicitly impose the non-negative constraint on the self-representation learning, ensuring that each data point is approximated from a group of homogeneous samples and enhancing the distinguishability of data representation. Meanwhile, an adaptive graph is developed so that both representation and the geometric structure of data are simultaneously learned in a unified procedure. Moreover, the learned representation is less sensitive to data noise imposed by the ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> -norm, so the adaptive graph will be further improved. An efficient optimization procedure is developed to find the optimal solution. Extensive experiments on subspace clustering and the extension application to image segmentation validate the advantages of our method against state-of-the-art methods.

Affine Subspace Robust Low-Rank Self-Representation: from Matrix to Tensor.

Low-rank self-representation based subspace learning has confirmed its great effectiveness in a broad range of applications. Nevertheless, existing studies mainly focus on exploring the global linear subspace structure, and cannot commendably handle the case where the samples approximately (i.e., the samples contain data errors) lie in several more general affine subspaces. To overcome this drawback, in this paper, we innovatively propose to introduce affine and nonnegative constraints into low-rank self-representation learning. While simple enough, we provide their underlying theoretical insight from a geometric perspective. The union of two constraints geometrically restricts each sample to be expressed as a convex combination of other samples in the same subspace. In this way, when exploring the global affine subspace structure, we can also consider the specific local distribution of data in each subspace. To comprehensively demonstrate the benefits of introducing two constraints, we instantiate three low-rank self-representation methods ranging from single-view low-rank matrix learning to multi-view low-rank tensor learning. We carefully design the solution algorithms to efficiently optimize the proposed three approaches. Extensive experiments are conducted on three typical tasks, including single-view subspace clustering, multi-view subspace clustering, and multi-view semi-supervised classification. The notably superior experimental results powerfully verify the effectiveness of our proposals.

Anomaly Detection With Representative Neighbors.

Identifying anomalies from data has attracted increasing attention in recent years due to its broad range of potential applications. Although many efforts have been made for anomaly detection, how to effectively handle high-dimensional data and how to exactly explore neighborhood information, a fundamental issue in anomaly detection, have not yet received sufficient concerns. To circumvent these challenges, in this article, we propose an effective anomaly detection method with representative neighbors for high-dimensional data. Specifically, it projects the high-dimensional data into a low-dimensional space via a sparse operation and explores representative neighbors with a self-representation learning technique. The neighborhood information is then transformed into similarity relations, making the data converge or disperse. Eventually, anomalies are discriminated by a tailored graph clustering technique, which can effectively reveal structural information of the data. Extensive experiments were conducted on ten public real-world datasets with 11 popular anomaly detection algorithms. The results show that the proposed method has encouraging and promising performance compared to the state-of-the-art anomaly detection algorithms.

Spatially aware self-representation learning for tissue structure characterization and spatial functional genes identification.

Spatially resolved transcriptomics (SRT) enable the comprehensive characterization of transcriptomic profiles in the context of tissue microenvironments. Unveiling spatial transcriptional heterogeneity needs to effectively incorporate spatial information accounting for the substantial spatial correlation of expression measurements. Here, we develop a computational method, SpaSRL (spatially aware self-representation learning), which flexibly enhances and decodes spatial transcriptional signals to simultaneously achieve spatial domain detection and spatial functional genes identification. This novel tunable spatially aware strategy of SpaSRL not only balances spatial and transcriptional coherence for the two tasks, but also can transfer spatial correlation constraint between them based on a unified model. In addition, this joint analysis by SpaSRL deciphers accurate and fine-grained tissue structures and ensures the effective extraction of biologically informative genes underlying spatial architecture. We verified the superiority of SpaSRL on spatial domain detection, spatial functional genes identification and data denoising using multiple SRT datasets obtained by different platforms and tissue sections. Our results illustrate SpaSRL's utility in flexible integration of spatial information and novel discovery of biological insights from spatial transcriptomic datasets.

Multi-View Clustering for Integration of Gene Expression and Methylation Data With Tensor Decomposition and Self-Representation Learning.

The accumulated DNA methylation and gene expression provide a great opportunity to exploit the epigenetic patterns of genes, which is the foundation for revealing the underlying mechanisms of biological systems. Current integrative algorithms are criticized for undesirable performance because they fail to address the heterogeneity of expression and methylation data, and the intrinsic relations among them. To solve this issue, a novel multi-view clustering with self-representation learning and low-rank tensor constraint (MCSL-LTC) is proposed for the integration of gene expression and DNA methylation data, which are treated as complementary views. Specifically, MCSL-LTC first learns the low-dimensional features for each view with the linear projection, and then these features are fused in a unified tensor space with low-rank constraints. In this case, the complementary information of various views is precisely captured, where the heterogeneity of omic data is avoided, thereby enhancing the consistency of different views. Finally, MCSL-LTC obtains a consensus cluster of genes reflecting the structure and features of various views. Experimental results demonstrate that the proposed approach outperforms state-of-the-art baselines in terms of accuracy on both the social and cancer data, which provides an effective and efficient method for the integration of heterogeneous genomic data.

Latent energy preserving embedding for unsupervised feature selection

As is a fundamental yet challenging topic in machine learning and pattern recognition, feature selection has received much attention. Since data are often unlabeled in real-world applications, unsupervised feature selection (UFS) methods have aroused extensive attention. In this article, we present a novel latent energy preserving embedding (LEPE) method for UFS. First, we rewrite subspace learning into the form of energy preserving. Then, a novel self-representation learning strategy is utilized in the feature selection framework, in which the low-rank and sparse constraints are imposed on the representation matrix. In addition, we utilize a graph Laplacian to mine the manifold information of data. Meanwhile, we use an ℓ2,1-norm for feature selection. To validate the effectiveness of the proposed LEPE method, we conduct extensive experiments on six real-world datasets. Experimental results illustrate the effectiveness of the proposed method.

Incomplete multi-view clustering via independent self-representation learning

Incomplete multi-view clustering is a learning paradigm that combines heterogeneous and incomplete characteristics of multi-view data to obtain data structure and improve clustering performance. In practical application, each view will also be affected by missing values and outliers, in addition to missing some complete samples, which makes most traditional incomplete multi-view clustering methods ineffective. An incomplete multi-view clustering method based on independent self-presentation learning is proposed to solve the above issue. The proposed method fills in the missing features and learns the unique self-presentation matrix of each view using self-representation reconstruction. Then, to better mine the structure, low-rank constraints are added to the self-presentation matrix, and the diversity among different views is measured by introducing the Hilbert-Schmidt independence criterion. Experimental results on multiple data sets show that the proposed method can achieve better clustering than other advanced methods in most cases.

Generalized Multi-View Collaborative Subspace Clustering

In real-world applications, complete or incomplete multi-view data are common, which leads to the problem of generalized multi-view clustering. Recently, researchers attempt to learn the latent representation in the common subspace from heterogeneous data, which usually suffers from feature degeneration. Moreover, there are limited efforts on simultaneously revealing the underlying subspace structure and exploring the complementary information from incomplete multiple views. In this paper, we introduce a novel Generalized Multi-view Collaborative Subspace Clustering (GMCSC) framework to address the above issues, in which consensus subspace structure of all views and embedding subspaces for each view are jointly learned to benefit each other. Specifically, we develop a novel collaborative subspace learning strategy based on self-representation learning, which provides a brand-new way of pursuing the complete subspace structure directly from multi-view data. Furthermore, we explore complementary information by enforcing the consistency across different views and preserving the view-specific information of each view, which can alleviate the problem of feature degeneration and enhance the reasonability of using a consensus representation for multiple views. Experimental results on six benchmark datasets demonstrate that the proposed method can significantly outperform the state-of-the-art algorithms.

Multiview Multi-Instance Multilabel Active Learning.

Multiview multi-instance multilabel learning (M3L) is a framework for modeling complex objects. In this framework, each object (or bag) contains one or more instances, is represented with different feature views, and simultaneously annotated with a set of nonexclusive semantic labels. Given the multiplicity of the studied objects, traditional M3L methods generally demand a large number of labeled bags to train a predictive model to annotate bags (or instances) with semantic labels. However, annotating sufficient bags is very expensive and often impractical. In this article, we present an active learning-based M3L approach (M3AL) to reduce the labeling costs of bags and to improve the performance as much as possible. M3AL first adapts the multiview self-representation learning to evacuate the shared and individual information of bags and to learn the shared/individual similarities between bags across/within views. Next, to avoid scrutinizing all the possible labels, M3AL introduces a new query strategy that leverages the shared and individual information, and the diverse instance distribution of bags across views, to select the most informative bag-label pair for the query. Experimental studies on benchmark data sets show that M3AL can significantly reduce the query costs while achieving a better performance than other related competitive methods at the same cost.