Low-rank Representation Method Research Articles

Laplacian regularized low-rank representation for cancer samples clustering

Cancer samples clustering based on biomolecular data has been becoming an important tool for cancer classification. The recognition of cancer types is of great importance for cancer treatment. In this paper, in order to improve the accuracy of cancer recognition, we propose to use Laplacian regularized Low-Rank Representation (LLRR) to cluster the cancer samples based on genomic data. In LLRR method, the high-dimensional genomic data are approximately treated as samples extracted from a combination of several low-rank subspaces. The purpose of LLRR method is to seek the lowest-rank representation matrix based on a dictionary. Because a Laplacian regularization based on manifold is introduced into LLRR, compared to the Low-Rank Representation (LRR) method, besides capturing the global geometric structure, LLRR can capture the intrinsic local structure of high-dimensional observation data well. And what is more, in LLRR, the original data themselves are selected as a dictionary, so the lowest-rank representation is actually a similar expression between the samples. Therefore, corresponding to the low-rank representation matrix, the samples with high similarity are considered to come from the same subspace and are grouped into a class. The experiment results on real genomic data illustrate that LLRR method, compared with LRR and MLLRR, is more robust to noise and has a better ability to learn the inherent subspace structure of data, and achieves remarkable performance in the clustering of cancer samples.

Sparse and Low-Rank Subspace Data Clustering with Manifold Regularization Learned by Local Linear Embedding

Data clustering is an important research topic in data mining and signal processing communications. In all the data clustering methods, the subspace spectral clustering methods based on self expression model, e.g., the Sparse Subspace Clustering (SSC) and the Low Rank Representation (LRR) methods, have attracted a lot of attention and shown good performance. The key step of SSC and LRR is to construct a proper affinity or similarity matrix of data for spectral clustering. Recently, Laplacian graph constraint was introduced into the basic SSC and LRR and obtained considerable improvement. However, the current graph construction methods do not well exploit and reveal the non-linear properties of the clustering data, which is common for high dimensional data. In this paper, we introduce the classic manifold learning method, the Local Linear Embedding (LLE), to learn the non-linear structure underlying the data and use the learned local geometry of manifold as a regularization for SSC and LRR, which results the proposed LLE-SSC and LLE-LRR clustering methods. Additionally, to solve the complex optimization problem involved in the proposed models, an efficient algorithm is also proposed. We test the proposed data clustering methods on several types of public databases. The experimental results show that our methods outperform typical subspace clustering methods with Laplacian graph constraint.

Robust structure low-rank representation in latent space

Subspace clustering algorithms are usually used when processing high-dimensional data, such as in computer vision. This paper presents a robust low-rank representation (LRR) method that incorporates structure constraints and dimensionality reduction for subspace clustering. The existing LRR and its extensions use noise data as the dictionary, while this influences the final clustering results. The method proposed in this paper uses a discriminant dictionary for matrix recovery and completion in order to find the lowest rank representation of the data matrix. As the algorithm performs clustering operations in low-dimensional latent space, the computational efficiency of the algorithm is higher, which is also a major advantage of the proposed algorithm in this paper. A large number of experiments on standard datasets show the efficiency and effectiveness of the proposed method in subspace clustering problems.

Localized LRR on Grassmann Manifold: An Extrinsic View

Subspace data representation has recently become a common practice in many computer vision tasks. Low-rank representation (LRR) is one of the most successful models for clustering vectorial data according to their subspace structures. This paper explores the possibility of extending LRR for subspace data on Grassmann manifold. Rather than directly embedding the Grassmann manifold into the symmetric matrix space, an extrinsic view is taken to build the self-representation in the local area of the tangent space at each Grassmannian point, resulting in a localized LRR method on Grassmann manifold. A novel algorithm for solving the proposed model is investigated and implemented. The performance of the new clustering algorithm is assessed through experiments on several real-world data sets including MNIST handwritten digits, ballet video clips, SKIG action clips, and DynTex++ data set and highway traffic video clips. The experimental results show that the new method outperforms a number of state-of-the-art clustering methods.

LRR for Subspace Segmentation via Tractable Schatten-$p$ Norm Minimization and Factorization.

Recently, nuclear norm-based low rank representation (LRR) methods have been popular in several applications, such as subspace segmentation. However, there exist two limitations: one is that nuclear norm as the relaxation of rank function will lead to the suboptimal solution since nuclear norm-based minimization subproblem tends to the over-relaxations of singular value elements and treats each of them equally; the other is that solving LRR problems may cause more time consumption due to involving singular value decomposition of the large scale matrix at each iteration. To overcome both disadvantages, this paper mainly considers two tractable variants of LRR: one is Schatten-p norm minimization-based LRR (i.e., SpNM_LRR) and the other is Schatten-p norm factorization-based LRR (i.e., SpNFLRR) for p=1, 2/3 and 1/2. By introducing two or more auxiliary variables in the constraints, the alternating direction method of multiplier (ADMM) with multiple updating variables can be devised to solve these variants of LRR. Furthermore, both computational complexity and convergence property are given to evaluate nonconvex multiblocks ADMM algorithms. Several experiments finally validate the efficacy and efficiency of our methods on both synthetic data and real world data.

Self-Supervised Low-Rank Representation (SSLRR) for Hyperspectral Image Classification

Low-rank representation (LRR) can construct the relationships among pixels for hyperspectral image (HSI) classification with a given dictionary and a noise term. However, the accuracy of HSI classification based on LRR methods is degraded with the redundant and noise information existed in pixels. The neglect of semantic information around pixels in the LRR methods may cause “salt-and-pepper” problem in HSI classification. To avoid the aforementioned problems, a novel self-supervised low-rank representation method called SSLRR is developed. In SSLRR, the LRR and spectral–spatial graph regularization are developed as the pixel-level constraints to remove the redundant and noise information in HSIs. Superpixel constraints including data structure and relationship construction are further utilized to provide supervised feedback information to the subspace learning to avoid the “salt-and-pepper” problem generated in the pixel-based classification methods, and simultaneously enhance the performance of LRR. The pixel-level and superpixel-level regularizations are explicitly integrated into a unified objective function for LRR. By means of the linearized alternating direction method with adaptive penalty, the solution to the objective function is achieved by employing a customized iterative algorithm. We perform comprehensive evaluation of the proposed method on three challenging public HSI data sets. We obtain new state-of-the-art performance on these data sets, and achieve improvements of 44.3%, 13.4%, and 30.1% in overall accuracy compared to the best LRR method.

Iterative Reconstrained Low-Rank Representation via Weighted Nonconvex Regularizer

Benefiting from the joint consideration of geometric structures and low-rank constraint, graph low-rank representation (GLRR) method has led to the state-of-the-art results in many applications. However, it faces the limitations that the structure of errors should be known a prior, the isolated construction of graph Laplacian matrix, and the over shrinkage of the leading rank components. To improve GLRR in these regards, this paper proposes a new LRR model, namely iterative reconstrained LRR via weighted nonconvex regularization, using three distinguished properties on the concerned representation matrix. The first characterizes various distributions of the errors into an adaptively learned weight factor for more flexibility of noise suppression. The second generates an accurate graph matrix from weighted observations for less afflicted by noisy features. The third employs a parameterized rational function to reveal the importance of different rank components for better approximation to the intrinsic subspace structure. Following a deep exploration of automatic thresholding, parallel update, and partial SVD operation, we derive a computationally efficient low-rank representation algorithm using an iterative reconstrained framework and accelerated proximal gradient method. Comprehensive experiments are conducted on synthetic data, image clustering, and background subtraction to achieve several quantitative benchmarks as clustering accuracy, normalized mutual information, and execution time. Results demonstrate the robustness and efficiency of IRWNR compared with other state-of-the-art models.

Textile fabric defect detection based on low-rank representation

In this paper, we propose a novel and robust fabric defect detection method based on the low-rank representation (LRR) technique. Due to the repeated texture structure we model a defects-free fabric image as a low-rank structure. In addition, because defects, if exist, change only the texture of fabric locally, we model them with a sparse structure. Based on the above idea, we represent a fabric image into the sum of a low-rank matrix which expresses fabric texture and a sparse matrix which expresses defects. Then, the LRR method is applied to obtain the corresponding decomposition. Especially, in order to make better use of low-rank structure characteristics we propose LRREB (low-rank representation based on eigenvalue decomposition and blocked matrix) method to improve LRR. LRREB is implemented by dividing a image into some corresponding blocked matrices to reduce dimensions and applying eigen-value decomposition (EVD) on blocked matrix instead of singular value decomposition (SVD) on original fabric image, which improves the accuracy and efficiency. No training samples are required in our methods. Experimental results show that the proposed fabric defect detection method is feasible, effective, and simple to be employed.

A Mixed-Norm Laplacian Regularized Low-Rank Representation Method for Tumor Samples Clustering.

Tumor samples clustering based on biomolecular data is a hot issue of cancer classifications discovery. How to extract the valuable information from high dimensional genomic data is becoming an urgent problem in tumor samples clustering. In this paper, we introduce manifold regularization into low-rank representation model and present a novel method named Mixed-norm Laplacian regularized Low-Rank Representation (MLLRR) to identify the differentially expressed genes for tumor clustering based on gene expression data. Then, in order to advance the accuracy and stability of tumor clustering, we establish the clustering model based on Penalized Matrix Decomposition (PMD) and propose a novel cluster method named MLLRR-PMD. In this method, the cancer clustering research includes three steps. First, the matrix of gene expression data is decomposed into a low rank representation matrix and a sparse matrix by MLLRR. Second, the differentially expressed genes are identified based on the sparse matrix. Finally, the PMD is applied to cluster the samples based on the differentially expressed genes. The experiment results on simulation data and real genomic data illustrate that MLLRR method enhances the robustness to outliers and achieves remarkable performance in the extraction of differentially expressed genes.

Implicit Block Diagonal Low-Rank Representation

While current block diagonal constrained subspace clustering methods are performed explicitly on the original data space, in practice, it is often more desirable to embed the block diagonal prior into the reproducing kernel Hilbert feature space by kernelization techniques, as the underlying data structure in reality is usually nonlinear. However, it is still unknown how to carry out the embedding and kernelization in the models with block diagonal constraints. In this paper, we shall take a step in this direction. First, we establish a novel model termed implicit block diagonal low-rank representation (IBDLR), by incorporating the implicit feature representation and block diagonal prior into the prevalent low-rank representation method. Second, mostly important, we show that the model in IBDLR could be kernelized by making use of a smoothed dual representation and the specifics of a proximal gradient-based optimization algorithm. Finally, we provide some theoretical analyses for the convergence of our optimization algorithm. Comprehensive experiments on synthetic and real-world data sets demonstrate the superiorities of our IBDLR over state-of-the-art methods.

Online Low-Rank Representation Learning for Joint Multi-Subspace Recovery and Clustering.

Benefiting from global rank constraints, the low-rank representation (LRR) method has been shown to be an effective solution to subspace learning. However, the global mechanism also means that the LRR model is not suitable for handling large-scale data or dynamic data. For large-scale data, the LRR method suffers from high time complexity, and for dynamic data, it has to recompute a complex rank minimization for the entire data set whenever new samples are dynamically added, making it prohibitively expensive. Existing attempts to online LRR either take a stochastic approach or build the representation purely based on a small sample set and treat new input as out-of-sample data. The former often requires multiple runs for good performance and thus takes longer time to run, and the latter formulates online LRR as an out-of-sample classification problem and is less robust to noise. In this paper, a novel online LRR subspace learning method is proposed for both large-scale and dynamic data. The proposed algorithm is composed of two stages: static learning and dynamic updating. In the first stage, the subspace structure is learned from a small number of data samples. In the second stage, the intrinsic principal components of the entire data set are computed incrementally by utilizing the learned subspace structure, and the LRR matrix can also be incrementally solved by an efficient online singular value decomposition algorithm. The time complexity is reduced dramatically for large-scale data, and repeated computation is avoided for dynamic problems. We further perform theoretical analysis comparing the proposed online algorithm with the batch LRR method. Finally, experimental results on typical tasks of subspace recovery and subspace clustering show that the proposed algorithm performs comparably or better than batch methods, including the batch LRR, and significantly outperforms state-of-the-art online methods.

Improving RPCA-Based Clutter Suppression in GPR Detection of Antipersonnel Mines

Detecting shallow buried antipersonnel mines (APMs) with a ground-penetrating radar (GPR) is a challenging task because of clutter contamination, which often obscures the APM response. In this letter, a novel method combining migration imaging with the low-rank and sparse representation method to suppress clutter and extract target image is presented. The proposed method first focuses and strengthens the target response with migration imaging. Then, since the focused target response and clutter, respectively, constitute the sparse component and the low-rank component of the recorded data, the recently proposed robust principal component analysis (RPCA) can be applied to the recorded data to separate the target response (sparse component) from the clutter (low-rank component). Numerical simulation and experiments with real GPR systems are conducted. Results demonstrate the effectiveness of the proposed method in improving signal-to-clutter ratio and retrieving geometrical information of the target, which permits a better APM identification in heavy clutter environment.

Enhanced speaker verification using an adaptive multiple low-rank representation based on the modified adaptive Gaussian mixture model framework

In this paper, a new method for the calculation of the observation-confidence value that is applied in the modified adaptive Gaussian mixture model framework is proposed for speaker verification. First, an adaptive version of the multiple low-rank representation method, for which a weighted decomposition that incorporates the prior information regarding the speech/non-speech content is considered, is proposed to find the enhanced speech and for the estimation of the frame signal-to-noise ratio (SNR) values. Then, a simple sigmoid function is applied to convert the frame SNR values into the observation-confidence values. To verify the accuracy of the system, we use utterances from the Korean movie You Came From The Stars. The experiment results show that our proposed approach achieves a greater accuracy compared with the other well-known baseline methods, such as the GMM-based universal background model, the GMM supervector-based support vector machine (SVM), the i-vector-based SVM, and the sparse representation, under the noisy environment.

Discriminative Block-Diagonal Representation Learning for Image Recognition.

Existing block-diagonal representation studies mainly focuses on casting block-diagonal regularization on training data, while only little attention is dedicated to concurrently learning both block-diagonal representations of training and test data. In this paper, we propose a discriminative block-diagonal low-rank representation (BDLRR) method for recognition. In particular, the elaborate BDLRR is formulated as a joint optimization problem of shrinking the unfavorable representation from off-block-diagonal elements and strengthening the compact block-diagonal representation under the semisupervised framework of LRR. To this end, we first impose penalty constraints on the negative representation to eliminate the correlation between different classes such that the incoherence criterion of the extra-class representation is boosted. Moreover, a constructed subspace model is developed to enhance the self-expressive power of training samples and further build the representation bridge between the training and test samples, such that the coherence of the learned intraclass representation is consistently heightened. Finally, the resulting optimization problem is solved elegantly by employing an alternative optimization strategy, and a simple recognition algorithm on the learned representation is utilized for final prediction. Extensive experimental results demonstrate that the proposed method achieves superb recognition results on four face image data sets, three character data sets, and the 15 scene multicategories data set. It not only shows superior potential on image recognition but also outperforms the state-of-the-art methods.

Robust subspace clustering via joint weighted Schatten- p norm and Lq norm minimization

Low-rank representation (LRR) has been successfully applied to subspace clustering. However, the nuclear norm in the standard LRR is not optimal for approximating the rank function in many real-world applications. Meanwhile, the L21 norm in LRR also fails to characterize various noises properly. To address the above issues, we propose an improved LRR method, which achieves low rank property via the new formulation with weighted Schatten-p norm and Lq norm (WSPQ). Specifically, the nuclear norm is generalized to be the Schatten-p norm and different weights are assigned to the singular values, and thus it can approximate the rank function more accurately. In addition, Lq norm is further incorporated into WSPQ to model different noises and improve the robustness. An efficient algorithm based on the inexact augmented Lagrange multiplier method is designed for the formulated problem. Extensive experiments on face clustering and motion segmentation clearly demonstrate the superiority of the proposed WSPQ over several state-of-the-art methods.

Multiple Laplacian graph regularised low‐rank representation with application to image representation

Recently, low-rank representation (LRR)-based techniques have manifested remarkable results for data representation. To exploit the latent manifold structure of data, the graph regulariser is incorporated into the model of LRR. However, it is critical to construct an appropriate graph model and set the corresponding parameters. In addition, this procedure is usually time-consuming and proved to be overfitting when using cross validation or discrete grid search. Two novel LRR-based methods, called multiple graph regularised LRR and multiple hypergraph regularised LLR, are proposed to represent the high-dimensional data. To guarantee the smoothness along the estimated manifold, the multiple graph regulariser and the multiple hypergraph regulariser are incorporated into the traditional LRR method, respectively, which results in a unified framework. Moreover, the augmented Lagrange multiplier is adopted to solve the proposed models. Extensive experiments on real image datasets show the effectiveness of the proposed methods.

Unsupervised Classification of Polarimetirc SAR Image Via Improved Manifold Regularized Low-Rank Representation With Multiple Features

In this paper, a novel polarimetric synthetic aperture radar (PolSAR) image unsupervised classification method is proposed. It combines three typical features, including polarimetric data features (coherent matrix), polarimetric decomposition features (Krogager, Freeman, Yamaguqi, Neuman, and H/A/α decomposition), and gray-level co-occurrence matrix features to comprehensively describe the data characteristics. And it also proposes a symmetric revised Wishart (SRW) distance-derived manifold regularized low-rank representation (SRWM_LRR) method to deeply exploit the geometry data structure. The low-rank representation (LRR) is used to capture the intrinsic global structure of PolSAR data and the manifold regularization is employed to detect the local structure of the data, in which SRW distance is introduced to measure the similarity between different pixels for describing the local manifold structure. This algorithm considers the specific statistics property in PolSAR data and simultaneously integrates multiple features in perspective of data geometry structure to represent pixels for achieving a better classification performance. The effectiveness and practicability of the proposed method are demonstrated by datasets obtained either in spaceborne or airborne SAR system, including the Flevoland dataset (AIRSAR L-Band) extensively used in land classification cover, and Xi'an dataset (RadarSAT-2 C-Band). Compared with the traditional Wishart classifier, Euclidean and SRW distance-based spectral clustering and LRR, the proposed method shows improvement in accuracy and efficiency as well as a better visualization result.

Constrained Low-Rank Learning Using Least Squares-Based Regularization.

Low-rank learning has attracted much attention recently due to its efficacy in a rich variety of real-world tasks, e.g., subspace segmentation and image categorization. Most low-rank methods are incapable of capturing low-dimensional subspace for supervised learning tasks, e.g., classification and regression. This paper aims to learn both the discriminant low-rank representation (LRR) and the robust projecting subspace in a supervised manner. To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization. Naturally, the data label structure tends to resemble that of the corresponding low-dimensional representation, which is derived from the robust subspace projection of clean data by low-rank learning. Moreover, the low-dimensional representation of original data can be paired with some informative structure by imposing an appropriate constraint, e.g., Laplacian regularizer. Therefore, we propose a novel constrained LRR method. The objective function is formulated as a constrained nuclear norm minimization problem, which can be solved by the inexact augmented Lagrange multiplier algorithm. Extensive experiments on image classification, human pose estimation, and robust face recovery have confirmed the superiority of our method.

Differentially expressed genes selection via Laplacian regularized low-rank representation method

With the rapid development of DNA microarray technology and next-generation technology, a large number of genomic data were generated. So how to extract more differentially expressed genes from genomic data has become a matter of urgency. Because Low-Rank Representation (LRR) has the high performance in studying low-dimensional subspace structures, it has attracted a chunk of attention in recent years. However, it does not take into consideration the intrinsic geometric structures in data.In this paper, a new method named Laplacian regularized Low-Rank Representation (LLRR) has been proposed and applied on genomic data, which introduces graph regularization into LRR. By taking full advantages of the graph regularization, LLRR method can capture the intrinsic non-linear geometric information among the data. The LLRR method can decomposes the observation matrix of genomic data into a low rank matrix and a sparse matrix through solving an optimization problem. Because the significant genes can be considered as sparse signals, the differentially expressed genes are viewed as the sparse perturbation signals. Therefore, the differentially expressed genes can be selected according to the sparse matrix. Finally, we use the GO tool to analyze the selected genes and compare the P-values with other methods.The results on the simulation data and two real genomic data illustrate that this method outperforms some other methods: in differentially expressed gene selection.

Laplacian Regularized Low-Rank Representation and Its Applications.

Low-rank representation (LRR) has recently attracted a great deal of attention due to its pleasing efficacy in exploring low-dimensional subspace structures embedded in data. For a given set of observed data corrupted with sparse errors, LRR aims at learning a lowest-rank representation of all data jointly. LRR has broad applications in pattern recognition, computer vision and signal processing. In the real world, data often reside on low-dimensional manifolds embedded in a high-dimensional ambient space. However, the LRR method does not take into account the non-linear geometric structures within data, thus the locality and similarity information among data may be missing in the learning process. To improve LRR in this regard, we propose a general Laplacian regularized low-rank representation framework for data representation where a hypergraph Laplacian regularizer can be readily introduced into, i.e., a Non-negative Sparse Hyper-Laplacian regularized LRR model (NSHLRR). By taking advantage of the graph regularizer, our proposed method not only can represent the global low-dimensional structures, but also capture the intrinsic non-linear geometric information in data. The extensive experimental results on image clustering, semi-supervised image classification and dimensionality reduction tasks demonstrate the effectiveness of the proposed method.