Subspace Segmentation Research Articles

Projection subspace clustering

Gene expression data is a kind of high dimension and small sample size data. The clustering accuracy of conventional clustering techniques is lower on gene expression data due to its high dimension. Because some subspace segmentation approaches can be better applied in the high-dimensional space, three new subspace clustering models for gene expression data sets are proposed in this work. The proposed projection subspace clustering models have projection sparse subspace clustering, projection low-rank representation subspace clustering and projection least-squares regression subspace clustering which combine projection technique with sparse subspace clustering, low-rank representation and least-square regression, respectively. In order to compute the inner product in the high-dimensional space, the kernel function is used to the projection subspace clustering models. The experimental results on six gene expression data sets show these models are effective.

Constrained Low-Rank Learning Using Least Squares-Based Regularization.

Low-rank learning has attracted much attention recently due to its efficacy in a rich variety of real-world tasks, e.g., subspace segmentation and image categorization. Most low-rank methods are incapable of capturing low-dimensional subspace for supervised learning tasks, e.g., classification and regression. This paper aims to learn both the discriminant low-rank representation (LRR) and the robust projecting subspace in a supervised manner. To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization. Naturally, the data label structure tends to resemble that of the corresponding low-dimensional representation, which is derived from the robust subspace projection of clean data by low-rank learning. Moreover, the low-dimensional representation of original data can be paired with some informative structure by imposing an appropriate constraint, e.g., Laplacian regularizer. Therefore, we propose a novel constrained LRR method. The objective function is formulated as a constrained nuclear norm minimization problem, which can be solved by the inexact augmented Lagrange multiplier algorithm. Extensive experiments on image classification, human pose estimation, and robust face recovery have confirmed the superiority of our method.

LRSR: Low-Rank-Sparse representation for subspace clustering

High-dimensional data in the real world often resides in low-dimensional subspaces. The state-of-the-art methods for subspace segmentation include Low Rank Representation (LRR) and Sparse Representation (SR). The former seeks the global lowest rank representation but restrictively assumes the independence among subspaces, whereas the latter seeks the clustering of disjoint or overlapped subspaces through locality measure, which may cause failure in the case of large noises. To this end, a Low Rank subspace Sparse Representation framework, hereafter referred to as LRSR, is proposed in this paper to recover and segment embedding subspaces simultaneously. Three major contributions can be claimed in this paper: First, a clean dictionary is constructed by optimizing its nuclear norm, low-rank-sparse coefficient matrix obtained using linearized alternating direction method (LADM). Second, both the convergence proof and the complexity analysis are given to prove the effectiveness and efficiency of our proposed LRSR algorithm. Third, the experiments on synthetic data and two benchmark datasets further verify that the LRSR enjoys the capability of clustering disjoint subspaces as well as the robustness against large noises, thanks to its considerations of both global and local subspace information. Therefore, it has been demonstrated in this research that our proposed LRSR algorithm outperforms the state-of-the-art subspace clustering methods, verified by both theoretical analysis and the empirical studies.

Local graph regularized coding for salient object detection

Subspace segmentation based salient object detection has received increasing interests in recent years. To preserve the locality and similarity of regions, a grouping effect of representation is introduced to segment the salient object and background in subspace. Then a new saliency map is calculated by incorporating this local graph regularizer into coding, which explicitly explores the data self-representation model and thus locate more accurate salient regions. Moreover, a heuristic object-based dictionary from background superpixels is obtained in border set removing the image regions within the potential object regions. Experimental results on four large benchmark databases demonstrate that the proposed method performs favorably against eight recent state-of-the-art methods in terms of three evaluation criterions, with a reduction of MAE by 19.8% than GR and 29.3% than CB in the two SED datasets, respectively. Meanwhile, our method also runs faster than the comparative detection approaches.

Tensor LRR and Sparse Coding-Based Subspace Clustering.

Subspace clustering groups a set of samples from a union of several linear subspaces into clusters, so that the samples in the same cluster are drawn from the same linear subspace. In the majority of the existing work on subspace clustering, clusters are built based on feature information, while sample correlations in their original spatial structure are simply ignored. Besides, original high-dimensional feature vector contains noisy/redundant information, and the time complexity grows exponentially with the number of dimensions. To address these issues, we propose a tensor low-rank representation (TLRR) and sparse coding-based (TLRRSC) subspace clustering method by simultaneously considering feature information and spatial structures. TLRR seeks the lowest rank representation over original spatial structures along all spatial directions. Sparse coding learns a dictionary along feature spaces, so that each sample can be represented by a few atoms of the learned dictionary. The affinity matrix used for spectral clustering is built from the joint similarities in both spatial and feature spaces. TLRRSC can well capture the global structure and inherent feature information of data and provide a robust subspace segmentation from corrupted data. Experimental results on both synthetic and real-world data sets show that TLRRSC outperforms several established stateof- the-art methods.

Spectral clustering steered low-rank representation for subspace segmentation

Low-rank representation (LRR) and its variations have achieved great successes in subspace segmentation tasks. However, the segmentation processes of the existing LRR-related methods are all divided into two separated steps: affinity graphs construction and segmentation results obtainment. In the second step, normalize cut (Ncut) algorithm is used to get the final results based on the constructed graphs. This implies that the affinity graphs obtained by LRR-related algorithms may not be most suitable for Ncut, and the best results are not guaranteed to be achieved. In this paper, we propose a spectral clustering steered LRR representation algorithm (SCSLRR) which combines the objection functions of Ncut, K-means and LRR together. By solving a joint optimization problem, SCSLRR is able to find low-rank affinity matrices which are most beneficial for Ncut to get best segmentation results. The extensive experiments of subspace segmentation on several benchmark datasets show that SCSLRR dominates the related methods.

Robust subspace segmentation via nonconvex low rank representation

Recently, low rank representation (LRR) has been successfully applied to explore subspace segmentation of data. In this paper, we propose a nonconvex formulation to determine the LRR from contaminated data. Unlike in traditional methods, which directly utilize the nuclear norm to approximate the rank function and penalize noise using the ℓ2,1-norm, our method introduces the Ky Fan p-k-norm and the ℓ2,q-norm, to better approximate the rank minimization problem and enhance the robustness against noise. An efficient algorithm is derived for solving the novel objective function, and this is followed by a rigorous theoretical proof of the convergence. Extensive experiments on face datasets clearly demonstrate that the proposed methods are more robust to illumination variations, corruptions, and occlusions.

Bayesian Low-Rank and Sparse Nonlinear Representation for Manifold Clustering

Linear representation usually used by the optimization model about low-rankness and sparsity limits their applications to some extent. In this paper, we propose Bayesian low-rank and sparse nonlinear representation (BLSN) model exploiting nonlinear representation. Different from the optimization model, BLSN can be solved by traditional algorithm in Bayesian statistics easily without knowing the explicit mapping by kernel trick. Moreover, it can learn the parameters adaptively to choose the low-rank and sparse properties and also provides a way to enforce more properties on one quantity in a Bayesian model. Based on the observation that the data points drawn from a union of manifolds may gain more meaningful linear structure after a nonlinear mapping, we apply BLSN for manifold clustering. It can handle different problems by constructing various kernels. With respect to the case of linear manifold, known as subspace segmentation, we propose a kernel by the Veronese mapping. In addition, we also design the kernel matrices for the case of nonlinear manifold. Experimental results confirm the effectiveness and the potential of our model for manifold clustering.

Subspace segmentation by dense block and sparse representation

Subspace segmentation is a fundamental topic in computer vision and machine learning. However, the success of many popular methods is about independent subspace segmentation instead of the more flexible and realistic disjoint subspace segmentation. Focusing on the disjoint subspaces, we provide theoretical and empirical evidence of inferior performance for popular algorithms such as LRR. To solve these problems, we propose a novel dense block and sparse representation (DBSR) for subspace segmentation and provide related theoretical results. DBSR minimizes a combination of the 1,1-norm and maximum singular value of the representation matrix, leading to a combination of dense block and sparsity. We provide experimental results for synthetic and benchmark data showing that our method can outperform the state-of-the-art.

Dual Graph Regularized Latent Low-Rank Representation for Subspace Clustering.

Low-rank representation (LRR) has received considerable attention in subspace segmentation due to its effectiveness in exploring low-dimensional subspace structures embedded in data. To preserve the intrinsic geometrical structure of data, a graph regularizer has been introduced into LRR framework for learning the locality and similarity information within data. However, it is often the case that not only the high-dimensional data reside on a non-linear low-dimensional manifold in the ambient space, but also their features lie on a manifold in feature space. In this paper, we propose a dual graph regularized LRR model (DGLRR) by enforcing preservation of geometric information in both the ambient space and the feature space. The proposed method aims for simultaneously considering the geometric structures of the data manifold and the feature manifold. Furthermore, we extend the DGLRR model to include non-negative constraint, leading to a parts-based representation of data. Experiments are conducted on several image data sets to demonstrate that the proposed method outperforms the state-of-the-art approaches in image clustering.

Quality point cloud normal estimation by guided least squares representation

In this paper, we present a quality point cloud normal estimation method via subspace segmentation based on guided least squares representation. A structure guided low-rank subspace segmentation model has been employed in normal estimation (LRRSGNE). In order to select a consistent sub-neighborhood for a point, the subspace segmentation model is adopted to analyze the underlying structure of its neighborhood. LRRSGNE generates more faithful normals than previous methods but at the price of a long runtime which may take hours. Following its framework, two improvements are proposed. We first devise a novel least squares representation based subspace segmentation model with structure guiding (LSRSG) and design a numerical algorithm which has a natural parallelism for solving it. It segments subspaces as quality as the low-rank model used in LRRSGNE but with less runtime. We prove that, no matter whether the subspaces are independent or disjoint, it generates a block-diagonal solution which leads to a quality subspace segmentation. To reduce the computational cost of the normal estimation framework further, we develop a subspace structure propagation algorithm. Only parts of the candidate feature points’ neighborhoods are segmented by LSRSG and those of the rest candidate points are inferred via the propagation algorithm which is faster than LSRSG. The experiments exhibit that our method and LRRSGNE generate comparable normals and are more faithful than other state-of-the-art methods. Furthermore, hours of runtime of LRRSGNE is reduced to just minutes.

Latent space robust subspace segmentation based on low-rank and locality constraints

Low-rank representation (LRR) and its extensions have proven to be effective methods to handle different kinds of subspace segmentation applications. In this paper, we propose a new subspace segmentation algorithm, termed latent space robust subspace segmentation based on low-rank and locality constraints (LSRS2). Different from LRR, LSRS2 learns a low-dimensional space and a coefficient matrix for a data set simultaneously. In the obtained latent space, the coefficient matrix can faithfully reveal both the global and local structures for the data set. Furthermore, we build the connections between LSRS2 and robust coding methods, and show LSRS2 can be regarded as a kind of robust LRR method. Therefore, it can be guaranteed in theory that LSRS2 shows good performance. In addition, an efficient optimization method for solving LSRS2 is presented and its convergence is also proven. Extensive experiments show that the proposed algorithm outperforms the related subspace segmentation methods.

Study on Different Representation Methods for Subspace Segmentation

With many engineering and science application problems, we must deal with a lot of high-dimensional data, such as videos, images, web documents, text, etc. In the areas of computer vision, image processing and machine learning, high-dimensional data are widespread. However, it is very hard for obtaining meaningful learning and inference from these high-dimensional data directly, the computational complexity of highdimensional data is often exponential. However, under many conditions, highdimensional data lie in low-dimensional data corresponding to some classes of the data. Thus, finding the low-dimensional structure from the high-dimensional data is very important. The aim of subspace segmentation is to cluster data that lie in a union of lowdimensional subspaces. In recent years, based on the research of representation methods, many subspace segmentation algorithms appeared. Although these methods are all effective for handling subspace segmentation problems, they all have advantages and disadvantages. This paper focuses on the performance comparison of different subspace segmentation algorithms currently used in handling subspace segmentation problems and views other conventional methods that can be applied in this field.

Structure-constrained low-rank representation.

Benefiting from its effectiveness in subspace segmentation, low-rank representation (LRR) and its variations have many applications in computer vision and pattern recognition, such as motion segmentation, image segmentation, saliency detection, and semisupervised learning. It is known that the standard LRR can only work well under the assumption that all the subspaces are independent. However, this assumption cannot be guaranteed in real-world problems. This paper addresses this problem and provides an extension of LRR, named structure-constrained LRR (SC-LRR), to analyze the structure of multiple disjoint subspaces, which is more general for real vision data. We prove that the relationship of multiple linear disjoint subspaces can be exactly revealed by SC-LRR, with a predefined weight matrix. As a nontrivial byproduct, we also illustrate that SC-LRR can be applied for semisupervised learning. The experimental results on different types of vision problems demonstrate the effectiveness of our proposed method.

Sparse fixed-rank representation for robust visual analysis

Robust visual analysis plays an important role in a great variety of computer vision tasks, such as motion segmentation, pose and face analysis. One of the promising real-world applications is to recover the clean data representation from the corrupted data points for subspace segmentation. Recently, low-rank based methods have gained considerable popularity in solving this problem, such as Low-Rank Representation (LRR) and Fixed-Rank Representation (FRR). They both learn a low-rank data matrix and a sparse error matrix. Each new data representation is learnt using the whole dictionary covering all data points. However, they neglect a common fact that each point can be represented by a linear combination of only a few other points w.r.t. a given dictionary, which has been shown in sparse learning. Motivated by this, we explicitly impose the sparsity constraint on the learnt low-rank representation. To be more efficient, we adopt a fixed-rank scheme by minimizing the Frobenius norm of the new representation. Hence, in this paper we propose a novel Sparse Fixed-Rank Representation (SFRR) approach for robust visual analysis. Specifically, we model the corruptions by enforcing a sparse regularizer. This way, we can obtain a new data representation with both low-rankness and sparseness robustly. Furthermore, we present a generalized alternating direction method (ADM) to optimize the objective function. Extensive experiments on both synthetic and real-world databases have suggested the effectiveness and the robustness of the proposed method.

Semi-supervised classification via kernel low-rank representation graph

Sparse Representation based Graphs (SRGs) have attracted increasing interests in very recent years. However, for lacking global constraints on solutions to sparse representation, SRGs cannot accurately reveal data structure when data are grossly corrupted. In this paper, in order to achieve robust classification of wide range of datasets when only a small number of labeled samples are available, we advance a new semi-supervised kernel low-rank representation graph (SKLRG), by combining low-rank representation (LRR) with graphs and kernel trick. A kernel projection is first learned to find high-dimensional space where data have possible low-rank structure. Then a low-rank representation of the projected data is calculated from which we can derive a SKLRG matrix to evaluate data affinity and classify corrupted patterns. The proposed SKLRG can naturally reveal the relationship among data in the projected space, and can capture the global structure of complex data and implements more robust subspace segmentation. Moreover, connected weights of SKLRG are refined by pairwise constrains where label information is explored to further improve the classification results. Some experiments are taken on some benchmark datasets and Synthetic Aperture Radar (SAR) images that are corrupted by speckle noise. The results show that the proposed SKLRG can achieve better performance than its counterparts when there are only a small number of labeled samples.

Gene expression data clustering based on graph regularized subspace segmentation

Gene expression data clustering offers a powerful approach to detect cancers. Specifically, gene expression data clustering based on nonnegative matrix factorization (NMF) has been widely applied to identify tumors. However, traditional NMF methods cannot deal with negative data and easily lead to local optimum because the iterative methods are adopted to solve the optimal problem. To avoid these problems of NMF methods, we propose graph regularized subspace segmentation method (GRSS) for clustering gene expression data. The global optimal solution of GRSS can be achieved by solving a Sylvester equation. Experimental results on eight gene expression data sets show that GRSS has significant performance improvement compared with other subspace segmentation methods, traditional clustering methods and various extensions of NMF.

Fast Low-Rank Subspace Segmentation

Subspace segmentation is the problem of segmenting (or grouping) a set of $n$ data points into a number of clusters, with each cluster being a (linear) subspace. The recently established algorithms such as Sparse Subspace Clustering (SSC), Low-Rank Representation (LRR) and Low-Rank Subspace Segmentation (LRSS) are effective in terms of segmentation accuracy, but computationally inefficient as they possess a complexity of $O(n^{3})$ , which is too high to afford for the case where $n$ is very large. In this paper we devise a fast subspace segmentation algorithm with complexity of $O(n\log (n))$ . This is achieved by firstly using partial Singular Value Decomposition (SVD) to approximate the solution of LRSS, secondly utilizing Locality Sensitive Hashing (LSH) to build a sparse affinity graph that encodes the subspace memberships, and finally adopting a fast Normalized Cut (NCut) algorithm to produce the final segmentation results. Besides of high efficiency, our algorithm also has comparable effectiveness as the original LRSS method.

Reduced row echelon form and non-linear approximation for subspace segmentation and high-dimensional data clustering

Given a set of data W={w1,…,wN}∈RD drawn from a union of subspaces, we focus on determining a nonlinear model of the form U=⋃i∈ISi, where {Si⊂RD}i∈I is a set of subspaces, that is nearest to W. The model is then used to classify W into clusters. Our approach is based on the binary reduced row echelon form of data matrix, combined with an iterative scheme based on a non-linear approximation method. We prove that, in absence of noise, our approach can find the number of subspaces, their dimensions, and an orthonormal basis for each subspace Si. We provide a comprehensive analysis of our theory and determine its limitations and strengths in presence of outliers and noise.

Forward Basis Selection for Pursuing Sparse Representations over a Dictionary

The forward greedy selection algorithm of Frank and Wolfe has recently been applied with success to coordinate-wise sparse learning problems, characterized by a tradeoff between sparsity and accuracy. In this paper, we generalize this method to the setup of pursuing sparse representations over a prefixed dictionary. Our proposed algorithm iteratively selects an atom from the dictionary and minimizes the objective function over the linear combinations of all the selected atoms. The rate of convergence of this greedy selection procedure is analyzed. Furthermore, we extend the algorithm to the setup of learning nonnegative and convex sparse representation over a dictionary. Applications of the proposed algorithms to sparse precision matrix estimation and low-rank subspace segmentation are investigated with efficiency and effectiveness validated on benchmark datasets.