Sparse Subspace Research Articles

Sparsity induced locality preserving projection approaches for dimensionality reduction

We consider the problem of sparse subspace learning for data classification and face recognition. New approaches called lα-regularization-based sparse locality preserving projection (α-SLPP) and structural sparse locality preserving projection (SSLPP) are proposed by incorporating theories of sparse representation and structural sparse regularization into spectral embedding. The proposed methods can efficiently exploit the local geometric information of the data. Also, by inducing sparsity, they facilitate the interpretation of the projection results and the detection of more discriminating features for classification and recognition. In addition, α-SLPP induces sparsity by using non-convex lα-norm regularizer, which is much closer to l0-norm. SSLPP derives a more organized sparse pattern through structural sparse regularization, and thus overcomes the problem that merely decreasing the cardinality may not be enough in certain situations. We formulate the sparse subspace learning problem as feasible optimization problems and present efficient methods to solve them. Experiments in data classification, face recognition, and pixel-corrupted face recognition are carried out to verify the feasibility and effectiveness of the proposed approaches.

Constructing the L2-Graph for Robust Subspace Learning and Subspace Clustering.

Under the framework of graph-based learning, the key to robust subspace clustering and subspace learning is to obtain a good similarity graph that eliminates the effects of errors and retains only connections between the data points from the same subspace (i.e., intrasubspace data points). Recent works achieve good performance by modeling errors into their objective functions to remove the errors from the inputs. However, these approaches face the limitations that the structure of errors should be known prior and a complex convex problem must be solved. In this paper, we present a novel method to eliminate the effects of the errors from the projection space (representation) rather than from the input space. We first prove that l1 -, l2 -, l∞ -, and nuclear-norm-based linear projection spaces share the property of intrasubspace projection dominance, i.e., the coefficients over intrasubspace data points are larger than those over intersubspace data points. Based on this property, we introduce a method to construct a sparse similarity graph, called L2-graph. The subspace clustering and subspace learning algorithms are developed upon L2-graph. We conduct comprehensive experiment on subspace learning, image clustering, and motion segmentation and consider several quantitative benchmarks classification/clustering accuracy, normalized mutual information, and running time. Results show that L2-graph outperforms many state-of-the-art methods in our experiments, including L1-graph, low rank representation (LRR), and latent LRR, least square regression, sparse subspace clustering, and locally linear representation.

Decentralized Robust Subspace Clustering

We consider the problem of subspace clustering using the SSC (Sparse Subspace Clustering) approach, which has several desirable theoretical properties and has been shown to be effective in various computer vision applications.We develop a large scale distributed framework for the computation of SSC via an alternating direction method of multiplier (ADMM) algorithm. The proposed framework solves SSC in column blocks and only involves parallel multivariate Lasso regression subproblems and sample-wise operations. This appealing property allows us to allocate multiple cores/machines for the processing of individual column blocks.We evaluate our algorithm on a shared-memory architecture. Experimental results on real-world datasets confirm that the proposed block-wise ADMM framework is substantially more efficient than its matrix counterpart used by SSC,without sacrificing accuracy. Moreover, our approach is directly applicable to decentralized neighborhood selection for Gaussian graphical models structure estimation.

Sparse time-frequency representation for seismic noise reduction using low-rank and sparse decomposition

Attenuation of random noise is a major concern in seismic data processing. This kind of noise is usually characterized by random oscillation in seismic data over the entire time and frequency. We introduced and evaluated a low-rank and sparse decomposition-based method for seismic random noise attenuation. The proposed method, which is a trace by trace algorithm, starts by transforming the seismic signal into a new sparse subspace using the synchrosqueezing transform. Then, the sparse time-frequency representation (TFR) matrix is decomposed into two parts: (a) a low-rank component and (b) a sparse component using bilateral random projection. Although seismic data are not exactly low-rank in the sparse TFR domain, they can be assumed as being of semi-low-rank or approximately low-rank type. Hence, we can recover the denoised seismic signal by minimizing the mixed [Formula: see text] norms’ objective function by considering the intrinsically semilow-rank property of the seismic data and sparsity feature of random noise in the sparse TFR domain. The proposed method was tested on synthetic and real data. In the synthetic case, the data were contaminated by random noise. Denoising was carried out by means of the [Formula: see text] classical singular spectrum analysis (SSA) and [Formula: see text] deconvolution method for comparison. The [Formula: see text] deconvolution and the classical [Formula: see text] SSA method failed to properly reduce the noise and to recover the desired signal. We have also tested the proposed method on a prestack real data set from an oil field in the southwest of Iran. Through synthetic and real tests, the proposed method is determined to be an effective, amplitude preserving, and robust tool that gives superior results over classical [Formula: see text] SSA as conventional algorithm for denoising seismic data.

LSDT: Latent Sparse Domain Transfer Learning for Visual Adaptation.

We propose a novel reconstruction-based transfer learning method called latent sparse domain transfer (LSDT) for domain adaptation and visual categorization of heterogeneous data. For handling cross-domain distribution mismatch, we advocate reconstructing the target domain data with the combined source and target domain data points based on ℓ1-norm sparse coding. Furthermore, we propose a joint learning model for simultaneous optimization of the sparse coding and the optimal subspace representation. In addition, we generalize the proposed LSDT model into a kernel-based linear/nonlinear basis transformation learning framework for tackling nonlinear subspace shifts in reproduced kernel Hilbert space. The proposed methods have three advantages: 1) the latent space and the reconstruction are jointly learned for pursuit of an optimal subspace transfer; 2) with the theory of sparse subspace clustering, a few valuable source and target data points are formulated to reconstruct the target data with noise (outliers) from source domain removed during domain adaptation, such that the robustness is guaranteed; and 3) a nonlinear projection of some latent space with kernel is easily generalized for dealing with highly nonlinear domain shift (e.g., face poses). Extensive experiments on several benchmark vision data sets demonstrate that the proposed approaches outperform other state-of-the-art representation-based domain adaptation methods.

Discriminant Sparsity Preserving Analysis for Face Recognition

Sparse subspace learning has drawn more and more attentions recently, however, most of them are unsupervised and unsuitable for classification tasks. In this paper, a new discriminant sparsity preserving analysis (DSPA) method by integrating sparse reconstructive weighting into Fisher criterion is proposed for face recognition. We first get sparsity preserving space spanned by the eigenvectors of sparsity preserving projections (SPP). Then, the optimal projection can be obtained by solving an eigenvalue and eigenvector problem of the between-class scatter matrix in sparsity preserving space. The method not only preserves the sparse reconstructive relationship of the data, but also encodes the discriminant information. Extensive experiments on four face image datasets (Yale, ORL, AR and CMU PIE) demonstrate the effectiveness of the proposed DSPA method.

Discriminative sparse subspace learning and its application to unsupervised feature selection

In order to efficiently use the intrinsic data information, in this study a Discriminative Sparse Subspace Learning (DSSL) model has been investigated for unsupervised feature selection. First, the feature selection problem is formulated as a subspace learning problem. In order to efficiently learn the discriminative subspace, we investigate the discriminative information in the subspace learning process. Second, a two-step TDSSL algorithm and a joint modeling JDSSL algorithm are developed to incorporate the clusters׳ assignment as the discriminative information. Then, a convergence analysis of these two algorithms is provided. A kernelized discriminative sparse subspace learning (KDSSL) method is proposed to handle the nonlinear subspace learning problem. Finally, extensive experiments are conducted on real-world datasets to show the superiority of the proposed approaches over several state-of-the-art approaches.

3D human pose regression via robust sparse tensor subspace learning

In this paper, we present a novel algorithm called robust sparse tensor subspace learning (RSTSL) for 3D human pose regression, and it further extends the latest presented tensor learning to a sparse case. A set of interrelated sparse discriminant projection matrices for feature extraction can be obtained by introducing k-mode optimization and elastic-net algorithm into the objective function of RSTSL and each non-zero element in each discriminant projection matrix is selected from the most typical variables. Thus, the most important low-dimensional tensor feature (LDTF) that corresponds to a test image (i.e., high-order tensor) is extracted through sparse projection transformation. Moreover, we present a novel regression model called optimal-order support tensor regression (OOSTR) to build a finest mapping function between LDTF and 3D human pose configuration. Extensive simulations are conducted on two human motion databases, HumanEva and Brown databases, experimental results show that our proposed RSTSL can not only weaken the sensitivity to incoherent human motions caused by transient occlusion of cameras, sudden change in human velocity and low-frame rate but also strengthen the robustness to silhouette ambiguity, obstacle occlusion and random noise. All the results have confirmed that our tracking system achieves the most significant performance against the compared the state-of-the-art approaches, especially in the complicated human motion databases with different clustered backgrounds, human movements, clothing-style, illumination and subjects like HumanEva database.

Noisy sparse subspace clustering

This paper considers the problem of subspace clustering under noise. Specifically, we study the behavior of Sparse Subspace Clustering (SSC) when either adversarial or random noise is added to the ...

Static-Map and Dynamic Object Reconstruction in Outdoor Scenes Using 3-D Motion Segmentation

This letter aims to build the static-map of a dynamic scene using a mobile robot equipped with 3-D sensors. The sought static-map consists of only the static scene parts, which has a vital role in scene understanding and landmark based navigation. Building static-map requires the categorization of moving and static objects. In this work, we propose a Sparse subspace clustering-based motion segmentation method that categories the static scene parts and the multiple moving objects using their 3D motion trajectories. Our motion segmentation method uses the raw trajectory data, allowing the objects to move in direct 3-D space, without any projection model assumption or whatsoever. We also propose a complete pipeline for static-map building, which estimates the interframe motion parameters by exploiting the minimal 3-point random sample consensus algorithm on the feature correspondences only from the static scene parts. The proposed method has been especially designed and tested for large scene in real outdoor environments. On one hand, our 3-D motion segmentation approach outperforms its 2-D-based counterparts, for extensive experiments on KITTI dataset. On the other hand, separately reconstructed static-maps and moving objects for various dynamic scenes are very satisfactory.

Global and local structure preserving sparse subspace learning: An iterative approach to unsupervised feature selection

As we aim at alleviating the curse of high-dimensionality, subspace learning is becoming more popular. Existing approaches use either information about global or local structure of the data, and few studies simultaneously focus on global and local structures as the both of them contain important information. In this paper, we propose a global and local structure preserving sparse subspace learning (GLoSS) model for unsupervised feature selection. The model can simultaneously realize feature selection and subspace learning. In addition, we develop a greedy algorithm to establish a generic combinatorial model, and an iterative strategy based on an accelerated block coordinate descent is used to solve the GLoSS problem. We also provide whole iterate sequence convergence analysis of the proposed iterative algorithm. Extensive experiments are conducted on real-world datasets to show the superiority of the proposed approach over several state-of-the-art unsupervised feature selection approaches.

Multiplexed fluorescence tomography with spectral and temporal data: demixing with intrinsic regularization

We consider the joint use of spectral and temporal data for multiplexed fluorescence molecular tomography to enable high-throughput imaging of multiple fluorescent targets in bulk tissue. This is a challenging problem due to the narrow near-infrared diagnostic window and relatively broad emission spectra of common fluorophores, and the distortion ("redshift") that the fluorophore signals undergo as they propagate through tissue. We show through a Cramér-Rao lower bound analysis that demixing with spectral-temporal data could result in an order of magnitude improvement in performance over either modality alone. To cope with the resulting large data set, we propose a novel two-stage algorithm that decouples the demixing and tomographic reconstruction operations. In this work we concentrate on the demixing stage. We introduce an approach which incorporates ideas from sparse subspace clustering and compressed sensing and does not require a regularization parameter. We report on simulations in which we simultaneously demixed four fluorophores with closely overlapping spectral and temporal profiles in a 25 mm diameter cross-sectional area with a root-mean-square error of less than 3% per fluorophore, as well as on studies of sensitivity of the method to model mismatch.

Ordered Subspace Clustering With Block-Diagonal Priors.

Many application scenarios involve sequential data, but most existing clustering methods do not well utilize the order information embedded in sequential data. In this paper, we study the subspace clustering problem for sequential data and propose a new clustering method, namely ordered sparse clustering with block-diagonal prior (BD-OSC). Instead of using the sparse normalizer in existing sparse subspace clustering methods, a quadratic normalizer for the data sparse representation is adopted to model the correlation among the data sparse coefficients. Additionally, a block-diagonal prior for the spectral clustering affinity matrix is integrated with the model to improve clustering accuracy. To solve the proposed BD-OSC model, which is a complex optimization problem with quadratic normalizer and block-diagonal prior constraint, an efficient algorithm is proposed. We test the proposed clustering method on several types of databases, such as synthetic subspace data set, human face database, video scene clips, motion tracks, and dynamic 3-D face expression sequences. The experiments show that the proposed method outperforms state-of-the-art subspace clustering methods.

Locality Structured Sparsity Preserving Embedding

In recent years, the theory of sparse representation (SR) has been widely exploited in sparse subspace learning (SSL). Among all these methods, SR is a parameter-free global algorithm in nature which is mostly utilized to construct the correlations between samples to avoid some negative effects incurred by k-nearest neighbor (KNN) or some other methods. However, these SSL algorithms always lack obvious discrimination because of the ignorance of samples distribution. Meanwhile, some incorrect correlations are taken into consideration owing to the global feature of SR. To solve these two problems, a new SSL algorithm called locality structured sparsity preserving embedding (LSPE) is proposed in this paper. We add the local structured information to SR and construct correlations between samples. However, LSPE is an unsupervised method which wastes all label information. Therefore, LSPE is extended to semi-supervised LSPE (SLSPE) in this paper. SLSPE not only makes good use of the label information but also enhances the discriminative power of LSPE. Extensive experiments have been performed on three image datasets (CMU, COIL20, ORL) and two UCI datasets (Glass, Segment) to prove the efficiency of the LSPE and SLSPE.

Multispectral image fusion based on joint sparse subspace recovery

The aim of multimodal image fusion is to enhance the perception of a scene by combining prominent features of images captured by different sensors. Using joint sparse subspace recovery (JSSR), this paper proposes an image fusion method. We consider each source image as projecting the original scene into a specified low-dimensional subspace that can be learned by the orthogonal matching pursuit (OMP) algorithm. We then reconstruct the fused image from a union of these subspaces. Considering the high computational complexity of the OMP algorithm, we provide an optimized OMP implementation for a large set of signals on the same dictionary. We evaluate the proposed JSSR fusion method on different spectral images, and compare its performance with the other state-of-the-art methods in terms of visual effect and quantitative fusion evaluation metrics. The experimental results demonstrate that our approach can enhance the visual quality of the fused images.

Minimum Error Entropy Based Sparse Representation for Robust Subspace Clustering

This paper addresses the problem of clustering data points that are approximately drawn from multiple subspaces. Recently a large family of spectral clustering based methods, such as sparse subspace clustering (SSC) and low-rank representation (LRR), have been proposed. In this paper, we present a general formulation to unify many of them within a common framework based on atomic representation. Since mean square error (MSE) relies heavily on the Gaussianity assumption, the previous MSE based subspace clustering methods have the limitation of being sensitive to non-Gaussian noise. In this paper, we develop a novel subspace clustering method, termed MEESSC, by specifying the minimum error entropy (MEE) as the loss function and the sparsity inducing atomic set. We show that MEESSC can well overcome the above limitation. The experimental results on both synthetic and real data verify the effectiveness of the proposed method.

Constrained Multi-View Video Face Clustering.

In this paper, we focus on face clustering in videos. To promote the performance of video clustering by multiple intrinsic cues, i.e., pairwise constraints and multiple views, we propose a constrained multi-view video face clustering method under a unified graph-based model. First, unlike most existing video face clustering methods which only employ these constraints in the clustering step, we strengthen the pairwise constraints through the whole video face clustering framework, both in sparse subspace representation and spectral clustering. In the constrained sparse subspace representation, the sparse representation is forced to explore unknown relationships. In the constrained spectral clustering, the constraints are used to guide for learning more reasonable new representations. Second, our method considers both the video face pairwise constraints as well as the multi-view consistence simultaneously. In particular, the graph regularization enforces the pairwise constraints to be respected and the co-regularization penalizes the disagreement among different graphs of multiple views. Experiments on three real-world video benchmark data sets demonstrate the significant improvements of our method over the state-of-the-art methods.

Robust Visual Tracking via Sparsity-Induced Subspace Learning.

Target representation is a necessary component for a robust tracker. However, during tracking, many complicated factors may make the accumulated errors in the representation significantly large, leading to tracking drift. This paper aims to improve the robustness of target representation to avoid the influence of the accumulated errors, such that the tracker only acquires the information that facilitates tracking and ignores the distractions. We observe that the locally mutual relations between the feature observations of temporally obtained targets are beneficial to the subspace representation in visual tracking. Thus, we propose a novel subspace learning algorithm for visual tracking, which imposes joint row-wise sparsity structure on the target subspace to adaptively exclude distractive information. The sparsity is induced by exploiting the locally mutual relations between the feature observations during learning. To this end, we formulate tracking as a subspace sparsity inducing problem. A large number of experiments on various challenging video sequences demonstrate that our tracker outperforms many other state-of-the-art trackers.

Band Selection Using Improved Sparse Subspace Clustering for Hyperspectral Imagery Classification

An improved sparse subspace clustering (ISSC) method is proposed to select an appropriate band subset for hyperspectral imagery (HSI) classification. The ISSC assumes that band vectors are sampled from a union of low-dimensional orthogonal subspaces and each band can be sparsely represented as a linear or affine combination of other bands within its subspace. First, the ISSC represents band vectors with sparse coefficient vectors by solving the L2-norm optimization problem using the least square regression (LSR) algorithm. The sparse and block diagonal structure of the coefficient matrix from LSR leads to correct segmentation of band vectors. Second, the angular similarity measurement is presented and utilized to construct the similarity matrix. Third, the distribution compactness (DC) plot algorithm is used to estimate an appropriate size of the band subset. Finally, spectral clustering is implemented to segment the similarity matrix and the desired ISSC band subset is found. Four groups of experiments on three widely used HSI datasets are performed to test the performance of ISSC for selecting bands in classification. In addition, the following six state-of-the-art band selection methods are used to make comparisons: linear constrained minimum variance-based band correlation constraint (LCMV-BCC), affinity propagation (AP), spectral information divergence (SID), maximum-variance principal component analysis (MVPCA), sparse representation-based band selection (SpaBS), and sparse nonnegative matrix factorization (SNMF). Experimental results show that the ISSC has the second shortest computational time and also outperforms the other six methods in classification accuracy when using an appropriate band number obtained by the DC plot algorithm.

Reweighted sparse subspace clustering

Motion segmentation and human face clustering are two fundamental problems in computer vision. The state-of-the-art algorithms employ the subspace clustering scheme when processing the two problems. Among these algorithms, sparse subspace clustering (SSC) achieves the state-of-the-art clustering performance via solving a ℓ1 minimization problem and employing the spectral clustering technique for clustering data points into different subspaces. In this paper, we propose an iterative weighting (reweighted) ℓ1 minimization framework which largely improves the performance of the traditional ℓ1 minimization framework. The reweighted ℓ1 minimization framework makes a better approximation to the ℓ0 minimization than tradition ℓ1 minimization framework. Following the reweighted ℓ1 minimization framework, we propose a new subspace clustering algorithm, namely, reweighted sparse subspace clustering (RSSC). Through an extensive evaluation on three benchmark datasets, we demonstrate that the proposed RSSC algorithm significantly reduces the clustering errors over the SSC algorithm while the additional reweighted step has a moderate impact on the computational cost. The proposed RSSC also achieves lowest clustering errors among recently proposed algorithms. On the other hand, as majority of the algorithms were evaluated on the Hopkins155 dataset, which is insufficient of non-rigid motion sequences, the dataset can hardly reflect the ability of the existing algorithms on processing non-rigid motion segmentation. Therefore, we evaluate the performance of the proposed RSSC and state-of-the-art algorithms on the Freiburg-Berkeley Motion Segmentation Dataset, which mainly contains non-rigid motion sequences. The performance of these state-of-the-art algorithms, as well as RSSC, will drop dramatically on this dataset with mostly non-rigid motion sequences. Though the proposed RSSC achieves the better performance than other algorithms, the results suggest that novel algorithms that focus on segmentation of non-rigid motions are still in need.