Joint Sparse Regularization Research Articles

Pairwise-Constraint-Guided Multi-View Feature Selection by Joint Sparse Regularization and Similarity Learning

Feature selection is a basic and important step in real applications, such as face recognition and image segmentation. In this paper, we propose a new weakly supervised multi-view feature selection method by utilizing pairwise constraints, i.e., the pairwise constraint-guided multi-view feature selection (PCFS for short) method. In this method, linear projections of all views and a consistent similarity graph with pairwise constraints are jointly optimized to learning discriminative projections. Meanwhile, the l2,0-norm-based row sparsity constraint is imposed on the concatenation of projections for discriminative feature selection. Then, an iterative algorithm with theoretically guaranteed convergence is developed for the optimization of PCFS. The performance of the proposed PCFS method was evaluated by comprehensive experiments on six benchmark datasets and applications on cancer clustering. The experimental results demonstrate that PCFS exhibited competitive performance in feature selection in comparison with related models.

Random Manifold Sampling and Joint Sparse Regularization for Multi-Label Feature Selection

Random Manifold Sampling and Joint Sparse Regularization for Multi-Label Feature Selection

A novel joint sparse regularization model to structural damage identification by the generalized fused lasso penalty

The sparse regularization (SReg) model is widely used for structural damage identification by employing the sparsity peculiarity of the structural damage. The conventional SReg model often separately conducts damage identification on each measurement, while ignores the utilization of the similarity information among different measurements. In this paper, we propose a novel joint sparse regularization model for structural damage identification. In detail, combined with the sparsity of the structural damage by the conventional SReg model, our model employs the similarity of different measurements to further improve the damage identification performance, which is realized by the generalized fused lasso penalty. Numerical simulations on the six-bay planar truss structure and experimental studies on the cantilever beam structure illustrate that the damage identification accuracy of our model is promoted 3.59% and 20.75% than the conventional SReg one, respectively.

Random Graph Embedding and Joint Sparse Regularization for Multi-Label Feature Selection

Random Graph Embedding and Joint Sparse Regularization for Multi-Label Feature Selection

Manifold constrained joint sparse learning via non-convex regularization

The traditional robust principal component analysis (RPCA) via decomposition into low-rank plus sparse matrices offers a powerful framework for a large variety of applications in computer vision. However, the reconstructed image experiences serious interference by Gaussian noise, resulting in the degradation of image quality during the denoising process. Thus, a novel manifold constrained joint sparse learning (MCJSL) via non-convex regularization approach is proposed in this paper. Morelly, the manifold constraint is adopted to preserve the local geometric structures and the non-convex joint sparsity is introduced to capture the global row-wise sparse structures. To solve MCJSL, an efficient optimization algorithm using the manifold alternating direction method of multipliers (MADMM) is designed with closed-form solutions and it achieves a fast and convergent procedure. Moreover, the convergence is analyzed mathematically and numerically. Comparisons among the proposed MCJSL and some state-of-the-art solvers, on several accessible datasets, are presented to demonstrate its superiority in image denoising and background subtraction. The results indicate the importance to incorporate the manifold learning and non-convex joint sparse regularization into a general RPCA framework.

Locally Joint Sparse Marginal Embedding for Feature Extraction

Classical linear discriminant analysis (LDA) has the limitation that it requires the within-class scatter matrix to be nonsingular so that it can perform eigen-decomposition to obtain optimal solutions. To break through this limitation, many methods based on LDA have been proposed. However, these methods are either sensitive to outliers or lack joint sparsity for effective feature extraction. To release these problems, this paper proposes a locally joint sparse marginal embedding (LJSME) method. LJSME reconstructs the scatter matrices and utilizes the locality graph to weigh each pair of data, such that it is robust to outliers and able to preserve the neighborhood relationship of the data. Moreover, LJSME can easily avoid the small sample-size problem by a maximum margin criterion and obtain joint sparsity for effective feature extraction by using joint sparse regularization. The comprehensive analysis between the proposed LJSME and the related methods is presented, which indicates the advantages of the proposed method. A series of experiments was conducted to evaluate the performance of LJSME when compared with the state-of-the-art methods. The MATLAB code of LJSME can be downloaded from https://github.com/TungmeeMo/LJSME.git .

Joint Sparse Regularization for Dictionary Learning

As a powerful data representation framework, dictionary learning has emerged in many domains, including machine learning, signal processing, and statistics. Most existing dictionary learning methods use the l0 or l1 norm as regularization to promote sparsity, which neglects the redundant information in dictionary. In this paper, a class of joint sparse regularization is introduced to dictionary learning, leading to a compact dictionary. Unlike previous works which obtain sparse representations independently, we consider all representations in dictionary simultaneously. An efficient iterative solver based on ConCave-Convex Procedure (CCCP) framework and Lagrangian dual is developed to tackle the resulting model. Further, based on the dictionary learning with joint sparse regularization, we consider the multi-layer structure, which can extract the more abstract representation of data. Numerical experiments are conducted on several publicly available datasets. The experimental results demonstrate the effectiveness of joint sparse regularization for dictionary learning.

Robust auto-weighted projective low-rank and sparse recovery for visual representation

Most existing low-rank and sparse representation models cannot preserve the local manifold structures of samples adaptively, or separate the locality preservation from the coding process, which may result in the decreased performance. In this paper, we propose an inductive Robust Auto-weighted Low-Rank and Sparse Representation (RALSR) framework by joint feature embedding for the salient feature extraction of high-dimensional data. Technically, the model of our RALSR seamlessly integrates the joint low-rank and sparse recovery with robust salient feature extraction. Specifically, RALSR integrates the adaptive locality preserving weighting, joint low-rank/sparse representation and the robustness-promoting representation into a unified model. For accurate similarity measure, RALSR computes the adaptive weights by minimizing the joint reconstruction errors over the recovered clean data and salient features simultaneously, where L1-norm is also applied to ensure the sparse properties of learnt weights. The joint minimization can also potentially enable the weight matrix to have the power to remove noise and unfavorable features by reconstruction adaptively. The underlying projection is encoded by a joint low-rank and sparse regularization, which can ensure it to be powerful for salient feature extraction. Thus, the calculated low-rank sparse features of high-dimensional data would be more accurate for the subsequent classification. Visual and numerical comparison results demonstrate the effectiveness of our RALSR for data representation and classification.

Spatial-aware hyperspectral image classification via multifeature kernel dictionary learning

Sparse representation based on dictionary learning has yielded impressive effects on hyperspectral image (HSI) classification. But most of these methods utilize only the single spectral feature of HSI and advanced features are not considered, such that the discriminability of sparse representation coefficients is relatively weak. In this paper, we propose a novel multifeature spatial aware dictionary learning model by incorporating complementary across-feature and contextual information obtaining from HSI. The newly developed model, by designing a joint sparse regularization term for pixels represented by several complementary yet correlated features in a contextual group, makes the learning sparse coefficients have enough discriminability. Also, in order to further improve the discrimination ability of coding coefficients, utilizing kernel trick, we design the corresponding kernel extension of the newly proposed model. Based on the newly presented models, we give two corresponding discriminant dictionary learning algorithms. The experimental results on Indian Pines and University of Pavia images show that the effectiveness of the proposed algorithms, which also validate that our algorithms can obtain more discriminant coding coefficients.

Ultra-Low Dose 4D-CBCT Image Reconstruction Via Joint Sparse and Nonlocal Regularization

4D cone-beam CT (4D-CBCT) is becoming an indispensable tool in cancer radiotherapy, image-guided interventions, and many other clinical applications. However, excessive radiation exposure presents a major concern for its widespread clinical applications due to its prolonged scanning duration. While intense efforts have been made in the development of 4D-CBCT under sparse-view projections, few progress has been made to address the noise problem in ultra-low dose 4D-CBCT when the projection data are also subject to photon starvation. The purpose of this work is to establish a joint sparse and nonlocal regularization approach for improved 4D-CBCT image quality with order of magnitude less radiation dose as compared to the existing techniques. To address the under-sampling problem in 4D-CBCT, we adopted the spatiotemporal tensor framelet (STF) regularization model to characterize the spatiotemporal sparse property of patient anatomy by effective use of inter-phase correlation of the projections. To simultaneously tackle the imaging noise problem caused by photon starvation, the spatiotemporal nonlocal total variation (SNTV) regularization model was also incorporated to take into account the spatiotemporal nonlocal self-recursive property of anatomical structures in 4D images. The proposed joint STF-SNTV regularization model in conjunction with the projection data fidelity constraint was optimized via the Bregman operator splitting technique. The proposed approach was evaluated first using dynamic digital phantoms and then using physical experiment data in the low-dose context of both sparse and noisy projections. Root mean squared error (RMSE), signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) were utilized to quantitatively evaluate the reconstructed image quality. With at least one order of magnitude less radiation dose, the presented joint regularization approach achieved improved image quality in terms of noise suppression when comparing with the existing STF regularization approach. Our approach also outperformed the existing SNTV approach in terms of resolution and detail preservation. Simulation studies showed that the improvement over the STF approach is 22.41% in RMSE and 10.07% in SNR; the improvement over the SNTV approach is 51.32% in RMSE and 34.32% in SNR. While for the experimental data, the improvement over the STF is 4.42% in CNR, and the improvement over the SNTV is 14.51% in CNR. All quantitative metrics indicated the superior performance of the proposed method. The presented approach is not only effective in suppressing the view-aliasing artifacts due to angular under-sampling, but also superior in suppressing noise artifacts caused by photon starvation. With improved image quality and reduced imaging dose, our approach represents a significant step forward in 4D-CBCT guided radiation therapy.

Spectral–Spatial Classification of Hyperspectral Images With a Superpixel-Based Discriminative Sparse Model

A novel superpixel-based discriminative sparse model (SBDSM) for spectral-spatial classification of hyperspectral images (HSIs) is proposed. Here, a superpixel in a HSI is considered as a small spatial region whose size and shape can be adaptively adjusted for different spatial structures. In the proposed approach, the SBDSM first clusters the HSI into many superpixels using an efficient oversegmentation method. Then, pixels within each superpixel are jointly represented by a set of common atoms from a dictionary via a joint sparse regularization. The recovered sparse coefficients are utilized to determine the class label of the superpixel. In addition, instead of directly using a large number of sampled pixels as dictionary atoms, the SBDSM applies a discriminative K-SVD learning algorithm to simultaneously train a compact representation dictionary, as well as a discriminative classifier. Furthermore, by utilizing the class label information of training pixels and dictionary atoms, a class-labeled orthogonal matching pursuit is proposed to accelerate the K-SVD algorithm while still enforcing high discriminability on sparse coefficients when training the classifier. Experimental results on four real HSI datasets demonstrate the superiority of the proposed SBDSM algorithm over several well-known classification approaches in terms of both classification accuracies and computational speed.

Information integration for ground-based cloud classification using joint consistent sparse coding in heterogeneous sensor network

Although sparsity-based algorithm has emerged as an extremely powerful tool for information integration, it neglects the relationship of heterogeneous features and coding coefficients from the same class in the training stage, which may cause declining of the classification performance. In this paper, we focus on information integration for ground-based cloud classification in heterogeneous sensor network (HSN), and propose a novel coding strategy named joint consistent sparse coding (JCSC) to overcome the drawbacks of traditional sparse coding. In order to integrate information effectively, we add a joint sparse regularization to learn the relationship of heterogeneous features. Moreover, we utilize the consistent constraint on coding coefficients. In this way, coding coefficients from the same class can be forced to their mean vector, and therefore they are more compact and discriminative. The experimental results demonstrate that our method achieves better performance than the state-of-the-art methods.

Joint sparse regularization based Sparse Semi-Supervised Extreme Learning Machine (S3ELM) for classification

Extreme Learning Machine (ELM) has received increasing attention for its simple principle, low computational cost and excellent performance. However, a large number of labeled instances are often required, and the number of hidden nodes should be manually tuned, for better learning and generalization of ELM. In this paper, we propose a Sparse Semi-Supervised Extreme Learning Machine (S3ELM) via joint sparse regularization for classification, which can automatically prune the model structure via joint sparse regularization technology, to achieve more accurate, efficient and robust classification, when only a small number of labeled training samples are available. Different with most of greedy-algorithms based model selection approaches, by using ℓ2,1-norm, S3ELM casts a joint sparse constraints on the training model of ELM and formulate a convex programming. Moreover, with a Laplacian, S3ELM can make full use of the information from both the labeled and unlabeled samples. Some experiments are taken on several benchmark datasets, and the results show that S3ELM is computationally attractive and outperforms its counterparts.

Object Tracking via Robust Multitask Sparse Representation

Sparse representation has been applied to the object tracking problem. Mining the self-similarities between particles via multitask learning can improve tracking performance. However, some particles may be different from others when they are sampled from a large region. Imposing all particles share the same structure may degrade the results. To overcome this problem, we propose a tracking algorithm based on robust multitask sparse representation (RMTT) in this letter. When we learn the particle representations, we decompose the sparse coefficient matrix into two parts in our algorithm. Joint sparse regularization is imposed on one coefficient matrix while element-wise sparse regularization is imposed on another matrix. The former regularization exploits self-similarities of particles while the later one considers the differences between them. Experiments on the benchmark data show the superior performance over other state-of-art algorithms.

Robust non-negative matrix factorization via joint sparse and graph regularization for transfer learning

In real-world applications, we often have to deal with some high-dimensional, sparse, noisy, and non-independent identically distributed data. In this paper, we aim to handle this kind of complex data in a transfer learning framework, and propose a robust non-negative matrix factorization via joint sparse and graph regularization model for transfer learning. First, we employ robust non-negative matrix factorization via sparse regularization model (RSNMF) to handle source domain data and then learn a meaningful matrix, which contains much common information between source domain and target domain data. Second, we treat this learned matrix as a bridge and transfer it to target domain. Target domain data are reconstructed by our robust non-negative matrix factorization via joint sparse and graph regularization model (RSGNMF). Third, we employ feature selection technique on new sparse represented target data. Fourth, we provide novel efficient iterative algorithms for RSNMF model and RSGNMF model and also give rigorous convergence and correctness analysis separately. Finally, experimental results on both text and image data sets demonstrate that our REGTL model outperforms existing start-of-art methods.