Estimation Of Sparse Coefficients Research Articles

Trust your source: quantifying source condition elements for variational regularisation methods

Abstract Source conditions are a key tool in regularisation theory that are needed to derive error estimates and convergence rates for ill-posed inverse problems. In this paper, we provide a recipe to practically compute source condition elements as the solution of convex minimisation problems that can be solved with first-order algorithms. We demonstrate the validity of our approach by testing it on two inverse problem case studies in machine learning and image processing: sparse coefficient estimation of a polynomial via LASSO regression and recovering an image from a subset of the coefficients of its discrete Fourier transform. We further demonstrate that the proposed approach can easily be modified to solve the machine learning task of identifying the optimal sampling pattern in the Fourier domain for a given image and variational regularisation method, which has applications in the context of sparsity promoting reconstruction from magnetic resonance imaging data.

Sparse Density Estimation with Measurement Errors

This paper aims to estimate an unknown density of the data with measurement errors as a linear combination of functions from a dictionary. The main novelty is the proposal and investigation of the corrected sparse density estimator (CSDE). Inspired by the penalization approach, we propose the weighted Elastic-net penalized minimal -distance method for sparse coefficients estimation, where the adaptive weights come from sharp concentration inequalities. The first-order conditions holding a high probability obtain the optimal weighted tuning parameters. Under local coherence or minimal eigenvalue assumptions, non-asymptotic oracle inequalities are derived. These theoretical results are transposed to obtain the support recovery with a high probability. Some numerical experiments for discrete and continuous distributions confirm the significant improvement obtained by our procedure when compared with other conventional approaches. Finally, the application is performed in a meteorology dataset. It shows that our method has potency and superiority in detecting multi-mode density shapes compared with other conventional approaches.

Hyperspectral image super-resolution through clustering-based sparse representation

Promoting the spatial resolution of hyperspectral sensors is expected to improve computer vision tasks. However, due to the physical limitations of imaging sensors, the hyperspectral image is often of low spatial resolution. In this paper, we propose a new hyperspectral image super-resolution method from a low-resolution (LR) hyperspectral image and a high resolution (HR) multispectral image of the same scene. The reconstruction of HR hyperspectral image is formulated as a joint estimation of the hyperspectral dictionary and the sparse codes based on the spatial-spectral sparsity of the hyperspectral image. The hyperspectral dictionary is learned from the LR hyperspectral image. The sparse codes with respect to the learned dictionary are estimated from LR hyperspectral image and the corresponding HR multispectral image. To improve the accuracy, both spectral dictionary learning and sparse coefficients estimation exploit the spatial correlation of the HR hyperspectral image. Experiments show that the proposed method outperforms several state-of-art hyperspectral image super-resolution methods in objective quality metrics and visual performance.

Multiview Automatic Target Recognition for Infrared Imagery Using Collaborative Sparse Priors

The low resolution of infrared (IR) images makes feature extraction for classification of a challenging work. Learning-based methods, therefore, are preferred to be used on such raw imagery. In this article, in order to avoid difficulties in feature extraction, a novel multitask extension of the widely used sparse-representation-classification (SRC) method is proposed in both single and multiview set-ups. That is, the test sample could be a single IR image or images from different views. In both single-view and multiview scenarios, we try to employ collaborative spike and slab priors. This is because the traditional sparsity-inducing measures such as the $l_{0}$ -row pseudonorm makes it hard to capture the sparse structure of the coefficient matrix when expanded in terms of a training dictionary, and the priors are proved to be able to capture fairly general sparse structures. Furthermore, a joint prior and sparse coefficient estimation method (JPCEM) is proposed for the first time in this article in order to alleviate the need to handpick prior parameters required before classification. Multiple experiments are conducted on a synthetic Comanche Forward Looking IR (FLIR) Automatic Target Recognition (ATR) database collected by Army Research Lab and a challenging mid-wave IR (MWIR) image ATR database made available by the U.S. Army Night Vision and Electronic Sensors Directorate. The final results substantiate the merits of the proposed JPCEM through comparisons with other state-of-the-art methods, including both the ones based on SRC and the ones constructed using deep learning frameworks.

Super-Resolution of Optical Coherence Tomography Images by Scale Mixture Models.

In this paper, a new statistical model is proposed for the single image super-resolution of retinal Optical Coherence Tomography (OCT) images. OCT imaging relies on interfero-metry, which explains why OCT images suffer from a high level of noise. Moreover, data subsampling is carried out during the acquisition of OCT A-scans and B-scans. So, it is necessary to utilize effective super-resolution algorithms to reconstruct high-resolution clean OCT images. In this paper, a nonlocal sparse model-based Bayesian framework is proposed for OCT restoration. For this reason, by characterizing nonlocal patches with similar structures, known as a group, the sparse coefficients of each group of OCT images are modeled by the scale mixture models. In this base, the coefficient vector is decomposed into the point-wise product of a random vector and a positive scaling variable. Estimation of the sparse coefficients depends on the proposed distribution for the random vector and scaling variable where the Laplacian random vector and Generalized Extreme-Value (GEV) scale parameter (Laplacian+GEV model) show the best goodness of fit for each group of OCT images. Finally, a new OCT super-resolution method based on this new scale mixture model is introduced, where the maximum a posterior estimation of both sparse coefficients and scaling variables are calculated efficiently by applying an alternating minimization method. Our experimental results prove that the proposed OCT super-resolution method based on the Laplacian+GEV model outperforms other competing methods in terms of both subjective and objective visual qualities.

Image Denoising Algorithm based on Improved NCSR Model

The Nonlocally Centralized Sparse Representation (NCSR) algorithm aims to reduce sparse coding noise and utilizes the Nonlocal Self-similarity of the image to improve the effectiveness and accuracy of the image denoising algorithm, which has achieved good denoising effect. However, in the local dictionary learning and sparse coefficient estimation, the algorithm directly uses the Euclidean distance of the noise image to measure the similarity, which will have a certain impact on the final denoising effect. Aiming at this deficiency, it is proposed to use the Gaussian Mixed Model (GMM) to train nonlocal self-similar image patch of the external clean image in the local dictionary learning, and use the learned GMM to guide the internal image clustering; when performing the sparse coefficient estimation, the structural similarity index(SSIM) is introduced into the Euclidean distance and the improved Euclidean distance is introduced into the weight calculation. The experimental results show that the improved algorithm proposed in this paper has improved the PSNR and the visual effect of the image.

Image Denoising via Improved Simultaneous Sparse Coding with Laplacian Scale Mixture

Image denoising is a well-studied problem closely related to sparse coding. Noticing that the Laplacian distribution has a strong sparseness, we use Laplacian scale mixture to model sparse coefficients. With the observation that prior information of an image is relevant to the estimation of sparse coefficients, we introduce the prior information into maximum a posteriori (MAP) estimation of sparse coefficients by an appropriate estimate of the probability density function. Extending to structured sparsity, a nonlocal image denoising model: Improved Simultaneous Sparse Coding with Laplacian Scale Mixture (ISSC-LSM) is proposed. The centering preprocessing, which admits biased-mean of sparse coefficients and saves expensive computation, is done firstly. By alternating minimization and learning an orthogonal PCA dictionary, an efficient algorithm with closed-form solutions is proposed. When applied to noise removal, our proposed ISSC-LSM can capture structured image features, and the adoption of image prior information leads to highly competitive denoising performance. Experimental results show that the proposed method often provides higher subjective and objective qualities than other competing approaches. Our method is most suitable for processing images with abundant self-repeating patterns by effectively suppressing undesirable artifacts while maintaining the textures and edges.

3D sound source localization based on coherence-adjusted monopole dictionary and modified convex clustering

In this paper, a sound source localization method for simultaneously estimating both direction-of-arrival (DOA) and distance from the microphone array is proposed. For estimating distance, the off-grid problem must be overcome because the range of distance to be considered is quite broad and even not bounded. The proposed method estimates the positions based on a modified version of the convex clustering method combined with the sparse coefficients estimation. A method for constructing a suitable monopole dictionary based on the coherence is also proposed so that the convex clustering based method can appropriately estimate distance of the sound sources. Numerical and measurement experiments were performed to investigate the performance of the proposed method.

MAP-based image denoising with structured sparsity and Gaussian scale mixture

Image denoising is a classical problem in image processing and is known to be closely related to sparse coding. In this work, based on the key observation that the probability density function (PDF) of image patch is relevant to the maximum a posteriori estimation of sparse coefficients, using an efficient approximation of the PDF of image patch, a nonlocal image denoising method: improved simultaneous sparse coding with Gaussian scale mixture (ISSC-GSM) is proposed. The preprocessing of centering for a collection of similar patches saves expensive computation and admits biased-mean of sparse coefficients. Our formulation can be efficiently computed by alternating minimization, and both subproblems have analytical solutions using the orthogonal PCA dictionary. When applied to noise removal, the proposed ISSC-GSM has achieved highly competitive denoising performance with often higher subjective and objective qualities than other competing approaches. Experimental results have shown that our method often provides the best visual quality by effectively suppressing undesirable artifacts while maintaining the textures and edges, which is most suitable for processing images with abundant self-repeating patterns.

CS-MRI reconstruction via group-based eigenvalue decomposition and estimation

Abstract This paper proposes a novel method for compressed sensing MRI (CS-MRI) reconstruction that combines both the sparse representation and statistical estimation. In this work, the low-rank property is observed and utilized to sparsely represent the similar patches based on group singular value decomposition (SVD), and the linear minimum mean square error (LMMSE) estimation is exploited to perform sparse coefficients estimation. Based on this, the proposed approach is named group-based eigenvalue decomposition and estimation (GEDE). Furthermore, in order to improve the estimation accuracy of the coefficients, the original problem is reformulated into an equivalent noise model and a novel method is proposed to assess the noise variance of similar patches. Extensive experimental results on the MRI data demonstrate that the proposed method outperforms the state-of-the-art reconstruction methods in terms of removing artifacts and reconstruction errors.

Joint MR image super-resolution reconstruction and sparse coefficients estimation

The spatial resolution of magnetic resonance (MR) image is determined by various instrumental limitations and physical considerations. This paper presents a new approach to MR image super-resolution reconstruction. Specifically, we incorporate nonlocal similarity, natural image prior, and sparsity prior into a unified, non-convex minimisation functional that contains both the unknown high-resolution image and the sparse coefficients. We also develop a majorisation-minimisation (MM) approach to solve the target non-convex functional. Experiments on MR image super-resolution reconstruction validate the efficiency of the proposed method.

Simultaneous sparse model selection and coefficient estimation for heavy-tailed autoregressive processes

We propose a sparse coefficient estimation and automated model selection procedure for autoregressive processes with heavy-tailed innovations based on penalized conditional maximum likelihood. Under mild moment conditions on the innovation processes, the penalized conditional maximum likelihood estimator satisfies a strong consistency, OP(N−1/2) consistency, and the oracle properties, where N is the sample size. We have the freedom in choosing penalty functions based on the weak conditions on them. Two penalty functions, least absolute shrinkage and selection operator and smoothly clipped average deviation, are compared. The proposed method provides a distribution-based penalized inference to AR models, which is especially useful when the other estimation methods fail or under perform for AR processes with heavy-tailed innovations [Feigin, Resnick. Pitfalls of fitting autoregressive models for heavy-tailed time series. Extremes. 1999;1:391–422]. A simulation study confirms our theoretical results. At the end, we apply our method to a historical price data of the US Industrial Production Index for consumer goods, and obtain very promising results.

Compressed sensing of superimposed chirps with adaptive dictionary refinement

The compressed sensing (CS) theory shows that accurate signal reconstruction depends on presetting an appropriate signal sparsifying dictionary. For CS of superimposed chirps, this dictionary is typically taken to be a waveform-matched dictionary formed by blindly discretizing the frequency-chirp rate plane. However, since practical target parameters do not lie exactly on gridding points of the assumed dictionary, there is always mismatch between the assumed and the actual sparsifying dictionaries, which will cause the performance of conventional CS reconstruction methods to degrade considerably. To address this, we model the waveform-matched sparsifying dictionary as a parameterized one by treating its sampled frequency-chirp rate grid points as the underlying parameters. As a consequence, the sparsifying dictionary becomes refinable and its refinement can be achieved by optimizing the underlying parameters. Based on this, we develop a novel reconstruction algorithm for CS of superimposed chirps by utilizing the variational expectation-maximization (EM) algorithm. By alternating between steps of sparse coefficients estimation and dictionary parameters optimization, the algorithm integrates the process for dictionary refinement into that of signal reconstruction, and thus can achieve sparse reconstruction and dictionary optimization simultaneously. Experimental results demonstrate that the algorithm effectively deals with the performance degradation incurred by dictionary mismatch, and also outperforms the state-of-the-art CS reconstruction methods both in compressing the signal measurements and in suppressing the measurement noise.