Sparsity-based Techniques Research Articles

Single-image super-resolution via patch-based and group-based local smoothness modeling

Local smoothness and nonlocal self-similarity of natural images are two main priors in the image restoration (IR) problem. Many IR methods have widely used patch-based modeling. Recently, the concept of grouping-based technique, the nonlocal patches with similar structures, has been introduced as the basic unit of sparse representation. In the group-based methods, the nonlocal self-similarity and the local sparsity properties are combined in a unified framework using the sparsity-based techniques. In this paper, a new model is proposed which utilizes both the patch and the group as the basic units of image modeling, called patch-based and group-based local smoothness modeling (PGLSM). More, precisely, in the proposed PGLSM scheme, the local smoothness in the patch-based unit is exploited by an isotropic total variation method and the local smoothness in the group-based unit is exploited by group-based sparse representation method. In this way, a novel technique for high-fidelity single-image super-resolution (SISR) via PGLSM is proposed, called SR-PGLSM. By adding nonlocal means (NLM) as the complementary regularization term to PGLSM, another technique for SISR is modeled, called SR_PGLSM_NLM. In order to efficiently solve the above variational problems, the split Bergman iterative technique has been leveraged. Extensive experimental results validate the effectiveness and robustness of the proposed methods. Our proposed schemes can recover more fine structures and achieve better results than the competing methods with the scaling factor of 2 and 3 and for noisy images both subjectively and objectively in most cases.

Correcting for unknown errors in sparse high-dimensional function approximation

We consider sparsity-based techniques for the approximation of high-dimensional functions from random pointwise evaluations. To date, almost all the works published in this field contain some a priori assumptions about the error corrupting the samples that are hard to verify in practice. In this paper, we instead focus on the scenario where the error is unknown. We study the performance of four sparsity-promoting optimization problems: weighted quadratically-constrained basis pursuit, weighted LASSO, weighted square-root LASSO, and weighted LAD-LASSO. From the theoretical perspective, we prove uniform recovery guarantees for these decoders, deriving recipes for the optimal choice of the respective tuning parameters. On the numerical side, we compare them in the pure function approximation case and in applications to uncertainty quantification of ODEs and PDEs with random inputs. Our main conclusion is that the lesser-known square-root LASSO is better suited for high-dimensional approximation than the other procedures in the case of bounded noise, since it avoids (both theoretically and numerically) the need for parameter tuning.

Joint Sparsity-Based Imaging and Motion Error Estimation for BFSAR

Due to its flexibility and low cost, the bistatic forward-looking synthetic aperture radar (BFSAR) which employs side-looking transmitter and forward-looking receiver has been studied in recent years. Sparsity-based techniques have been applied in the field of BFSAR imaging and show great potential. In sparsity-based BFSAR imaging, compensation of the motion errors is crucial to get a well-focused image. For fields that admit a sparse representation, we propose a sparsity-based imaging approach integrated with motion error estimation and compensation in this paper. First, a novel joint phase-amplitude compensation-based motion error correction scheme is developed to cope with the spatial variance of motion error. Then, an inversion observation model of the range-Doppler algorithm combined with motion error correction is derived, based on which a joint problem of BFSAR imaging and motion error estimation is formulated as a sparse recovery problem and solved in an iterative way, where in each iteration, both image formation and motion error correction are carried out. Experiments on both the simulated and real BFSAR data show that the proposed method can obtain a more accurate estimation result, and generate better focused images compared with the existing methods.

Sensor array calibration with joint-block-sparsity in the presence of multiple separable observations

In sparsity-based optimization problems, one of the major issue is computational complexity, especially when the unknown signal is represented in multi-dimensions such as in the problem of 2-D (azimuth and elevation) direction-of-arrival (DOA) estimation. In this paper, a low-complexity sparsity-based method is proposed for DOA estimation in the presence of array imperfections such as mutual coupling. In order to reduce the complexity of the optimization problem, this paper introduces a new sparsity structure that can be used to model the optimization problem in case of multiple data snapshots and multiple separable observations where the dictionary can be decomposed into two parts: azimuth and elevation dictionaries. The proposed sparsity structure is called joint-block-sparsity which exploits the sparsity in both multiple dimensions, namely azimuth and elevation, and data snapshots. In order to model the joint-block-sparsity in the optimization problem, triple mixed norms are used. In the simulations, the proposed method is compared with both sparsity-based techniques and subspace-based methods as well as the Cramer–Rao lower bound. It is shown that the proposed method effectively calibrates the sensor array with significantly low complexity and sufficient accuracy.

Joint-block-sparsity for efficient 2-D DOA estimation with multiple separable observations

In sparsity-based optimization problems, one of the major issue is computational complexity, especially when the unknown signal is represented in multi-dimensions such as in the problem of 2-D (azimuth and elevation) direction-of-arrival (DOA) estimation. In order to cope with this issue, this paper introduces a new sparsity structure that can be used to model the optimization problem in case of multiple data snapshots and multiple separable observations where the dictionary can be decomposed into two parts: azimuth and elevation dictionaries. The proposed sparsity structure is called joint-block-sparsity which enforces the sparsity in multiple dimensions, namely azimuth, elevation and data snapshots. In order to model the joint-block-sparsity in the optimization problem, triple mixed norms are used. In the simulations, the proposed method is compared with both sparsity-based techniques and subspace-based methods as well as the Cramer–Rao lower bound. It is shown that the proposed method effectively solves the 2-D DOA estimation problem with significantly low complexity and sufficient accuracy.

An Automatic Dictionary Construction Framework for Sparsity-Based Hyperspectral Target Detectors

The mechanisms behind the sparsity-based techniques for hyperspectral target detection and classifications applications are quite similar except the construction methods of the dictionaries used by the algorithms. In hyperspectral image classification, the dictionaries are formed using some known labeled training samples for each class. In contrast, the only a priori target spectrum information is available for the target detection applications. In addition, most of the time, the background materials are unknown in an arbitrary scene. Although some practical approaches such as sliding window exist, their performances are highly unsatisfactory. In order to increase the detection performance of the sparsity-based methods, we propose an automatic dictionary construction framework which is based on a couple of stages consisting of dimension reduction, k-means clustering, connected component labeling via spatial techniques, and spectral methods such as constrained energy minimization filtering. Our experiments show that the proposed approach outperforms the conventional methods, and it is a promising framework for hyperspectral target detection applications.

Removal of Eye Blink Artifacts From EEG Signals Using Sparsity

Neural activities recorded using electroencephalography (EEG) are mostly contaminated with eye blink (EB) artifact. This results in undesired activation of brain-computer interface (BCI) systems. Hence, removal of EB artifact is an important issue in EEG signal analysis. Of late, several artifact removal methods have been reported in the literature and they are based on independent component analysis (ICA), thresholding, wavelet transformation, etc. These methods are computationally expensive and result in information loss which makes them unsuitable for online BCI system development. To address the above problems, we have investigated sparsity-based EB artifact removal methods. Two sparsity-based techniques namely morphological component analysis (MCA) and K-SVD-based artifact removal method have been evaluated in our work. MCA-based algorithm exploits the morphological characteristics of EEG and EB using predefined Dirac and discrete cosine transform (DCT) dictionaries. Next, in K-SVD-based algorithm an overcomplete dictionary is learned from the EEG data itself and is designed to model EB characteristics. To substantiate the efficacy of the two algorithms, we have carried out our experiments with both synthetic and real EEG data. We observe that the K-SVD algorithm, which uses a learned dictionary, delivers superior performance for suppressing EB artifacts when compared to MCA technique. Finally, the results of both the techniques are compared with the recent state-of-the-art FORCe method. We demonstrate that the proposed sparsity-based algorithms perform equal to the state-of-the-art technique. It is shown that without using any computationally expensive algorithms, only with the use of over-complete dictionaries the proposed sparsity-based algorithms eliminate EB artifacts accurately from the EEG signals.

Convolutional Neural Networks for Inverse Problems in Imaging: A Review

In this survey paper, we review recent uses of convolution neural networks (CNNs) to solve inverse problems in imaging. It has recently become feasible to train deep CNNs on large databases of images, and they have shown outstanding performance on object classification and segmentation tasks. Motivated by these successes, researchers have begun to apply CNNs to the resolution of inverse problems such as denoising, deconvolution, super-resolution, and medical image reconstruction, and they have started to report improvements over state-of-the-art methods, including sparsity-based techniques such as compressed sensing. Here, we review the recent experimental work in these areas, with a focus on the critical design decisions: Where does the training data come from? What is the architecture of the CNN? and How is the learning problem formulated and solved? We also bring together a few key theoretical papers that offer perspective on why CNNs are appropriate for inverse problems and point to some next steps in the field.

A comparative study of signal transformation techniques in automated spectral unmixing of infrared spectra for remote sensing applications

ABSTRACTFrom geological and planetary exploration perspectives, automated sub-pixel classification of hyperspectral data is the most difficult task as it involves blind unmixing with library spectra of minerals. In this study, we demonstrate a procedure involving spectral transformation and linear unmixing to achieve the above task. For this purpose, infrared spectra of rocks from the spectral library, field, and remotely sensed hyperspectral image cube were used. Potential spectra of minerals for unmixing rock spectra were drawn from the library based on similarity of absorption features measured using Pearson correlation coefficient. Eight transformation techniques namely, first derivative, fast Fourier transform, discrete wavelet transform, Hilbert–Huang transform, crude low pass filter, S-transform, binary encoding, spectral effective peak matching, and two sparsity-based techniques (orthogonal matching pursuit, sparse unmixing via variable splitting, and augmented Lagrangian) were evaluated. Subsequently, minerals identified by above techniques were unmixed by linear mixture model (LMM) to decipher mineralogical composition and abundance. Results of LMM achieved using fully constrained least-square-estimation-based quadratic programming optimization approach were evaluated by conventional procedures such as X-ray diffraction and microscopy. In the case of image cube, endmembers derived using minimum noise fraction and pixel purity index were subjected to above procedure. It is evident that the discrete-wavelet-transformation-based approach produced excellent and meaningful results due to its flexibility in scaling the data and capability to handle noisy spectra. It is interesting to note that the adopted procedure could perform sub-pixel classification of image cube automatically and identify predominance of dolomite in limestone and sodium in alunite based on subtle differences in absorption positions.

The Race to Improve Radar Imagery: An overview of recent progress in statistical sparsity-based techniques

The exploitation of sparsity has significantly advanced the field of radar imaging over the last few decades, leading to substantial improvements in the resolution and quality of the processed images. More recent developments in compressed sensing (CS) suggest that statistical sparsity can lead to further performance benefits by imposing sparsity as a statistical prior on the considered signal. In this article, a comprehensive survey is made of recent progress on statistical sparsitybased techniques for various radar imagery applications.

Online Sparsifying Transform Learning—Part II: Convergence Analysis

Sparsity-based techniques have been widely popular in signal processing applications such as compression, denoising, and compressed sensing. Recently, the learning of sparsifying transforms for data has received interest. The advantage of the transform model is that it enables cheap and exact computations. In Part I of this work, efficient methods for online learning of square sparsifying transforms were introduced and investigated (by numerical experiments). The online schemes process signals sequentially, and can be especially useful when dealing with big data, and for real-time, or limited latency signal processing applications. In this paper, we prove that although the associated optimization problems are non-convex, the online transform learning algorithms are guaranteed to converge to the set of stationary points of the learning problem. The guarantee relies on a few simple assumptions. In practice, the algorithms work well, as demonstrated by examples of applications to representing and denoising signals.

Joint Sparsity Model for Multilook Hyperspectral Image Unmixing

Recent work on hyperspectral image (HSI) unmixing has addressed the use of overcomplete dictionaries by employing sparse models. In essence, this approach exploits the fact that HSI pixels can be associated with a small number of constituent pure materials. However, unlike traditional least-squares-based methods, sparsity-based techniques do not require a preselection of endmembers and are thus able to simultaneously estimate the underlying active materials along with their respective abundances. In addition, this perspective has been extended so as to exploit the spatial homogeneity of abundance vectors. As a result, these techniques have been reported to provide improved estimation accuracy. In this letter, we present an alternative approach that is able to relax, yet exploit, the assumption of spatial homogeneity by introducing a model that captures both similarities and differences between neighboring abundances. In order to validate this approach, we analyze our model using simulated as well as real hyperspectral data acquired by the HyMap sensor.

Low-complexity sparse reconstruction for high-resolution multi-static passive SAR imaging*

Abstract Bi-static passive synthetic aperture radar (SAR) systems using ground broadcast and wireless network signals suffer from a low spatial resolution due to the narrow bandwidths and low carrier frequencies. By exploiting multiple distributed illuminators, multi-static passive radar has the possibility of producing high-resolution SAR images. In this paper, a two-stage image formation approach, which combines the Fourier transform and sparse reconstruction strategies, is proposed to process multi-static SAR data. This method exploits the group sparsity of the sparse scene, i.e., the observations associated with different bi-static pairs share the same support of the sparse scene but correspond to aspect-dependent scattering coefficients. Such observations are described as a number of generally disjoint sub-bands in the two-dimensional spatial frequency domain. In each sub-band, the sampling satisfies the Nyquist criterion, whereas different sub-bands are sparsely distributed. In the proposed approach, Fourier-based reconstruction is applied to the sub-band data to produce the coarse-resolution images, which are then combined to produce a high-resolution image through the exploitation of sparse reconstruction techniques. The proposed approach greatly improves the imaging quality as compared to Fourier-based reconstruction, whereas it exhibits significant reduction of the computational complexity when compared to direct application of sparse reconstruction techniques. The exploitation of block sparsity-based techniques also permits practical treatment of the angle-dependent target scattering characteristics in SAR image reconstruction. The advantages of the proposed approach are delineated using analysis and simulations.

Near-field acoustic holography using sparse regularization and compressive sampling principles

Regularization of the inverse problem is a complex issue when using near-field acoustic holography (NAH) techniques to identify the vibrating sources. This paper shows that, for convex homogeneous plates with arbitrary boundary conditions, alternative regularization schemes can be developed based on the sparsity of the normal velocity of the plate in a well-designed basis, i.e., the possibility to approximate it as a weighted sum of few elementary basis functions. In particular, these techniques can handle discontinuities of the velocity field at the boundaries, which can be problematic with standard techniques. This comes at the cost of a higher computational complexity to solve the associated optimization problem, though it remains easily tractable with out-of-the-box software. Furthermore, this sparsity framework allows us to take advantage of the concept of compressive sampling; under some conditions on the sampling process (here, the design of a random array, which can be numerically and experimentally validated), it is possible to reconstruct the sparse signals with significantly less measurements (i.e., microphones) than classically required. After introducing the different concepts, this paper presents numerical and experimental results of NAH with two plate geometries, and compares the advantages and limitations of these sparsity-based techniques over standard Tikhonov regularization.