Iterative Shrinkage Algorithm Research Articles

Restoration of remote sensing images based on nonconvex constrained high-order total variation regularization

Convex total variation (TV) regularization models have been widely used in remote sensing image restoration problems; however, these models tend to produce staircase effects. We consider a nonconvex second-order TV regularization model with linear constraints for remote sensing image restoration. To solve the nonconvex second-order TV regularization model, we propose an efficient alternating minimization algorithm based on generalized iterated shrinkage algorithm and alternating direction method of multipliers. Experimental results demonstrate the effectiveness of the proposed model, which can reduce staircase effects while preserving edges. In terms of signal-to-noise ratio and structural similarity index measure, the experimental results show that our proposed model and algorithm can give better performance compared with some state-of-the-art methods.

Remote Sensing Image Sharpening by Integrating Multispectral Image Super-Resolution and Convolutional Sparse Representation Fusion

In remote sensing, it is quite necessary to fuse spectral information of low-resolution multispectral (LRMS) images and spatial information of panchromatic (PAN) images for obtaining high-resolution multispectral (HRMS) images. In this paper, an effective fusion method integrating multispectral (MS) image super-resolution and convolutional sparse representation (CSR) fusion is proposed to make full use of the spatial information of remote sensing images. First, for enhancing the spatial information of LRMS images with suitable sizes, a fast iterative image super-resolution algorithm based on the learned iterative shrinkage and thresholding algorithm (LISTA) is exploited in the first stage. It employs a feed-forward neural network to simplify the solution of sparse coefficients in the process of super-resolution. In the fusion stage, we propose a CSR-based image fusion framework, in which each MS super-resolution image and PAN image is decomposed into a basic layer and a detail layer, then we fuse the basic layers and the detail layers of the images, respectively. This hierarchical fusion strategy guarantees great performance in detail preservation. The experimental results on QuickBird, WorldView-2, and Landsat ETM+ datasets demonstrate that the proposed method outperforms other methods in terms of both objective evaluation and visual effect.

Estimation With Low-Rank Time–Frequency Synthesis Models

Many state-of-the-art signal decomposition techniques rely on a low-rank factorization of a time-frequency (t-f) transform. In particular, nonnegative matrix factorization (NMF) of the spectrogram has been considered in many audio applications. This is an analysis approach in the sense that the factorization is applied to the squared magnitude of the analysis coefficients returned by the t-f transform. In this paper we instead propose a synthesis approach, where low-rankness is imposed to the synthesis coefficients of the data signal over a given t-f dictionary (such as a Gabor frame). As such we offer a novel modeling paradigm that bridges t-f synthesis modeling and traditional analysis-based NMF approaches. The proposed generative model allows in turn to design more sophisticated multi-layer representations that can efficiently capture diverse forms of structure. Additionally, the generative modeling allows to exploit t-f low-rankness for compressive sensing. We present efficient iterative shrinkage algorithms to perform estimation in the proposed models and illustrate the capabilities of the new modeling paradigm over audio signal processing examples.

Robust Single-Image Super-Resolution Based on Adaptive Edge-Preserving Smoothing Regularization.

Single-image super-resolution (SR) reconstruction via sparse representation has recently attracted broad interest. It is known that a low-resolution (LR) image is susceptible to noise or blur due to the degradation of the observed image, which would lead to a poor SR performance. In this paper, we propose a novel robust edge-preserving smoothing SR (REPS-SR) method in the framework of sparse representation. An EPS regularization term is designed based on gradient-domain-guided filtering to preserve image edges and reduce noise in the reconstructed image. Furthermore, a smoothing-aware factor adaptively determined by the estimation of the noise level of LR images without manual interference is presented to obtain an optimal balance between the data fidelity term and the proposed EPS regularization term. An iterative shrinkage algorithm is used to obtain the SR image results for LR images. The proposed adaptive smoothing-aware scheme makes our method robust to different levels of noise. Experimental results indicate that the proposed method can preserve image edges and reduce noise and outperforms the current state-of-the-art methods for noisy images.

Motion Adaptive Wavelet Thresholding for Recovery of Compressively Sampled Static and Dynamic MR Images

Iterative shrinkage algorithms like parallel coordinate descent and separable surrogate functional use wavelet thresholding with uniform and empirically selected threshold values to recover the under-sampled magnetic resonance (MR) images. In this paper, an adaptive thresholding parameter, for the recovery of static and dynamic MR images, is derived and used in wavelet domain shrinkage. A modified iterative shrinkage thresholding algorithm based on the derived parameter is also proposed. Simulation results show that adaptive wavelet thresholding yields significantly higher signal-to-noise ratio and correlation than the fixed thresholding value. The algorithm based on the adaptive threshold is experimentally tested for static and dynamic MR images with varying acceleration rates, and it has been shown that it outperforms the fixed thresholding value algorithm.

Efficient Multi-Dimensional Tensor Sparse Coding Using t-Linear Combination

In this paper, we propose two novel multi-dimensional tensor sparse coding (MDTSC) schemes using the t-linear combination. Based on the t-linear combination, the shifted versions of the bases are used for the data approximation, but without need to store them. Therefore, the dictionaries of the proposed schemes are more concise and the coefficients have richer physical explanations. Moreover, we propose an efficient alternating minimization algorithm, including the tensor coefficient learning and the tensor dictionary learning, to solve the proposed problems. For the tensor coefficient learning, we design a tensor-based fast iterative shrinkage algorithm. For the tensor dictionary learning, we first divide the problem into several nearly-independent subproblems in the frequency domain, and then utilize the Lagrange dual to further reduce the number of optimization variables. Experimental results on multi-dimensional signals denoising and reconstruction (3DTSC, 4DTSC, 5DTSC) show that the proposed algorithms are more efficient and outperform the state-of-the-art tensor-based sparse coding models.

A New Family of Model-Based Impulsive Wavelets and Their Sparse Representation for Rolling Bearing Fault Diagnosis

The localized faults of rolling bearings can be diagnosed by the extraction of the impulsive feature. However, the approximately periodic impulses may be submerged in strong interferences generated by other components and the background noise. To address this issue, this paper explores a new impulsive feature extraction method based on the sparse representation. According to the vibration model of an impulse generated by the bearing fault, a novel impulsive wavelet is constructed, which satisfies the admissibility condition. As a result, this family of model-based impulsive wavelets can form a Parseval frame. With the model-based impulsive wavelet basis and Fourier basis, a convex optimization problem is formulated to extract the repetitive impulses. Based on the splitting idea, an iterative thresholding shrinkage algorithm is proposed to solve this problem, and it has a fast convergence rate. Via the simulated signal and real vibration signals with bearing fault information, the performance of the proposed approach for repetitive impulsive feature extraction is validated and compared with the noted spectral kurtosis method, the optimized spectral kurtosis method based on simulated annealing, and the resonance-based signal decomposition method. The results demonstrate its advantage and superiority in weak repetitive transient feature extraction.

Adaptively Detecting the Transient Feature of Faulty Wind Turbine Planetary Gearboxes by the Improved Kurtosis and Iterative Thresholding Algorithm

It is significant for diagnosing the early faults of wind turbine planetary gearboxes. However, the weak transient feature generated by the early gear failure is often submerged in other vibration signals and noise, thus this paper discusses a new automatic sparse representation method for detecting weak transients. An improved Morlet wavelet which strictly satisfies the admissibility condition is constructed, and it is highly beneficial to sparsely separating signal component. By the use of the improved Morlet wavelet dictionary and Fourier dictionary, the transient component can be extracted by solving a sparse problem. Aiming at the drawback of the kurtosis and squared envelope spectrum (SES) negentropy in evaluating the similarity between the extracted transient and the original transient, an improved kurtosis index is designed, which can be used to search the optimal sparse parameters. Then, by the use of the improved kurtosis index and iterative thresholding method, an adaptively iterative thresholding shrinkage algorithm is proposed to detect the repetitive transients from the planetary gearbox signal. The comparative results in simulation and experiments show that the proposed method can more effectively and accurately perform the fault diagnosis of wind turbine planetary gearboxes than the infogram method and the resonance-based signal decomposition method.

Lobbes: An Algorithm for Sparse-Spike Deconvolution

This letter proposes an algorithm for solving the sparse-spike deconvolution problem, named Lobbes (Lasso-based binary search for parameter selection). It improves the fast iterative shrinkage and threshold algorithm for Toeplitz-sparse matrix factorization by performing three steps to find a suitable regularization parameter: 1) a normalization procedure over the input data; 2) a binary search step based on the least absolute shrinkage and selection operator; and 3) the elimination of consecutive peaks similar to non-maximum suppression. Such parameter allows us to find a solution with a specified sparsity. We compare our results against the original algorithm and with the known sparse-inducing greedy approach of orthogonal matching pursuit. Relative to state-of-the-art, results demonstrate that Lobbes generates better results: better signal-to-noise ratio of the reconstructed signal and better result for reflectivity peaks. We also derive a new way to measure the quality of the deconvolution.

Group sparsity residual constraint for image denoising with external nonlocal self-similarity prior

Abstract Nonlocal image representation has been successfully used in many image-related inverse problems including denoising, deblurring and deblocking. However, most existing methods only consider the nonlocal self-similarity (NSS) prior of degraded observation image, and few methods use the NSS prior from natural images. In this paper we propose a novel method for image denoising via group sparsity residual constraint with external NSS prior (GSRC-ENSS). Different from the previous NSS prior-based denoising methods, two kinds of NSS prior (e.g., NSS priors of noisy image and natural images) are used for image denoising. In particular, to enhance the performance of image denoising, the group sparsity residual is proposed, and thus the problem of image denoising is translated into reducing the group sparsity residual. Because the groups contain a large amount of NSS information of natural images, to reduce the group sparsity residual, we obtain a good estimation of the group sparse coefficients of the original image by the external NSS prior based on Gaussian Mixture Model (GMM) learning, and the group sparse coefficients of noisy image are used to approximate the estimation. To combine these two NSS priors better, an effective iterative shrinkage algorithm is developed to solve the proposed GSRC-ENSS model. Experimental results demonstrate that the proposed GSRC-ENSS not only outperforms several state-of-the-art methods, but also delivers the best qualitative denoising results with finer details and less ringing artifacts.

A proximal difference-of-convex algorithm with extrapolation

We consider a class of difference-of-convex (DC) optimization problems whose objective is level-bounded and is the sum of a smooth convex function with Lipschitz gradient, a proper closed convex function and a continuous concave function. While this kind of problems can be solved by the classical difference-of-convex algorithm (DCA) [26], the difficulty of the subproblems of this algorithm depends heavily on the choice of DC decomposition. Simpler subproblems can be obtained by using a specific DC decomposition described in [27]. This decomposition has been proposed in numerous work such as [18], and we refer to the resulting DCA as the proximal DCA. Although the subproblems are simpler, the proximal DCA is the same as the proximal gradient algorithm when the concave part of the objective is void, and hence is potentially slow in practice. In this paper, motivated by the extrapolation techniques for accelerating the proximal gradient algorithm in the convex settings, we consider a proximal difference-of-convex algorithm with extrapolation to possibly accelerate the proximal DCA. We show that any cluster point of the sequence generated by our algorithm is a stationary point of the DC optimization problem for a fairly general choice of extrapolation parameters: in particular, the parameters can be chosen as in FISTA with fixed restart [15]. In addition, by assuming the Kurdyka-{\L}ojasiewicz property of the objective and the differentiability of the concave part, we establish global convergence of the sequence generated by our algorithm and analyze its convergence rate. Our numerical experiments on two difference-of-convex regularized least squares models show that our algorithm usually outperforms the proximal DCA and the general iterative shrinkage and thresholding algorithm proposed in [17].

A Novel Iterative Shrinkage Algorithm for CS-MRI via Adaptive Regularization

A new algorithm is proposed for compressed sensing-magnetic resonance imaging (CS-MRI). The $l_p$ -norm $(0 based adaptive regularization model is used for MRI. The algorithm is established by using a novel iterative shrinkage scheme. In the iteration, the quasi-Newton method is employed. In the shrinkage, the threshold is defined varyingly. Also, the parameter $p$ is selected dynamically in the algorithm. Comparing with some certain state-of-the-art methods for the noisy case, the proposed algorithm provides a higher accuracy of the MR image reconstruction. The performance of the proposed algorithm is validated by the theoretical analysis as well as some experimental results.

Patch-Based Image Inpainting via Two-Stage Low Rank Approximation.

To recover the corrupted pixels, traditional inpainting methods based on low-rank priors generally need to solve a convex optimization problem by an iterative singular value shrinkage algorithm. In this paper, we propose a simple method for image inpainting using low rank approximation, which avoids the time-consuming iterative shrinkage. Specifically, if similar patches of a corrupted image are identified and reshaped as vectors, then a patch matrix can be constructed by collecting these similar patch-vectors. Due to its columns being highly linearly correlated, this patch matrix is low-rank. Instead of using an iterative singular value shrinkage scheme, the proposed method utilizes low rank approximation with truncated singular values to derive a closed-form estimate for each patch matrix. Depending upon an observation that there exists a distinct gap in the singular spectrum of patch matrix, the rank of each patch matrix is empirically determined by a heuristic procedure. Inspired by the inpainting algorithms with component decomposition, a two-stage low rank approximation (TSLRA) scheme is designed to recover image structures and refine texture details of corrupted images. Experimental results on various inpainting tasks demonstrate that the proposed method is comparable and even superior to some state-of-the-art inpainting algorithms.

Terahertz Superresolution Stratigraphic Characterization of Multilayered Structures Using Sparse Deconvolution

Terahertz sparse deconvolution based on an iterative shrinkage algorithm is presented in this study to characterize multilayered structures. With an upsampling approach, sparse deconvolution with superresolution is developed to overcome the time resolution limited by the sampling period in the measurement and increase the precision of the estimation of echo arrival times. A simple but effective time-domain model for describing the temporal pulse spreading due to the frequency-dependent loss is also designed and introduced into the algorithm, which greatly improves the performance of sparse deconvolution in processing time-varying pulses during the propagation of terahertz waves in materials. Numerical simulations and experimental measurements verify the algorithms and show that sparse deconvolution can be considered as an effective tool for terahertz nondestructive characterization of multilayered structures.

Near field 3-D imaging approach for joint high-resolution imaging and phase error correction

This paper combines compressed sensing (CS) imaging theory and range migration algorithm (RMA), and then proposes a near-field three-dimensional (3-D) imaging approach for joint high-resolution imaging and phase error correction. Firstly, a sparse measurement matrix construction method based on a logistic sequence is proposed, which conducts nonlinear transformation for the determined logistic sequence, making it obey uniform distribution, then conducts sign function mapping, and generates the pseudorandom sequence with Bernoulli distribution, thus leading to good signal recovery under down-sampling and easy availability for engineering realization. Secondly, in combination with the RMA imaging approach, the dictionary with all scene information and phase error correction is constructed for CS signal recovery and error correction. Finally, the non-quadratic solution model jointing imaging and phase error correction based on regularization is built, and it is solved by two steps - the separable surrogate functionals (SSF) iterative shrinkage algorithm is adopted to realize target scattering estimate; the iteration mode is adopted for the correction of the dictionary model, so as to achieve the goal of error correction and highly-focused imaging. The proposed approach proves to be effective through numerical simulation and real measurement in anechoic chamber. The results show that, the proposed approach can realize high-resolution imaging in the case of less data; the designed measurement matrix has better non-coherence and easy availability for engineering realization. The proposed approach can effectively correct the phase error, and achieve highly-focused target image.

Coupled image restoration model with non‐convex non‐smooth ℓ p wavelet frame and total variation regularisation

In this study, the authors propose a coupled analysis-based image restoration model regularised by total variation(TV) and wavelet frame coefficients penalty terms imposed on non-convex non-smooth lp-norm (0<p<1). The highlighted contributions to this model are: (i) the intrinsic quality of preserving piecewise smooth areas of TV and the amazing sparse representing capability of the wavelet frame to the underlying image alternately interact, will lead to better experimental results; and (ii) the non-convex non-smooth lp-norm (0<p<1) regularisation is more amenable to the marginal distributions of gradients of natural images than l1-norm, which will suppress staircase effects more effectively. By alternative direction method of multipliers, the objective function is first divided into three subproblems that are solved by the fast iterative shrinkage-thresholding algorithm (FISTA) and the generalised iterated shrinkage algorithm (GISA) respectively. The GISA solution is computationally more efficient than a diversity of algorithms such as iteratively reweighted L1( IRL1), iteratively reweighted least squares (IRLS) restricted to solve non-convex non-decreasing function; and the FISTA solution also has a faster convergence rate than iterative shrinkage-thresholding algorithm. The extensive experimental results show that the proposed model exhibit an amazing image restoration capability.

Intensity non-uniformity correction of aerothermal images via ℓp-regularized minimization.

Aerothermal-induced intensity non-uniformity (NU) effects severely influence the effective performance of infrared (IR) imaging systems in high-speed flight. In this paper we propose a ℓp-regularized minimization method to remove intensity NU in IR images. Different from the existing NU correction methods, we consider and study important priors from the NU noise and the IR image, respectively. We assume spatial smoothness of the NU noise and piecewise continuity of the IR image, where the ℓp regularization term is employed in the correction model. A computationally efficient numerical algorithm based on half-quadratic regularization is adopted to solve the optimization problem. To tackle the non-convex ℓp-norm minimization sub-problem in this scheme, a generalized iterated shrinkage algorithm is used. Significant improvement on the image quality is obtained on both simulation and experimental studies. Both quantitative and qualitative comparisons to specialized state-of-the-art algorithms demonstrate its superiority.

Weighted Schatten &lt;inline-formula&gt; &lt;tex-math notation="LaTeX"&gt;$p$ &lt;/tex-math&gt; &lt;/inline-formula&gt;-Norm Minimization for Image Denoising and Background Subtraction

Low rank matrix approximation (LRMA), which aims to recover the underlying low rank matrix from its degraded observation, has a wide range of applications in computer vision. The latest LRMA methods resort to using the nuclear norm minimization (NNM) as a convex relaxation of the nonconvex rank minimization. However, NNM tends to over-shrink the rank components and treats the different rank components equally, limiting its flexibility in practical applications. We propose a more flexible model, namely the Weighted Schatten $p$-Norm Minimization (WSNM), to generalize the NNM to the Schatten $p$-norm minimization with weights assigned to different singular values. The proposed WSNM not only gives better approximation to the original low-rank assumption, but also considers the importance of different rank components. We analyze the solution of WSNM and prove that, under certain weights permutation, WSNM can be equivalently transformed into independent non-convex $l_p$-norm subproblems, whose global optimum can be efficiently solved by generalized iterated shrinkage algorithm. We apply WSNM to typical low-level vision problems, e.g., image denoising and background subtraction. Extensive experimental results show, both qualitatively and quantitatively, that the proposed WSNM can more effectively remove noise, and model complex and dynamic scenes compared with state-of-the-art methods.

Single image super-resolution reconstruction based on genetic algorithm and regularization prior model

Single image super-resolution (SR) reconstruction is an ill-posed inverse problem because the high-resolution (HR) image, obtained from the low-resolution (LR) image, is non-unique or unstable. In this paper, single image SR reconstruction is treated as an optimization problem, and a new single image SR method, based on a genetic algorithm and regularization prior model, is proposed. In the proposed method, the optimization problem is constructed with a regularization prior model which consists of the non-local means (NLMs) filter, total variation (TV) and adaptive sparse domain selection (ASDS) scheme for sparse representation. In order to avoid local optimization, we combine the genetic algorithm and the iterative shrinkage algorithm to deal with the regularization prior model. Compared with several other state-of-the-art algorithms, the proposed method demonstrates better performances in terms of both numerical analysis and visual effect.

Hyperspectral Image Restoration via Iteratively Regularized Weighted Schatten &lt;inline-formula&gt; &lt;tex-math notation="LaTeX"&gt;$p$&lt;/tex-math&gt; &lt;/inline-formula&gt;-Norm Minimization

Hyperspectral images (HSIs) are inevitably corrupted by mixture noise during their acquisition process, in which various kinds of noise, e.g., Gaussian noise, impulse noise, dead lines, and stripes, may exist concurrently. In this paper, mixture noise removal is well illustrated by the task of recovering the low-rank and sparse components of a given matrix, which is constructed by stacking vectorized HSI patches from all the bands at the same position. Instead of applying a traditional nuclear norm, a nonconvex low-rank regularizer, i.e., weighted Schatten p -norm (WSN), is introduced to not only give better approximation to the original low-rank assumption but also to consider the importance of different rank components. The resulted nonconvex low-rank matrix approximation (LRMA) model falls into the applicable scope of an augmented Lagrangian method, and its WSN minimization subproblem can be efficiently solved by generalized iterated shrinkage algorithm. Moreover, the proposed model is integrated into an iterative regularization schema to produce final results, leading to a completed HSI restoration framework. Extensive experimental testing on simulated and real data shows, both qualitatively and quantitatively, that the proposed method has achieved highly competent objective performance compared with several state-of-the-art HSI restoration methods.