Nonlocal Low-rank Regularization Research Articles

Multispectral and hyperspectral images fusion based on subspace representation and nonlocal low-rank regularization

ABSTRACT Multispectral image (MSI) and hyperspectral image (HSI) fusion is a popular method for HSI super-resolution reconstruction (HSI-SR). MSI-HSI fusion problem is ill-posed and demands several image priors or regularization terms to solve accurately, which is a challenging issue. In this paper, we propose an MSI-HSI fusion model via subspace representation and nonlocal low-rank regularization (SRNLRR). SRNLRR model incorporates the global spectral correlations and spatial nonlocal similarities of HSI to improve the fusion results, where the priors complement each other. First, we use the mode-n tensor-matrix product to project latent high spatial resolution HSI (HR-HSI) into spectral subspace, which can capture spectral low-rankness and reduce computational complexity. Then, based on low-rank representation (LRR) and nonlocal processing strategy, we design a spatial nonlocal LRR regularization (spa-NLRR) and a spectral global LRR regularization (spe-GLRR). These two regularizations analyze the spatial nonlocal similarities and spectral global correlations from intermediate-level vision. Finally, we use the residual regularization program to obtain more image information and input it into the fusion model. We use alternate minimization (AM) methods to optimize the SRNLRR model and employ the alternating direction method (ADM) on spatial/spectral LRR learning. Comparison experiments between the SRNLRR method and six advanced methods on three HSI datasets indicate the superiority of our method.

Deep nonlocal low-rank regularization for complex-domain pixel super-resolution.

Pixel super-resolution (PSR) has emerged as a promising technique to break the sampling limit for phase imaging systems. However, due to the inherent nonconvexity of phase retrieval problem and super-resolution process, PSR algorithms are sensitive to noise, leading to reconstruction quality inevitably deteriorating. Following the plug-and-play framework, we introduce the nonlocal low-rank (NLR) regularization for accurate and robust PSR, achieving a state-of-the-art performance. Inspired by the NLR prior, we further develop the complex-domain nonlocal low-rank network (CNLNet) regularization to perform nonlocal similarity matching and low-rank approximation in the deep feature domain rather than the spatial domain of conventional NLR. Through visual and quantitative comparisons, CNLNet-based reconstruction shows an average 1.4 dB PSNR improvement over conventional NLR, outperforming existing algorithms under various scenarios.

Multiplicative noise removal using complementary total generalized variation and nonlocal low rank regularization

The problem of multiplicative image denoising is a typical ill-posed inverse problem which has been extensively studied in the past few decades and is still an open problem. Exploiting appropriate image priors is critical to solve ill-posed inverse image restoration problem. The existing total generalized variation (TGV) based approaches have achieved significant performance in removing the multiplicative noise. However, these TGV regularized approaches only consider the local smoothness prior of the potential clean image, which limits their ability for image restoration. Other than the local smoothness property, the nonlocal self-similarity prior of images can also be exploited to offer helpful remedy for better image reconstruction. In this paper, to simultaneously make use of both the local smoothness prior and nonlocal self-similarity prior, we integrate the TGV regularizer with the nonlocal low rank regularizer to propose a novel variational model for multiplicative image denoising. Specifically, in the objective function, the nonlocal low rank regularization is expressed by the celebrated weighted nuclear norm minimization (WNNM) which has recently been applied to image denoising and presents state-of-art result. Then, we apply the split Bregman algorithm to efficiently solve the resulting model. Numerical experiments demonstrate that the proposed method achieve superior performance over the competing methods, including the state-of-the-art SAR-BM3D method.

Boosting quantification accuracy of chemical exchange saturation transfer MRI with a spatial-spectral redundancy-based denoising method.

Chemical exchange saturation transfer (CEST) is a versatile technique that enables noninvasive detections of endogenous metabolites present in low concentrations in living tissue. However, CEST imaging suffers from an inherently low signal-to-noise ratio (SNR) due to the decreased water signal caused by the transfer of saturated spins. This limitation challenges the accuracy and reliability of quantification in CEST imaging. In this study, a novel spatial-spectral denoising method, called BOOST (suBspace denoising with nOnlocal lOw-rank constraint and Spectral local-smooThness regularization), was proposed to enhance the SNR of CEST images and boost quantification accuracy. More precisely, our method initially decomposes the noisy CEST images into a low-dimensional subspace by leveraging the global spectral low-rank prior. Subsequently, a spatial nonlocal self-similarity prior is applied to the subspace-based images. Simultaneously, the spectral local-smoothness property of Z-spectra is incorporated by imposing a weighted spectral total variation constraint. The efficiency and robustness of BOOST were validated in various scenarios, including numerical simulations and preclinical and clinical conditions, spanning magnetic field strengths from 3.0 to 11.7 T. The results demonstrated that BOOST outperforms state-of-the-art algorithms in terms of noise elimination. As a cost-effective and widely available post-processing method, BOOST can be easily integrated into existing CEST protocols, consequently promoting accuracy and reliability in detecting subtle CEST effects.

Remote Sensing Image of The Landsat 8–9 Compressive Sensing via Non-Local Low-Rank Regularization with the Laplace Function

Utilizing low-rank prior data in compressed sensing (CS) schemes for Landsat 8-9 remote sensing images (RSIs) has recently received widespread attention. Nevertheless, most CS algorithms focus on the sparsity of an RSI and ignore its low-rank (LR) nature. Therefore, this paper proposes a new CS reconstruction algorithm for Landsat 8-9 remote sensing images based on a non-local optimization framework (NLOF) that is combined with non-convex Laplace functions (NCLF) used for the low-rank approximation (LAA). Since the developed algorithm is based on an approximate low-rank model of the Laplace function, it can adaptively assign different weights to different singular values. Moreover, exploiting the structural sparsity (SS) and low-rank (LR) between the image patches enables the restored image to obtain better CS reconstruction results of Landsat 8-9 RSI than the existing models. For the proposed scheme, first, a CS reconstruction model is proposed using the non-local low-rank regularization (NLLRR) and variational framework. Then, the image patch grouping and Laplace function are used as regularization/penalty terms to constrain the CS reconstruction model. Finally, to effectively solve the rank minimization problem, the alternating direction multiplier method (ADMM) is used to solve the model. Extensive numerical experimental results demonstrate that the non-local variational framework (NLVF) combined with the low-rank approximate regularization (LRAR) method of non-convex Laplace function (NCLF) can obtain better reconstruction results than the more advanced image CS reconstruction algorithms. At the same time, the model preserves the details of Landsat 8-9 RSIs and the boundaries of the transition areas.

Mixed noise removal based on Stokes residual noise removal for division of focal plane polarimetric images

The division of focal plane (DoFP) polarimeter has the advantages of simultaneous imaging and compact optical structure. It is inevitable to introducing the noise in the process of micro-polarizer array integration, polarization image acquisition and transmission. In this paper, we propose a novel mixed noise suppression method based on Stokes residual noise removal (SRNR) for mixed additive white Gaussian noise (AWGN) and impulse noise (IN) removal in DoFP polarimetric images. The Laplacian scale mixture (LSM) model is introduced to estimate the IN and the nonlocal low-rank regularization (NLR) is adopted to enhance the denoising performance. The residual noise of the Stokes parameters is taken into consideration. This crucial processing achieves the acquisition of high-quality polarization data. The experiments demonstrate the superior denoising performance for polarimetric imagery in terms of objective assessment and visual evaluation.

A new nonlocal low-rank regularization method with applications to magnetic resonance image denoising

Magnetic resonance (MR) images are frequently corrupted by Rician noise during image acquisition and transmission. And it is very challenging to restore MR data because Rician noise is signal-dependent. By exploring the nonlocal self-similarity of natural images and further using the low-rank prior of the matrices formed by nonlocal similar patches for 2D data or cubes for 3D data, we propose in this paper a new nonlocal low-rank regularization (NLRR) method including an optimization model and an efficient iterative algorithm to remove Rician noise. The proposed mathematical model consists of a data fidelity term derived from a maximum a posteriori estimation and a NLRR term using the log-det function. The resulting model in terms of approximated patch/cube matrices is non-convex and non-smooth. To solve this model, we propose an alternating reweighted minimization (ARM) algorithm using the Lipschitz-continuity of the gradient of the fidelity term and the concavity of the logarithmic function in the log-det function. The subproblems of the ARM algorithm have closed-form solutions and its limit points are first-order critical points of the problem. The ARM algorithm is further integrated with a two-stage scheme to enhance the denoising performance of the proposed NLRR method. Experimental results tested on 2D and 3D MR data, including simulated and real data, show that the NLRR method outperforms existing state-of-the-art methods for removing Rician noise.

Iterative self-consistent parallel magnetic resonance imaging reconstruction based on nonlocal low-rank regularization

Iterative self-consistent parallel imaging reconstruction (SPIRiT) is an effective self-calibrated reconstruction model for parallel magnetic resonance imaging (PMRI). The joint L1 norm of wavelet or tight frame coefficients and joint total variation (TV) regularization terms are incorporated into the SPIRiT model to improve the reconstruction performance. The simultaneous two-directional low-rankness (STDLR) in k-space data is incorporated into SPIRiT to realize improved reconstruction. Recent methods have exploited the nonlocal self-similarity (NSS) of images by imposing nonlocal low-rankness of similar patches to achieve a superior performance. To fully utilize both the NSS in Magnetic resonance (MR) images and calibration consistency in the k-space domain, we propose a nonlocal low-rank (NLR)-SPIRiT model by incorporating NLR regularization into the SPIRiT model. We apply the weighted nuclear norm (WNN) as a surrogate of the rank and employ the Nash equilibrium (NE) formulation and alternating direction method of multipliers (ADMM) to efficiently solve the NLR-SPIRiT model. The experimental results demonstrate the superior performance of NLR-SPIRiT over the state-of-the-art methods via three objective metrics and visual comparison.

CFAR Detection Based on the Nonlocal Low-Rank and Sparsity-Driven Laplacian Regularization for HFSWR

Target detection in high-frequency surface-wave radar (HFSWR) systems is a challenging task due to various sources of interference. In this article, we propose a robust constant false alarm rate (CFAR) detector to solve the problem of target detection for the HFSWR by exploiting the prior information on the detection environment. Clutter amplitudes in the HFSWR are commonly assumed to follow the Weibull distribution. Additionally, distribution parameters of the clutter have nonlocal self-similarity, and targets have sparsity as well as a certain geometric structure. We design an objective function based on the Bayesian framework, which consists of a data fidelity term and some regularization terms. Specifically, we use the nonlocal low-rank (NLR) regularization to represent the prior information regarding distribution parameters of clutter, and we adopt the combination of the sparsity and Laplacian regularization to represent the prior information regarding targets. The proposed objective function inherits advantages of the NLR, sparsity, and Laplacian regularization, thereby improving the performance of the proposed detector in the nonhomogeneous clutter. By optimizing the proposed objective function, we obtain estimates of clutter statistics for every cell under test and perform the CFAR detection. Experiments on both simulated data and experimental HFSWR data demonstrate that the proposed detector has a good false alarm regulation property and detection performance in the nonhomogeneous clutter.

Nonlocal low-rank regularized two-phase approach for mixed noise removal

Removing mixed noise in images is a difficult problem which has been discussed in many recent papers. In this paper, we tackle the problem of having mixed additive Gaussian white noise and impulse noise. We propose to remove this mixed noise through a nonlocal low-rank regularized two-phase approach. In the first phase, we identify and label the pixels that are likely to be corrupted by the impulse noise. In the second phase, the image is restored through the unlabeled observed data. The restored image is achieved via solving an optimization problem whose objective function has an ℓ 1/ℓ 2 combined content-dependent fidelity term and a nonconvex nonlocal low-rank regularization term. Both terms are built on patch matrices formed from similar patches. Each patch matrix is considered to be low-rank according to the prior knowledge of images. We solve this nonconvex optimization through an iterative adaptive nuclear norm minimization algorithm and provide its convergence analysis. Our experiments show the proposed method outperforms the existing state-of-the-art algorithms in terms of three quantitative metrics, namely, the peak signal-to-noise ratio, the structural similarity and the feature similarity, and visual quality of the restored images.

Image compressive sensing recovery via group residual based nonlocal low-rank regularization

To address the ill-posed nature of image compressive sensing (CS) recovery, it is of great importance to properly exploit image priors. In this paper, we propose a novel group residual based nonlocal low-rank regularization for image CS recovery, namely GRNLR-CS, which performs nonlocal low-rank modeling based on the residual of patch group rather than on the patch group itself as conventional methods. In our GRNLR-CS, the weighted Schatten p-norm is adopted as a non-convex surrogate function to estimate the rank of the group residual. To make our GRNLR-CS method tractable and robust, the split Bregman iteration technique is employed to develop an efficient algorithm to solve the optimization problem of GRNLR-CS. Extensive experimental results demonstrate that our GRNLR-CS method outperforms some state-of-the-art optimization based or deep learning based methods for image CS recovery.

Joint low-rank prior and difference of Gaussian filter for magnetic resonance image denoising.

The low-rank matrix approximation (LRMA) is an efficient image denoising method to reduce additive Gaussian noise. However, the existing low-rank matrix approximation does not perform well in terms of Rician noise removal for magnetic resonance imaging (MRI). To this end, we propose a novel MR image denoising approach based on the extended difference of Gaussian (DoG) filter and nonlocal low-rank regularization. In the proposed method, a novel nonlocal self-similarity evaluation with the tight frame is exploited to improve the patch matching. To remove the Rician noise and preserve the edge details, the extended DoG filter is exploited to the nonlocal low-rank regularization model. The experimental results demonstrate that the proposed method can preserve more edge and fine structures while removing noise in MR image as compared with some of the existing methods.

Nonlocal Low-Rank Regularization Combined with Bilateral Total Variation for Compressive Sensing Image Reconstruction

The use of non-local self-similarity prior between image blocks can improve image reconstruction performance significantly. We propose a compressive sensing image reconstruction algorithm that combines bilateral total variation and nonlocal low-rank regularization to overcome over-smoothing and degradation of edge information which result from the prior reconstructed image. The proposed algorithm makes use of the preservation of image edge information by bilateral total variation operator to enhance the edge details of the reconstructed image. In addition, we use weighted nuclear norm regularization as a low-rank constraint for similar blocks of the image. To solve this convex optimization problem, the Alternating Direction Method of Multipliers (ADMM) is employed to optimize and iterate the algorithm model effectively. Experimental results show that the proposed algorithm can obtain better image reconstruction quality than conventional algorithms with using total variation regularization or considering the nonlocal structure of the image only. At 10% sampling rate, the peak signal-to-noise ratio gain is up to 2.39 dB in noiseless measurements compared with Nonlocal Low-rank Regularization (NLR-CS). Reconstructed image comparison shows that the proposed algorithm retains more high frequency components. In noisy measurements, the proposed algorithm is robust to noise and the reconstructed image retains more detail information.

A mixed model with multi-fidelity terms and nonlocal low rank regularization for natural image noise removal

Reconstructing an original image from its corrupted observation is an important and fundamental problem in many image processing applications. Generally, the L1-norm or L2-norm combined with a regularization term (the total variation (TV), total generalized variation (TGV) or nuclear norm) is used to fit the impulse noise and Gaussian noise, respectively. However, these methods can only be used to remove a single type of noise from images, and traditional regularization terms often have difficulties in capturing some important prior knowledge of images, such as nonlocal self-similarity, low rank and sparsity. To overcome the above issues, we propose a mixed noise removal model with L1-L2 fidelity terms and a popular nonlocal low-rank regularization term, which has been shown to have more effective image denoising performance than traditional regularization methods. To solve this model, the split Bregman iteration method (SBIM) is adopted to decompose the difficult minimization optimization problem into four simple subproblems. Extensive experiments on natural images demonstrate that the effectiveness of the proposed method is better than that of other state-of-the-art methods.

Hyperspectral Image Denoising Based on Nonlocal Low-Rank and TV Regularization

Hyperspectral image (HSI) acquisitions are degraded by various noises, among which additive Gaussian noise may be the worst-case, as suggested by information theory. In this paper, we present a novel tensor-based HSI denoising approach by fully identifying the intrinsic structures of the clean HSI and the noise. Specifically, the HSI is first divided into local overlapping full-band patches (FBPs), then the nonlocal similar patches in each group are unfolded and stacked into a new third order tensor. As this tensor shows a stronger low-rank property than the original degraded HSI, the tensor weighted nuclear norm minimization (TWNNM) on the constructed tensor can effectively separate the low-rank clean HSI patches. In addition, a regularization strategy with spatial–spectral total variation (SSTV) is utilized to ensure the global spatial–spectral smoothness in both spatial and spectral domains. Our method is designed to model the spatial–spectral non-local self-similarity and global spatial–spectral smoothness simultaneously. Experiments conducted on simulated and real datasets show the superiority of the proposed method.

SAR Image Despeckling Based on Combination of Fractional-Order Total Variation and Nonlocal Low Rank Regularization

Regularization method is an effective tool for synthetic aperture radar (SAR) image despeckling. Design of the effective regularization terms describing the image priors plays a vital role in this kind of method. In this article, a new combinational regularization model for speckle reduction (CRM-SR) is proposed, in which a regularization term is elaborately designed to contain both a fractional-order total variation (FrTV) regularization and a nonlocal low rank (NLR) regularization. The new regularization model inherits both the advantages of FrTV and NLR and improves the performance of SAR despeckling and, therefore, better preserves the edges and geometrical features of the images during the despeckling process. An efficient algorithm based on alternating direction optimization is derived to solve the proposed combinational regularization model. Experimental results show that the proposed model can effectively remove SAR image speckle and preserve the geometrical features of images according to both subjective visual assessment of image quality and objective evaluation.

Weighted Nonlocal Low-Rank Tensor Decomposition Method for Sparse Unmixing of Hyperspectral Images

The low spatial resolution of hyperspectral images leads to the coexistence of multiple ground objects in a single pixel (called mixed pixels). A large number of mixed pixels in a hyperspectral image hinders the subsequent analysis and application of the image. In order to solve this problem, a novel sparse unmixing method, which considers highly similar patches in nonlocal regions of a hyperspectral image, is proposed in this article. This method exploits spectral correlation by using collaborative sparsity regularization and spatial information by employing total variation and weighted nonlocal low-rank tensor regularization. To effectively utilize the tensor decomposition, nonlocal similar patches are first grouped together. Then, these nonlocal patches are stacked to form a patch group tensor. Finally, weighted low-rank tensor regularization is enforced to constrain the patch group to obtain an estimated low-rank abundance image. Experiments on simulated and real hyperspectral datasets validated the superiority of the proposed method in better maintaining fine details and obtaining better unmixing results.

Robust Image Compressive Sensing Based on Truncated Cauchy Loss and Nonlocal Low-Rank Regularization

This work presents a novel robust image compressive sensing reconstruction approach. In contrast to the existing work, we employ the truncated Cauchy loss function to measure the errors induced during the measurement, showing strong robustness to impulsive noise and outliers. To ensure high quality reconstructed images, we utilize a non-local low rank regularizer - with truncated Schatten-p norm being the surrogate function of rank - to capture the self-similar property inherent in most natural images. Considering the fact that the whole optimization model is neither convex nor smooth, to solve it effectively, we firstly use the half-quadratic strategy to transform the loss function into a quadratic objective by introducing some auxiliary variables, and then iteratively and alternatively optimize different groups of variables. Extensive experimental results demonstrate its effectiveness in terms of both quantitative indexes of Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM), and visual quality under impulsive noise.

Video deraining via nonlocal low-rank regularization

Outdoor videos captured in rainy weather may be significantly corrupted by the undesired rain streaks, which severely affect the video processing tasks in outdoor computer vision systems. In this paper, we propose a tensor-based video rain streaks removal method using the nonlocal low-rank regularization. Specifically, we first divide videos into overlapped spatial–temporal patches. Then for each patch, we group its nonlocal similar spatial–temporal patches to form a third-order tensor. To model the clean videos, we characterize the wealth redundancy by adopting the tensor nuclear norm to regularize the low-rankness of the third-order tensors formed by similar spatial–temporal patches of clean videos. We also consider the piecewise smoothness and the temporal continuity of clean videos and utilize the unidirectional total variation to enhance the smoothness and continuity. Moreover, as rain streaks are sparse and smooth along the rain direction, we model the rain streaks by employing an ℓ1 norm and the unidirectional total variation penalty to boost the sparsity and directional smoothness, respectively. We develop an efficient alternating direction method of multipliers to solve the proposed model. Experimental results on both synthetic and real rainy videos show that our method outperforms the state-of-the-art methods quantitatively and qualitatively.

SAR Image Despeckling Based on Nonlocal Low-Rank Regularization

In this paper, we propose a new synthetic aperture radar (SAR) image despeckling method based on the nonlocal low-rank minimization model. First, some similar image patches are selected for each pixel to construct the patch group matrix (PGM). Then, a new low-rank minimization model, called Fisher–Tippett distribution (FT)-weighted nuclear norm minimization (WNNM), is proposed to recover the underlying low-rank component from the PGM. Specifically, the FT-WNNM is developed by reformulating the despeckling problem as the maximizing a posterior probability problem. The new model consists of a data fidelity term and a regularization term (also called prior term). The data fidelity term is derived from the statistical distribution of SAR images in the logarithm domain, which is known as the Fisher–Tippett distribution, and the regularization term is the recent weighted nuclear norm. Then, the alternating direction method of multipliers (ADMM) is introduced to solve the corresponding optimization problem. Under ADMM framework, the resulting subproblems can be solved efficiently and the convergence can be guaranteed. Extensive experiments on both simulated and real SAR images demonstrate that the proposed method can achieve comparable or even better despeckling performance than some state-of-the-art despeckling algorithms.