Nonlocal Total Variation Regularization Research Articles

Image Restoration using Nonlocal Regularized Variational Model with Spatially Adapted Regularization Parameter

Restoration of blurred images corrupted with impulse noise has attracted increasing attention due to its great potential toward practical applications. To guarantee the noisy image deblurring performance, total variation (TV), one of the most popular regularizers, has been extensively exploited in the literature. This local regularizer is known for its capability of preserving edges and penalizing oscillations but not sharp discontinuities. However, TV regularized variational models commonly suffer from undesirable staircase-like artifacts in homogeneous regions. The reason behind this phenomenon is that TV regularizer favors solutions that are piecewise constant in numerical experiments. To overcome the model-dependent deficiency, we proposed to replace the commonly used local regularizer by its nonlocal extension, i.e., nonlocal total variation (NLTV) regularizer. By taking advantage of the high degree of self-similarity within images, the proposed nonlocal variational model is able to remove both blurring and impulse noise while preserving meaningful image details. To guarantee solution stability and efficiency, the resulting nonsmooth image restoration problem was effectively dealt with using an alternating optimization algorithm. In addition, to further enhance the image quality, a local variance estimator-based automatic method was introduced to calculate the spatially adapted regularization parameters during image restoration. Extensive experiments have demonstrated that the proposed method could achieve superior imaging performance compared with several state-of-the-art image restoration methods.

A second-order nonlocal regularized variational model for multiframe image super-resolution

Abstract In this paper, we propose a new nonlocal super-resolution (SR) model which is a combination of the nonlocal total variation (TV) regularization and the nonlocal p-Laplacian term (with p = 2). This choice is motivated by the success of the nonlocal TV term in preserving image edges and the efficiency of the nonlocal p-Laplacian term in preserving the image texture. To ensure the convergence of the proposed optimization SR problem, we prove the existence and uniqueness of a solution in a well-posed framework. In addition, to resolve the encountered minimization problem, we proposed a modified primal-dual algorithm and numerical results are also given to show the performance of the proposed approach.

Nonlocal total variation regularization with Shape Adaptive Patches for image denoising via Split Bregman method

Non-Local or patch-based approaches are used in most of the state-of-the-art methods for image denoising. Denoising with regular square patches may cause noise halos around edges, particularly high contrasted edges. In order to overcome this drawback, this work presents an extension of the Non-Local TV framework that effectively exploits the potential local geometric features of the image by replacing the simple square patches with different oriented shapes. We first solve the denoised models of ROF-NLTV with different oriented patch shapes by Split Bregman algorithm. Then, use exponentially weighted aggregation method based on Stein’s unbiased risk estimate to combine the estimators obtained in the first step. The numerical results show our method outperforms some previous ones. Moreover, common noise halos around edges usually observed by denoising with Non-Local TV method are reduced thanks our approach.

Sparse Representation based Super-Resolution of MRI Images with Non-Local Total Variation Regularization

Diffusion-weighted (DW) and spectroscopic MR (MRS) images are found to be very helpful for diagnostic purposes as they provide complementary information to that provided by conventional MRI. These images are also acquired at a faster rate, but with low signal-to-noise ratio. This limitation can be overcome by applying image super-resolution techniques. In this paper, we propose sparse representation over a learned overcomplete dictionary based single-image super-resolution (SISR) technique for DW and MRS images. The proposed SISR method incorporates patch-wise sparsity constraint based on external HR information together with the non-local total variation (NLTV) as internal information to make the regularization problem more robust. Experiments are conducted for both DW and MRS test images and results are compared with some of the recent methods. Results indicate the potential of the proposed method for clinical MRI applications.

Noninvasive electrocardiographic imaging with low-rank and non-local total variation regularization

Abstract The reconstruction of epicardial and endocardial extracellular potentials (EEP) by noninvasive methods has become a significant topic in cardiac electrophysiology over recent years. It is of great importance for the diagnosis of arrhythmia and for guidance of radiofrequency ablation, based on the difference in potentials between different locations on the heart's surface. In this study, we propose a non-local regularization of total variation (TV) in a low-rank (LR) and sparse decomposition framework, suitable for the rank-deficient problem of EEP reconstruction. LR and sparse decomposition can be utilized to extract the spatial-temporal information resulting from the sparse properties of EEP data, and the non-local similarities in the LR part can be a constraint for a non-local total variation regularization. The proposed method is implemented in simulated myocardial infarction (MI), interventional, and clinical premature ventricular contraction (PVC) experiments to verify its feasibility and reliability. Compared with the existing LR and TV methods, the proposed method performs better at potential reconstruction as well as PVC localization, particularly in the boundary of the MI region, while the results of this method are also consistent with those of invasive measurements using an EnSite 3000 system in the clinical experiment.

Analytical Low-Dose CBCT Reconstruction Using Non-local Total Variation Regularization for Image Guided Radiation Therapy.

Purpose: Conventional iterative low-dose CBCT reconstruction techniques are slow and tend to over-smooth edges through uniform weighting of the image penalty gradient. In this study, we present a non-iterative analytical low-dose CBCT reconstruction technique by restoring the noisy low-dose CBCT projection with the non-local total variation (NLTV) method.Methods: We modeled the low-dose CBCT reconstruction as recovering high quality, high-dose CBCT x-ray projections (100 kVp, 1.6 mAs) from low-dose, noisy CBCT x-ray projections (100 kVp, 0.1 mAs). The restoration of CBCT projections was performed using the NLTV regularization method. In NLTV, the x-ray image is optimized by minimizing an energy function that penalizes gray-level difference between pair of pixels between noisy x-ray projection and denoising x-ray projection. After the noisy projection is restored by NLTV regularization, the standard FDK method was applied to generate the final reconstruction output.Results: Significant noise reduction was achieved comparing to original, noisy inputs while maintaining the image quality comparable to the high-dose CBCT projections. The experimental validations show the proposed NLTV algorithm can robustly restore the noise level of x-ray projection images while significantly improving the overall image quality. The improvement in normalized mean square error (NMSE) and peak signal-to-noise ratio (PSNR) measured from the non-local total variation-gradient projection (NLTV-GPSR) algorithm is noticeable compared to that of uncorrected low-dose CBCT images. Moreover, the difference of CNRs from the gains from the proposed algorithm is noticeable and comparable to high-dose CBCT.Conclusion: The proposed method successfully restores noise degraded, low-dose CBCT projections to high-dose projection quality. Such an outcome is a considerable improvement to the reconstruction result compared to the FDK-based method. In addition, a significant reduction in reconstruction time makes the proposed algorithm more attractive. This demonstrates the potential use of the proposed algorithm for clinical practice in radiotherapy.

Nonlocal TV-Gaussian prior for Bayesian inverse problems with applications to limited CT reconstruction

Bayesian inference methods have been widely applied in inverse problems due to the ability of uncertainty characterization of the estimation. The prior distribution of the unknown plays an essential role in the Bayesian inference, and a good prior distribution can significantly improve the inference results. In this paper, we propose a hybrid prior distribution on combining the nonlocal total variation regularization (NLTV) and the Gaussian distribution, namely NLTG prior. The advantage of this hybrid prior is two-fold. The proposed prior models both texture and geometric structures present in images through the NLTV. The Gaussian reference measure also provides a flexibility of incorporating structure information from a reference image. Some theoretical properties are established for the hybrid prior. We apply the proposed prior to limited tomography reconstruction problem that is difficult due to severe data missing. Both maximum a posteriori and conditional mean estimates are computed through two efficient methods and the numerical experiments validate the advantages and feasibility of the proposed NLTG prior.

Hyperspectral Image Denoising With Total Variation Regularization and Nonlocal Low-Rank Tensor Decomposition

Hyperspectral images (HSIs) are normally corrupted by a mixture of various noise types, which degrades the quality of the acquired image and limits the subsequent application. In this article, we propose a novel denoising method for the HSI restoration task by combining nonlocal low-rank tensor decomposition and total variation regularization, which we refer to as TV-NLRTD. To simultaneously capture the nonlocal similarity and high spectral correlation, the HSI is first segmented into overlapping 3-D cubes that are grouped into several clusters by the $k$ -means++ algorithm and exploited by low-rank tensor approximation. Spatial–spectral total variation (SSTV) regularization is then investigated to restore the clean HSI from the denoised overlapping cubes. Meanwhile, the $\ell _{1} $ -norm facilitates the separation of the clean nonlocal low-rank tensor groups and the sparse noise. The proposed TV-NLRTD method is optimized by employing the efficient alternating direction method of multipliers (ADMM) algorithm. The experimental results obtained with both simulated and real hyperspectral data sets confirm the validity and superiority of the proposed method compared with the current state-of-the-art HSI denoising algorithms.

A convex nonlocal total variation regularization algorithm for multiplicative noise removal

This study proposes a nonlocal total variation restoration method to address multiplicative noise removal problems. The strictly convex, objective, nonlocal, total variation effectively utilizes prior information about the multiplicative noise and uses the maximum a posteriori estimator (MAP). An efficient iterative multivariable minimization algorithm is then designed to optimize our proposed model. Finally, we provide a rigorous convergence analysis of the alternating multivariable minimization iteration. The experimental results demonstrate that our proposed model outperforms other currently related models both in terms of evaluation indices and image visual quality.

Nonconvex-sparsity and Nonlocal-smoothness Based Blind Hyperspectral Unmixing.

Blind hyperspectral unmixing (HU), as a crucial technique for hyperspectral data exploitation, aims to decompose mixed pixels into a collection of constituent materials weighted by the corresponding fractional abundances. In recent years, nonnegative matrix factorization (NMF) based methods have become more and more popular for this task and achieved promising performance. Among these methods, two types of properties upon the abundances, namely the sparseness and the structural smoothness, have been explored and shown to be important for blind HU. However, all of previous methods ignores another important insightful property possessed by a natural hyperspectral images (HSI), non-local smoothness, which means that similar patches in a larger region of an HSI are sharing the similar smoothness structure. Based on previous attempts on other tasks, such a prior structure reflects intrinsic configurations underlying a HSI, and is thus expected to largely improve the performance of the investigated HU problem. In this paper, we firstly consider such prior in HSI by encoding it as the nonlocal total variation (NLTV) regularizer. Furthermore, by fully exploring the intrinsic structure of HSI, we generalize NLTV to non-local HSI TV (NLHTV) to make the model more suitable for the bind HU task. By incorporating these two regularizers, together with a non-convex log-sum form regularizer characterizing the sparseness of abundance maps, to the NMF model, we propose novel blind HU models named NLTV/NLHTV and log-sum regularized NMF (NLTV-LSRNMF/NLHTV-LSRNMF), respectively. To solve the proposed models, an efficient algorithm is designed based on alternative optimization strategy (AOS) and alternating direction method of multipliers (ADMM). Extensive experiments conducted on both simulated and real hyperspectral data sets substantiate the superiority of the proposed approach over other competing ones for blind HU task.

Single-image super-resolution via joint statistic models-guided deep auto-encoder network

Recent researches on super-resolution (SR) with deep learning networks have achieved amazing results. However, most of the existing studies neglect the internal distinctiveness of an image and the output of most methods tends to be of blurring, smoothness and implausibility. In this paper, we proposed a unified model which combines the deep model with the image restoration model for single-image SR. This model can not only reconstruct the SR image, but also keep the distinct fine structures for the low-resolution image. Two statistic priors are used to guide the updating of the output of the deep neural network: One is the non-local similarity and the other is the local smoothness. The former is modeled as the non-local total variation regularization, and the latter as the steering kernel regression total variation regularization. For this unified model, a new optimization function is formulated under a regularization framework. To optimize the total variation problem, a novel algorithm based on split Bregman iteration is developed with the theoretical proof of convergence. The experimental results demonstrate that the proposed unified model improves the peak signal-to-noise ratio of the deep SR model. Quantitative and qualitative results on four benchmark datasets show that the proposed model achieves better performance than the deep SR model without regularization terms.

Non-local total variation regularization approach for image restoration under a Poisson degradation

ABSTRACTPoisson noise (also known as shot or photon noise) arises due to the lack of information during the image acquisition phase, it is quite common in the field of microscopic or astronomical imaging applications. In this paper, we propose a non-local total variation regularization framework with a p-norm driven data-fidelity for denoising the Poissonian images. In precise, the energy functional is derived using a Maximum A Posteriori estimator of the Poisson probability density function. The diffusion amounts to a non-local total variation minimization process, which eventually preserves fine structures while denoising the data. The numerical solution is sought under a fast converging split-Bregman iterative scheme. The proposed model is compared visually and statistically with the state-of-the-art Poisson denoising models.

A tensor-based nonlocal total variation model for multi-channel image recovery

Abstract In this paper, a new nonlocal total variation (NLTV) regularizer is proposed for solving the inverse problems in multi-channel image processing. Different from the existing nonlocal total variation regularizers that rely on the graph gradient, the proposed nonlocal total variation involves the standard image gradient and simultaneously exploits three important properties inherent in multi-channel images through a tensor nuclear norm, hence we call this proposed functional as tensor-based nonlocal total variation (TenNLTV). In specific, these three properties are the local structural image regularity, the nonlocal image self-similarity, and the image channel correlation, respectively. By fully utilizing these three properties, TenNLTV can provide a more robust measure of image variation. Then, based on the proposed regularizer TenNLTV, a novel regularization model for inverse imaging problems is presented. Moreover, an effective algorithm is designed for the proposed model, and a closed-form solution is derived for a two-order complex eigen system in our algorithm. Extensive experimental results on several inverse imaging problems demonstrate that the proposed regularizer is systematically superior over other competing local and nonlocal regularization approaches, both quantitatively and visually.

Single Image Dehazing With Depth-aware Non-local Total Variation Regularization.

Single image dehazing can benefit many computer vision applications hence has attracted much more attention in recent years. However, it still remains a challenging task due to its double uncertainty of scene transmission and scene radiance. The existing image dehazing methods usually impair edges in the estimated transmission which leads to halo effects in the dehazing results. Besides, most existing methods suffer from noise and artifacts amplification in dense haze region after dehazing. To address these challenges, we propose a transmission adaptive regularized image recovery method for high quality single image dehazing. An initial transmission map is first obtained by a boundary constraint on the haze model. Then it is refined by applying a non-local total variation (NLTV) regularization to keep depth structures while smoothing excessive details. Noticing that the artifacts amplification effect depends on scene transmission, a transmission adaptive regularized recovery method based on NLTV is proposed to simultaneously suppress visual artifacts and preserve image details in the final dehazing result. An efficient alternating optimization algorithm is also proposed to solve the regularization model. Thorough experimental results demonstrate that the proposed method can effectively suppress visual artifacts for degraded hazy images, and yields high-quality results comparative to the state-of-the-art dehazing methods both quantitatively and qualitatively.

Non-local total variation regularization models for image restoration

Restoration of images corrupted by data-correlated Rayleigh noise distribution has not been studied much extensively in the literature, unlike the other noise distributions. In this paper, we analyze the degradations due to a data-correlated Rayleigh noise and a linear blurring artifact. This work employs a variance stabilization approach and two variational approaches for restoring images from their noisy and blurred observations. The split-Bregman iterative scheme is used for numerically solving the models to improve their convergence rates. Furthermore, non-local total variation and non-local total bounded variation priors are being used as regularizers in these models to improve their restoration efficiency. Various synthetic and real images (such as ultrasound and synthetic aperture radar images) are tested to show the performance of these models.

A new local and nonlocal total variation regularization model for image denoising

The total variation (TV) method for image deblurring is effective for sharpening image details in noisy images although this method tends to over-smooth the image details and inevitably results in staircase effects in smooth areas of the image. The nonlocal total variation (NLTV) method overcomes these drawbacks and retains fine details. However, it is not suitable for detecting similar patches and usually blurs edges in the image. Considering that the TV and NLTV are complementary, we propose a new local and nonlocal total variation (LNLTV) model. In this model, we first decompose the original image into a cartoon component and a detail component, then respectively apply the TV and NLTV to both components. To optimize the hybrid model, the Bregman iteration-based multivariable minimization (BIMM) method and the fast iteration-based multivariable minimization (FIMM) method are respectively employed to minimize the LNLTV energy function. The experimental results clearly demonstrate that the LNLTV model has better performance than some other state-of-the-art models with regard to evaluation indices and visual quality, and the FIMM method has a faster convergence rate and requires less time than the BIMM method.

Hyperspectral Image Super-Resolution via Nonlocal Low-Rank Tensor Approximation and Total Variation Regularization

Hyperspectral image (HSI) possesses three intrinsic characteristics: the global correlation across spectral domain, the nonlocal self-similarity across spatial domain, and the local smooth structure across both spatial and spectral domains. This paper proposes a novel tensor based approach to handle the problem of HSI spatial super-resolution by modeling such three underlying characteristics. Specifically, a noncovex tensor penalty is used to exploit the former two intrinsic characteristics hidden in several 4D tensors formed by nonlocal similar patches within the 3D HSI. In addition, the local smoothness in both spatial and spectral modes of the HSI cube is characterized by a 3D total variation (TV) term. Then, we develop an effective algorithm for solving the resulting optimization by using the local linear approximation (LLA) strategy and the alternative direction method of multipliers (ADMM). A series of experiments are carried out to illustrate the superiority of the proposed approach over some state-of-the-art approaches.

Poisson noise removal based on nonlocal total variation with Euler’s elastica pre-processing

An enhancement-based Poisson denoising method for photon-limited images is presented. The noisy image is firstly pre-processed for enhancing incomplete object information, and then it is denoised while preserving the restored structural details. A variational regularization model based on Euler’s elastica (EE) is proposed for image enhancement pre-processing. A nonlocal total variation (NLTV) regularization model is then employed in the second stage of image denoising. The above two optimization problems are solved by the alternating direction method of multipliers (ADMM). For Poissonian images with low image peak values, experiments demonstrate the validity and efficiency of the proposed method for both restoring geometric structure and removing noise.

Ultra-Low Dose 4D-CBCT Image Reconstruction Via Joint Sparse and Nonlocal Regularization

4D cone-beam CT (4D-CBCT) is becoming an indispensable tool in cancer radiotherapy, image-guided interventions, and many other clinical applications. However, excessive radiation exposure presents a major concern for its widespread clinical applications due to its prolonged scanning duration. While intense efforts have been made in the development of 4D-CBCT under sparse-view projections, few progress has been made to address the noise problem in ultra-low dose 4D-CBCT when the projection data are also subject to photon starvation. The purpose of this work is to establish a joint sparse and nonlocal regularization approach for improved 4D-CBCT image quality with order of magnitude less radiation dose as compared to the existing techniques. To address the under-sampling problem in 4D-CBCT, we adopted the spatiotemporal tensor framelet (STF) regularization model to characterize the spatiotemporal sparse property of patient anatomy by effective use of inter-phase correlation of the projections. To simultaneously tackle the imaging noise problem caused by photon starvation, the spatiotemporal nonlocal total variation (SNTV) regularization model was also incorporated to take into account the spatiotemporal nonlocal self-recursive property of anatomical structures in 4D images. The proposed joint STF-SNTV regularization model in conjunction with the projection data fidelity constraint was optimized via the Bregman operator splitting technique. The proposed approach was evaluated first using dynamic digital phantoms and then using physical experiment data in the low-dose context of both sparse and noisy projections. Root mean squared error (RMSE), signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) were utilized to quantitatively evaluate the reconstructed image quality. With at least one order of magnitude less radiation dose, the presented joint regularization approach achieved improved image quality in terms of noise suppression when comparing with the existing STF regularization approach. Our approach also outperformed the existing SNTV approach in terms of resolution and detail preservation. Simulation studies showed that the improvement over the STF approach is 22.41% in RMSE and 10.07% in SNR; the improvement over the SNTV approach is 51.32% in RMSE and 34.32% in SNR. While for the experimental data, the improvement over the STF is 4.42% in CNR, and the improvement over the SNTV is 14.51% in CNR. All quantitative metrics indicated the superior performance of the proposed method. The presented approach is not only effective in suppressing the view-aliasing artifacts due to angular under-sampling, but also superior in suppressing noise artifacts caused by photon starvation. With improved image quality and reduced imaging dose, our approach represents a significant step forward in 4D-CBCT guided radiation therapy.

Low-dose 4D cone-beam CT via joint spatiotemporal regularization of tensor framelet and nonlocal total variation

Excessive radiation exposure is still a major concern in 4D cone-beam computed tomography (4D-CBCT) due to its prolonged scanning duration. Radiation dose can be effectively reduced by either under-sampling the x-ray projections or reducing the x-ray flux. However, 4D-CBCT reconstruction under such low-dose protocols is prone to image artifacts and noise. In this work, we propose a novel joint regularization-based iterative reconstruction method for low-dose 4D-CBCT. To tackle the under-sampling problem, we employ spatiotemporal tensor framelet (STF) regularization to take advantage of the spatiotemporal coherence of the patient anatomy in 4D images. To simultaneously suppress the image noise caused by photon starvation, we also incorporate spatiotemporal nonlocal total variation (SNTV) regularization to make use of the nonlocal self-recursiveness of anatomical structures in the spatial and temporal domains. Under the joint STF–SNTV regularization, the proposed iterative reconstruction approach is evaluated first using two digital phantoms and then using physical experiment data in the low-dose context of both under-sampled and noisy projections. Compared with existing approaches via either STF or SNTV regularization alone, the presented hybrid approach achieves improved image quality, and is particularly effective for the reconstruction of low-dose 4D-CBCT data that are not only sparse but noisy.