Regularization By Denoising Research Articles

Self-Supervised Deep Unrolled Reconstruction Using Regularization by Denoising.

Deep learning methods have been successfully used in various computer vision tasks. Inspired by that success, deep learning has been explored in magnetic resonance imaging (MRI) reconstruction. In particular, integrating deep learning and model-based optimization methods has shown considerable advantages. However, a large amount of labeled training data is typically needed for high reconstruction quality, which is challenging for some MRI applications. In this paper, we propose a novel reconstruction method, named DURED-Net, that enables interpretable self-supervised learning for MR image reconstruction by combining a self-supervised denoising network and a plug-and-play method. We aim to boost the reconstruction performance of Noise2Noise in MR reconstruction by adding an explicit prior that utilizes imaging physics. Specifically, the leverage of a denoising network for MRI reconstruction is achieved using Regularization by Denoising (RED). Experiment results demonstrate that the proposed method requires a reduced amount of training data to achieve high reconstruction quality among the state-of-the-art approaches utilizing Noise2Noise.

Integrating Learning-based Priors with Physics-based Models in Ultrasound Elasticity Reconstruction.

Ultrasound elastography images which enable quantitative visualization of tissue stiffness can be reconstructed by solving an inverse problem. Classical model-based methods are usually formulated in terms of constrained optimization problems. To stabilize the elasticity reconstructions, regularization techniques such as Tikhonov method are used with the cost of promoting smoothness and blurriness in the reconstructed images. Thus, incorporating a suitable regularizer is essential for reducing the elasticity reconstruction artifacts while finding the most suitable one is challenging. In this work, we present a new statistical representation of the physical imaging model which incorporates effective signal-dependent colored noise modeling. Moreover, we develop a learning-based integrated statistical framework which combines a physical model with learning-based priors. We use a dataset of simulated phantoms with various elasticity distributions and geometric patterns to train a denoising regularizer as the learning-based prior. We use fixed-point approaches and variants of gradient descent for solving the integrated optimization task following learning-based plug-and-play (PnP) prior and regularization by denoising (RED) paradigms. Finally, we evaluate the performance of the proposed approaches in terms of relative mean square error (RMSE) with nearly 20% improvement for both piece-wise smooth simulated phantoms and experimental phantoms compared to the classical model-based methods and 12% improvement for both spatially-varying breast-mimicking simulated phantoms and an experimental breast phantom, demonstrating the potential clinical relevance of our work. Moreover, the qualitative comparisons of reconstructed images demonstrate the robust performance of the proposed methods even for complex elasticity structures that might be encountered in clinical settings.

Blind Restoration of a Single Real Turbulence-Degraded Image Based on Self-Supervised Learning

Turbulence-degraded image frames are distorted by both turbulent deformations and space–time varying blurs. Restoration of the atmospheric turbulence-degraded image is of great importance in the state of affairs, such as remoting sensing, surveillance, traffic control, and astronomy. While traditional supervised learning uses lots of simulated distorted images for training, it has poor generalization ability for real degraded images. To address this problem, a novel blind restoration network that only inputs a single turbulence-degraded image is presented, which is mainly used to reconstruct the real atmospheric turbulence distorted images. In addition, the proposed method does not require pre-training, and only needs to input a single real turbulent degradation image to output a high-quality result. Meanwhile, to improve the self-supervised restoration effect, Regularization by Denoising (RED) is introduced to the network, and the final output is obtained by averaging the prediction of multiple iterations in the trained model. Experiments are carried out with real-world turbulence-degraded data by implementing the proposed method and four reported methods, and we use four non-reference indicators for evaluation, among which Average Gradient, NIQE, and BRISQUE have achieved state-of-the-art effects compared with other methods. As a result, our method is effective in alleviating distortions and blur, restoring image details, and enhancing visual quality. Furthermore, the proposed approach has a certain degree of generalization, and has an excellent restoration effect for motion-blurred images.

Untrained deep network powered with explicit denoiser for phase recovery in inline holography

Single-shot reconstruction of the inline hologram is highly desirable as a cost-effective and portable imaging modality in resource-constrained environments. However, the twin image artifacts, caused by the propagation of the conjugated wavefront with missing phase information, contaminate the reconstruction. Existing end-to-end deep learning-based methods require massive training data pairs with environmental and system stability, which is very difficult to achieve. Recently proposed deep image prior (DIP) integrates the physical model of hologram formation into deep neural networks without any prior training requirement. However, the process of fitting the model output to a single measured hologram results in the fitting of interference-related noise. To overcome this problem, we have implemented an untrained deep neural network powered with explicit regularization by denoising (RED), which removes twin images and noise in reconstruction. Our work demonstrates the use of alternating directions of multipliers method (ADMM) to combine DIP and RED into a robust single-shot phase recovery process. The use of ADMM, which is based on the variable splitting approach, made it possible to plug and play different denoisers without the need of explicit differentiation. Experimental results show that the sparsity-promoting denoisers give better results over DIP in terms of phase signal-to-noise ratio (SNR). Considering the computational complexities, we conclude that the total variation denoiser is more appropriate for hologram reconstruction.

Fourier Ptychography Reconstruction Based on Reweighted Amplitude Flow With Regularization by Denoising and Deep Decoder

Fourier ptychography (FP) is a computational imaging technique with the advantage that it can obtain large field-of-view (FOV) and high-resolution (HR) imaging. We propose an algorithm for Fourier ptychography based on reweighted amplitude flow (RAF) with regularization by denoising (RED) and deep decoder (DD), which is an untrained deep generative model. The proposed method includes two loops, using reweighted amplitude flow with regularization by denoising as an inner loop for phase retrieval and deep decoder for further denoising as an outer loop in the Fourier ptychography recovery system. The proposed method does not need any training dataset, just adds a little computer time during the image recovery process. The proposed method has no bias due to training images, which is different from other deep learning methods. The experimental results show that the proposed method can improve the reconstruction quality in both PSNR and SSIM.

Unsupervised Bayesian PET Reconstruction

Positron emission tomography (PET) reconstruction has become an ill-posed inverse problem due to low-count projection data, and a robust algorithm is urgently required to improve imaging quality. Recently, the deep image prior (DIP) has drawn much attention and has been successfully applied in several image restoration tasks, such as denoising and inpainting, since it does not need any labels (reference image). However, overfitting is a vital defect of this framework. Hence, many methods have been proposed to mitigate this problem, and DeepRED is a typical representation that combines DIP and regularization by denoising (RED). In this article, we leverage DeepRED from a Bayesian perspective to reconstruct PET images from a single corrupted sinogram without any supervised or auxiliary information. In contrast to the conventional denoisers customarily used in RED, a DnCNN-like denoiser, which can add an adaptive constraint to DIP and facilitate the computation of derivation, is employed. Moreover, to further enhance the regularization, Gaussian noise is injected into the gradient updates, deriving a Markov chain Monte Carlo (MCMC) sampler. Experimental studies on brain and whole-body datasets demonstrate that our proposed method can achieve better performance in terms of qualitative and quantitative results compared to several classic and state-of-the-art methods.

Ultrasound elasticity reconstruction with inaccurate forward model using integrated data-driven correction of data-fidelity gradient

Ultrasound elasticity images, which enable the visualization of quantitative maps of tissue stiffness, can be reconstructed by solving an inverse problem. Classical model-based approaches for ultrasound elastography use deterministic finite element methods (FEMs) to incorporate the governing physical laws leading to poor performance in low SNR conditions. Moreover, these approaches utilize approximate linear forward models discretized by FEMs to describe the underlying physics governed by partial differential equations (PDEs). To achieve highly accurate stiffness images, it is essential to compensate the error induced by noisy measurements and inaccurate forward models. In this regard, we propose a joint model-based and learning-based framework for estimating the elasticity distribution by solving a regularized optimization problem. To address noise, we introduce a statistical representation of the imaging system, which incorporates the noise statistics as a signal-dependent correlated noise model. Moreover, in order to compensate for the model errors, we introduce an explicit data-driven correction model, which can be integrated with any regularization term. This constrained optimization problem is solved using fixed-point gradient descent where the analytical gradient of the inaccurate data-fidelity term is corrected using a neural network, while regularization is achieved by data-driven unrolled regularization by denoising (RED). Both networks are jointly trained in an end-to-end manner.

Wasserstein Distance-Based Full-Waveform Inversion With a Regularizer Powered by Learned Gradient

Full-waveform inversion (FWI) is a powerful technique for building high-quality subsurface geological structures. It is known to suffer from local minima problems when a good starting model is lost. To obtain a desirable solution, regularization constraints are needed to impose suitable priors, in particular for salt models. Recent studies have allowed cooperating the denoising operator as a specific prior with optimization algorithms to effectively solve various image reconstruction tasks. Inspired by the promising performance of regularization by denoising (RED), we propose a flexible unsupervised learning FWI framework, in which the regularizer is an extensive variant of RED that powered by learned gradient for addressing salt models inversion. To further mitigate local minima issues, the Wasserstein distance induced by optimal transport theory with a new preprocessing transformation is applied as a measure in data domain. Integrating the physical constraints featured by the wave equation and the model priors featured by the deep convolutional neural network, our method is able to produce measurement consistent and high-resolution results. We experimentally compare the proposed method with traditional total variation regularized FWI and RED regularized FWI on the well-known geological models. The numerical results demonstrate the effectiveness of our method for handling the inversion task of high-contrast media in the presence of a free-surface case. Moreover, our framework is adaptive to restrict the solutions by leveraging the regularizer with learned gradient, whose training does not need any geological images, thus paving the way to develop FWI algorithms with state-of-the-art deep learning techniques in interdisciplinary research.

SLN-RED: Regularization by Simultaneous Local and Nonlocal Denoising for Image Restoration

Regularization by denoising (RED) framework has shown impressive performance for many imaging inverse problems, by leveraging the denoising method in defining an explicit regularization. In this letter, we propose a novel SLN-RED scheme for image restoration by exploiting the local and nonlocal denoisers simultaneously. Theoretically, we proves that for <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">bounded</i> denoisers, the SLN-RED under ADMM scheme with a continuation strategy converges to a fixed-point. Numerical experiments on deblurring and super-resolution tasks demonstrate promising performance of the proposed algorithm.

Inverse Problem of Ultrasound Beamforming With Denoising-Based Regularized Solutions.

During the past few years, inverse problem formulations of ultrasound beamforming have attracted growing interest. They usually pose beamforming as a minimization problem of a fidelity term resulting from the measurement model plus a regularization term that enforces a certain class on the resulting image. Here, we take advantage of alternating direction method of multipliers to propose a flexible framework in which each term is optimized separately. Furthermore, the proposed beamforming formulation is extended to replace the regularization term with a denoising algorithm, based on the recent approaches called plug-and-play (PnP) and regularization by denoising (RED). Such regularizations are shown in this work to better preserve speckle texture, an important feature in ultrasound imaging, than sparsity-based approaches previously proposed in the literature. The efficiency of the proposed methods is evaluated on simulations, real phantoms, and in vivo data available from a plane-wave imaging challenge in medical ultrasound. Furthermore, a comprehensive comparison with existing ultrasound beamforming methods is also provided. These results show that the RED algorithm gives the best image quality in terms of contrast index while preserving the speckle statistics.

Model‐guided boosting for image denoising

Boosting algorithms have demonstrated their effectiveness in improving the restoration quality of existing image denoising methods by extracting the residual signal or removing the noise leftover iteratively. Unlike existing boosting algorithms that focus on designing an ingenious recursive step by making use of the residual signal or the noise leftover, in this paper, we propose a novel model-guided boosting framework. Specifically, we derive the recursive step from an overall restoration model constructed with the technique of Regularization by Denoising (RED) towards an interpretable, extensible and flexible boosting mechanism. By using the RED, we can apply explicit regularization equipped with powerful image denoising engine to establish the global minimization problem, making the obtained model is clearly defined and well optimized. The framework enjoys the advantage of easily extending to the case of composite denoising via superadding a regularization term. As such, we develop a simultaneous model through the joint use of deep neural network and low-rank regularization to fully utilize both external and internal image properties. The resulting restoration models are capable of being flexibly solved with fixed-point strategy and steepest-descent method, leading to two types of denoising boosters. It is shown that the proposed schemes have promise results due to the improvement in signal-to-noise ratio of input signal, and are guaranteed to converge. Experiments verify the validity of the boosters for several denoising algorithms, and show that combining the power of internal and external denoising based on our framework achieves enhancement in denoising performance.

Restoration of speckle noise corrupted SAR images using regularization by denoising

Speckle noise removal is a well-established problem in synthetic aperture radar (SAR) image processing. Among different methods focused on the reconstruction of SAR images, variational models have achieved state-of-the-art performance. In this paper, a Rayleigh based speckle reduction algorithm is developed using the variational framework. The forward model is combined with recently proposed regularization by denoising (RED) prior. However, RED has been proposed in literature for the additive noise model. Multiplicative noise in SAR images prevents the direct application of RED to variational models. Hence, logarithm transformation is applied to change the multiplicative noise model to additive model, and the forward model from Rayleigh to Fisher–Tippett distribution. The resulting optimization problem is solved using the alternating direction method of multipliers. Further, the proof of the convergence analysis is carried out for the above framework. Simulations convey that the proposed method has better despeckling performance compared to that of state-of-the-art methods.

Anomaly Detection in Hyperspectral Images via Regularization by Denoising

In a recent work, Fu et al. proposed an anomaly detector for hyperspectral images called DeCNN-AD, which decomposes the image in a low rank representation of the background and the anomalies. DeCNN-AD is regularized by an implicit Plug and Play Prior (PnP), providing state-of-the-art anomaly detection performance in hyperspectral images. In this paper, we propose a different anomaly detector using the Regularization by DEnoising (RED) framework. The regularizer that emerges in RED algorithms is advantageous in the sense that it can be minimized by many solvers, unlike PnP, which has to be rederived for each specific solver like the Alternating Direction Method of Multipliers or the Proximal Gradient Method. The proposed detector was compared to DeCNN-AD in several experiments, and was shown to present a more stable behavior better convergence properties, to be less sensitive to the choice of parameters, while leading to very similar performance in terms of optimal AUC of Receiver Operating Characteristic curves.

On the Convergence of Non-Convex Phase Retrieval With Denoising Priors

Denoising priors have achieved empirical success in solving non-convex phase retrieval but still lack convergence guarantees in theory. In this paper, we provide a novel insight on the convergence guarantees for a general class of non-convex phase retrieval with denoising priors. Specifically, we propose a Wirtinger flow based framework, named DWF, that allows monotone and contractive iterative optimization. We demonstrate in theory that, under a milder demi-Lipschitz condition on denoisers, the proposed DWF framework converges to the actual signal (up to a global sign) at a geometric rate. The derived convergence guarantees and rates are general to accommodate to complex-valued and real-valued signals in the presence of noise perturbation. Furthermore, we exemplify the proposed framework with two prevailing denoising priors, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , plug-and-play priors (PnP) and regularization by denoising (RED), including specific conditions on denoisers and convergence rates. To our best knowledge, this paper is the first attempt to provide theoretical guarantees of convergence for non-convex phase retrieval with denoising priors. Numerical evaluations demonstrate the theoretical results for analytic denoiser like arithmetic mean filter and deep learning based denoiser DnCNN. Furthermore, extensive experiments under the Gaussian model and coded diffraction pattern show that the proposed framework outperforms existing denoising prior-based methods and evidently reduces the necessary sampling rate for stable reconstruction with a guarantee of convergence in theory.

A regularization by denoising (RED) scheme for 3-D FWI model updates in large-contrast media

SUMMARY Full waveform inversion (FWI) endeavours to estimate high-resolution physical properties of subsurface structures. The technique minimizes the data misfit between observed and modelled seismograms. Despite its success, the application of FWI in areas with high-velocity contrasts remains a challenging problem. Often, quadratic regularization methods are chosen to stabilize inverse problems. Unfortunately, quadratic regularization does not preserve edges and sharp discontinuities adequately. Conversely, a regularization term that uses the l1 norm of the gradient of model parameters can preserve discontinuities. The latter leads to edge-preserving methods based on total variation regularization. This work adopts the framework named regularization by denoising (RED) to solve the FWI problem in high-contrast media. The RED technique only requires an image denoising engine, which, in our case, is a modified weighted total variation filter. One advantage of adopting the RED algorithm for solving FWI problems is its simplicity in the numerical implementation and selection of trade-off parameters. We have benchmarked our algorithm via the 2-D BP/EAGE model, a model with significant velocity contrasts and complex salt bodies. We have also tested the proposed regularization method with the 3-D SEG/EAGE overthrust P-wave velocity model. We also compare the proposed RED method and FWI with total variation regularization.

Computational inverse imaging method by machine learning-informed physical model for electrical capacitance tomography

The solution of the imaging inversion problem is an important step in the electrical capacitance tomography developed for process parameter measurements. Many studies have been carried out to improve the reconstruction quality, but the discrepancy between ground-truth imaging prototypes and recovered tomograms is still significant. To address the challenge, the regularization by denoising (RED) is introduced in this work, turning the denoising algorithm into a regularizer. Measurement physics, RED and sparsity prior are coupled into a new imaging model. A new numerical method is developed to solve the established imaging model by integrating the split Bregman algorithm and the forward backward splitting technique. To improve the performance of RED, the multiple output least squares support vector machine is combined with the low-dimensional representation method, and the training problem is solved by a new distributed computational method. The nonnegative matrix factorization method is extended into a new low-dimensional representation method, and a powerful optimizer is developed to solve the model. The performance evaluations clearly imply that the new method achieves more significant reconstruction performance gain and better robustness than popular imaging algorithms. This study improves the measurement physics based imaging method by machine learning techniques, and provides new perspectives and insights into the development of the image reconstruction paradigm.

Regularization by denoising for simultaneous source separation

Denoisers can help solve inverse problems via a recently proposed framework known as regularization by denoising (RED). The RED approach defines the regularization term of the inverse problem via explicit denoising engines. Simultaneous source separation techniques, themselves being a combination of inversion and denoising methods, provide a formidable field to explore RED. We investigate the applicability of RED to simultaneous-source data processing and introduce a deblending algorithm called REDeblending (RDB). The formulation permits developing deblending algorithms in which the user can select any denoising engine that satisfies RED conditions. Two popular denoisers are tested, but the method is not limited to them: frequency-wavenumber thresholding and singular spectrum analysis. We offer synthetically blended data examples to showcase the performance of RDB.

A Regularization by Denoising super-resolution method based on genetic algorithms

Increasing the resolution of an image is an actual and extensively studied problem in image processing. Recently, Regularization by Denoising (RED) showing that any inverse problem can be handled by sequentially applying image denoising steps, including the image super-resolution (SR) task, which facilitate the resolution of the encountered optimization problem. In this paper, we propose a new configuration of genetic algorithms to resolve the super-resolution problem using a Non-Local Means filter as a denoiser function with a rigorous proof of the existence of a unique minimizer. In fact, since the SR algorithms always skip the complex spatial interactions within images, a more consistent model is then needed. The use of the genetic algorithms with the RED techniques guaranteed, in high intensity of noise and blur, the convergence to the globally optimal solution. As a result, the proposed algorithm shows efficient and consistent results, in terms of edges and feature preservation, compared with other SR approaches.

Dynamic PET Image Denoising Using Deep Image Prior Combined With Regularization by Denoising

The quantitative accuracy of positron emission tomography (PET) is affected by several factors, including the intrinsic resolution of the imaging system and inherently noisy data, which result in a low signal-to-noise ratio (SNR) of PET image. To address this problem, in this paper, we proposed a novel deep learning denoising framework aiming to enhance the quantitative accuracy of dynamic PET images via introduction of deep image prior (DIP) combined with Regularization by Denoising (RED), as such the method is labeled as DeepRED denoising. The network structure is based on encoder-decoder architecture and uses skip connections to combine hierarchical features to generate the estimated image. The network input can be random noise or other prior images (such as the patient’s own static PET image), avoiding the need of high quality noiseless images, which is limited in PET clinical practice due to high radiation dose. Based on simulated data and real patient data, the quantitative performance of the proposed method was compared with conventional Gaussian filtering (GF), non-local mean (NLM), block-matching and 3D filtering (BM3D), DIP and stochastic gradient Langevin dynamics (SGLD) method. Overall, the proposed method can outperform other conventional methods in substantial visual as well as quantitative accuracy improvements (in terms of noise versus bias performance) with and without prior images.

Block Coordinate Regularization by Denoising

We consider the problem of reconstructing an image from its noisy measurements using a prior specified only with an image denoiser. Recent work on plug-and-play priors (PnP) and regularization by denoising (RED) has shown the state-of-the-art performance of image reconstruction algorithms under such priors in a range of imaging problems. In this work, we develop a new block coordinate RED algorithm that decomposes a large-scale estimation problem into a sequence of updates over a small subset of the unknown variables. We theoretically analyze the convergence of the algorithm and discuss its relationship to the traditional proximal optimization. Our analysis complements and extends recent theoretical results for RED-based estimation methods. We numerically validate our method using several denoising priors, including those based on deep neural nets.