Nonlocal Patch-based Method Research Articles

Guided Nonlocal Patch Regularization and Efficient Filtering-Based Inversion for Multiband Fusion

In multiband fusion, an image with a high spatial and low spectral resolution is combined with an image with a low spatial but high spectral resolution to produce a single multiband image having high spatial and spectral resolutions. This comes up in remote sensing applications such as pansharpening (MS+PAN), hyperspectral sharpening (HS+PAN), and HS-MS fusion (HS+MS). Remote sensing images are textured and have repetitive structures. Motivated by nonlocal patch-based methods for image restoration, we propose a convex regularizer that (i) takes into account long-distance correlations, (ii) penalizes patch variation, which is more effective than pixel variation for capturing texture information, and (iii) uses the higher spatial resolution image as a guide image for weight computation. We come up with an efficient ADMM algorithm for optimizing the regularizer along with a standard least-squares loss function derived from the imaging model. The novelty of our algorithm is that by expressing patch variation as filtering operations and by judiciously splitting the original variables and introducing latent variables, we are able to solve the ADMM subproblems efficiently using FFT-based convolution and soft-thresholding. As far as the reconstruction quality is concerned, our method is shown to outperform state-of-the-art variational and deep learning techniques.

Video Denoising by Combining Patch Search and CNNs

Non-local patch-based methods were until recently the state of the art for image denoising but are now outperformed by CNNs. In video denoising, however, they are still competitive with CNNs, as they can effectively exploit the video temporal redundancy, which is a key factor to attain high denoising performance. The problem is that CNN architectures are not compatible with the search for self-similarities. In this work, we propose a simple, yet efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed method to image and video denoising. In the case of video, the patches are searched for in a 3D spatiotemporal volume. The proposed method achieves state-of-the-art results.

Image denoising via a non-local patch graph total variation.

Total variation (TV) based models are very popular in image denoising but suffer from some drawbacks. For example, local TV methods often cannot preserve edges and textures well when they face excessive smoothing. Non-local TV methods constitute an alternative, but their computational cost is huge. To overcome these issues, we propose an image denoising method named non-local patch graph total variation (NPGTV). Its main originality stands for the graph total variation method, which combines the total variation with graph signal processing. Schematically, we first construct a K-nearest graph from the original image using a non-local patch-based method. Then the model is solved with the Douglas-Rachford Splitting algorithm. By doing so, the image details can be well preserved while being denoised. Experiments conducted on several standard natural images illustrate the effectiveness of our method when compared to some other state-of-the-art denoising methods like classical total variation, non-local means filter (NLM), non-local graph based transform (NLGBT), adaptive graph-based total variation (AGTV).

A target-oriented segmentation method for specific tissues in MRI images of the brain

The multi-atlas based segmentation method can achieve the accurate segmentation of specific tissues of the human brain in the magnetic resonance imaging (MRI). The correct image registration and fusion scheme used in this method have an impact on the accuracy of segmentation. Similar to any traditional rigid registration method, we use the same method in our proposed target-oriented registration for the coarse registration between the target image and atlas image. However, to improve the registration accuracy in the area to be segmented, we propose a target-oriented image registration method for the refinement. We employ the distribution probability of the tissue (to be segmented) in the sparse patch-based label fusion process. Our aim is to determine if the proposed registration method can contribute the segmentation accuracy and which label fusion method is a good fit with this target-oriented registration. To evaluate the efficiency of our proposed method, we compare the performance of the majority voting method (MV), the nonlocal patch-based method (Nonlocal-PBM) and the sparse patch-based method (Sparse-PBM). Experimental results show that more accurate segmentation results can be obtained with the proposed registration method in this study. This result can provide more accurate clinical diagnosis information.

Image Denoising via Multiscale Nonlinear Diffusion Models

Image denoising is a fundamental operation in image processing and holds considerable practical importance for various real-world applications. Arguably several thousands of papers are dedicated to image denoising. In the past decade, state-of-the-art denoising algorithms have been clearly dominated by nonlocal patch-based methods, which explicitly exploit patch self-similarity within the targeted image. However, in the past two years, discriminatively trained local approaches have started to outperform previous nonlocal models and have been attracting increasing attention due to the additional advantage of computational efficiency. Successful approaches include cascade of shrinkage fields (CSF) and trainable nonlinear reaction diffusion (TNRD). These two methods are built on the filter response of linear filters of small size using feed forward architectures. Due to the locality inherent in local approaches, the CSF and TNRD models become less effective when the noise level is high and consequently introduce some noise artifacts. In order to overcome this problem, in this paper we introduce a multiscale strategy. To be specific, we build on our newly developed TNRD model, adopting the multiscale pyramid image representation to devise a multiscale nonlinear diffusion process. As expected, all the parameters in the proposed multiscale diffusion model, including the filters and the influence functions across scales, are learned from training data through a loss-based approach. Numerical results on Gaussian and Poisson denoising substantiate that the exploited multiscale strategy can successfully boost the performance of the original TNRD model with a single scale. As a consequence, the resulting multiscale diffusion models can significantly suppress the typical incorrect features for those noisy images with heavy noise. It turns out that multiscale TNRD variants achieve better performance than state-of-the-art denoising methods.

A Nonlocal Denoising Algorithm for Manifold-Valued Images Using Second Order Statistics

Nonlocal patch-based methods, in particular the Bayesian approach of Lebrun, Buades, and Morel [SIAM J. Imaging Sci., 6 (2013), pp. 1665--1688], are considered to be state-of-the-art methods for denoising (color) images corrupted by white Gaussian noise of moderate variance. This paper is the first attempt to generalize this technique to manifold-valued images. Such images, for example, images with phase or directional entries or with values in the manifold of symmetric positive definite matrices, are frequently encountered in real-world applications. Generalizing the normal law to manifolds is not canonical, and different attempts have been considered. Here, we focus on a straightforward intrinsic model and discuss the relation to other approaches for specific manifolds. We reinterpret the Bayesian approach of Lebrun, Buades, and Morel [SIAM J. Imaging Sci., 6 (2013), pp. 1665--1688] in terms of minimum mean squared error estimation, which motivates our definition of a corresponding estimator on the mani...

Local denoising applied to RAW images may outperform non-local patch-based methods applied to the camera output

State-of-the-art denoising methods achieve impressive results, even for large noise levels. However, they can not be implemented in camera hardware, mainly due to the fact that they are computationally too intensive. The aim of this paper is then to show that we can obtain comparable denoising results to the ones obtained with state-of-art methods by inserting a well-chosen fast denoising method at the right location in the camera processing pipeline. We evaluate our results visually and with respect to objective measures.

Nonlocal patch-based method on spatially-variant amoeba morphology for image restoration

In view of the selection of structuring elements problem in morphological filters, this paper presents a new method to generate structuring elements for spatially-variant morphology. This method takes the theory of amoeba morphology as a foundation and new amoeba distances to generate structuring elements for spatially-variant filters are defined. The proposed strategy essentially consists in an amoeba kernel is divided into two parts: one is the patch-based center; another is the pixel-based boundary, the two parts combining both nonlocal patch-based distance and local pixel-based distance. This nonlocal and local configurations make nonlocal patch-based processing method becomes local processing on amoeba structuring element. New amoeba kernels are less flexible at patch-based center than traditional amoebas and their shape is less affected by noises in pilot image. By designing new amoeba structuring element, a new family of morphological filters are derived that have better performance in removing the noise while adaptively preserving the main structures compared with traditional amoeba filters.

Influence of the training library composition on a patch-based label fusion method: Application to hippocampus segmentation on the ADNI dataset

The atrophy of medial temporal lobe structures is may serve as early biomarkers of Alzheimer's disease. The evaluation of hippocampus (HC) atrophy is estimated by volumetric studies requiring a segmentation step that can be very time consuming when done manually. This limitation can be overcome by using automatic segmentation methods. In this study, we propose to validate our nonlocal patch-based method on the Alzheimer's Disease Neuroimaging Initiative (ADNI) database by segmenting the HC of Cognitively Normal (CN) subjects and patients with early Alzheimer's Disease (AD).