Nonlocal Total Variation Research Articles

A mean curvature flow arising in adversarial training

We connect adversarial training for binary classification to a geometric evolution equation for the decision boundary. Relying on a perspective that recasts adversarial training as a regularization problem, we introduce a modified training scheme that constitutes a minimizing movements scheme for a nonlocal perimeter functional. We prove that the scheme is monotone and consistent as the adversarial budget vanishes and the perimeter localizes, and as a consequence we rigorously show that the scheme approximates a weighted mean curvature flow. This highlights that the efficacy of adversarial training may be due to locally minimizing the length of the decision boundary. In our analysis, we introduce a variety of tools for working with the subdifferential of a supremal-type nonlocal total variation and its regularity properties.

Bilevel optimal parameter learning for a high-order nonlocal multiframe super-resolution problem

This work elaborated an improved method to multiframe super-resolution (SR), which involves a nonlocal first-order regularization combined with a nonlocal p-Laplacian term. The nonlocal TV term excels at edge preserving, whilst the nonlocal p-Laplacian is commonly used to perfectly reconstruct image textures. Firstly, we discuss the existence and uniqueness of a solution to our new model in a well posed framework. Then, we derive a modified Primal-dual iteration to compute the super-resolved solution. Furthermore, we introduce a new bilevel optimization approach to learn two regularization parameters. The included tests validate that the introduced optimization procedure performs favorably compared to numerous SR approaches in terms of efficiency and accuracy.

Color Remote Sensing Image Restoration through Singular-Spectra-Derived Self-Similarity Metrics

Color remote sensing images have key features of pronounced internal similarity characterized by numerous repetitive local patterns, so the capacity to effectively harness these self-similarity features plays a key role in the enhancement of color images. The main novelty of this study lies in that we utilized an unusual technique (singular spectrum) to derive brand-new similarity metrics inside the quaternion representation of color images and then incorporated these metrics into denoising algorithms. Color image denoising experiments demonstrated that compared with seven mainstream image restoration algorithms (homomorphic filtering (HPF), wavelet transforms (WT), non-local means (NLM), non-local total variation (NLTV), the color adaptation of non-local means (NLMC), quaternion Euclidean metric (QNLM), and quaternion Euclidean metric total variation (QNLTV)), our algorithms with two novel self-similarity metrics achieved maximum peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), average gradient (AG), and information entropy index (IE) values, with average increases of 1.98 dB /2.12 dB, 0.1168/0.1244, 1.824/1.897, and 0.158/0.135. Moreover, for a complex, mixed-noise scenario, two versions of our algorithms also achieved average increases of 0.382 dB/0.394 dB and 0.0207/0.0210 under Motion and Gaussian mixed noise and average increases of 0.129 dB/0.154 dB and 0.0154/0.0158 under Average and Gaussian mixed noise compared with three quaternion-based restoration algorithms (QNLM, QNLTV, and quantization weighted nuclear norm minimization (QWNNM)).

HPHR-SAR-Net: Hyperpixel High-Resolution SAR Imaging Network Based on Nonlocal Total Variation

HPHR-SAR-Net: Hyperpixel High-Resolution SAR Imaging Network Based on Nonlocal Total Variation

A GPU-accelerated light-field super-resolution framework based on mixed noise model and weighted regularization

Light-field (LF) super-resolution (SR) plays an essential role in alleviating the current technology challenge in the acquisition of a 4D LF, which assembles both high-density angular and spatial information. Due to the algorithm complexity and data-intensive property of LF images, LFSR demands a significant computational effort and results in a long CPU processing time. This paper presents a GPU-accelerated computational framework for reconstructing high-resolution (HR) LF images under a mixed Gaussian-Impulse noise condition. The main focus is on developing a high-performance approach considering processing speed and reconstruction quality. From a statistical perspective, we derive a joint ell ^1-ell ^2 data fidelity term for penalizing the HR reconstruction error taking into account the mixed noise situation. For regularization, we employ the weighted non-local total variation approach, which allows us to effectively realize LF image prior through a proper weighting scheme. We show that the alternating direction method of the multipliers algorithm (ADMM) can be used to simplify the computation complexity and results in a high-performance parallel computation on the GPU Platform. An extensive experiment is conducted on both synthetic 4D LF dataset and natural image dataset to validate the proposed SR model’s robustness and evaluate the accelerated optimizer’s performance. The experimental results show that our approach achieves better reconstruction quality under severe mixed-noise conditions as compared to the state-of-the-art approaches. In addition, the proposed approach overcomes the limitation of the previous work in handling large-scale SR tasks. While fitting within a single off-the-shelf GPU, the proposed accelerator provides an average speedup of 2.46{times } and 1.57{times } for {times }2 and {times }3 SR tasks, respectively. In addition, a speedup of 77{times } is achieved as compared to CPU execution.

Image Restoration using Nonlocal Regularized Variational Model with Spatially Adapted Regularization Parameter

Restoration of blurred images corrupted with impulse noise has attracted increasing attention due to its great potential toward practical applications. To guarantee the noisy image deblurring performance, total variation (TV), one of the most popular regularizers, has been extensively exploited in the literature. This local regularizer is known for its capability of preserving edges and penalizing oscillations but not sharp discontinuities. However, TV regularized variational models commonly suffer from undesirable staircase-like artifacts in homogeneous regions. The reason behind this phenomenon is that TV regularizer favors solutions that are piecewise constant in numerical experiments. To overcome the model-dependent deficiency, we proposed to replace the commonly used local regularizer by its nonlocal extension, i.e., nonlocal total variation (NLTV) regularizer. By taking advantage of the high degree of self-similarity within images, the proposed nonlocal variational model is able to remove both blurring and impulse noise while preserving meaningful image details. To guarantee solution stability and efficiency, the resulting nonsmooth image restoration problem was effectively dealt with using an alternating optimization algorithm. In addition, to further enhance the image quality, a local variance estimator-based automatic method was introduced to calculate the spatially adapted regularization parameters during image restoration. Extensive experiments have demonstrated that the proposed method could achieve superior imaging performance compared with several state-of-the-art image restoration methods.

A Robust Image Segmentation Framework Based on Nonlocal Total Variation Spectral Transform

Image segmentation plays an important role in various computer vision tasks. Nevertheless, noise always inevitably appears in images and brings a big challenge to image segmentation. To handle the problem, we study the nonlocal total variation (NLTV) spectral theory and build up an image segmentation framework with NLTV spectral transform to segment images with noise. Firstly, we decompose an image into the NLTV flow in the NLTV spectral transform, with which the max response time of each pixel is calculated. Secondly, a separation surface is constructed with the max response time to distinguish the objects and preserve the structure details in segmentation. Thirdly, the image is filtered by the surface in the NLTV spectral domain, and a rough segmentation result is obtained by means of an inverse transform. Finally, we use a binary process and morphological operations to refine the segmentation result. Experiments illustrate that our method can preserve edge structures effectively and has the ability to achieve competitive segmentation performance compared with the state-of-the-art approaches.

The lower bound of nonlocal gradient for non-convex and non-smooth image patches based regularization

In the inverse problem of image processing, we have witnessed that the non-convex and non-smooth regularizers can produce clearer image edges than convex ones such as total variation (TV). This fact can be explained by the uniform lower bound theory of the local gradient in non-convex and non-smooth regularization. In recent years, although it has been numerically shown that the nonlocal regularizers of various image patches based nonlocal methods can recover image textures well, we still desire a theoretical interpretation. To this end, we propose a non-convex non-smooth and block nonlocal regularization model based on image patches. By integrating the advantages of the non-convex and non-smooth potential function in the regularization term, the uniform lower bound theory of the image patches based nonlocal gradient is given. This approach partially explains why the proposed method can produce clearer image textures and edges. Compared to some classical regularization methods, such as TV, non-convex and non-smooth regularization, nonlocal total variation and block nonlocal total variation, our experimental results show that the proposed method improves restoration quality.

Color image denoising using adaptive non-local 3D total variation

Color image denoising using adaptive non-local 3D total variation

SAR Time-Series Despeckling via Nonlocal Total Variation Regularized Robust PCA

Through the development of Synthetic Aperture Radar (SAR) technology, it is now possible to observe dynamic processes on the earth with fine temporal resolution by forming SAR time series. Nonetheless, such sequential images remain difficult to interpret due to the speckle effect. Despeckling them is further complicated by outliers caused by abrupt changes in weather conditions or the appearance of objects. In spite of the fact that many state-of-the-art methods can achieve excellent filtering performances over stable areas, they often result in artifacts in those areas where outliers existed at the time of acquisition. To simultaneously mitigate the speckle noise and extract outliers, we propose a novel SAR time series despeckling method based on nonlocal total variation regularized robust principle component analysis, which is termed SAR-NL-TVRPCA. By comparing it to other state-of-the-art methods, the effectiveness of its despeckling has been validated in real data experiments. Furthermore, the extracted outliers can provide insight into abrupt changes occurring throughout the observation period, which provides byproducts for further analysis.

Hyperspectral linear unmixing based on collaborative sparsity and multi-band non-local total variation

ABSTRACT In recent years, spectral unmixing is a hot issue in the hyperspectral remote sensing technology and applications. Within various methods, sparse regression is one of the most commonly used methods in the hyperspectral unmixing community. In order to overcome the limitation of spatial correlation of existing sparse unmixing methods and improve further the accuracy of sparsity representation, a novel hyperspectral linear unmixing method via fusion of collaborative sparsity and multi-band non-local total variation is proposed. This method analyses the sparsity and spatially geometrical structure of hyperspectral pixels by studying the linear spectral mixed model of hyperspectral images, and reveals the interior spatial correlation of hyperspectral images with the non-local variation framework. Specifically, the sparsity and spatial correlation of hyperspectral imagery are discussed, and a hyperspectral linear sparse unmixing model is afterwards constructed by combining the relaxation technique and non-local total variation framework. Second, the solution procedure of the above spectral unmixing model is divided into two steps under the variable separation method: fractional abundance estimation and abundance map restoration. The alternating direction method of multipliers (ADMM) and split Bregman operator are exploited to obtain the spectral unmixing results. Finally, the proposed hyperspectral unmixing algorithm is evaluated with synthetic and real hyperspectral datasets. In the experiments with a synthetic hyperspectral dataset, the feasibility and effectiveness of the method are analysed quantitatively and qualitatively. The quantitative metrics and visual examination of the estimated fractional abundance map are also better than the performance of the current mainstream hyperspectral sparse unmixing algorithms. Furthermore, two real hyperspectral datasets are applied to the algorithm of this work to prove its practicability.

Mutual Information-Based Non-Local Total Variation Denoiser for Low-Dose Cone-Beam Computed Tomography.

Conventional non-local total variation (NLTV) approaches use the weight of a non-local means (NLM) filter, which degrades performance in low-dose cone-beam computed tomography (CBCT) images generated with a low milliampere-seconds (mAs) parameter value because a local patch used to determine the pixel weights comprises noisy-damaged pixels that reduce the similarity between corresponding patches. In this paper, we propose a novel type of NLTV based on a combination of mutual information (MI): MI-NLTV. It is based on a statistical measure for a similarity calculation between the corresponding bins of non-local patches vs. a reference patch. The weight is determined in terms of a statistical measure comprising the MI value between corresponding non-local patches and the reference-patch entropy. The MI-NLTV denoising process is applied to CBCT images generated by the analytical reconstruction algorithm using a ray-driven backprojector (RDB). The MI-NLTV objective function is minimized based on the steepest gradient descent optimization to augment the difference between a real structure and noise, cleaning noisy pixels without significant loss of the fine structure and details that remain in the reconstructed images. The proposed method was evaluated using patient data and actual phantom measurement data acquired with lower mAs. The results show that integrating the RDB further enhances the MI-NLTV denoising-based analytical reconstruction algorithm to achieve a higher CBCT image quality when compared with those generated by NLTV denoising-based approach, with an average of 15.97% higher contrast-to-noise ratio, 2.67% lower root mean square error, 0.12% lower spatial non-uniformity, 1.14% higher correlation, and an average of 18.11% higher detectability index. These quantitative results indicate that the incorporation of MI makes the NLTV more stable and robust than the conventional NLM filter for low-dose CBCT imaging. In addition, achieving clinically acceptable CBCT image quality despite low-mAs projection acquisition can reduce the burden on common online CBCT imaging, improving patient safety throughout the course of radiotherapy.

A second-order nonlocal regularized variational model for multiframe image super-resolution

Abstract In this paper, we propose a new nonlocal super-resolution (SR) model which is a combination of the nonlocal total variation (TV) regularization and the nonlocal p-Laplacian term (with p = 2). This choice is motivated by the success of the nonlocal TV term in preserving image edges and the efficiency of the nonlocal p-Laplacian term in preserving the image texture. To ensure the convergence of the proposed optimization SR problem, we prove the existence and uniqueness of a solution in a well-posed framework. In addition, to resolve the encountered minimization problem, we proposed a modified primal-dual algorithm and numerical results are also given to show the performance of the proposed approach.

Frequency domain despeckling technique for medical ultrasound images

Abstract This work proposes a novel frequency domain despeckling technique pertaining to the enhancement of the quality of medical ultrasound images. The results of the proposed method have been validated in comparison to both the time-domain and the frequency-domain projections of the schur decomposition as well as with several other benchmark schemes such as frost, lee, probabilistic non-local means (PNLM) and total variation filtering (TVF). The proposed algorithm has shown significant improvements in edge detection and signal to noise ratio (SNR) levels when compared with the performance of the other techniques. Both real and simulated medical ultrasound images have been used to evaluate the numerical and visual effects of each algorithm used in this work.

Center pixel weight based on Wiener filter for non-local means image denoising

In non-local means (NLM) method for image denoising, the weight of the center pixel is called the center weight (CW). The CW plays an important role for the performance of NLM. This paper proposes a novel CW by studying the relation between Perona-Malik anisotropic diffusion (PMAD) and NLM. The proposed CW is called the Wiener filter center weight (WFCW) since Wiener filter is introduced in the proposed CW computation. At the same time, the relation between non-local total variation and NLM is further disclosed. Test results show that the proposed method can achieve better results and is very efficient compared with the related NLM methods.

Robust optical flow estimation via edge preserving filtering

It is known that optical flow estimation techniques suffer from the issues of ill-defined edges and boundaries of the moving objects. Traditional variational methods for optical flow estimation are not robust to handle these issues since the local filters in these methods do not hold the robustness near the edges. In this paper, we propose a non-local total variation NLTV-L1 optical flow estimation method based on robust weighted guided filtering. Specifically, first, the robust weighted guided filtering objective function is proposed to preserve motion edges. The proposed objective function is based on the linear model which is computationally efficient and edge-preserving in complex natural scenarios. Second, the proposed weighted guided filtering objective function is incorporated into the non-local total variation NLTV-L1 energy function. Finally, the novel NLTV-L1 optical flow method is performed using the coarse-to-fine process. Additionally, we modify some state-of-the-art variational optical flow estimation methods by the robust weighted guided filtering objective function to verify the performance on Middlebury, MPI-Sintel, and Foggy Zurich sequences. Experimental results show that the proposed method can preserve edges and improve the accuracy of optical flow estimation compared with several state-of-the-art methods.

Nonlocal total variation regularization with Shape Adaptive Patches for image denoising via Split Bregman method

Non-Local or patch-based approaches are used in most of the state-of-the-art methods for image denoising. Denoising with regular square patches may cause noise halos around edges, particularly high contrasted edges. In order to overcome this drawback, this work presents an extension of the Non-Local TV framework that effectively exploits the potential local geometric features of the image by replacing the simple square patches with different oriented shapes. We first solve the denoised models of ROF-NLTV with different oriented patch shapes by Split Bregman algorithm. Then, use exponentially weighted aggregation method based on Stein’s unbiased risk estimate to combine the estimators obtained in the first step. The numerical results show our method outperforms some previous ones. Moreover, common noise halos around edges usually observed by denoising with Non-Local TV method are reduced thanks our approach.

Nonlocal adaptive direction-guided structure tensor total variation for image recovery

A common strategy in variational image recovery is utilizing the nonlocal self-similarity property, when designing energy functionals. One such contribution is nonlocal structure tensor total variation (NLSTV), which lies at the core of this study. This paper is concerned with boosting the NLSTV regularization term through the use of directional priors. More specifically, NLSTV is leveraged so that, at each image point, it gains more sensitivity in the direction that is presumed to have the minimum local variation. The actual difficulty here is capturing this directional information from the corrupted image. In this regard, we propose a method that employs anisotropic Gaussian kernels to estimate directional features to be later used by our proposed model. The experiments validate that our entire two-stage framework achieves better results than the NLSTV model and two other competing local models, in terms of visual and quantitative evaluation.

Low-dose CBCT reconstruction via joint non-local total variation denoising and cubic B-spline interpolation

This study develops an improved Feldkamp–Davis–Kress (FDK) reconstruction algorithm using non-local total variation (NLTV) denoising and a cubic B-spline interpolation-based backprojector to enhance the image quality of low-dose cone-beam computed tomography (CBCT). The NLTV objective function is minimized on all log-transformed projections using steepest gradient descent optimization with an adaptive control of the step size to augment the difference between a real structure and noise. The proposed algorithm was evaluated using a phantom data set acquired from a low-dose protocol with lower milliampere-seconds (mAs).The combination of NLTV minimization and cubic B-spline interpolation rendered the enhanced reconstruction images with significantly reduced noise compared to conventional FDK and local total variation with anisotropic penalty. The artifacts were remarkably suppressed in the reconstructed images. Quantitative analysis of reconstruction images using low-dose projections acquired from low mAs showed a contrast-to-noise ratio with spatial resolution comparable to images reconstructed using projections acquired from high mAs. The proposed approach produced the lowest RMSE and the highest correlation. These results indicate that the proposed algorithm enables application of the conventional FDK algorithm for low mAs image reconstruction in low-dose CBCT imaging, thereby eliminating the need for more computationally demanding algorithms. The substantial reductions in radiation exposure associated with the low mAs projection acquisition may facilitate wider practical applications of daily online CBCT imaging.

Bilateral Spectrum Weighted Total Variation for Noisy-Image Super-Resolution and Image Denoising

In this paper, we propose a regularization technique for noisy-image super-resolution and image denoising. Total variation (TV) regularization is adopted in many image processing applications to preserve the local smoothness. However, TV prior is prone to oversmoothness, staircasing effect, and contrast losses. Nonlocal TV (NLTV) mitigates the contrast losses by adaptively weighting the smoothness based on the similarity measure of image patches. Although it suppresses the noise effectively in the flat regions, it might leave residual noise surrounding the edges especially when the image is not oversmoothed. To address this problem, we propose the bilateral spectrum weighted total variation (BSWTV). Specially, we apply a locally adaptive shrink coefficient to the image gradients and employ the eigenvalues of the covariance matrix of the weighted image gradients to effectively refine the weighting map and suppress the residual noise. In conjunction with the data fidelity term derived from a mixed Poisson-Gaussian noise model, the objective function is decomposed and solved by the alternating direction method of multipliers (ADMM) algorithm. In order to remove outliers and facilitate the convergence stability, the weighting map is smoothed by a Gaussian filter with an iteratively decreased kernel width and updated in a momentum-based manner in each ADMM iteration. We benchmark our method with the state-of-the-art approaches on the public real-world datasets for super-resolution and image denoising. Experiments show that the proposed method obtains outstanding performance for super-resolution and achieves promising results for denoising on real-world images.