Nonlocal Similarity Research Articles

A novel denoising algorithm for medical images based on the non‐convex non‐local similar adaptive regularization

Sparse representation is a powerful statistical image modelling technique and has been successfully applied to image denoising. For a given patch, a non-convex non-local similarity adaptive method is adopted for sparse representation of images. First, it uses the autoregressive model to perform dictionary learning from sample patch datasets. Second, the sparse representation of an image introduces non-convex non-local self-similarity as the regularization term. In order to make better use of the sparse regularization method for image denoising, the parameters used in this study are estimated using adaptive methods. This model is more efficient and accurate, Compared with K-means singular value decomposition (KSVD) algorithm, a generalized K-means clustering method, total variation of population sparsity (GSTV) algorithm, adaptive sparse domain selection (ASDS) algorithm, forward denoising convolutional neural network (DnCNNs), a fast and flexible Convolutional Neural Network image denoising method (FFNNet) and operator-splitting algorithm to minimize the Euler elastica functional (OSEEF). Image noise-reduction experiments confirmed that using the adaptive regularization method, the results in peak signal to noise ratio (PSNR) and visual opinion are better than other algorithms.

Hierarchical Dynamic Time Warping methodology for aggregating multiple geological time series

Coherent investigation of paleo-records relies on the interpretation of multiple time series of climate proxies which requires the application of signal matching techniques between separate records and within different proxy measurements inside a single record. However, current methodologies, such as correlation matrices or manual tuning using prominent signal features, result in considerable signal manipulation. Here, we extend Dynamic Time Warping (DTW), a widely used tool for measuring similarity between two signals, to include a Hierarchical aggregation (HDTW) for applying DTW on more than two input signals. Our approach to HDTW is not limited to aggregating up the hierarchies but also indexes a unified ”path matrix” for the original inputs, thus emphasising non-local similarities between the input signals and highlighting non-local outliers, eventually extrapolating the optimal match between all signals. As a use case for paleo-reconstructions, we apply an HDTW-based peak finding algorithm on two published micron-scale measurements of speleothems from water-limited environments, where annual growth cycles are inconsistent. By HTDW-aligning and then stacking several coeval time axes of parallel proxy measurements (petrographic input for the first test sample and elemental measurements for the second) we were able to identify and rank prominence of local and non-local features on the sample. The results of the sub-annual calibrations agree with published age models and are within known age constraints of those samples. The output of the sub-annual calibration provides insights into local features which were not ranked high enough to be included in the model. The presented age model provides the researcher with an in-depth understanding of signal conformity while highlighting unconformities for domain-specific analysis. Future implementations may explore similar geoscience applications in different scales (e.g. regional, global), and may benefit the general case of DTW-based applications outside of geosciences.

Non-uniform low-light image enhancement via non-local similarity decomposition model

Many low-light image enhancement methods ignore the characteristic of non-uniform low-light images, thereby causing the enhanced results to lose their naturalness. In this paper, a non-local similarity decomposition model based on the Retinex theory is proposed to obtain high-quality enhanced results for non-uniform low-light images. For the reflectance layer, we explore a weighted L1-norm regularization according to the similarity measure of non-local patches. The similarity measure contains color, contrast and texture information, which are the intrinsic features of images. For the illumination layer, we introduce a weight matrix to constrain its gradient based on the edge information of non-uniform low-light images. By applying these constraints, it is more precise to estimate the reflectance and illumination layers simultaneously. Finally, the decomposed illumination is further adjusted. Furthermore, an alternating direction method of multipliers and preconditioned conjugate gradient are utilized to accelerate the solution of the proposed model. Experimental results demonstrate that our method is better for enhancing non-uniform low-light image both in subjective and objective assessments.

Nonlocal weighted sparse unmixing based on global search and parallel optimization

Sparse unmixing (SU) can represent an observed image using pure spectral signatures and corresponding fractional abundance from a large spectral library and is an important technique in hyperspectral unmixing. However, the existing SU algorithms mainly exploit spatial information from a fixed neighborhood system, which is not sufficient. To solve this problem, we propose a nonlocal weighted SU algorithm based on global search (G-NLWSU). By exploring the nonlocal similarity of the hyperspectral image, the weights of pixels are calculated to form a matrix to weight the abundance matrix. Specifically, G-NLWSU first searches for a similar group of each pixel in the global scope then uses singular value decomposition to denoise and finally obtains the weight matrix by considering correlations between similar pixels. To reduce the execution burden of G-NLWSU, we propose a parallel computing version of G-NLWSU, named PG-NLWSU, which integrates compute unified device architecture-based parallel computing into G-NLWSU to accelerate the search for groups of nonlocally similar pixels. Our proposed algorithms shed new light on SU by considering a new exploitation process of spatial information and parallel computing scenario. Experimental results conducted on simulated and real datasets show that PG-NLWSU is superior to comparison algorithms.

Post-stack seismic data interpolation using a fast non-local similarity matching algorithm

In the process of seismic data acquisition, there are often missing seismic traces in seismic records, so it is necessary to reconstruct the missing data to provide high-quality data for subsequent seismic data migration and reservoir inversion. Traditional interpolation methods for post-stack seismic data are based on the sparse constraint in the frequency-wavenumber (f-k) domain. However, the data completed using the interpolation method usually leads to the loss of some weak signals when the dip of the post-stack seismic profile is complex. In this paper, the missing data could be regarded as the result of irregular noise with the same waveform and the original signal but with the opposite polarity. The non-local similarity in the denoising algorithm is introduced as a low-rank promoting transform of the low-rank regularization term, and an interpolation method based on non-local similarity is proposed (NLS-WNNM). Furthermore, a fast matching algorithm is developed to search and match the non-local similarity of missing seismic traces (abbreviation FNLS-WNNM), which reduces the loss of weak signals during interpolation. The traditional interpolation method based on f-k domain is compared with the NLS-WNNM to highlight the advancement of the method. Finally, the interpolation test applied to field data confirmes the robustness of the proposed method.

NSTMR: Super Resolution of Sentinel-2 Images Using Nonlocal Nonconvex Surrogate of Tensor Multirank

In this article, we address the super-resolution problems, which estimate the high-resolution multispectral images from the multispectral Sentinel-2 (S2) images with different resolutions. Since S2 images can be naturally represented by tensors, we reformulate the degradation process as the tensor-based form. Based on the degradation mechanism, we build a tensor-based optimization model for S2 images super-resolution problem, which fully exploits intrinsic nonlocal spatial similarity and global spectral redundancy. Specifically, the model consists of the data fidelity term and the low-multirank regularizer tailored to thoroughly mining the inherent spatial-nonlocal and spectral redundancy. Then, we develop an efficient alternating direction method of multipliers algorithm with theoretically guaranteed convergence to tackle the resulting tensor optimization problem. Numerical experiments including simulated and real data demonstrate that our method outperforms the competing methods visually and qualitatively.

Hyperspectral-Multispectral Image Fusion via Tensor Ring and Subspace Decompositions

Fusion from a spatially low resolution hyperspectral image (LR-HSI) and a spectrally low resolution multispectral image (MSI) to produce a high spatial-spectral HSI (HR-HSI), known as hyperspectral super resolution, has risen to a preferred topic for reinforcing the spatial-spectral resolution of HSI in recent years. In this work, we propose a new model, namely, low-rank tensor ring decomposition based on tensor nuclear norm (LRTRTNN), for HSI-MSI fusion. Specifically, for each spectrally subspace cube, similar patches are grouped to exploit both the global low-rank property of LR-HSI and the nonlocal similarity of HR-MSI. Afterward, a joint optimization of all groups via the presented LRTRTNN approximation is implemented in a unified cost function. With the introduced tensor nuclear norm (TNN) constraint, all 3D tensor ring factors are no longer unfolded to suit the matrix nuclear norm used in conventional methods, and the internal tensor structure can be naturally retained. The alternating direction method of multipliers is introduced for coefficients update. Numerical and visual experiments on real data show that our LRTRTNN method outperforms most state-of-the-art algorithms in terms of fusing performance.

Artistic Color Virtual Reality Implementation Based on Similarity Image Restoration

In this paper, exploratory and innovative research is done on the implementation technique of artistic color virtual reality for similarity image recovery. Based on similarity images, a nonlocal natural image before the regular term is proposed to deal with the single‐image blind deblurring problem. This paper designs a new artistic color virtual reality realization technology based on similarity image restoration, which exploits the low‐rank property between nonlocal similarity blocks in images and combines a strong convex term to enhance the convexity of the artistic color virtual reality model. We analyze virtual reality interaction design from the perspective of art color design, sort out the concept and content of design, analyze the elements, design principles, and evaluation criteria included in virtual reality interaction art color design, and explore the conceptual principles of virtual reality interaction art color design. A full understanding of the characteristics of the medium of the virtual reality interaction can help us to better use this medium as a tool to create works that aim to bring higher quality and experiential feeling with a perceptual communication method that is closer to natural interaction. Combining the power of technology, artistic colourful thinking, and a design approach paves the way forward. The study shows that virtual reality technology can effectively improve the status quo and promote the cultivation of professional practice ability in art color design, which is conducive to the cultivation of applied design talents.

Mixed Noise Removal by Processing of Patches

Sonar images are degraded by mixed noise which has an adverse impact on detection and classification of underwater objects. Existing denoising methods of sonar images remove either additive noise or multiplicative noise. In this study, the mixed noise in sonar images, the additive Gaussian noise and the multiplicative speckle effect are handled by the data adaptive methods. A patch based denoising is applied in two phases to remove the additive Gaussian and multiplicative speckle noises. In the first phase, the adaptive processing of local patches is used to remove the additive Gaussian noise by exploiting the sonar image local sparsity. The PCA and SVD methods are used for denoising the noisy image patches and blocks of patches. In the second phase, the weighted maximum likelihood denoising of the nonlocal patches reduces the speckle effect by exploiting the non-local similarity in a probability distribution. Experiments on side scan sonar images are conducted and the results show the importance of removing both the additive and multiplicative components from the sonar images.

Signal Reconstruction Based on Probabilistic Dictionary Learning Combined with Group Sparse Representation Clustering

In order to make full use of nonlocal and local similarity and improve the efficiency and adaptability of the NPB-DL algorithm, this paper proposes a signal reconstruction algorithm based on dictionary learning algorithm combined with structure similarity clustering. Nonparametric Bayesian for Dirichlet process is firstly introduced into the prior probability modeling of clustering labels, and then, Dirichlet prior distribution is applied to the prior probability of cluster labels so as to ensure the analyticity and conjugation of the probability model. Experimental results show that the proposed algorithm is not only superior to other comparison algorithms in numerical evaluation indicators but also closer to the original image in terms of visual effects.

Total variation and block-matching 3D filtering-based image reconstruction for single-shot compressed ultrafast photography

Compressed ultrafast photography (CUP), as the fastest receive-only ultrafast imaging technology by combining steak imaging and compressed sensing (CS), has shown to be a powerful tool to measure ultrafast dynamic scenes. Through a reconstruction algorithm based on CS, CUP can capture the three-dimensional image information of non-repetitive transient events with a single exposure. However, it still suffers from poor image reconstruction quality on account of the super-high data compression ratio induced by the undersampling strategy. Here, we propose a total variation (TV) combined with block matching and 3D filtering (BM3D) reconstruction algorithm to improve the image quality of CUP, which is named as the TV-BM3D algorithm. The proposed algorithm can simultaneously exploit gradient sparsity and non-local similarity for image reconstruction by incorporating TV and BM3D denoisers. Both the numerical simulations and experimental results show that, compared with the two conventional two-step iterative shrinkage/thresholding and augmented Lagrangian algorithms in CUP, the TV-BM3D algorithm can not only improve the image reconstruction quality, but also strengthen the noise immunity of this technique. It is prospected that these improvements in image reconstruction will further promote the practical applications of CUP in capturing complex physical and biological dynamics.

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-Similarity

To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by the reliance on limited external training or mismatch of internal similarity patch instances, this study proposes a novel video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with the joint internal and external constraints is established utilizing the complementary advantages of both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A novel spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by the patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy, which achieves a better tradeoff between super-resolution accuracy and speed. Experimental results demonstrate that the DLSS-VSR algorithm achieves competitive super-resolution quality compared to other state-of-the-art algorithms in both subjective and objective evaluations.

Robust Nonlocal Low-Rank SAR Time Series Despeckling Considering Speckle Correlation by Total Variation Regularization

Outliers and speckle both corrupt time series of synthetic aperture radar (SAR) acquisitions. Owing to the coherence between SAR acquisitions, their speckle can no longer be regarded as independent. In this study, we propose an algorithm for nonlocal low-rank time series despeckling, which is robust against outliers and also specifically addresses speckle correlation between acquisitions. By imposing total variation regularization on the signal’s speckle component, the correlation between acquisitions can be identified, facilitating the extraction of outliers from unfiltered signals and the correlated speckle. This robustness against outliers also addresses matching errors and inaccuracies in the nonlocal similarity search. Such errors include mismatched data in the nonlocal estimation process, which degrade the denoising performance of conventional similarity-based filtering approaches. Multiple experiments on real and synthetic data assess the performance of the approach by comparing it with state-of-the-art methods. It provides filtering results of comparable quality but is not adversely affected by outliers. The source code is available at https://github.com/gbaier/nllrtv .

Recovery of image and video based on compressive sensing via tensor approximation and Spatio-temporal correlation

In recent years, block-based compressive sensing (BCS) has been extensively studied because it can reduce computational complexity and data storage by dividing the image into smaller patches, but the performance of the reconstruction algorithm is not satisfactory. In this paper, a new reconstruction model for image and video is proposed. The model makes full use of spatio-temporal correlation and utilizes low-rank tensor approximation to improve the quality of the reconstructed image and video. For image recovery, the proposed model obtains a low-rank approximation of a tensor formed by non-local similar patches, and improves the reconstruction quality from a spatial perspective by combining non-local similarity and low-rank property. For video recovery, the reconstruction process is divided into two phases. In the first phase, each frame of the video sequence is regarded as an independent image to be reconstructed by taking advantage of spatial property. The second phase performs tensor approximation through searching similar patches within frames near the target frame, to achieve reconstruction by putting the spatio-temporal correlation into full play. The resulting model is solved by an efficient Alternating Direction Method of Multipliers (ADMM) algorithm. A series of experiments show that the quality of the proposed model is comparable to the current state-of-the-art recovery methods.

Single Satellite Imagery Superresolution Based on Hybrid Nonlocal Similarity Constrained Convolution Sparse Coding

The traditional superresolution methods based on image patches often ignore the consistency between the overlapped patches, causing block effects in produced images. The convolutional sparse coding based superresolution method uses the translation invariance of the convolution filter to directly encode the entire image, maintaining consistency and good performance. In this article, we propose a novel approach to single-image superresolution reconstruction based on hybrid nonlocal similarity constrained convolution sparse coding. We first decompose the input image into a smooth part and a texture part. The Bayesian nonparametric model can use more prior information of the original image, so we replace the previous bicubic interpolation with this method to better reconstruct the residual high-frequency information in the smooth part. When reconstructing the texture part, this article proposes a nonlocal similarity constrained convolutional sparse coding method, which transforms the reconstruction of the texture part to minimize the convolution sparse coding noise of the feature maps and classifies the image patches in the search space by using the correlation coefficients as the structural information, avoiding unnecessary weight calculation. Several methods were tested on satellite images extensively. Both visual inspection and quantitative analysis results demonstrate that our method outperforms other state-of-the-art methods and increases noise immunity effectively.

Material decomposition for simulated dual-energy breast computed tomography via hybrid optimization method.

Dual-energy breast CT reconstruction has a potential application that includes separation of microcalcification from healthy breast tissue for assisting early breast cancer detection. To investigate and validate the noise suppression algorithm applied in the decomposition of the simulated breast phantom into microcalcification and healthy breast. The proposed hybrid optimization method (HOM) uses a simultaneous algebraic reconstruction technique (SART) output as a prior image, which is then incorporated into the self-adaptive dictionary learning. This self-adaptive dictionary learning seeks each group of patches to faithfully represent the learned dictionary, and the sparsity and non-local similarity of group patches are used to enforce the image regularization term of the prior image. We simulate a numerical phantom by adding different levels of Gaussian noise to test performance of the proposed method. The mean value of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and root mean square error (RMSE) for the proposed method are (49.043±1.571), (0.997±0.002), (0.003±0.001) and (51.329±1.998), (0.998±0.002), (0.003±0.001) for 35 kVp and 49 kVp, respectively. The PSNR of the proposed method shows greater improvement over TWIST (5.2%), SART (34.6%), FBP (40.4%) and TWIST (3.7%), SART (39.9%), FBP (50.3%) for 35 kVp and 49 kVp energy images, respectively. For the proposed method, the signal-to-noise ratio (SNR) of decomposed normal breast tissue (NBT) is (22.036±1.535), which exceeded that of TWIST, SART, and FBP by 7.5%, 49.6%, and 96.4%, respectively. The results reveal that the proposed algorithm achieves the best performance in both reconstructed and decomposed images under different levels of noise and the performance is due to the high sparsity and good denoising ability of minimization exploited to solve the convex optimization problem. This study demonstrates the potential of applying dual-energy reconstruction in breast CT to detect and separate clustered MCs from healthy breast tissues without noise amplification. Compared to other competing methods, the proposed algorithm achieves the best noise suppression performance for both reconstructed and decomposed images.

Multi-energy CT reconstruction using tensor nonlocal similarity and spatial sparsity regularization.

Multi-energy computed tomography (MECT) based on a photon-counting detector is an emerging imaging modality that collects projections at several energy bins with a single scan. However, the limited number of photons collected into the divided, narrow energy bins results in high quantum noise levels in reconstructed images. This study aims to improve MECT image quality by minimizing noise levels while retaining image details. A novel MECT reconstruction method was proposed by exploiting the nonlocal tensor similarity among interchannel images and spatial sparsity in single-channel images. Similar patches were initially extracted from the interchannel images in spectral and spatial domains, then stacked into a new three-order tensor. Intrinsic tensor sparsity regularization that combined the Tuker and canonical polyadic (CP) low-rank decomposition techniques were applied to exploit the nonlocal similarity of the formulated tensor. Spatial sparsity in single-channel images was modeled by total variation (TV) regularization that utilizes the compressibility of gradient image. A new MECT reconstruction model was established by simultaneously incorporating the intrinsic tensor sparsity and TV regularizations. The iterative alternating minimization method was utilized to solve the reconstruction model based on a flexible framework. The proposed method was applied to the digital phantom and real mouse data to assess its feasibility and reliability. The reconstruction and decomposition results in the mouse data were encouraging and demonstrated the ability of the proposed method in noise suppression while preserving image details, not observed with other methods. Imaging data from the digital phantom illustrated this method as achieving the best intuitive reconstruction and decomposition results among all compared methods. They reduced the root mean square error (RMSE) by 89.75%, 50.75%, and 36.54% on the reconstructed images compared with analytic, TV-based, and tensor-based methods, respectively. This phenomenon was also observed with decomposition results, where the RMSE was also reduced by 97.96%, 67.74%, 72.05%, respectively. In this study, we proposed a reconstruction method for photon counting detector-based MECT, using the intrinsic tensor sparsity and TV regularizations. Improvements in noise suppression and detail preservation in the digital phantom and real mouse data were validated by the qualitative and quantitative evaluations on the reconstruction and decomposition results, verifying the potential of the proposed method in MECT reconstruction.

Improving semantic part features for person re-identification with supervised non-local similarity

In person re-IDentification (re-ID) task, the learning of part-level features benefits from fine-grained information. To facilitate part alignment, which is a prerequisite for learning part-level features, a popular approach is to detect semantic parts with the use of human parsing or pose estimation. Such methods of semantic partition do offer cues to good part alignment but are prone to noisy part detection, especially when they are employed in an off-the-shelf manner. In response, this paper proposes a novel part feature learning method for re-ID, that suppresses the impact of noisy semantic part detection through Supervised Non-local Similarity (SNS) learning. Given several detected semantic parts, SNS first locates their center points on the convolutional feature maps for use as a set of anchors and then evaluates the similarity values between these anchors and each pixel on the feature maps. The non-local similarity learning is supervised such that: each anchor should be similar to itself and simultaneously dissimilar to any other anchors, thus yielding the SNS. Finally, each anchor absorbs features from all of the similar pixels on the convolutional feature maps to generate a corresponding part feature (SNS feature). We evaluate our method with extensive experiments conducted under both holistic and partial re-ID scenarios. Experimental results confirm that SNS consistently improves re-ID accuracy using human parsing or pose estimation, and that our results are on par with state-of-the-art methods.

Real‐time speech enhancement using optimised empirical mode decomposition and non‐local means estimation

In this study, the authors present a novel speech enhancement method by exploring the benefits of non-local means (NLM) estimation and optimised empirical mode decomposition (OEMD) adopting cubic-spline interpolation. The optimal parameters responsible for improving the performance are estimated using the path-finder algorithm. At first, the noisy speech signal is decomposed into many scaled signals called intrinsic-mode functions (IMFs) through the use of a temporary decomposition method is called sifting process in OEMD approach. The obtained IMFs are processed by NLM estimation technique in terms of non-local similarities present in each IMF, to reduce the ill-effects caused by interfering noise. The proposed NLM-based method is effective to eliminate the noise of less-frequency. Each IMF contains essential information about the signals, on some scale or frequency band. Field programmable gate array architecture is implemented on a Xilinx ISE 14.5 and the result of the proposed method offers good performance with a high signal-to-noise ratio (SNR) and low mean-square error compared to other approaches. The performance evolution is carried out for different speech signals taken from the TIMIT database and noises taken from the NOISEX-92 database in different SNR stages of 0, 5 and 10 dB, respectively.

A Modified Higher-Order Singular Value Decomposition Framework With Adaptive Multilinear Tensor Rank Approximation for Three-Dimensional Magnetic Resonance Rician Noise Removal.

The magnetic resonance (MR) images are acknowledged to be inevitably corrupted by Rician distributed noise, which adversely affected the image quality for diagnosis purpose. However, the traditional denoising methods may recover the images from corruptions with severe loss of detailed structure and edge information, which would affect the lesion detections and diagnostic decision making. In this study, we challenged improving the Rician noise removal from three-dimensional (3D) MR volumetric data through a modified higher-order singular value decomposition (MHOSVD) method. The proposed framework of MHOSVD involved a parameterized logarithmic nonconvex penalty function for low-rank tensor approximation (LRTA) algorithm optimization to suppress the image noise in MR dataset. Reference cubes were extracted from the noisy image volume, and block matching was performed according to nonlocal similarity for a fourth-order tensor construction. Then the LRTA problem was implemented by tensor factorization approaches, and the ranks of unfolding matrices along different modes of the tensor were estimated utilizing an adaptive nonconvex low-rank method. The denoised MR images were finally restored through aggregating all recovered cubes. We investigated the proposed algorithm MHOSVD on both the synthetic and real clinic 3D MR images for Rician noise removal, and relative results demonstrated that the MHOSVD can recover images with fine structures and detailed edge preservation with heavy noise even as high as 15% of the maximum intensity. The experimental results were also compared along with several classical denoising methods; the MHOSVD exhibited a sufficient improvement in noise-removal performance at various noise conditions in terms of different measurement indices such as peak signal-to-noise ratio and structural similarity index metrics. Based upon the comparison, the proposed MHOSVD has proved a relative state-of-the-art performance with excellent detailed structure reservation.