Similar Patches Research Articles

RBDN: Residual Bottleneck Dense Network for Image Super-Resolution

Recent studies have shown that a super-resolution generative adversarial network (SRGAN) can significantly improve the quality of single-image super-resolution. However, existing SRGAN methods also have certain drawbacks, such as an insufficient feature utilization, a large number of parameters. To further enhance the visual quality, we thoroughly studied three key components of SRGAN, i.e., the network architecture, adversarial loss, and perceptual loss, and propose a DenseNet with Residual-in-Residual Bottleneck Block (RRBB), called a residual bottleneck dense network (RBDN), for single-image super-resolution. First, to improve the utilization of features between the various layers of the network, we adopted a dense cascading connection between layers. At the same time, to reduce the computational cost, we added a bottleneck structure to each layer, greatly reducing the number of network parameters and accelerating the convergence speed of the training process. Second, the proposed RRBB, as the basic network building unit, removes the batch normalization (BN) layer and employs the ELU function to reduce the opposite effects in the absence of BN. In addition, we applied an improved overall loss function during the model training process to stably train the model and further improve the realism of the reconstructed high-resolution image. To prove the superiority of our proposed model, we conducted a comprehensive and objective evaluation of the Peak Signal-to-Noise Ratio, structural similarity, learned perceptual image patch similarity, and other evaluation indicators obtained from the three test sets, i.e., Set5, Set14, and BSD100, from the recent state-of-the-art model. Finally, we conducted qualitative and quantitative analyses of the results obtained in terms of the evaluation indicators, the authenticity of the restored HR images, and textural details, which show the superiority of the RBDN model.

Accelerating patch-based low-rank image restoration using kd-forest and Lanczos approximation

Patch-based low-rank approximation (PLRA) via truncated singular value decomposition is a powerful and effective tool for recovering the underlying low-rank structure in images. Generally, it first performs an approximate nearest neighbors (ANN) search algorithm to group similar patches into a collection of matrices with reshaping them as vectors. The inherent correlation among similar patches makes these matrices have a low-rank structure. Then the singular value decomposition (SVD) is used to derive a low-rank approximation of each matrix by truncating small singular values. However, the conventional implementation of patch-based low-rank image restoration suffers from high computational cost of the ANN search and full SVD. To address this limitation, we propose a fast approximation method that accelerates the computation of PLRA using multiple kd-trees and Lanczos approximation. The basic idea of this method is to exploit an index kd-tree built from patch samples of the observed image and several small kd-trees built from overlapping regions of the image to accelerate the search for similar patches, and apply the Lanczos bidiagonalization procedure to obtain a fast low-rank approximation of patch matrix without computing the full SVD. Experimental results on image denoising and inpainting tasks demonstrate the efficiency and accuracy of our method.

A nonlocal low rank model for poisson noise removal

<p style='text-indent:20px;'>Patch-based methods, which take the advantage of the redundancy and similarity among image patches, have attracted much attention in recent years. However, these methods are mainly limited to Gaussian noise removal. In this paper, the Poisson noise removal problem is considered. Unlike Gaussian noise which has an identical and independent distribution, Poisson noise is signal dependent, which makes the problem more challenging. By incorporating the prior that a group of similar patches should possess a low-rank structure, and applying the maximum a posterior (MAP) estimation, the Poisson noise removal problem is formulated as an optimization one. Then, an alternating minimization algorithm is developed to find the minimizer of the objective function efficiently. Convergence of the minimizing sequence will be established, and the efficiency and effectiveness of the proposed algorithm will be demonstrated by numerical experiments.

A new nonlocal means based framework for mixed noise removal

Many image-denoising approaches seek to remove either additive white Gaussian noise (AWGN) or impulse noise (IN), because both types are easier to process when considered separately. However, images can be corrupted by a mixture of AWGN and IN during image acquisition and transmission. The major difficulty of mixed noise removal arises through the complex distribution of noise, which cannot be fitted by a simple parametric model. In this paper, a new nonlocal means based framework (NMF) is proposed. A median-type filter is used to detect the locations of outlier pixels; these pixels are then replaced by their nonlocal means, which makes the mixed noise distribution approximately Gaussian. To prove the effectiveness of our NMF, a low rank approximation combined with NMF (LRNM) model is presented for mixed noise removal. In the LRNM, we group similar nonlocal patches in a matrix and apply a low rank approximation to reconstruct the clean image. Gradient regularization is added to better preserve the image texture details. A convolutional neural network (CNN) combined with the NMF (NMF-CNN) is also presented, to prove the generality of the NMF. Experimental results show that LRNM and NMF-CNN achieve a strong mixed noise removal performance and also produce visually pleasing denoising results.

Digital Reconfiguration of a Single Arm 3-D Bowtie Antenna

This communication proposes the reconfiguration of a coplanar-fed single arm bowtie antenna structure by relying on the integration of a digital tunable capacitor (DTC). The coplanar-fed single arm bowtie antenna is presented in two configurations. The first configuration is a 2-D topology with a modified ground plane arrangement. The second configuration employs vertical folding for the ground plane as well as the radiating arm. In addition, vertical copper inserts are embedded within the bowtie single arm to further enhance the compactness of the structure and transform it into a 3-D topology. Both antennas are initially designed to operate at 868 MHz for Internet of Things (IoT) applications, with a digitally reconfigurable frequency operation for wireless communication. The 2-D antenna structure exhibits a 77% size reduction at the IoT operational frequency of 868 MHz in comparison with a similar rectangular patch antenna operating at the same frequency. On the other hand, the 3-D antenna structure reaches 84% of size reduction at the same frequency. A figure of merit (FOM) is derived to prove the superior performance of the presented antenna structure.

Heavy-Tailed Self-Similarity Modeling for Single Image Super Resolution.

Self-similarity is a prominent characteristic of natural images that can play a major role when it comes to their denoising, restoration or compression. In this paper, we propose a novel probabilistic model that is based on the concept of image patch similarity and applied to the problem of Single Image Super Resolution. Based on this model, we derive a Variational Bayes algorithm, which super resolves low-resolution images, where the assumed distribution for the quantified similarity between two image patches is heavy-tailed. Moreover, we prove mathematically that the proposed algorithm is both an extended and superior version of the probabilistic Non-Local Means (NLM). Its prime advantage remains though, which is that it requires no training. A comparison of the proposed approach with state-of-the-art methods, using various quantitative metrics shows that it is almost on par, for images depicting rural themes and in terms of the Structural Similarity Index (SSIM) with the best performing methods that rely on trained deep learning models. On the other hand, it is clearly inferior to them, for urban themed images and in terms of all metrics, especially for the Mean-Squared-Error (MSE). In addition, qualitative evaluation of the proposed approach is performed using the Perceptual Index metric, which has been introduced to better mimic the human perception of the image quality. This evaluation favors our approach when compared to the best performing method that requires no training, even if they perform equally in qualitative terms, reinforcing the argument that MSE is not always an accurate metric for image quality.

Depth Image Denoising Using Nuclear Norm and Learning Graph Model

Depth image denoising is increasingly becoming the hot research topic nowadays, because it reflects the three-dimensional scene and can be applied in various fields of computer vision. But the depth images obtained from depth camera usually contain stains such as noise, which greatly impairs the performance of depth-related applications. In this article, considering that group-based image restoration methods are more effective in gathering the similarity among patches, a group-based nuclear norm and learning graph (GNNLG) model was proposed. For each patch, we find and group the most similar patches within a searching window. The intrinsic low-rank property of the grouped patches is exploited in our model. In addition, we studied the manifold learning method and devised an effective optimized learning strategy to obtain the graph Laplacian matrix, which reflects the topological structure of image, to further impose the smoothing priors to the denoised depth image. To achieve fast speed and high convergence, the alternating direction method of multipliers is proposed to solve our GNNLG. The experimental results show that the proposed method is superior to other current state-of-the-art denoising methods in both subjective and objective criterion.

The erythema Q-score, an imaging biomarker for redness in skin inflammation.

Physician rating of cutaneous erythema is central to clinical dermatological assessment as well as quantification of outcome measures in clinical trials in a number of dermatologic conditions. However, issues with inter‐rater reliability and variability in the setting of higher Fitzpatrick skin types make visual erythema assessment unreliable. We developed and validated a computer‐assisted image‐processing algorithm (EQscore) to reliably quantify erythema (across a range of skin types) in the dermatology clinical setting.Our image processing algorithm evaluated erythema based upon green light suppression differentials between affected and unaffected skin. A group of four dermatologists used a 4‐point Likert scale as a human evaluation of similar erythematous patch tests. The algorithm and dermatologist scores were compared across 164 positive patch test reactions. The intra‐class correlation coefficient of groups and the correlation coefficient between groups were calculated. The EQscore was validated on and independent image set of psoriasis, minimal erythema dose testing and steroid‐induced blanching images. The reliability of the erythema quantification method produced an intra‐class correlation coefficient of 0.84 for the algorithm and 0.67 for dermatologists. The correlation coefficient between groups was 0.85. The EQscore demonstrated high agreement with clinical scoring and superior reliability compared with clinical scoring, avoiding the pitfalls of erythema underrating in the setting of pigmentation. The EQscore is easily accessible (http://lab.rockefeller.edu/krueger/EQscore), user‐friendly, and may allow dermatologists to more readily and accurately rate the severity of dermatological conditions and the response to therapeutic treatments.

Multiplex Networks to Characterize Seizure Development in Traumatic Brain Injury Patients.

Traumatic brain injury (TBI) may cause secondary debilitating problems, such as post-traumatic epilepsy (PTE), which occurs with unprovoked recurrent seizures, months or even years after TBI. Currently, the Epilepsy Bioinformatics Study for Antiepileptogenic Therapy (EpiBioS4Rx) has been enrolling moderate-severe TBI patients with the goal to identify biomarkers of epileptogenesis that may help to prevent seizure occurrence and better understand the mechanism underlying PTE. In this work, we used a novel complex network approach based on segmenting T1-weighted Magnetic Resonance Imaging (MRI) scans in patches of the same dimension (network nodes) and measured pairwise patch similarities using Pearson's correlation (network connections). This network model allowed us to obtain a series of single and multiplex network metrics to comprehensively analyze the different interactions between brain components and capture structural MRI alterations related to seizure development. We used these complex network features to train a Random Forest (RF) classifier and predict, with an accuracy of 70 and a 95% confidence interval of [67, 73%], which subjects from EpiBioS4Rx have had at least one seizure after a TBI. This complex network approach also allowed the identification of the most informative scales and brain areas for the discrimination between the two clinical groups: seizure-free and seizure-affected subjects, demonstrating to be a promising pilot study which, in the future, may serve to identify and validate biomarkers of PTE.

Image Inpainting Based on Structural Tensor Edge Intensity Model

In the exemplar-based image inpainting approach, there are usually two major problems: the unreasonable calculation of priority and only considering the color features in the patch lookup strategy. In this paper, we propose an image inpainting approach based on the structural tensor edge intensity model. First, we use the progressive scanning inpainting method to avoid the image filling order being affected by the priority function. Then, we use the edge intensity model to build the patches similarity function for correctly identifying the local image structure. Finally, the balance operator is used to restrict the excessive propagation of structural information to ensure the correct structural reconstruction. The experimental results show that the our approach is comparable and even superior to some state-of-the-art inpainting algorithms.

Non-local mean denoising using multiple PET reconstructions

ObjectivesNon-local mean (NLM) filtering has been broadly used for denoising of natural and medical images. The NLM filter relies on the redundant information, in the form of repeated patterns/textures, in the target image to discriminate the underlying structures/signals from noise. In PET (or SPECT) imaging, the raw data could be reconstructed using different parameters and settings, leading to different representations of the target image, which contain highly similar structures/signals to the target image contaminated with different noise levels (or properties). In this light, multiple-reconstruction NLM filtering (MR-NLM) is proposed, which relies on the redundant information provided by the different reconstructions of the same PET data (referred to as auxiliary images) to conduct the denoising process.MethodsImplementation of the MR-NLM approach involved the use of twelve auxiliary PET images (in addition to the target image) reconstructed using the same iterative reconstruction algorithm with different numbers of iterations and subsets. For each target voxel, the patches of voxels at the same location are extracted from the auxiliary PET images based on which the NLM denoising process is conducted. Through this, the exhaustive search scheme performed in the conventional NLM method to find similar patches of voxels is bypassed. The performance evaluation of the MR-NLM filter was carried out against the conventional NLM, Gaussian and bilateral post-reconstruction approaches using the experimental Jaszczak phantom and 25 whole-body PET/CT clinical studies.ResultsThe signal-to-noise ratio (SNR) in the experimental Jaszczak phantom study improved from 25.1 when using Gaussian filtering to 27.9 and 28.8 when the conventional NLM and MR-NLM methods were applied (p value < 0.05), respectively. Conversely, the Gaussian filter led to quantification bias of 35.4%, while NLM and MR-NLM approaches resulted in a bias of 32.0% and 31.1% (p value < 0.05), respectively. The clinical studies further confirm the superior performance of the MR-NLM method, wherein the quantitative bias measured in malignant lesions (hot spots) decreased from − 12.3 ± 2.3% when using the Gaussian filter to − 3.5 ± 1.3% and − 2.2 ± 1.2% when using the NLM and MR-NLM approaches (p value < 0.05), respectively.ConclusionThe MR-NLM approach exhibited promising performance in terms of noise suppression and signal preservation for PET images, thus translating into higher SNR compared to the conventional NLM approach. Despite the promising performance of the MR-NLM approach, the additional computational burden owing to the requirement of multiple PET reconstruction still needs to be addressed.

Color and direction-invariant nonlocal self-similarity prior and its application to color image denoising

Nonlocal self-similarity (NSS) is one of the most commonly used priors in computer vision and image processing. It aims to make use of the fact that a natural image often possesses many repetitive local patterns, and thus a local image patch always has many similar patches across the image. Through compensatively integrating these similar image patches, their insightful patterns hiding under corrupted noises can be intrinsically extracted. However, for using this prior knowledge, current methods search the similar patches by using simple block matching strategy with Euclidean distance, which largely ignores those patches containing similar local patterns but with different texture-directions and colors. To more sufficiently explore similar patches over an image, in this paper, we propose two new representations for image patches, which facilitate an easy NSS prior for measuring direction-invariant and color-invariant nonlocal self-similarity possessed by image patches. Specifically, based on this prior term, we formulate the color image denoising problem as a concise Bayesian posterior estimation framework, and design an efficient expectation-maximization (EM) algorithm to solve it. A series of experiments implemented on simulated and real noisy color images demonstrate the superiority of the proposed method as compared with the state-of-the-arts both visually and quantitatively, verifying the potential usefulness of this new NSS prior.

Nonlocal Tensor-Based Sparse Hyperspectral Unmixing

Sparse unmixing is an important technique for analyzing and processing hyperspectral images (HSIs). Simultaneously exploiting spatial correlation and sparsity improves substantially abundance estimation accuracy. In this article, we propose to exploit nonlocal spatial information in the HSI for the sparse unmixing problem. Specifically, we first group similar patches in the HSI, and then unmix each group by imposing simultaneous a low-rank constraint and joint sparsity in the corresponding third-order abundance tensor. To this end, we build an unmixing model with a mixed regularization term consisting of the sum of the weighted tensor trace norm and the weighted tensor <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{2,1}$ </tex-math></inline-formula> -norm of the abundance tensor. The proposed model is solved under the alternating direction method of multipliers framework. We term the developed algorithm as the nonlocal tensor-based sparse unmixing algorithm. The effectiveness of the proposed algorithm is illustrated in experiments with both simulated and real hyperspectral data sets.

Video Denoising by Combining Patch Search and CNNs

Non-local patch-based methods were until recently the state of the art for image denoising but are now outperformed by CNNs. In video denoising, however, they are still competitive with CNNs, as they can effectively exploit the video temporal redundancy, which is a key factor to attain high denoising performance. The problem is that CNN architectures are not compatible with the search for self-similarities. In this work, we propose a simple, yet efficient way to feed video self-similarities to a CNN. The non-locality is incorporated into the network via a first non-trainable layer which finds for each patch in the input image its most similar patches in a search region. The central values of these patches are then gathered in a feature vector which is assigned to each image pixel. This information is presented to a CNN which is trained to predict the clean image. We apply the proposed method to image and video denoising. In the case of video, the patches are searched for in a 3D spatiotemporal volume. The proposed method achieves state-of-the-art results.

Recovery of image and video based on compressive sensing via tensor approximation and Spatio-temporal correlation

In recent years, block-based compressive sensing (BCS) has been extensively studied because it can reduce computational complexity and data storage by dividing the image into smaller patches, but the performance of the reconstruction algorithm is not satisfactory. In this paper, a new reconstruction model for image and video is proposed. The model makes full use of spatio-temporal correlation and utilizes low-rank tensor approximation to improve the quality of the reconstructed image and video. For image recovery, the proposed model obtains a low-rank approximation of a tensor formed by non-local similar patches, and improves the reconstruction quality from a spatial perspective by combining non-local similarity and low-rank property. For video recovery, the reconstruction process is divided into two phases. In the first phase, each frame of the video sequence is regarded as an independent image to be reconstructed by taking advantage of spatial property. The second phase performs tensor approximation through searching similar patches within frames near the target frame, to achieve reconstruction by putting the spatio-temporal correlation into full play. The resulting model is solved by an efficient Alternating Direction Method of Multipliers (ADMM) algorithm. A series of experiments show that the quality of the proposed model is comparable to the current state-of-the-art recovery methods.

Aquatic beetles influence colonization of disparate taxa in small lentic systems.

Structure of natural communities is shaped by both abiotic characteristics and the ongoing processes of community assembly. Important to this process are the habitat selection behaviors and subsequent survival of colonists, both in the context of temporal changes in the abiotic characteristics and priority effects driven by earlier colonists. Aquatic beetles are prevalent in temporary freshwater systems, form speciose assemblages, and are often early colonists of temporary ponds. While beetles have the potential to influence community structure through post‐colonization interactions (predation and competition), our goal was to determine whether the presence of beetle assemblages (versus patches without beetles) influences the colonization and oviposition of a diverse group of animals in a naturally colonized experimental landscape. We established mesocosms that either contained existing beetle assemblages or contained no beetles and assessed abundances of subsequent colonists. Treefrogs, Hyla chrysoscelis, and mosquitoes, Culex restuans, both deposited fewer eggs in patches containing beetle assemblages, while two beetles, Copelatus glyphicus and Paracymus, colonized those patches at lower rates. One beetle, Helophorus linearis, colonized patches containing beetle assemblages at higher rates, while two beetles, Berosus infuscatus and Tropisternus lateralis, exhibited no colonization differences between treatments. Overall, there were no differences in the assemblage structure or richness of beetles that colonized patches. Our results illustrate the importance of species‐specific habitat selection behavior in determining the species composition of habitat patches, while emphasizing the role of priority effects in influencing patterns of community assembly. Habitat selection in response to abiotic and biotic characteristics of habitat patches can potentially create greater spatiotemporal niche separation among the numerous, often closely related species (phylogenetically and trophically), that can be simultaneously found in similar patches across landscapes.

Multi‐scale patches based image denoising using weighted nuclear norm minimisation

As a prior knowledge, non-local self-similarity (NSS) has been widely utilised in ill-posed problems. Actually, similar textures appear not only in a single scale, but also in different scales. Unlike most existing patch-based methods that only explore NSS in the same scale, a multi-scale patches based image denoising algorithm is proposed in this study. The authors have designed a multi-scale strategy to expand the search space of block-matching, which will increase the probability of finding more similar patches. After that, the weighted nuclear norm minimisation (WNNM) algorithm is employed to reveal latent clean patches. With the join of the multi-scale framework, the performance of WNNM can be improved. The proposed algorithm can be used to solve NSS-based image restoration tasks. In this study, mainly image denoising is studied, and its effectiveness is derived through experiments on widely used test images.

Adaptive iterative global image denoising method based on SVD

Based on the image self-similarity and singular value decomposition (SVD) techniques, the authors propose an iterative adaptive global denoising method. For the structural differences between image patches, they adaptively determine the size of the search window. In each window, a similar image patch matrix is constructed based on the multi-scale similarity measure. In order to ensure the speed of the method, the adaptive step size and the number of image patches are introduced, and all image patches are denoised in different iterations. This not only ensures the speed of the method, suppresses residual noise, but also reduces the artefacts caused by the fixed step size and the number of image patches. Therefore, the problem of image denoising is converted to the estimation of low-rank matrix. New singular values are estimated according to the noise level, and similar image patch matrices without noise are estimated using them and corresponding singular vectors. Experimental results show that compared with the state-of-the-art denoising algorithms, this method has a higher PSNR and FSIM, and has a good visual effect. The new method can be applied to image and video restoration, target recognition and image classification.

Fusion of External and Internal Prior Information for the Removal of Gaussian Noise in Images.

In this paper, a new method for the removal of Gaussian noise based on two types of prior information is described. The first type of prior information is internal, based on the similarities between the pixels in the noisy image, and the other is external, based on the index or pixel location in the image. The proposed method focuses on leveraging these two types of prior information to obtain tangible results. To this end, very similar patches are collected from the noisy image. This is done by sorting the image pixels in ascending order and then placing them in consecutive rows in a new two-dimensional image. Henceforth, a principal component analysis is applied on the patch matrix to help remove the small noisy components. Since the restored pixels are similar or close in values to those in the clean image, it is preferable to arrange them using indices similar to those of the clean pixels. Simulation experiments show that outstanding results are achieved, compared to other known methods, either in terms of image visual quality or peak signal to noise ratio. Specifically, once the proper indices are used, the proposed method achieves PSNR value better than the other well-known methods by >1.5 dB in all the simulation experiments.

Material decomposition for simulated dual-energy breast computed tomography via hybrid optimization method.

Dual-energy breast CT reconstruction has a potential application that includes separation of microcalcification from healthy breast tissue for assisting early breast cancer detection. To investigate and validate the noise suppression algorithm applied in the decomposition of the simulated breast phantom into microcalcification and healthy breast. The proposed hybrid optimization method (HOM) uses a simultaneous algebraic reconstruction technique (SART) output as a prior image, which is then incorporated into the self-adaptive dictionary learning. This self-adaptive dictionary learning seeks each group of patches to faithfully represent the learned dictionary, and the sparsity and non-local similarity of group patches are used to enforce the image regularization term of the prior image. We simulate a numerical phantom by adding different levels of Gaussian noise to test performance of the proposed method. The mean value of peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and root mean square error (RMSE) for the proposed method are (49.043±1.571), (0.997±0.002), (0.003±0.001) and (51.329±1.998), (0.998±0.002), (0.003±0.001) for 35 kVp and 49 kVp, respectively. The PSNR of the proposed method shows greater improvement over TWIST (5.2%), SART (34.6%), FBP (40.4%) and TWIST (3.7%), SART (39.9%), FBP (50.3%) for 35 kVp and 49 kVp energy images, respectively. For the proposed method, the signal-to-noise ratio (SNR) of decomposed normal breast tissue (NBT) is (22.036±1.535), which exceeded that of TWIST, SART, and FBP by 7.5%, 49.6%, and 96.4%, respectively. The results reveal that the proposed algorithm achieves the best performance in both reconstructed and decomposed images under different levels of noise and the performance is due to the high sparsity and good denoising ability of minimization exploited to solve the convex optimization problem. This study demonstrates the potential of applying dual-energy reconstruction in breast CT to detect and separate clustered MCs from healthy breast tissues without noise amplification. Compared to other competing methods, the proposed algorithm achieves the best noise suppression performance for both reconstructed and decomposed images.