High Frequency Details Research Articles

A multi‐path residual network for image denoising based on edge prior information

AbstractEdge‐preserving denoising is important in image analysis. However, most existing methods suffer from smoothing and loss of high frequency detail at the edges. Aiming at this issue, a multi‐path residual network using edge prior information is proposed. Specifically, the edge is first recovered by a newly designed parallel edge extraction module. Then it is used as a priori to generate affine transform parameters to guide the weight distribution between noise and edge, so that the shallow feature extraction pays more attention to preserving edges. In the deep feature extraction stage, a multi‐scale attention unit is designed to provide the spatial neighbourhood information of the feature points to filter and activate the deep features, which further enhances the ability of the network to extract fine‐grained noisy features. Finally, by taking the difference between the noisy image and the noise feature obtained with residual learning, the denoised image is produced. Extensive experiments show that the PSNR is improved by 0.00–0.78 dB on grey image denoising and 0.00–2.69 dB on colour image denoising compared with others. The denoised image has a better ability to preserve image edges, high frequency details and visual perception.

Diabetic retinopathy data augmentation and vessel segmentation through deep learning based three fully convolution neural networks

ProblemThe eye fundus imaging is used for early diagnosis of most damaging concerns such as diabetic retinopathy, retinal detachments and vascular occlusions. However, the presence of noise, low contrast between background and vasculature during imaging, and vessel morphology lead to uncertain vessel segmentation. AimThis paper proposes a novel retinalblood vessel segmentation method for fundus imaging using a Difference of Gaussian (DoG) filter and an ensemble of three fully convolutional neural network (FCNN) models. MethodsA Gaussian filter with standard deviation σ1 is applied on the preprocessed grayscale fundus image and is subtracted from a similarly applied Gaussian filter with standard deviation σ2 on the same image. The resultant image is then fed into each of the three fully convolutional neural networks as the input. The FCNN models' output is then passed through a voting classifier, and a final segmented vessel structure is obtained.The Difference of Gaussian filter played an essential part in removing the high frequency details (noise) and thus finely extracted the blood vessels from the retinal fundus with underlying artifacts. ResultsThe total dataset consists of 3832 augmented images transformed from 479 fundus images. The result shows that the proposed method has performed extremely well by achieving an accuracy of 96.50%, 97.69%, and 95.78% on DRIVE, CHASE,and real-time clinical datasets respectively. ConclusionThe FCNN ensemble model has demonstrated efficacy in precisely detecting retinal vessels and in the presence of various pathologies and vasculatures.

Super-Resolution Neural Radiance Field via Learning High Frequency Details for High-Fidelity Novel View Synthesis

Super-Resolution Neural Radiance Field via Learning High Frequency Details for High-Fidelity Novel View Synthesis

SF-SRGAN: PROGRESSIVE GAN-BASED FACE HALLUCINATION

Abstract. Facial hallucination is a technique that has emerged recently thanks to advances in deep learning. It can be used in various tasks such as face recognition in the wild, human identification, pedestrian re-identification, face analysis, and so on. We propose a wavelet-integrated trained face hallucination model to synthesize photorealistic face images called SF-SRGAN. The multi-stage progressive hallucination strategy is based on GAN architecture. The proposed generator consists of sequential cascade modules, each of which increases the scale by 2×. Each module has a complex structure of two branches: a progressive face hallucination branch for feature extraction and reconstruction and edge-preserving branch for high frequency detail extraction. The main difference from other progressive GAN-based face hallucination networks is that the two branches fuse followed by each cascade 2×. The model is trained and tested on popular public face datasets such as the CelebA-HQ dataset, the LFW dataset, and the Helen dataset with promising photorealistic results.

A new limited-angle CT reconstruction algorithm based on the local anisotropic total variation restoration of continuity

A new limited-angle computed tomography(CT) reconstruction algorithm based on local detail information restoration in the CT image projection domain was proposed in this paper. The lost projection information could be repaired with the new algorithm called local anisotropic total variation repair method of sinogram (LATV). According to the continuity of local gradient in sinogram, local high frequency characteristics can be improved which are blurred by conventional TVM. A series of numerical reconstruction experiments were conducted to validate the new algorithm. The overall quality of images reconstructed by new algorithm was better than conventional TVM algorithm when quantitatively evaluated. Then, a local region with rich details was selected in order to evaluate the reconstruction effect of the internal details. From the perspective of three image quality evaluation parameters, it was shown that the local high frequency details were well repaired. Finally, the reconstruction quality was evaluated by actual CT experimental data. The new method also performed better than conventional method. Therefore, the new LATV algorithm proposed in this paper may be a new method suitable for limited-angle CT reconstruction problems.

Combining Photogrammetry and Photometric Stereo to Achieve Precise and Complete 3D Reconstruction.

Image-based 3D reconstruction has been employed in industrial metrology for micro-measurements and quality control purposes. However, generating a highly-detailed and reliable 3D reconstruction of non-collaborative surfaces is still an open issue. In this paper, a method for generating an accurate 3D reconstruction of non-collaborative surfaces through a combination of photogrammetry and photometric stereo is presented. On one side, the geometric information derived with photogrammetry is used in areas where its 3D measurements are reliable. On the other hand, the high spatial resolution capability of photometric stereo is exploited to acquire a finely detailed topography of the surface. Finally, three different approaches are proposed to fuse both geometric information and high frequency details. The proposed method is tested on six different non-collaborative objects with different surface characteristics. To evaluate the accuracy of the proposed method, a comprehensive cloud-to-cloud comparison between reference data and 3D points derived from the proposed fusion methods is provided. The experiments demonstrated that, despite correcting global deformation up to an average RMSE of less than 0.1 mm, the proposed method recovers the surface topography at the same high resolution as the photometric stereo.

Super resolution image reconstruction via dual dictionary learning in sparse environment

<span lang="EN-US">Patch-based super resolution is a method in which spatial features from a low-resolution (LR) patch are used as references for the reconstruction of high-resolution (HR) image patches. Sparse representation for each patch is extracted. These coefficients obtained are used to recover HR patch. One dictionary is trained for LR image patches, and another dictionary is trained for HR image patches and both dictionaries are jointly trained. In the proposed method, high frequency (HF) details required are treated as combination of main high frequency (MHF) and residual high frequency (RHF). Hence, dual-dictionary learning is proposed for main dictionary learning and residual dictionary learning. This is required to recover MHF and RHF respectively for recovering finer image details. Experiments are carried out to test the proposed technique on different test images. The results illustrate the efficacy of the proposed algorithm.</span>

High-quality facial-expression image generation for UAV pedestrian detection

For UAV pedestrian detection in the wild with perturbed parameters, such as lighting, distance, poor pixel and uneven distribution, traditional methods of image generation cannot accurately generate facial-expression images for UAV pedestrian detection. In this study, we propose an improved PR-SGAN (perceptual-remix-star generative adversarial network) method, which combines the improved interpolation method, perceptual loss function, and StarGAN to achieve high-quality facial-expression image generation. Experimental results show that the proposed method for discriminator-parameter update improves the generated facial-expression images in terms of image-generation evaluation indexes (5.80 dB in PSNR and 24% in SSIM); the generated images for generator-parameter update have high robustness against color. Compared to the traditional StarGAN method, the generated images are significantly improved in high frequency details and textures.

The effect of loss function on conditional generative adversarial networks

Conditional Generative Adversarial Network (cGAN) is a general purpose approach for many image-to-image translation tasks, which aims to translate images from one form to another resulting in high-quality translated images. In this paper, the loss function of the cGAN model is modified by combining the adversarial loss of state-of-the-art Generative Adversarial Network (GAN) models with a new combination of non-adversarial loss functions to enhance model performance and generate more realistic images. Specifically, the effect of the Wasserstein GAN (WGAN), the WGAN with Gradient Penalty (WGAN-GP), and least Squared GAN (lsGAN) adversarial loss functions are explored. Several comparisons are performed to select an optimized combination of L1 with structure, gradient, content-based, Kullback-Leibler divergence, and softmax non-adversarial loss functions. For experimentation purposes, the Facades dataset is used in case of image-to-image translation task. Peak-signal-to-noise-ratio (PSNR), Structural Similarity Index (SSIM), Universal Quality Index (UQI), and Visual Information Fidelity (VIF) are used to quantitatively evaluate the translated images. Based on our experimental results, the best combination of the loss functions for image-to-image translation on facade dataset is (WGAN) adversarial loss with (L1 and content) non-adversarial loss functions. The model generates fine structure images, and captures both high and low frequency details of translated images. Image in-painting and lesion segmentation is investigated to demonstrate practicality of proposed work.

Generative Adversarial Network Based on Multi-feature Fusion Strategy for Motion Image Deblurring

<p>Deblurring of motion images is a part of the field of image restoration. The deblurring of motion images is not only difficult to estimate the motion parameters, but also contains complex factors such as noise, which makes the deblurring algorithm more difficult. Image deblurring can be divided into two categories: one is the non-blind image deblurring with known fuzzy kernel, and the other is the blind image deblurring with unknown fuzzy kernel. The traditional motion image deblurring networks ignore the non-uniformity of motion blurred images and cannot effectively recover the high frequency details and remove artifacts. In this paper, we propose a new generative adversarial network based on multi-feature fusion strategy for motion image deblurring. An adaptive residual module composed of deformation convolution module and channel attention module is constructed in the generative network. Where, the deformation convolution module learns the shape variables of motion blurred image features, and can dynamically adjust the shape and size of the convolution kernel according to the deformation information of the image, thus improving the ability of the network to adapt to image deformation. The channel attention module adjusts the extracted deformation features to obtain more high-frequency features and enhance the texture details of the restored image. Experimental results on public available GOPRO dataset show that the proposed algorithm improves the peak signal-to-noise ratio (PSNR) and is able to reconstruct high quality images with rich texture details compared to other motion image deblurring methods.</p> <p> </p>

Deep Region Adaptive Denoising for Texture Enhancement

Image denoising is a highly challenging problem yet important task in image processing. Recently, many CNN-based denoising methods have made great performances but they commonly denoise blindly texture and non-texture regions together. This frequently leads to excessive texture smoothing and detail loss. To address this issue, we propose a novel region adaptive denoising network that adjusts the denoising strength according to region textureness. The proposed network conducts denoising tasks for texture and non-texture areas independently to improve the visual quality of the resulting image. To this end, we first generate a texture map that separates the image into texture and non-texture region. Because the difference between texture and non-texture is more evident in the frequency domain than in the spatial domain, the classification is performed through discrete cosine transform (DCT). Second, guided by the texture map, denoising is performed independently in two subnets, corresponding to the texture and non-texture regions. This allows the texture subnet to avoid excessive smoothing of high frequency details, and the non-texture subnet to maximize noise reduction in flat regions. Finally, a cross fusion that takes into intra- and inter-relationship between two resulting features is proposed. The cross fusion highlights the discriminant features from two subnets without degradation when combining the output of two subnets, and thus helps enhancing the performance of regions adaptive denoising. The superiority of the proposed method is validated on both synthetic and real-world images. We demonstrate that our method outperforms the existing methods in both objective scores and subjective image quality, in particular showing outstanding results in the restoration of visually sensitive textures. Furthermore, ablation study shows that our network can adaptively control the noise removal strength by manually manipulating the texture map and that the details of the texture region can be further improved. This also can simplify the cumbersome noise tuning process when deploying deep neural networks (DNN) architectures into products.

Information redundancy across spatial scales modulates early visual cortical processing

Visual images contain redundant information across spatial scales where low spatial frequency contrast is informative towards the location and likely content of high spatial frequency detail. Previous research suggests that the visual system makes use of those redundancies to facilitate efficient processing. In this framework, a fast, initial analysis of low-spatial frequency (LSF) information guides the slower and later processing of high spatial frequency (HSF) detail. Here, we used multivariate classification as well as time-frequency analysis of MEG responses to the viewing of intact and phase scrambled images of human faces to demonstrate that the availability of redundant LSF information, as found in broadband intact images, correlates with a reduction in HSF representational dominance in both early and higher-level visual areas as well as a reduction of gamma-band power in early visual cortex. Our results indicate that the cross spatial frequency information redundancy that can be found in all natural images might be a driving factor in the efficient integration of fine image details.

Paired-unpaired Unsupervised Attention Guided GAN with transfer learning for bidirectional brain MR-CT synthesis.

Medical image acquisition plays a significant role in the diagnosis and management of diseases. Magnetic Resonance (MR) and Computed Tomography (CT) are considered two of the most popular modalities for medical image acquisition. Some considerations, such as cost and radiation dose, may limit the acquisition of certain image modalities. Therefore, medical image synthesis can be used to generate required medical images without actual acquisition. In this paper, we propose a paired-unpaired Unsupervised Attention Guided Generative Adversarial Network (uagGAN) model to translate MR images to CT images and vice versa. The uagGAN model is pre-trained with a paired dataset for initialization and then retrained on an unpaired dataset using a cascading process. In the paired pre-training stage, we enhance the loss function of our model by combining the Wasserstein GAN adversarial loss function with a new combination of non-adversarial losses (content loss and L1) to generate fine structure images. This will ensure global consistency, and better capture of the high and low frequency details of the generated images. The uagGAN model is employed as it generates more accurate and sharper images through the production of attention masks. Knowledge from a non-medical pre-trained model is also transferred to the uagGAN model for improved learning and better image translation performance. Quantitative evaluation and qualitative perceptual analysis by radiologists indicate that employing transfer learning with the proposed paired-unpaired uagGAN model can achieve better performance as compared to other rival image-to-image translation models.

Super-resolution method using generative adversarial network for Gaofen wide-field-view images

Accurate information on the spatial distribution of crops is of great significance for scientific research and production practices. Such accurate information can be extracted from high-spatial-resolution optical remote sensing images. However, acquiring these images with a wide coverage is difficult. We established a model named multispectral super-resolution generative adversarial network (MS_SRGAN) for generating high-resolution 4-m images using Gaofen 1 wide-field-view (WFV) 16-m images. The MS_SRGAN model contains a generator and a discriminator. The generator network is composed of feature extraction units and feature fusion units with a symmetric structure, and the attention mechanism is introduced to constrain the spectral value of the feature map during feature extraction. The generator loss introduces feature loss to describe the feature difference of the image. This is realized using pre-trained discriminator parameters and a partial discriminator network. In addition to realizing feature loss, the discriminator network, which is a simple convolutional neural network, also realizes adversarial loss. Adversarial loss can provide some fake high frequency details to the generator to get a more sharpened image. In the Gaofen 1 WFV image test, the performance of MS_SRGAN was compared with that of Bicubic, EDSR, SRGAN, and ESRGAN. The results show that the spectral angle mapper (3.387) and structural similarity index measure (0.998) of MS_SRGAN are higher than those of the other models. In addition, the image obtained by MS_SRGAN is more realistic; its texture details and color distribution are closer to the reference image to a greater extent.

Single-image de-raining using joint filter and multi-scale deep alternate-connection dense network

This paper aims at developing an effective approach to remove rain layer from an individual image. The proposed is a two-stage method named JF-MADN consisting of a Joint Filter (JF) extracting high frequency details from rainy image and a Multi-scale deep Alternate-connection Dense Network (MADN) separating rain streaks in a coarse-to-fine manner. In the first stage, rainy image is enhanced first through a high frequency emphasis filter. Next, a gradient operator is deduced based on trigonometric function and Laplacian operator to generate rain gradient prior. By combining it with rain dark channel prior, a rain-prior weighted statistic order filter could be constructed to separate high and low frequency components of rainy image. Then, the result image through a guided filter followed by would be subtracted by the original low frequency component to obtain residuals for final operation with high frequency component to generate high quality initial rain layer. In the second stage, by adopting multi-scale mechanism, this study develops two new alternate-connected fully convolutional DenseNets in different scale, WNet-129 and NNet-96, to implement de-raining. By combining element-wise addition in our alternate-connection structure with original concatenation in fully convolutional DenseNet, features could be constantly connected in our network. Experimental results demonstrate that the proposed framework outperforms existing single-image de-raining algorithms on testing rainy images. Code available at: http://www.imagetech-polynomials.com/derain.html.

Detail Reconstruction of 3D Face Model from Single Image

With the development of computer vision technology, 3D face model reconstruction technology has a great breakthrough. In view of the current popular deep learning method, we use an algorithm based on neural network to restore the details of 3D face model. We design a high frequency detail generation network to generate displacement map based on the texture extracted from the original image. The displacement map contains a large number of high frequency detail features of the target face. By embedding the displacement map into the face model, the details of the model can be restored.

Case Studies on Discrete Wavelet Denoising via Kernel Based Nonlinear Component Analysis

Complex real world problems are essentially nonlinear. Linear models are relatively simple but inaccurate to describe the nonlinear aspects of dynamic system behaviors. Denoising techniques have been broadly applied to numerous applications in the spatial domain, frequency domain, and time domain. To increase the adaptability of denoising techniques to signal processing of arbitrary nonlinear systems, kernel based nonlinear component analysis is proposed to enhance wavelet denoising. In the multilevel wavelet decomposition, the low frequency approximations and high frequency details are produced at each level. Discrete wavelet transform (DWT) will help to decompose low frequency approximations exclusively at all the succeeding levels, while wavelet packet transform decomposes both approximations and high frequency details at each level. DWT is selected for wavelet denoising in this study, where details at each level and the approximation at specified level are all subject to simplification using nonlinear component analysis. Case studies of typical nonlinear denoising problems in various domains are conducted. The results manifest strong feasibility and adaptability across diverse denoising problems of nonlinear systems.

High Frequency Detail Accentuation in CNN Image Restoration.

Given its nature of statistical inference, machine learning methods incline to downplay relatively rare events. But in many applications statistical outliers carry disproportional significance; they can, if being left without special treatment as of now, cause CNNs to perform unsatisfactorily on instances of interests. This is the reason why existing CNN image restoration methods all suffer from the problem of blurred details. To overcome this weakness, we advocate a new training methodology to sensitize the CNNs to desired events even they are atypical. Specifically for image restoration, we propose a so-called high frequency feature accentuation space that promotes image sharpness and clarity by maximally discriminating the ground truth image and the CNN-restored image in atypical but semantically important features. Then we force the restored image to agree with the ground truth image in the feature accentuation space by including an auxiliary loss term in the training process. This aims at a high degree of agreement of the two images on high frequency constructs such as sharp edges and fine textures, i.e., penalizes image blurs. The new CNN design method is implemented and tested for tasks of image super-resolution and denoising. Experimental results demonstrate the achievement of our design objective.

Single Image De-Raining Using Spinning Detail Perceptual Generative Adversarial Networks

In this paper, Spinning Detail Perceptual Generative Adversarial Networks (SDP-GAN) is proposed for single image de-raining. The proposed method adopts the Generative Adversarial Network (GAN) framework and consists of two following networks: the rain streaks generative network G and the discriminative network D. To reduce the background interference, we propose a rain streaks generative network which not only focuses on the high frequency detail map of rainy image, but also directly reduces the mapping range from input to output. To further improve the perceptual quality of generated images, we modify the perceptual loss by extracting high-level features from discriminative network D, rather than pre-trained networks. Furthermore, we introduce a new training procedure based on the notion of self spinning to improve the final de-raining performance. Extensive experiments on the synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods.

Hybridisation of single‐image super‐resolution with edge‐aware multi‐focus image fusion for edge enrichment

To break the curtailment of digital imaging and retrieve appropriate information with different focused images, a novel edge-aware multi-focus image fusion is proposed by integrating the single-image super-resolution (SISR) method along with the edge-preserving filters. Initially, the multi-focus images are converted to high resolution images by estimating missing high frequency details from its blurred versions. With acquired high resolution images, smoothing by median and the anisotropic diffusion filters are performed to extract focused regions. An initial weight map is constructed by using maximum selection of pixel intensities of the difference images obtained with filtering. The precision of estimated weight map is further improved by exhibiting morphological operations and guided filter. Finally, the images are fused based on the optimised decision map. Simulation results of proposed fusion work are evaluated with seven metrics and the values are compared with different state-of-the-art methods. Both the quantitative and qualitative analyses showed the excellence of proposed work over other fusion methods.