Clean Image Research Articles

Self-supervised Blind2Unblind deep learning scheme for OCT speckle reductions.

As a low-coherence interferometry-based imaging modality, optical coherence tomography (OCT) inevitably suffers from the influence of speckles originating from multiply scattered photons. Speckles hide tissue microstructures and degrade the accuracy of disease diagnoses, which thus hinder OCT clinical applications. Various methods have been proposed to address such an issue, yet they suffer either from the heavy computational load, or the lack of high-quality clean images prior, or both. In this paper, a novel self-supervised deep learning scheme, namely, Blind2Unblind network with refinement strategy (B2Unet), is proposed for OCT speckle reduction with a single noisy image only. Specifically, the overall B2Unet network architecture is presented first, and then, a global-aware mask mapper together with a loss function are devised to improve image perception and optimize sampled mask mapper blind spots, respectively. To make the blind spots visible to B2Unet, a new re-visible loss is also designed, and its convergence is discussed with the speckle properties being considered. Extensive experiments with different OCT image datasets are finally conducted to compare B2Unet with those state-of-the-art existing methods. Both qualitative and quantitative results convincingly demonstrate that B2Unet outperforms the state-of-the-art model-based and fully supervised deep-learning methods, and it is robust and capable of effectively suppressing speckles while preserving the important tissue micro-structures in OCT images in different cases.

Retinal image preprocessing techniques: Acquisition and cleaning perspective

AbstractImage preprocessing is a method to transform raw image data into clean image data. The objective of preprocessing is to improve the image data by suppressing undesired distortions. Enhancement of some image features which are relevant for further processing of image and analysis task is also done in preprocessing. Screening and diagnosis of various eye diseases like diabetic retinopathy, Choroidal Neovascularization(CNV), DRUSEN, etc. are possible using digital retinal images. This paper aims to provide a better understanding and knowledge of the computer algorithms used for retinal image preprocessing. In this paper, various image preprocessing techniques are incorporated such as color correction, color space selection, noise reduction, and contrast enhancement on retinal images. Retinal blood vessels are better seen in Green color space instead of Red or Blue color space. Noise reduction through Block matching and 3D(BM3D) techniques show a significant result as compared to Total Variation Filter (TVF) and Bilateral Filter (BLF). Contrast enhancement through Contrast Limited Adaptive Histogram Equalization (CLAHE) outperforms Global Equalization (GE) or Adaptive Histogram Equalization (AHE). Evaluation parameters such as Mean square error, Peak Signal Noise ratio, Structured similarity index measures, and Normalized root mean square error values for BM3D noise filtering are 0.0029, 25.3370, 0.6839 and 0.0998 respectively which shows that BM3D outperforms the others.

Using CNN with Multi-Level Information Fusion for Image Denoising

Deep convolutional neural networks (CNN) with hierarchical architectures have obtained good results for image denoising. However, in some cases where the noise level is unknown and the image background is complex, it is challenging to obtain robust information through CNN. In this paper, we present a multi-level information fusion CNN (MLIFCNN) in image denoising containing a fine information extraction block (FIEB), a multi-level information interaction block (MIIB), a coarse information refinement block (CIRB), and a reconstruction block (RB). In order to adapt to more complex image backgrounds, FIEB uses parallel group convolution to extract wide-channel information. To enhance the robustness of the obtained information, a MIIB uses residual operations to act in two sub-networks for implementing the interaction of wide and deep information to adapt to the distribution of different noise levels. To enhance the stability of the training denoiser, CIRB stacks common and group convolutions to refine the obtained information. Finally, RB uses a residual operation to act in a single convolution in order to obtain the resultant clean image. Experimental results show that our method is better than many other excellent methods, both in terms of quantitative and qualitative aspects.

DADRnet: Cross-domain image dehazing via domain adaptation and disentangled representation

Recent years have witnessed remarkable progress of learning-based methods in single image dehazing. Among them, dehazing methods trained on the synthetic images cannot adapt to real hazy ones due to the domain gap, and domain adaptation methods only concentrate on creating a mapping or extracting shared features regardless of representations of deep features. In this paper, we propose an innovative cross-domain dehazing architecture that integrates domain adaptation and disentangled representation. Specifically, we first construct a shared encoder to map synthetic and real hazy images to a latent space and narrow their domain gap at the feature level. Then we utilize a separator to separate the hazy image features into haze-free representations and haze ones. We introduce feature consistency loss and orthogonal loss to further guide the disentanglement process. And then, we utilize a decoder to produce clean images from haze-free features and introduce both supervised and unsupervised losses to guide the training process. Moreover, we also propose to reconstruct the images from both domains by recombining the separated features, which guarantees information completeness. Experiments demonstrate that our method is on par with state-of-the-art methods. Codes are available at https://github.com/lixiaopeng123456/DADRnet.

Stealthy and Flexible Trojan in Deep Learning Framework

Deep neural networks are increasingly used as the critical component of applications, bringing high computational costs. Many practitioners host their models on third-party platforms. This practice exposes DNNs to risks: A third party hosting the model may use a malicious deep learning framework to implement a backdoor attack. Our goal is to develop the realistic potential for backdoor attacks in third-party hosting platforms. We introduce a threatening and realistically implementable backdoor attack that is highly stealthy and flexible. We inject trojans by hijacking the built-in functions of the deep learning framework. Unlike existing backdoor attacks, the proposed sequential trigger is a specific sequence of clean image sets. Moreover, our attack is model agnostic and does not require retraining the model or modifying the parameters. Its stealthy is that injecting trojans will not change the models prediction for a clean image. Its flexibility lies in that adversary can remodify the trojan behavior at any time. Extensive experiments on multiple benchmarks with different frameworks demonstrate that our attack achieves a perfect success rate. And we can inject multiple trojans which do not affect each other at the same time. Analysis and experiments further show that state-of-the-art defenses are ineffective against our attacks.

Using High-Level Representation Difference Constraint and Relative Reconstruction Constraint for Defending against Adversarial Attacks

Adversarial examples in which imperceptible perturbations to the input can easily subvert a well-trained model’s prediction pose huge potential security threats to deep neural networks (DNNs). As an effective way to resist adversarial samples, input reconstruction can eliminate the antagonism of adversarial examples in the inference process without involving modifications to the target model’s structure and parameters. However, preprocessing inputs often results in some loss of the protected model’s prediction accuracy. In this paper, we introduce a new input reconstruction method that adopts the high-level representation difference constraint and the relative reconstruction constraint on a dual autoencoder to advance the prediction accuracy of the protected model. The high-level representation difference constraint utilizes the gap between the protected model’s high-level representations, activated by clean images, and their adversarial examples to guide the training of the dual autoencoder. Additionally, the relative reconstruction constraint is imposed on latent representations and their noisy versions to advance the robustness of the dual autoencoder to tiny perturbations. The extensive empirical experiments on two real datasets, CIFAR-10 and ImageNet, show that the presented approach demonstrates exceptional performance in resisting different types of attacks.

Mutual channel prior guided dual-domain interaction network for single image raindrop removal

Removing raindrops adhering to lenses or glass is challenging since raindrops have more complex internal structures and optical effects than rain streaks. Existing raindrop removal methods typically employ single-channel masks for raindrop representation. However, these methods ignore the complementarity of priors between different channels, thereby losing the color and transparency information of raindrops. Moreover, simply applying the same processing across different pixels is insufficient to deal with spatially-varying degradation. To address these limitation, we propose a novel Mutual channel prior guided Dual-domain Interaction Network (MDINet) for single image raindrop removal. Specifically, to effectively guide the restoration of the raindrop region, we propose a mutual channel prior by modeling the multi-channel raindrop soft mask, in which the enriched color and transparency information of raindrops are represented and learned by our model. To tackle the spatially-varying degradation, we design the dual-domain interaction block to capture global low-frequency content and local high-frequency details. Simultaneously, a spatially adaptive modulation block is devised to handle complicated and diverse raindrops. Moreover, we propose a contrastive discrimination strategy to motivate the restored image to approach the clean image and stay away from the raindrop image in the representation space. Extensive experiments on benchmark datasets and real-world images indicate that the proposed method achieves qualitative and quantitative superiority against state-of-the-art methods.

Self-supervised structural similarity-based convolutional neural network for cardiac diffusion tensor image denoising.

Diffusion tensor imaging (DTI) is a promising technique for non-invasively investigating the myocardial fiber structures of human heart. However, low signal-to-noise ratio (SNR) has been a major limit of cardiac DTI to prevent us from detecting myocardium structure accurately. Therefore, it is important to remove the effect of noise on diffusion weighted (DW) images. Although the conventional and deep learning-based denoising methods have shown the potential to deal with effectively the noise in DW images, most of them are redundant information dependent or require the noise-free images as golden standard. In addition, the existed DW image denoising methods often suffer from problems of over-smoothing. To address these issues, we propose a self-supervised learning model, structural similarity based convolutional neural network with edge-weighted loss (SSECNN), to remove the noise effectively in cardiac DTI. Considering that the DW images acquired along different diffusion directions have structural similarity, and the noise in these DW images is independent and identically distributed, the structural similarity-based matching algorithm is proposed to search for the most similar DW images. Such similar noisy DW image pairs are then used as the input and target of the denoising network SSECNN, which consists of several convolutional and residual blocks. Through the self-supervised training with these image pairs, the network can restore the clean DW images and retain the correlations between the denoised DW images along different directions. To avoid the over-smoothing problem, we design a novel edge-weighted loss which enables the network to adaptively adjust the loss weights with iterations and therefore to improve the detail preserve ability of the model. To verify the superiority of the proposed method, comparisons with state-of-the-art (SOTA) denoising methods are performed on both synthetic and real acquired DTI datasets. Experimental results show that SSECNN can effectively reduce the noise in the DW images while preserving detailed texture and edge information and therefore achieve better performance in DTI reconstruction. For synthetic dataset, compared to the SOTA method, the root mean square error (RMSE), peak signal-to-noise ratio (PSNR), and structure similarity index measure (SSIM) between the denoised DW images obtained with SSECNN and noise-free DW images are improved by 6.94%, 1.98%, and 0.76% respectively when the noise level is 10%. As for the acquired cardiac DTI dataset, the SSECNN method could significantly improve SNR and contrast to noise ratio (CNR) of cardiac DW images and achieve more regular helix angle (HA) and transverse angle (TA) maps. The ablation experimental results validate that using the structure similarity-based method to search the similar DW image pairs yield the smallest loss, and with the help of the edge-weighted loss, the denoised DW images and diffusion metric maps can preserve more details. The proposed SSECNN method can fully explore the similarity between the DW images along different diffusion directions. Using such similarity and an edge-weighted loss enable us to denoise cardiac DTI effectively in a self-supervised manner. Our method can overcome the redundancy information dependence and over-smoothing problem of the SOTA methods.

HLA-HOD: Joint High-Low Adaptation for Object Detection in Hazy Weather Conditions

Object detection remains challenging in hazy weather conditions due to the poor visibility of captured images. There are currently two types of detectors capable of adapting to varying weather conditions: (i) low-level adaptation methods that combine one detector with an additional dehazing network and (ii) high-level adaptation methods that explore various kinds of domain adaptation knowledge. However, neither of these approaches can achieve desirable performance due to their inherent limitations. We raise an intriguing question—if combining both low-level adaptation and high-level adaptation, can improve the generalization ability of a detector in hazy weather conditions? To answer it, we propose a Joint High-Low Adaptation Object Detection paradigm (HLA-HOD) in hazy weather conditions. By combining both low-level adaptation and high-level adaptation, HLA-HOD achieves superior performance on hazy images without requiring ground-truth bounding boxes or clean images. Extensive experiments demonstrate that our method outperforms state-of-the-art low-level and high-level adaptation methods by a large margin both quantitatively and qualitatively.

Residual dense network with non-residual guidance for blind image denoising

Residual learning is one of the most effective components in blind image denoising. It learns to estimate the noise instead of the clean image itself. A shortcoming of residual learning is that it cannot capture hierarchical features efficiently because there are no connections between layers to extract multi-level features. Therefore, residual dense network (RDN) adds dense connections among layers in the residual block to facilitate extraction of hierarchical features. Although RDN is a superior non-blind denoiser, it is only better than state-of-the-art methods in blind denoising when noise level is high. In this paper, we employ the concept of Wiener filters and add a non-residual network as a guidance for RDN. The non-residual network predicts the signal instead of the noise to assist RDN in achieving satisfactory performance across all noise levels. Then, a guidance network combines the outputs from both residual and non-residual networks to generate the final denoised output. Experimental results show that the new architecture performs better than state-of-the-art blind denoisers quantitatively and image quality of the new method is also better than existing methods.

Energy based denoising convolutional neural network for image enhancement

ABSTRACT Deep learning technologies like convolutional neural networks have recently become popular in the field of image denoising. A combined discrete wavelet transform (DWT) and denoising CNN (DnCNN)-based technique suggested in this paper can be used to adaptively boost image contrast. Images were captured in a wide variety of lighting circumstances, which means that not all of them are of the highest quality. As a result of the limited dynamic range of the pixel values, the overall image quality suffers when a shot is taken in low light. We employ a lossy method based on a structural similarity index to keep as much of the original texture in the images as feasible. Current advancements in image denoising, such as energy band analysis, very deep architecture, learning algorithms, dense-sparse-dense training, and regularization approaches, were applied in the hybrid DWT-Denoising convolutional neural networks. DnCNN eliminates the latent information in the hidden layers by design, resulting in a clean image. The DWT-DnCNN delivers SSIM values that are 2% higher than those of the standard RIDNet. Our hybrid DWT-DnCNN-based contrast enhancement strategy surpasses existing methods, according to simulation outcomes.

Image Denoising based on Deep Learning

Image denoising has always been one of the research hotspots in the field of image processing, which aims to remove the noise from the imaging device or external noise environment and other interfering factors in the image to restore the noisy image to the original clean and noise-free image. Mature algorithms and machine learning techniques have been developed previously for different application situations and specific computer vision works. Image denoising based on deep learning can adaptively learn image content and is suitable for image denoising tasks in high-noise environments. This paper builds a model based on the representative image denoising algorithm DnCNN, and discusses the performance difference with other denoising algorithms. All the results show the new methods are usually more efficient than traditional ones which can process pictures that are under more complex conditions.

Regularized Denoising Masked Visual Pretraining for Robust Embodied PointGoal Navigation

Embodied PointGoal navigation is a fundamental task for embodied agents. Recent works have shown that the performance of the embodied navigation agent degrades significantly in the presence of visual corruption, including Spatter, Speckle Noise, and Defocus Blur, showing the weak robustness of the agent. To improve the robustness of embodied navigation agents to various visual corruptions, we propose a navigation framework called Regularized Denoising Masked AutoEncoders Navigation (RDMAE-Nav). In a nutshell, RDMAE-Nav mainly consists of two modules: a visual module and a policy module. In the visual module, a self-supervised pretraining method, dubbed Regularized Denoising Masked AutoEncoders (RDMAE), is designed to enable the Vision Transformers (ViT)-based visual encoder to learn robust representations. The bidirectional Kullback-Leibler divergence is introduced in RDMAE as the regularization term for a denoising masked modeling task. Specifically, RDMAE mitigates the gap between clean and noisy image representations by minimizing the bidirectional Kullback-Leibler divergence. Then, the visual encoder is pretrained by RDMAE. In contrast to existing works, RDMAE-Nav applies denoising masked visual pretraining for PointGoal navigation to improve robustness to various visual corruptions. Finally, the pretrained visual encoder with frozen weights is applied to extract robust visual representations for policy learning in the RDMAE-Nav. Extensive experiments show that RDMAE-Nav performs competitively compared with state of the arts (SOTAs) on various visual corruptions. In detail, RDMAE-Nav performs the absolute improvement: 28.2% in SR and 23.68% in SPL under Spatter; 2.28% in SR and 6.41% in SPL under Speckle Noise; and 9.46% in SR and 9.55% in SPL under Defocus Blur.

Network Defense Model Based on Multi scale Feature Fusion for Countering Cycle Generation

The deep neural network has been applied well in many fields at present, but the security problem of the deep model has become increasingly prominent. Recent research shows that the neural network model will output the wrong classification by confronting samples. Therefore, confrontation samples are a major obstacle that must be overcome for the further development of deep neural networks. At present, the main task is to design an efficient and powerful defense model with strong robustness that can defend against multiple attack algorithms. This paper proposes a defense model based on multi-scale feature fusion circular confrontation generation network by combining the generation of generic adversarial networks (GAN) with existing attack algorithms. First, use the confrontation samples generated by the attack algorithm as the training samples of GAN, and use the unique network structure of CycleGan to make the reconstructed image closer to the clean image and remove potential disturbances. On the generator side, this paper uses multi feature fusion TernausNet structure to ensure that the feature information of the image can be restored as much as possible during the process of disturbance removal, and adds attention mechanism to the discriminator side to increase the receptive field, establish global dependency, and train a more robust discriminator to help GAN training through experiments on CIFAR-10 and ImageNet datasets. It is proved that after the training, the model can directly classify the original samples and confrontation samples correctly, and achieve good defense effect for all kinds of confrontation attack algorithms. Compared with the existing methods, the defense effect is good, and the robustness of the depth model is enhanced.

Self-supervised denoising of Nyquist-sampled volumetric images via deep learning.

Deep learning has demonstrated excellent performance enhancing noisy or degraded biomedical images. However, many of these models require access to a noise-free version of the images to provide supervision during training, which limits their utility. Here, we develop an algorithm (noise2Nyquist) that leverages the fact that Nyquist sampling provides guarantees about the maximum difference between adjacent slices in a volumetric image, which allows denoising to be performed without access to clean images. We aim to show that our method is more broadly applicable and more effective than other self-supervised denoising algorithms on real biomedical images, and provides comparable performance to algorithms that need clean images during training. We first provide a theoretical analysis of noise2Nyquist and an upper bound for denoising error based on sampling rate. We go on to demonstrate its effectiveness in denoising in a simulated example as well as real fluorescence confocal microscopy, computed tomography, and optical coherence tomography images. We find that our method has better denoising performance than existing self-supervised methods and is applicable to datasets where clean versions are not available. Our method resulted in peak signal to noise ratio (PSNR) within 1dB and structural similarity (SSIM) index within 0.02 of supervised methods. On medical images, it outperforms existing self-supervised methods by an average of 3dB in PSNR and 0.1 in SSIM. noise2Nyquist can be used to denoise any volumetric dataset sampled at at least the Nyquist rate making it useful for a wide variety of existing datasets.

ShuffleDetect: Detecting Adversarial Images against Convolutional Neural Networks

Recently, convolutional neural networks (CNNs) have become the main drivers in many image recognition applications. However, they are vulnerable to adversarial attacks, which can lead to disastrous consequences. This paper introduces ShuffleDetect as a new and efficient unsupervised method for the detection of adversarial images against trained convolutional neural networks. Its main feature is to split an input image into non-overlapping patches, then swap the patches according to permutations, and count the number of permutations for which the CNN classifies the unshuffled input image and the shuffled image into different categories. The image is declared adversarial if and only if the proportion of such permutations exceeds a certain threshold value. A series of 8 targeted or untargeted attacks was applied on 10 diverse and state-of-the-art ImageNet-trained CNNs, leading to 9500 relevant clean and adversarial images. We assessed the performance of ShuffleDetect intrinsically and compared it with another detector. Experiments show that ShuffleDetect is an easy-to-implement, very fast, and near memory-free detector that achieves high detection rates and low false positive rates.

Stylized image denoising via noise style transfer and Quasi Siamese network

Neural style transfer has attracted wide attention for the surprising quality of synthetic images. However, the visual quality of existing methods usually suffers from style transfer noise (STN). Unlike the real-world noises contaminated in natural image capturing, the STN results from the stylized image synthesizing process. This difference leads to two major challenges for stylized image denoising: (1) Synthesized stylized images have no corresponding ground-truth images, which makes it challenging to obtain noisy-clean sample pairs for training denoising networks. (2) STNs are usually color noises or accompanied by color distortions, so color correction should be considered in denoising. To tackle the first challenge, we propose a novel strategy called noise style transfer to produce noisy-clean sample pairs. Specifically, the noise style transfer takes noises as special style textures and transfers them from the noisy stylized image onto the clean style image. To tackle the second challenge, we design a Quasi Siamese denoising network, which takes the noisy sample and its high saturation (HS) version as inputs of two network branches in training. Through mutual supervision between the two branches, the “photo developer” effect of HS images on color noise is conducive to simultaneous noise removal and color correction. Extensive experimental results on various stylized images substantially demonstrate the superior performance of our approach. With great adaptiveness to different style transfer methods, our approach successfully removes distinctive STNs while keeping the artistic texture and achieves a further reduction in style loss of up to 57.7%.

Boosting Adversarial Attacks with Nadam Optimizer

Deep neural networks are extremely vulnerable to attacks and threats from adversarial examples. These adversarial examples deliberately crafted by attackers can easily fool classification models by adding imperceptibly tiny perturbations on clean images. This brings a great challenge to image security for deep learning. Therefore, studying and designing attack algorithms for generating adversarial examples is essential for building robust models. Moreover, adversarial examples are transferable in that they can mislead multiple different classifiers across models. This makes black-box attacks feasible for practical applications. However, most attack methods have low success rates and weak transferability against black-box models. This is because they often overfit the model during the production of adversarial examples. To address this issue, we propose a Nadam iterative fast gradient method (NAI-FGM), which combines an improved Nadam optimizer with gradient-based iterative attacks. Specifically, we introduce the look-ahead momentum vector and the adaptive learning rate component based on the Momentum Iterative Fast Gradient Sign Method (MI-FGSM). The look-ahead momentum vector is dedicated to making the loss function converge faster and get rid of the poor local maximum. Additionally, the adaptive learning rate component is used to help the adversarial example to converge to a better extreme point by obtaining adaptive update directions according to the current parameters. Furthermore, we also carry out different input transformations to further enhance the attack performance before using NAI-FGM for attack. Finally, we consider attacking the ensemble model. Extensive experiments show that the NAI-FGM has stronger transferability and black-box attack capability than advanced momentum-based iterative attacks. In particular, when using the adversarial examples produced by way of ensemble attack to test the adversarially trained models, the NAI-FGM improves the success rate by 8% to 11% over the other attack methods. Last but not least, the NAI-DI-TI-SI-FGM combined with the input transformation achieves a success rate of 91.3% on average.

"Got Milk Alternatives?" Understanding Key Factors Determining U.S. Consumers' Willingness to Pay for Plant-Based Milk Alternatives.

Milk is an important dairy product in U.S. food retail. Lifestyle changes toward climate-conscious consumption, animal welfare, and food safety concerns have increased the popularity of plant-based milk alternatives. This study is focused on such beverages and provides insights and best practice recommendations for marketing managers in the U.S. food retail sector. An online survey was distributed to explore factors explaining the intentions of U.S. consumers to purchase and pay a premium for plant-based milk alternatives. Food curiosity and food price inflation were identified as relevant for both willingness to buy and willingness to pay a price premium. In addition, animal welfare concerns and the green and clean product image of plant-based alternatives were relevant to the willingness to pay a premium for plant-based milk.

Revisiting video super-resolution: you only look outstanding frames

Video super-resolution (VSR) has been improved with various deep learning architectures and datasets that mostly contain clean images. However, most real-world videos are compressed and here arises a critical issue: each frame in a video usually varies in quality. We discover an important but simple coding prior that affects the performance of the existing VSR models because the prior can tell us which frame is outstanding than others in terms of quality, namely outstanding-frames. Exploiting the prior, we propose a method that allows you only look outstanding frames (YOLOF) to enhance the existing VSR models as a universal approach, which feeds VSR models the best quality of frames near the reference frame with given distance. Extensive evaluations with various VSR models show that our YOLOF method enhances existing VSR models substantially without harming original architectures.