Raindrop Removal Research Articles

Generative Adversarial Network for Image Raindrop Removal of Transmission Line Based on Unmanned Aerial Vehicle Inspection

In the process of UAV line inspection, there may be raindrops on the camera lens. Raindrops have a serious impact on the details of the image, reducing the identification of the target transmission equipment in the image, reducing the accuracy of the target detection algorithm, and hindering the practicability of UAV line inspection technology in cyber‐physical energy systems. In this paper, the principle of raindrop image formation is studied, and a method of raindrop removal based on generation countermeasure network is proposed. In this method, the attention recurrent network is used to generate the raindrop attention map, and the context code decoder is used to generate the raindrop image. The experimental results show that the proposed method can remove the raindrops in the image and repair the background image of raindrop coverage area and can generate a higher quality raindrop removal image than the traditional method.

Dual Attention-in-Attention Model for Joint Rain Streak and Raindrop Removal

Rain streaks and raindrops are two natural phenomena, which degrade image capture in different ways. Currently, most existing deep deraining networks take them as two distinct problems and individually address one, and thus cannot deal adequately with both simultaneously. To address this, we propose a Dual Attention-in-Attention Model (DAiAM) which includes two DAMs for removing both rain streaks and raindrops. Inside the DAM, there are two attentive maps - each of which attends to the heavy and light rainy regions, respectively, to guide the deraining process differently for applicable regions. In addition, to further refine the result, a Differential-driven Dual Attention-in-Attention Model (D-DAiAM) is proposed with a "heavy-to-light" scheme to remove rain via addressing the unsatisfying deraining regions. Extensive experiments on one public raindrop dataset, one public rain streak and our synthesized joint rain streak and raindrop (JRSRD) dataset have demonstrated that the proposed method not only is capable of removing rain streaks and raindrops simultaneously, but also achieves the state-of-the-art performance on both tasks.

Uncertainty Guided Multi-Scale Attention Network for Raindrop Removal From a Single Image.

Raindrops adhered to a glass window or camera lens appear in various blurring degrees and resolutions due to the difference in the degrees of raindrops aggregation. The removal of raindrops from a rainy image remains a challenging task because of the density and diversity of raindrops. The abundant location and blur level information are strong prior guide to the task of raindrop removal. However, existing methods use a binary mask to locate and estimate the raindrop with the value 1 (adhesion of raindrops) and 0 (no adhesion), which ignores the diversity of raindrops. Meanwhile, it is noticed that different scale versions of a rainy image have similar raindrop patterns, which makes it possible to employ such complementary information to represent raindrops. In this work, we first propose a soft mask with the value in [-1,1] indicating the blurring level of the raindrops on the background, and explore the positive effect of the blur degree attribute of raindrops on the task of raindrop removal. Secondly, we explore the multi-scale fusion representation for raindrops based on the deep features of the input multi-scale images. The framework is termed uncertainty guided multi-scale attention network (UMAN). Specifically, we construct a multi-scale pyramid structure and introduce an iterative mechanism to extract blur-level information about raindrops to guide the removal of raindrops at different scales. We further introduce the attention mechanism to fuse the input image with the blur-level information, which will highlight raindrop information and reduce the effects of redundant noise. Our proposed method is extensively evaluated on several benchmark datasets and obtains convincing results.

Selective generative adversarial network for raindrop removal from a single image

The removal of raindrops from a single image is still challenging because of the diversity and density of raindrops existing in the rainy image. Moreover, the colors of raindrops are constantly changing with the background which also makes the raindrops cannot be well removed by using the current methods. In this paper, we tackle these limitations by combining the raindrops shape features with the background structure features to guide the network to accurately remove raindrops. Specifically, we propose a selective skip connection GAN (SSCGAN) combining the selective skip connection and self-attention mechanism to restoring the clean image from a raindrop degraded one. Our main idea is selectively transmitting the information of raindrops to the decoder through Gated Recurrent Units (GRU) to better generate a clean image. During the training, the selective skip connection model (SSCM) extract raindrops binary mask from the rainy image and eliminate the interference of background noise. Simultaneously, we use self-attention blocks (SABs) to make the generator network pay more attention to global structure features of the rainy image and conversely correct the raindrops binary mask. Experiments show that our method has better performance than previous methods.

Raindrop-Aware GAN: Unsupervised Learning for Raindrop-Contaminated Coastal Video Enhancement

We propose an unsupervised network with adversarial learning, the Raindrop-aware GAN, which enhances the quality of coastal video images contaminated by raindrops. Raindrop removal from coastal videos faces two main difficulties: converting the degraded image into a clean one by visually removing the raindrops, and restoring the background coastal wave information in the raindrop regions. The components of the proposed network—a generator and a discriminator for adversarial learning—are trained on unpaired images degraded by raindrops and clean images free from raindrops. By creating raindrop masks and background-restored images, the generator restores the background information in the raindrop regions alone, preserving the input as much as possible. The proposed network was trained and tested on an open-access dataset and directly collected dataset from the coastal area. It was then evaluated by three metrics: the peak signal-to-noise ratio, structural similarity, and a naturalness-quality evaluator. The indices of metrics are 8.2% (+2.012), 0.2% (+0.002), and 1.6% (−0.196) better than the state-of-the-art method, respectively. In the visual assessment of the enhanced video image quality, our method better restored the image patterns of steep wave crests and breaking than the other methods. In both quantitative and qualitative experiments, the proposed method more effectively removed the raindrops in coastal video and recovered the damaged background wave information than state-of-the-art methods.

Weakly Supervised Learning for Raindrop Removal on a Single Image

In this paper, we address a problem of view-disturbing raindrop removal on a single image. In existing methods to tackle this problem, machine learning based ones seem promising but require elaborate pairwise images, i.e., the raindrop-degraded image and the corresponding clean image of the same scene, for training. To overcome this drawback, we propose a weakly supervised learning based model in the absence of pairwise training examples, which needs only a collection of images with image-level annotations indicating the presence/absence of raindrops for training. Specifically, we train a raindrop detector for highlighting regions of raindrops in a multi-task learning manner. Then, we propose an attention-based generative network for raindrop removal and introduce a weighted preservation loss to retain the non-raindrop details. Specially, our model can be mixedly trained with pairwise and unpaired samples, which enables us to conveniently adapt the model to a new domain. Experiments verify the effect of the proposed method. Especially, using only weakly-supervised learning, our method can achieve comparable results with state-of-the-art strongly-supervised learning methods.

Image Raindrop Removal Method for Generative Adversarial Network Based on Difference Learning

Due to the interference of the external environment such as rainy weather on the camera, raindrops can easily adhere to the lens and seriously affect the quality of the photos taken. Therefore, it is of great significance to remove raindrops from the image and improve the quality of the photo. In this paper, a raindrop method for generative adversarial network images based on differential learning is proposed. The general generative network is to input images with raindrops and output clean images. The generative network in this paper does not directly output clean images, but learning the difference between images with raindrops and without raindrops, then subtract the learned difference from the image with raindrops to generate a clean image. In order to learn this difference more effectively, adding reconstruction loss to the generative network, the pre-trained VGG-16 network is used to extract the difference between the generated image and the real image features and calculate the mean square error. The experimental results show, the method in this paper can not only remove the raindrops in the image well, but also reconstruct the image information of the part blocked by the raindrops. The image processed by the algorithm in this paper is tested using the yolov3 target detection algorithm, which can significantly improve the recognition accuracy of the detection algorithm.

Raindrop Removal With Light Field Image Using Image Inpainting

In this paper, we propose a method that removes raindrops with light field image using image inpainting. We first use the depth map generated from light field image to detect raindrop regions which are then expressed as a binary mask. The original image with raindrops is improved by refocusing on the far regions and filtering by a high-pass filter. With the binary mask and the enhanced image, image inpainting is then utilized to eliminate raindrops from the original image. We compare pre-trained models of several deep learning based image inpainting methods. A light field raindrop dataset is released to verify our method. Image quality analysis is performed to evaluate the proposed image restoration method. The recovered images are further applied to object detection and visual localization tasks.

A2Net: Adjacent Aggregation Networks for Image Raindrop Removal

Existing methods for single images raindrop removal either have poor robustness or suffer from parameter burdens. In this paper, we propose a new Adjacent Aggregation Network (A 2 Net) with lightweight architectures to remove raindrops from single images. Instead of directly cascading convolutional layers, we design an adjacent aggregation architecture to better fuse features for rich representations generation, which can lead to high quality images reconstruction. To further simplify the learning process, we utilize a problem-specific knowledge to force the network focus on the luminance channel in the YUV color space instead of all RGB channels. By combining adjacent aggregating operation with color space transformation, the proposed A 2 Net can achieve state-of-the-art performances on raindrop removal with significant parameters reduction.

Single-image raindrop removal using concurrent channel-spatial attention and long-short skip connections

Image raindrop removal refers to removing raindrops from the images taken through glass under rainy weather. It is a challenging problem due to the large variations of raindrop appearances as well as the nonlinear visual distortions caused by raindrops. This problem becomes much more difficult when only a single image is provided. This paper aims at tackling the single-image raindrop removal problem by leveraging the power of deep learning. We proposed an end-to-end approach using the deep neural network. To address the challenges of raindrop removal, we introduce a concurrent channel and spatial attention mechanism implemented by Squeeze & Excitation into the network. The channel attention mechanism allows individually selecting useful features in the network for handling the raindrops of different appearances and restoring different image patterns. The spatial attention enables different treatments to the image regions with different degrees of distortion and with different local image patterns. In addition, long-short skip connections are added for utilizing the intermediate features of the network for better restoration. The experimental results show that the proposed approach outperformed the existing ones.

적대적생성신경망을 이용한 연안 파랑 비디오 영상에서의 빗방울 제거 및배경 정보 복원

적대적생성신경망을 이용한 연안 파랑 비디오 영상에서의 빗방울 제거 및배경 정보 복원

Application research on improved CGAN in image raindrop removal

Rainy weather can greatly reduce the image quality and hinder the subsequent processing of the image. In order to achieve raindrop removal on rainy images, the single image raindrop removal method based on conditional generative adversarial networks (CGAN) is proposed. In this method, CGAN is used as the basic framework. The network receives the raindrop image as an additional condition information and adds Lipschitz constraint on the network. The network model is trained by the combination of condition adversarial loss, content loss, and perception loss to repair the raindrop area and reconstruct the image. The experimental results show that the proposed method has better raindrop removal effect than the existing algorithm and can avoid image blurring on the basis of ensuring the raindrop removal effect.

Review on Rain drop detection and removal using k-means clustering and Hough transformation

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Facile Patterning of Ag Nanowires Network by Micro-Contact Printing of Siloxane.

A simple, low-cost, scalable patterning method has been demonstrated for chemically welded Ag nanowires (AgNWs) network. The chemically welded network of AgNWs on substrates has been patterned by modified microcontact printing (μCP). As an ink for the μCP, uncured high-viscosity siloxane polymer has been applied. Using elastomeric polydimethylsiloxane (PDMS) stamp that has been replicated from micromachined Si master mold by metal-assisted chemical etching, the printed siloxane ink materials have been cured by simple UV/ozone exposure for 3 min, which acts as an etch barrier in ensuing wet-removal of exposed AgNWs network. The proposed patterning technique has no limitation in the choice of substrates and pattern shape, in addition to high resolution. The patterned AgNWs network electrodes have shown excellent optical, electrical, and mechanical performances, such as high flexibility (up to ∼10%) and stretchability (up to 40%). Finally, the patterned AgNWs network electrodes have been applied as a transparent heater, which can be used for rapid raindrop removal or deicing of car windows and outside mirrors. This can be a valuable help for driving safety under harsh weather conditions.

Pixel Based Temporal Analysis Using Chromatic Property for Removing Rain from Videos

<!-- /* Font Definitions */ @font-face {font-family:??; panose-1:2 1 6 0 3 1 1 1 1 1; mso-font-alt:SimSun; mso-font-charset:134; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:3 135135232 16 0 262145 0;} @font-face {font-family: