Rain Removal Research Articles

DWTN: deep wavelet transform network for lightweight single image deraining

On rainy days the uncertainty of the shape and distribution of rain streaks can cause the images captured by RGB image-based measurement tools to be blurred and distorted. Thanks to the wavelet transform ability to provide spatial and frequency domain information about an image and its multidirectional and multiscale nature, it is widely used in traditional image enhancement methods. In image deraining the distribution of rain streaks is not only related to spatial domain features but is also closely related to frequency domain spatial features. However, deep learning-based rain removal models mainly rely on the spatial features of the image, and RGB data can hardly distinguish rain marks from image details, which leads to the loss of crucial image information during rain removal. We have developed a lightweight single-image rain removal model called the deep wavelet transform network (DWTN) to address this limitation. This method separates image details from rain images and can more effectively remove rain marks. The proposed DWTN has three significant contributions. First, DWTN uses the feature components after the wavelet transform as the input to the model and assigns a separate frequency-aware enhancement block (FAEB) to each element. These blocks focus on specific frequency features that benefit the rain removal task. Second, we introduce a frequency feature fusion block (FFFB) that fuses different wavelet components to reduce noise and enhance the image background through a channel attention mechanism while attenuating rain streaks. Finally, we design a spatial feature enhancement block (SFEB), which uses a spatial attention mechanism to calibrate the spatial position of features to improve the rain removal performance. We evaluate the performance of DWTN using PSNR and SSIM on four synthetic datasets and NIQE and BRISQUE on two real datasets. The results of the evaluation of the six datasets and at least four performance metrics show that the proposed DWTN is superior to existing methods.

Single image deraining via wide rectangular regional blocks and dual attention complementary enhancement network

Rain is a common weather phenomenon, and the challenge of removing rain streaks from a single image is crucial due to its detrimental impact on image quality and the extraction of valuable background information. Existing methods commonly rely on specific assumptions regarding rain models, which restricts their ability to accommodate a wide range of real-world scenarios. To overcome this limitation, these methods often require complex optimization techniques or stepwise refinement strategies. In this paper, we propose a novel wide rectangular regional block and dual attention complementary enhancement deraining kernel prediction subnet to meet the challenge. The network called WRRDANet consists of a kernel prediction subnet and pixel-wise dilation filtering. In the kernel prediction subnet, we capture more specific contextual background information and complex pixel-wise kernels. Afterward, the learned pixel-wise multi-scale kernels from the kernel prediction subnet are used to perform dilation filtering on the original rainy image, effectively restoring richer background details by expanding the scope of deraining to a larger extent. We conducted a comprehensive evaluation using synthetic and real rainfall datasets to demonstrate the effectiveness of our approach. The results, both qualitatively and quantitatively, indicate that our approach outperforms other popular rain removal methods.

Mask-DerainGAN: Learning to remove rain streaks by learning to generate rainy images

Image deraining with unpaired data has been a challenging problem. Previous methods suffer from either the color distortion artifacts, due to the pixel-level cycle consistency loss, or the time-consuming training process. To address these problems, in this paper, we propose a novel method for rain removal based on using unpaired data. First, we obtain a rain streak mask from the derained result, which serves as a guidance for generating rainy images. Both the mask and the rain-free image are then fed into the proposed generator to obtain a high-quality rainy image, which implicitly helps improve the rain removal performance. In this way, the proposed learning framework simultaneously learns rain removal and rain generation in order to produce high-quality rain-free images and rainy images. Second, we propose a contrastive learning generator to preserve background texture details and ensure semantic consistency between the generated rain-free image and the original input. Experimental results demonstrate that our method surpasses most state-of-the-art unsupervised methods on multiple benchmark synthetic and real datasets.

Image Deraining Algorithm Based on Multi-Scale Features

In target detection, tracking, and recognition tasks, high-quality images can achieve better results. However, in actual scenarios, the visual effects and data quality of images are greatly reduced due to the influence of environmental factors, which affect subsequent detection, recognition, and other tasks. Therefore, this paper proposes an image rain removal algorithm based on multi-scale features, which can effectively remove rain streaks. First of all, this paper proposes a deraining algorithm that combines spatial information to improve the network’s generalization ability on real images, aiming at the problem of synthetic datasets used by previous deraining algorithms. Then, by proposing a multi-scale rain removal algorithm, it improves the feature extraction capabilities of existing deraining algorithms. Before extracting deep rain features, a preliminary fusion of multi-scale shallow features can be performed, which can show better performance in images of different sizes. In addition, a spatial attention module and channel are introduced. The attention module increases the ability to extract rain information at each scale; the resulting multi-scale feature image rain removal algorithm is called MFD. Finally, the rain removal algorithm is validated on the rain removal dataset, and the proposed method can effectively remove rain patterns, provide strong performance improvement on several datasets in the image rain removal task, and provide high-quality images for subsequent detection and recognition tasks.

Improving object detection in optical devices using a multi-hierarchical cyclable structure-aware rain removal network

Rain streaks pose a significant challenge to optical devices, impeding their ability to accurately recognize objects in images. To enhance the recognition capabilities of these devices, it is imperative to remove rain streaks from images prior to processing. While deep learning techniques have been adept at removing rain from the high-frequency components of images, they often neglect the low-frequency components, where residual rain streaks can persist. This oversight can severely limit the effectiveness of deraining methods and consequently, the object recognition rate in optical devices such as cameras and smartphones. To address this problem, we developed a novel multi-hierarchical cyclable structure-aware rain removal network (MCS-RRN), which effectively retains the background structure while removing rain streaks, improving the object recognition rate in images. Unlike state-of-the-art approaches that incorporate wavelet transform, our network maintained the low-frequency sub-images and integrated them into a structure-aware subnetwork. We also transferred low-frequency structural information to detail enhancement sub-networks to enhance detailed information and facilitate convergence; this enhanced the capability of our network to eliminate rain streaks in high frequency. In summary, we used a structure information blending module and inverse wavelet transform to fuse derained low-frequency sub-images and achieve rain removal while improving the object recognition rate with the combination of YOLO. Experimental results demonstrated that our method significantly enhances the object recognition rate in images.

AIMHNet: An Attribute-Insensitive Multiscale Hourglass Network for Rain Streak and Raindrop Removal

CNN-based methods have made great progress in single-image rain removal. Most recent methods improve performance by increasing the depth of the network. To fully extract local and global features while reducing inference time, we propose a top-to-down attribute-insensitive multiscale hourglass network for rain streak and raindrop removal. For the rain removal task, we expect that the constructed network can accurately identify the various attributes of the rain information characteristics of the small target. Considering the difference in the size, shape, direction and density of rain streak and raindrop, inspired by the performance of hourglass architecture to capture multiscale features in human pose estimation, we introduce an attribute-insensitive hourglass module to recognize the attributes of rain streak and raindrop in a unified framework. This feature extraction module could capture the characteristics of rain streak and raindrop with different attributes. This stacked hourglass blocks down-sample features and then up-samples them back to the original resolution based on discrete wavelet transform and inverse discrete wavelet transform. We perform extensive experiments on five synthetic and real-world de-raining datasets to validate the effectiveness of our proposed network on rain streak and raindrop removal. The qualitative and quantitative results show that our method is suitable for removing rain streak and raindrop in a unified framework. We present the results of generalization and ablation study for key components, we also report the accuracy of semantic segmentation after preprocessing with all rain removal methods. Our source code will be available on the GitHub: https://github.com/Ruini94/AIMHNet .

Experimental research on blowing-rain intensity and uniformity of rain tunnel for aircraft rain removal testing

The objective of the present study is to characterize a rain tunnel that is used for performance testing of aircraft windshield rain removal systems. The rain tunnel in the present study is an open jet wind tunnel with six full cone pressure swirl nozzles arranged at its outlet. Experiments are conducted with water at room temperature as the working medium. The influence of total water supply flow rate and air velocity on the blowing-rain intensity and uniformity is studied. The total water supply flow rate varies from 0.57 m3/h to 1.03 m3/h, while the pressure at the inlet of the nozzle ranges from 1.44 bar to 6.52 bar. The air velocity of 30 m/s, 40 m/s, and 50 m/s is studied. A special test setup has been designed for measurements of the blowing-rain intensity and uniformity. Experimental results indicate that the blowing-rain intensity and uniformity both increase with the increasing total water supply flow rate and decrease with the increasing air velocity.

Iterative image rain removal network using consecutive residual long short-term memory

Image rain removal is designed to effectively separate rain streaks from the background image layer. However, rain streaks in real-world scenarios vary in density, shape, and direction, making it difficult to decompose rainy images into clean backgrounds and rain layers. In this study, we introduce an iterative framework for image deraining to progressively enhance rainy images using a residual long short-term memory structure. The overall network comprises a multidomain residue channel, a fusion module, and a consecutive residual long short-term memory. We introduce multidomain residue channels by computing them in both the image and wavelet low-frequency domains. We propose a fusion module to combine the residue channel for guidance with wavelet domain features for rain removal. We also propose a feature extraction module based on successive residual long short-term memory to extract the main features in the wavelet domain. An iterative image restoration framework comprising three primary modules is introduced to progressively enhance rainy images. To evaluate the performance of the proposed approach, we conduct experiments using widely used benchmarks. The results demonstrate that our method outperforms state-of-the-art methods in image rain removal https://github.com/workofsu/.

Recursive attention collaboration network for single image de‐raining

AbstractSingle‐image rain removal is an important problem in the field of computer vision aimed at recovering clean images from rainy images. In recent years, data‐driven convolutional neural network (CNN)‐based rain removal methods have achieved significant results, but most of them cannot fully focus on the contextual information in rain‐containing images, which leads to the failure of recovering some of the background details of the images that have been corrupted due to the aggregation of rain streaks. With the success of Transformer‐based models in the field of computer vision, global features can be easily acquired to better help recover details in the background of an image. However, Transformer‐based models often require a large number of parameters during the training process, which makes the training process very difficult and makes it difficult to apply them to specific devices for execution in reality. The authors propose a Recursive Attention Collaboration Network, which consists of a recursive Swin‐transformer block (STB) and a CNN‐based feature fusion block. The authors designed the Recursively Integrate Transformer Block (RITB), which consists of several STBs recursively connected, that can effectively reduce the number of parameters of the model. The final part of the module can integrate the local information from the STBs. The authors also design the Feature Enhancement Block, which can better recover the details of the background information corrupted by rain streaks of different density shapes through the features passed from the RITB. Experiments show that the proposed network has an effective rain removal effect on both synthetic and real datasets and has fewer model parameters than other mainstream methods.

Deceptive deraining attack: a restorative adversarial attack by rain removal

Deceptive deraining attack: a restorative adversarial attack by rain removal

Synthetic rain image generation via CycleGAN

In the domain of autonomous driving and computer vision, the formidable challenge of adverse weather conditions, particularly rainy weather, profoundly impacts image quality and visibility. Rain streaks pose a significant impediment to accurate object detection, especially concerning pedestrians and vehicles on the road. While existing solutions have focused on training rain removal models on paired images, acquiring such data with congruent backgrounds has been a substantial hurdle. Three distinct images, each featuring a diverse background, were deliberately chosen for the rain removal experiment. The deliberate selection of these varied backgrounds allowed us to rigorously assess the efficacy of our rain removal method across diverse environmental contexts. The compelling results obtained from this experiment affirm the method's ability to effectively mitigate rain line across a spectrum of backgrounds, thus establishing its robustness and versatility in real-world scenarios. The resulting images are then employed to train our rain removal model, marking a significant advancement in our endeavor. Our innovative approach stands poised to revolutionize rain removal techniques, aligning synthetic and authentic rain images more closely in real-world scenarios.

Cross-domain attention-guided domain adaptive method for image real rain removal

Cross-domain attention-guided domain adaptive method for image real rain removal

Rethinking Multi-Scale Representations in Deep Deraining Transformer

Existing Transformer-based image deraining methods depend mostly on fixed single-input single-output U-Net architecture. In fact, this not only neglects the potentially explicit information from multiple image scales, but also lacks the capability of exploring the complementary implicit information across different scales. In this work, we rethink the multi-scale representations and design an effective multi-input multi-output framework that constructs intra- and inter-scale hierarchical modulation to better facilitate rain removal and help image restoration. We observe that rain levels reduce dramatically in coarser image scales, thus proposing to restore rain-free results from the coarsest scale to the finest scale in image pyramid inputs, which also alleviates the difficulty of model learning. Specifically, we integrate a sparsity-compensated Transformer block and a frequency-enhanced convolutional block into a coupled representation module, in order to jointly learn the intra-scale content-aware features. To facilitate representations learned at different scales to communicate with each other, we leverage a gated fusion module to adaptively aggregate the inter-scale spatial-aware features, which are rich in correlated information of rain appearances, leading to high-quality results. Extensive experiments demonstrate that our model achieves consistent gains on five benchmarks.

Wavelet-based Auto-Encoder for simultaneous haze and rain removal from images

Noise introduced due to weather can reduce the efficiency of computer vision applications as the visibility of the objects in images is greatly affected. Haze and rain are the most common weather conditions seen in nature. However, most of the algorithms found in the literature apply rain and haze removal approaches separately. To this end, in this paper, we propose a novel Wavelet-based deep Auto-encoder, called WAE, for simultaneously removing the haze and rain effects from images. The proposed network uses wavelet transformation and inverse wavelet transformation as an alternative to down-sampling and up-sampling operations, respectively, in order to add sparsity to the network. By training the model on both spatial and frequency domains, it learns non-stationary features that are found to be useful to remove haze and rain effects from images. The proposed model is tested on several rain and haze-affected image datasets, and it performs well in terms of standard evaluation metrics like structural similarity index measure and peak signal-to-noise ratio. The code can be found at : https://github.com/asfakali/WAE.git.

Context-detail-aware United Network for Single Image Deraining

Images captured outdoors are often affected by rainy days, resulting in a severe deterioration in the visual quality of the captured images and a decrease in the performance of related applications. Therefore, single image deraining has attracted attention as a challenging research topic. Nowadays, there are two common deraining architectures in single image deraining. The first one is to restore the rain-free image by deducting rain streaks learned by the model from the rain image, but the background structure is easily mistaken for rain streaks and subtracted. The other one is to directly learn the clean background structure through the model using rain images, but it is difficult to completely remove the rain streaks due to the complexity of the information in images with rain. Therefore, current methods cannot balance rain streak removal and rain-free image background restoration in a single architecture and achieve good results. To address this issue, we propose a novel framework, namely, Context-Detail-Aware United Network (CDaUNet), which combines the above two architectures in this study. More specifically, we divide the restoration of the background structure of rain-free images and the learning of rain streaks into two independent sub-networks. The proposed Structure-Aware Rain Removal Network (SaRRN) is to learn the background structure in images to reconstruct clean rain-free images, whereas Detail-Aware Rain Streak Learning Network (DaRLN) is proposed to learn the details of rain streaks in images. Finally, we fuse the results generated by the two sub-networks through our designed Dual Architecture Fusion Network (DAFN) to reconstruct original rain images to effectively fuse the results of the two sub-networks. The experimental results show that CDaUNet achieves satisfactory performance in comparison with the state-of-the-art approaches included in rain streak removal and rain-free image structure restoration architectures on both synthetic and real image datasets, confirming the effectiveness of our method.

Image quality enhancement and rain removal on images taken under different rain conditions

Image quality enhancement and rain removal on images taken under different rain conditions

Dual-attention U-Net and multi-convolution network for single-image rain removal

Dual-attention U-Net and multi-convolution network for single-image rain removal

Dual parallel multi-scale residual overlay network for single-image rain removal

Dual parallel multi-scale residual overlay network for single-image rain removal

From heavy rain removal to detail restoration: A faster and better network

The profound accumulation of precipitation during intense rainfall events can markedly degrade the quality of images, leading to the erosion of textural details. Despite the improvements observed in existing learning-based methods specialized for heavy rain removal, it is discerned that a significant proportion of these methods tend to overlook the precise reconstruction of the intricate details. In this work, we introduce a simple dual-stage progressive enhancement network, denoted as DPENet, aiming to achieve effective deraining while preserving the structural accuracy of rain-free images. This approach comprises two key modules, a rain streaks removal network (R2Net) focusing on accurate rain removal, and a details reconstruction network (DRNet) designed to recover the textural details of rain-free images. Firstly, we introduce a dilated dense residual block (DDRB) within R2Net, enabling the aggregation of high-level and low-level features. Secondly, an enhanced residual pixel-wise attention block (ERPAB) is integrated into DRNet to facilitate the incorporation of contextual information. To further enhance the fidelity of our approach, we employ a comprehensive loss function that accentuates both the marginal and regional accuracy of rain-free images. Extensive experiments conducted on publicly available benchmarks demonstrates the noteworthy efficiency and effectiveness of our proposed DPENet. The source code and pre-trained models are currently available at https://github.com/chdwyb/DPENet.

Progressive network based on detail scaling and texture extraction: A more general framework for image deraining

Many feature extraction components have been proposed for image deraining tasks, aiming to improve feature learning. However, few models have addressed the integration of multi-scale features from derain images. The fusion of multiple features at different scales in one model has the potential to significantly enhance the authenticity and detail of rainy images restoration. This study introduces a migratable multi-scale feature blending model, which is a progressive learning model based on detail dilation and texture extraction. First, the degraded image is sent to the detail dilation module, which is designed to increase the detailed outline and obtain the coarse image features. Second, the extracted feature maps are sent to the multi-scale feature extraction (MFE) module and the multi-scale hybrid strategy (MHS) module for improved texture restoration. Third, the simple convolution modules are replaced by an optimized transformer model to more efficiently extract contextual features and multi-scale information in images. Finally, a progressive learning strategy is employed to incrementally restore the degraded images. Empirical results show that our proposed module for progressive restoration achieves near state-of-the-art performance in several rain removal tasks. In particular, our model exhibits better rain removal realism compared to state-of-the-art models. The source code is available at https://github.com/JackAILab/DTPNet.