Single Image De-raining Research Articles

Complementary Parallel Residual Attention Network for Single Image Deraining

Complementary Parallel Residual Attention Network for Single Image Deraining

S2VSNet: Single stage V-shaped network for image deraining & dehazing

Producing high-quality, noise-free images from noisy or hazy inputs relies on essential tasks such as single image deraining and dehazing. In many advanced multi-stage networks, there is often an imbalance in contextual information, leading to increased complexity. To address these challenges, we propose a simplified method inspired by a U-Net structure, resulting in the “Single-Stage V-Shaped Network” (S2VSNet), capable of handling both deraining and dehazing tasks. A key innovation in our approach is the introduction of a Feature Fusion Module (FFM), which facilitates the sharing of information across multiple scales and hierarchical layers within the encoder-decoder structure. As the network progresses towards deeper layers, the FFM gradually integrates insights from higher levels, ensuring that spatial details are preserved while contextual feature maps are balanced. This integration enhances the image processing capability, producing noise-free, high-quality outputs. To maintain efficiency and reduce system complexity, we replaced or removed several non-essential non-linear activation functions, opting instead for simple multiplication operations. Additionally, we introduced a “Multi-Head Attention Integrated Module” (MHAIM) as an intermediary layer between encoder-decoder levels. This module addresses the limited receptive fields of traditional Convolutional Neural Networks (CNNs), allowing for the capture of more comprehensive feature-map information. Our focus on deraining and dehazing led to extensive experiments on a wide range of synthetic and real-world datasets. To further validate the robustness of our network, we implemented S2VSNet on a low-end edge device, achieving deraining in 2.46 seconds.

DWTN: deep wavelet transform network for lightweight single image deraining

On rainy days the uncertainty of the shape and distribution of rain streaks can cause the images captured by RGB image-based measurement tools to be blurred and distorted. Thanks to the wavelet transform ability to provide spatial and frequency domain information about an image and its multidirectional and multiscale nature, it is widely used in traditional image enhancement methods. In image deraining the distribution of rain streaks is not only related to spatial domain features but is also closely related to frequency domain spatial features. However, deep learning-based rain removal models mainly rely on the spatial features of the image, and RGB data can hardly distinguish rain marks from image details, which leads to the loss of crucial image information during rain removal. We have developed a lightweight single-image rain removal model called the deep wavelet transform network (DWTN) to address this limitation. This method separates image details from rain images and can more effectively remove rain marks. The proposed DWTN has three significant contributions. First, DWTN uses the feature components after the wavelet transform as the input to the model and assigns a separate frequency-aware enhancement block (FAEB) to each element. These blocks focus on specific frequency features that benefit the rain removal task. Second, we introduce a frequency feature fusion block (FFFB) that fuses different wavelet components to reduce noise and enhance the image background through a channel attention mechanism while attenuating rain streaks. Finally, we design a spatial feature enhancement block (SFEB), which uses a spatial attention mechanism to calibrate the spatial position of features to improve the rain removal performance. We evaluate the performance of DWTN using PSNR and SSIM on four synthetic datasets and NIQE and BRISQUE on two real datasets. The results of the evaluation of the six datasets and at least four performance metrics show that the proposed DWTN is superior to existing methods.

Non-local feature aggregation quaternion network for single image deraining

The existing deraining methods are based on convolutional neural networks (CNN) learning the mapping relationship between rainy and clean images. However, the real-valued CNN processes the color images as three independent channels separately, which fails to fully leverage color information. Additionally, sliding-window-based neural networks cannot effectively model the non-local characteristics of an image. In this work, we proposed a non-local feature aggregation quaternion network (NLAQNet), which is composed of two concurrent sub-networks: the Quaternion Local Detail Repair Network (QLDRNet) and the Multi-Level Feature Aggregation Network (MLFANet). Furthermore, in the subnetwork of QLDRNet, the Local Detail Repair Block (LDRB) is proposed to repair the backdrop of an image that has not been damaged by rain streaks. Finally, within the MLFANet subnetwork, we have introduced two specialized blocks, namely the Non-Local Feature Aggregation Block (NLAB) and the Feature Aggregation Block (Mix), specifically designed to address the restoration of rain-streak-damaged image backgrounds. Extensive experiments demonstrate that the proposed network delivers strong performance in both qualitative and quantitative evaluations on existing datasets. The code is available at https://github.com/xionggonghe/NLAQNet.

A two-stage-UNet network based on group normalization for single image deraining

A two-stage-UNet network based on group normalization for single image deraining

RCDNet: An Interpretable Rain Convolutional Dictionary Network for Single Image Deraining.

As common weather, rain streaks adversely degrade the image quality and tend to negatively affect the performance of outdoor computer vision systems. Hence, removing rains from an image has become an important issue in the field. To handle such an ill-posed single image deraining task, in this article, we specifically build a novel deep architecture, called rain convolutional dictionary network (RCDNet), which embeds the intrinsic priors of rain streaks and has clear interpretability. In specific, we first establish a rain convolutional dictionary (RCD) model for representing rain streaks and utilize the proximal gradient descent technique to design an iterative algorithm only containing simple operators for solving the model. By unfolding it, we then build the RCDNet in which every network module has clear physical meanings and corresponds to each operation involved in the algorithm. This good interpretability greatly facilitates an easy visualization and analysis of what happens inside the network and why it works well in the inference process. Moreover, taking into account the domain gap issue in real scenarios, we further design a novel dynamic RCDNet, where the rain kernels can be dynamically inferred corresponding to input rainy images and then help shrink the space for rain layer estimation with few rain maps, so as to ensure a fine generalization performance in the inconsistent scenarios of rain types between training and testing data. By end-to-end training such an interpretable network, all involved rain kernels and proximal operators can be automatically extracted, faithfully characterizing the features of both rain and clean background layers and, thus, naturally leading to better deraining performance. Comprehensive experiments implemented on a series of representative synthetic and real datasets substantiate the superiority of our method, especially on its well generality to diverse testing scenarios and good interpretability for all its modules, compared with state-of-the-art single image derainers both visually and quantitatively. Code is available at https://github.com/hongwang01/DRCDNet.

Feature decoupling and reorganization network for single image deraining

Feature decoupling and reorganization network for single image deraining

Contrastive Unfolding Deraining Network.

Due to the fact that the degradation of image quality caused by rain usually affects outdoor vision tasks, image deraining becomes more and more important. Focusing on the single image deraining (SID) task, in this article, we propose a novel Contrastive Unfolding DEraining Network (CUDEN), which combines the traditional iterative algorithm and deep network, exhibiting excellent performance and nice interpretability. CUDEN transforms the challenge of locating rain streaks into discovering rain features and defines the relationship between the image and feature domains in terms of mapping pairs. To obtain the mapping pairs efficiently, we propose a dynamic multidomain translation (DMT) module for decomposing the original mapping into sub-mappings. To enhance the feature extraction capability of networks, we also propose a new serial multireceptive field fusion (SMF) block, which extracts complex and variable rain features with convolution kernels of different receptive fields. Moreover, we are the first to introduce contrastive learning to the SID task and combine it with perceptual loss to propose a new contrastive perceptual loss (CPL), which is quite generalized and greatly helpful in identifying the appropriate gradient descent direction during training. Extensive experiments on synthetic and real-world datasets demonstrate that our proposed CUDEN outperforms the state-of-the-art (SOTA) deraining networks.

Dual-former: Hybrid self-attention transformer for efficient image restoration

Recently, image restoration transformers have achieved comparable performance with previous state-of-the-art CNNs. In this work, we present Dual-former whose critical insight is to combine the powerful global modeling ability of self-attention modules and the local modeling ability of convolutions in an overall architecture. With convolution-based Local Feature Extraction modules equipped in the encoder and the decoder, we only adopt a novel Hybrid Transformer Block in the latent layer to model the long-distance dependence in spatial dimensions and handle the uneven distribution between channels. Such a design eliminates the substantial computational complexity in previous image restoration transformers and achieves superior performance on multiple image restoration tasks. Experiments demonstrate that Dual-former achieves a 1.91 dB gain over the state-of-the-art MAXIM method on the Indoor dataset for single image dehazing while consuming only 4.2% GFLOPs as MAXIM. For single image deraining, it exceeds the SOTA method by 0.1 dB PSNR on the average results of five datasets with only 21.5% GFLOPs. Dual-former also substantially surpasses the latest methods on various tasks, with fewer parameters.

SRENet: Structure recovery ensemble network for single image deraining

SRENet: Structure recovery ensemble network for single image deraining

Internal and external transmission encoder–decoder network for single-image deraining

Internal and external transmission encoder–decoder network for single-image deraining

Residual Contextual Hourglass Network for Single-Image Deraining

Rain streaks could blur and distort images, significantly impacting further image processing. Single-image deraining is a hotspot and has practical application value, while most existing methods still have problems such as residual rain streaks and inadequate recovery of detail textures. To address these issues, we propose a Residual Contextual Hourglass Network (RCHNet) for single-image deraining, which could adapt to remove rain streaks in complex environments. Firstly, a contextual distillation block is presented to obtain local and global features across different scales. Further, residual downsampling block and residual upsampling block are used to maintain the residual nature of our architecture and better restore the details of the image. Finally, a dual attention mechanism is introduced to compensate for the spatial information lost by the downsampling. Extensive experiments on five synthetic datasets and a real-world dataset demonstrate that our proposed RCHNet outperforms existing state-of-the-art deraining approaches. On average across all synthetic datasets, the PSNR score of RCHNet is as high as 33.31 dB.

Frequency domain-enhanced transformer for single image deraining

Frequency domain-enhanced transformer for single image deraining

Context-detail-aware United Network for Single Image Deraining

Images captured outdoors are often affected by rainy days, resulting in a severe deterioration in the visual quality of the captured images and a decrease in the performance of related applications. Therefore, single image deraining has attracted attention as a challenging research topic. Nowadays, there are two common deraining architectures in single image deraining. The first one is to restore the rain-free image by deducting rain streaks learned by the model from the rain image, but the background structure is easily mistaken for rain streaks and subtracted. The other one is to directly learn the clean background structure through the model using rain images, but it is difficult to completely remove the rain streaks due to the complexity of the information in images with rain. Therefore, current methods cannot balance rain streak removal and rain-free image background restoration in a single architecture and achieve good results. To address this issue, we propose a novel framework, namely, Context-Detail-Aware United Network (CDaUNet), which combines the above two architectures in this study. More specifically, we divide the restoration of the background structure of rain-free images and the learning of rain streaks into two independent sub-networks. The proposed Structure-Aware Rain Removal Network (SaRRN) is to learn the background structure in images to reconstruct clean rain-free images, whereas Detail-Aware Rain Streak Learning Network (DaRLN) is proposed to learn the details of rain streaks in images. Finally, we fuse the results generated by the two sub-networks through our designed Dual Architecture Fusion Network (DAFN) to reconstruct original rain images to effectively fuse the results of the two sub-networks. The experimental results show that CDaUNet achieves satisfactory performance in comparison with the state-of-the-art approaches included in rain streak removal and rain-free image structure restoration architectures on both synthetic and real image datasets, confirming the effectiveness of our method.

Recovering a clean background: A parallel deep network architecture for single-image deraining

Deep convolutional neural networks have been popularly applied in single-image deraining recently. Nevertheless, as the network becomes deeper, it is easy to cause training over-fitting and performance saturation, particularly in the case of insufficient training data. In this paper, we report the design of a new network, namely parallel deraining convolutional neural network (PARDNet), for single-image deraining. Specifically, PARDNet adopts two parallel residual sub-networks based on different receptive fields to extract more comprehensive characteristics of the rain streaks, as well as decrease the depth of the network. The hybrid dilated convolution is employed to enlarge the sub-network’s receptive field to capture more context information. The efficient channel attention module is integrated into the proposed PARDNet to capture rain streaks more effectively and preserve more background details. Furthermore, to facilitate the network training, the residual learning is also fused into PARDNet in a holistic way. Extensive experiments on synthetic and real-world rainy image datasets demonstrate the superiority of PARDNet for single-image deraining.

An Efficient Single Image De-raining Model with Decoupled Deep Networks.

Single image de-raining is an emerging paradigm for many outdoor computer vision applications since rain streaks can significantly degrade the visibility and render the function compromised. The introduction of deep learning (DL) has brought about substantial advancement on de-raining methods. However, most existing DL-based methods use single homogeneous network architecture to generate de-rained images in a general image restoration manner, ignoring the discrepancy between rain location detection and rain intensity estimation. We find that this discrepancy would cause feature interference and representation ability degradation problems which significantly affect de-raining performance. In this paper, we propose a novel heterogeneous de-raining architecture aiming to decouple rain location detection and rain intensity estimation (DLINet). For these two subtasks, we provide dedicated network structures according to their differential properties to meet their respective performance requirements. To coordinate the decoupled subnetworks, we develop a high-order collaborative network learning the dynamic inter-layer interactions between rain location and intensity. To effectively supervise the decoupled subnetworks during training, we propose a novel training strategy that imposes task-oriented supervision using the label learned via joint training. Extensive experiments on synthetic datasets and real-world rainy scenes demonstrate that the proposed method has great advantages over existing state-of-the-art methods.

Multi-Scale Fusion and Decomposition Network for Single Image Deraining

Multi-Scale Fusion and Decomposition Network for Single Image Deraining

Layer Decomposition Learning Based on Discriminative Feature Group Split With Bottom-Up Intergroup Feature Fusion for Single Image Deraining

Layer Decomposition Learning Based on Discriminative Feature Group Split With Bottom-Up Intergroup Feature Fusion for Single Image Deraining

Exploring an efficient frequency-guidance transformer for single image deraining

Exploring an efficient frequency-guidance transformer for single image deraining

Wavelet Approximation-Aware Residual Network for Single Image Deraining.

It has been made great progress on single image deraining based on deep convolutional neural networks (CNN). In most existing deep deraining methods, CNNs aim to learn a direct mapping from rainy images to clean rain-less images, and their architectures are becoming more and more complex. However, due to the limitation of mixing rain with object edges and background, it is difficult to separate rain and object/background, and the edge details of the image cannot be effectively recovered in the reconstruction process. To address this problem, we propose a novel wavelet approximation-aware residual network (WAAR), wherein rain is effectively removed from both low-frequency structures and high-frequency details at each level separately, especially in low-frequency sub-images at each level. After wavelet transform, we propose novel approximation aware (AAM) and approximation level blending (ALB) mechanisms to further aid the low-frequency networks at each level recover the structure and texture of low-frequency sub-images recursively, while the high-frequency network can effectively eliminate rain streaks through block connection and achieve different degrees of edge detail enhancement by adjusting hyperparameters. In addition, we also introduce block connection to enrich the high-frequency details in the high-frequency network, which is favorable for obtaining potential interdependencies between high- and low-frequency features. Experimental results indicate that the proposed WAAR exhibits strong performance in reconstructing clean and rain-free images, recovering real and undistorted texture structures, and enhancing image edges in comparison with the state-of-the-art approaches on synthetic and real image datasets. It shows the effectiveness of our method, especially on image edges and texture details.