Network For Single Image De-raining Research Articles

SRENet: Structure recovery ensemble network for single image deraining

SRENet: Structure recovery ensemble network for single image deraining

Multi-Scale Fusion and Decomposition Network for Single Image Deraining

Multi-Scale Fusion and Decomposition Network for Single Image Deraining

Wavelet Approximation-Aware Residual Network for Single Image Deraining.

It has been made great progress on single image deraining based on deep convolutional neural networks (CNN). In most existing deep deraining methods, CNNs aim to learn a direct mapping from rainy images to clean rain-less images, and their architectures are becoming more and more complex. However, due to the limitation of mixing rain with object edges and background, it is difficult to separate rain and object/background, and the edge details of the image cannot be effectively recovered in the reconstruction process. To address this problem, we propose a novel wavelet approximation-aware residual network (WAAR), wherein rain is effectively removed from both low-frequency structures and high-frequency details at each level separately, especially in low-frequency sub-images at each level. After wavelet transform, we propose novel approximation aware (AAM) and approximation level blending (ALB) mechanisms to further aid the low-frequency networks at each level recover the structure and texture of low-frequency sub-images recursively, while the high-frequency network can effectively eliminate rain streaks through block connection and achieve different degrees of edge detail enhancement by adjusting hyperparameters. In addition, we also introduce block connection to enrich the high-frequency details in the high-frequency network, which is favorable for obtaining potential interdependencies between high- and low-frequency features. Experimental results indicate that the proposed WAAR exhibits strong performance in reconstructing clean and rain-free images, recovering real and undistorted texture structures, and enhancing image edges in comparison with the state-of-the-art approaches on synthetic and real image datasets. It shows the effectiveness of our method, especially on image edges and texture details.

A novel dual-stage progressive enhancement network for single image deraining

The dense rain accumulation in heavy rain can significantly wash out images and thus destroy the background details of images. Although existing deep rain removal models lead to improved performance for heavy rain removal, we find that most of them ignore the detail reconstruction accuracy of rain-free images. In this paper, we propose a dual-stage progressive enhancement network (DPENet-v2) to achieve effective deraining with structure-accurate rain-free images. Three main modules are included in our framework, namely a rain streaks removal network (R2Net), a details reconstruction network (DRNet) and a cross-stage feature interaction module (CFIM). The former aims to achieve accurate rain removal, and the latter is designed to recover the details of rain-free images. We introduce two main strategies within our networks to achieve trade-off between the effectiveness of deraining and the detail restoration of rain-free images. Firstly, a dilated dense residual block (DDRB) within the rain streaks removal network is presented to aggregate high/low level features of heavy rain. Secondly, an enhanced residual pixel-wise attention block (ERPAB) within the details reconstruction network is designed for context information aggregation. Meanwhile, CFIM learns the long-range dependencies and achieves cross-stage information communication. We also propose a comprehensive loss function to highlight the marginal and regional accuracy of rain-free images. Extensive experiments on benchmark public datasets show both efficiency and effectiveness of the proposed method in achieving structure-preserving rain-free images for heavy rain removal.

Progressive dense feature fusion network for single image deraining

Deep Learning (DL) has achieved significant progress in single image deraining methods. Most of current DL methods, however, are still weak in image detail recovery and feature inherent correlation learning. In this work, we explore the detail recovery mechanisms in both network architecture and loss function for image deraining. A U-net like architecture named progressive dense feature fusion network (PDFFN) is proposed to encode rain images and decode them as clean ones. Specifically, a residual dense connection unit (ReDCU) is designed to handle rain streaks of various blurring degrees and resolutions by enriching features. Moreover, a progressive feature fusion module (PFFM), which fuses the features in different stages of U-net progressively, is devised to not only capture the inherent correlations of features across encoder and decoder but also intensify the fine-gained details. To better evaluate the perceptual similarity between the ground-truth and derained image, we propose detail perceptual loss by focusing on low-level unactivated features. Apart from the global rain removal strategy, this paper applies contextual loss on target regions to conduct joint deraining and detection. Comprehensive experiments substantiate the superiority of the proposed method, especially its detail recovery capability, as compared with state-of-the-art methods both qualitatively and quantitatively.

Twin-stage Unet-like network for single image deraining

Twin-stage Unet-like network for single image deraining

Multispectral attention-based network for single image deraining

Multispectral attention-based network for single image deraining

Multi-Scale Hybrid Fusion Network for Single Image Deraining.

Deep learning models have been able to generate rain-free images effectively, but the extension of these methods to complex rain conditions where rain streaks show various blurring degrees, shapes, and densities has remained an open problem. Among the major challenges are the capacity to encode the rain streaks and the sheer difficulty of learning multi-scale context features that preserve both global color coherence and exactness of detail. To address the first problem, we design a non-local fusion module (NFM) and an attention fusion module (AFM), and construct the multi-level pyramids' architecture to explore the local and global correlations of rain information from the rain image pyramid. More specifically, we apply the non-local operation to fully exploit the self-similarity of rain streaks and perform the fusion of multi-scale features along the image pyramid. To address the latter challenge, we additionally design a residual learning branch that is capable of adaptively bridging the gaps (e.g., texture and color information) between the predicted rain-free image and the clean background via a hybrid embedding representation. Extensive results have demonstrated that our proposed method is able to generate much better rain-free images on several benchmark datasets than the state-of-the-art algorithms. Moreover, we conduct the joint evaluation experiments with respect to deraining performance and the detection/segmentation accuracy to further verify the effectiveness of our deraining method for downstream vision tasks/applications. The source code is available at https://github.com/kuihua/MSHFN.

Split frequency attention network for single image deraining

Split frequency attention network for single image deraining

Adaptive weighted rain streaks model-driven deep network for single image deraining

In the computer vision field, single image rain removal is an extremely important pre-processing task since rain streaks can degrade the performance of subsequent high-level tasks and outdoor image quality. In recent years, massive deep learning-based deraining methods have been proposed to remove rain streaks from rainy images, and they have achieved promising performance. However, most deep learning-based methods lack interpretability and have limited performance in image details restoration and rain streaks removal. In this paper, we propose a novel rain streaks model-driven deep network, MSANet, to alleviate these issues. First of all, instead of simply stacking those available convolutions, pooling, and attention blocks, the structure of MSANet is derived from a prior rain removal model and the structure is thus fully interpretable. Compared to prior models available in the literature, we introduce a weighted convolutional dictionary model to capture the shapes, sizes, directions, regions, and multi-scale information of rain streaks. And to reduce the bias associated with the L2-norm, we utilize an adaptive approach to constrain the fidelity term of the model. Finally, we employ the alternating direction method of multipliers (ADMM) algorithm to optimize this model and unfold the optimization procedure into a new neural network architecture by modeling each operation in the algorithm using network blocks. All the parameters can be automatically learned by end-to-end training. Extensive experiments on several benchmark datasets demonstrate that the proposed network architecture is more efficient than state-of-the-art methods based on both subjective and objective evaluations.

Multi-Scale Aggregation Residual Channel Attention Fusion Network for Single Image Deraining

Images captured on rainy days are prone to rain streaking on various scales. These images taken on a rainy day will be disturbed by rain streaks of varying degrees, resulting in degradation of image quality. This study sought to eliminate rain streaks from images using a two-stage network architecture involving progressive multi-scale recovery and aggregation. The proposed multi-scale aggregation residual channel attention fusion network (MARCAFNet) uses kernels of various scales to recover details at various levels of granularity to enhance the robustness of the model to streaks of various sizes, densities, and shapes. When applied to benchmark datasets, the proposed method outperformed other state-of-the-art schemes in the restoration of image details without distorting the image structure.

Progressive Hybrid-Modulated Network for Single Image Deraining

Rainy degeneration damages an image’s visual effect and influences the performance of subsequent vision tasks. Various deep learning methods for single image deraining have been proposed, obtaining appropriate recovery results. Unfortunately, most existing methods ignore the interaction between rain-layer and rain-free components when extracting relevant features, leading to undesirable results. To break the above limitations, we propose a progressive hybrid-modulated network (PHMNet) for single image deraining based on the two-branch and coarse-to-fine framework. Specifically, a hybrid-modulated module (HMM) with a two-branch framework is proposed to blend and modulate the feature of rain-free layers and rain streaks. After cascading several HMMs in the coarsest reconstructed stage of the PHMNet, a multi-level refined module (MLRM) is adopted to refine the final deraining results in the refined reconstructed stage. By being trained using loss functions such as contrastive learning, the PHMNet can obtain satisfactory deraining results. Extended experiments on several datasets and downstream tasks demonstrate that our method performs favorably against state-of-the-art methods in quantitative evaluation and visual effects.

Combining multiscale learning and attention mechanism densely connected network for single image deraining

Combining multiscale learning and attention mechanism densely connected network for single image deraining

Recurrent wavelet structure-preserving residual network for single image deraining

The combination of deep learning and image prior has been widely used in single image deraining since 2017. Recent studies have demonstrated an excellent deraining effect on the high-frequency part of rain images, but less attention was paid to the low-frequency part of rain images. The rain streaks remain in the low-frequency part of rain images, thus limiting the deraining effect. Since the rain streaks in rain images are often mixed with object edges and background scenes, it is challenging to separate rain from them by directly learning the deraining function in the image domain. To solve these problems, we propose a novel Recurrent Wavelet Structure-preserving Residual Network (RWSRNet), which mainly preserves and introduces the low-frequency sub-images of each level into the low-frequency rain removal sub-networks that are greatly different from the state-of-the-art approaches introducing wavelet transform. In addition, we also share the low-frequency structure information to the high-frequency sub-networks through block connection, which further enriches the detailed information, facilitates convergence, and strengthens the ability of our network to remove rain streaks in high frequency. Finally, we fuse the derained low-frequency sub-images of each level through the proposed image weighted blending module and finally reconstruct the low- and high-frequency sub-images into clean images through inverse wavelet transform recursively. The experimental results indicate that the proposed method achieves an excellent deraining effect on both low- and high-frequency parts of rain images and has better performance in low-frequency preservation and high-frequency enhancement in comparison with the state-of-the-art approaches on synthetic and real image datasets.

Recurrent context-aware multi-stage network for single image deraining

Single image rain streak removal is extremely necessary since rainy images can seriously affect many computer vision systems. In this paper, we propose a novel recurrent context-aware multi-stage network (ReCMN) for image rain removal that gradually predicts clean derained results. Specifically, the ReCMN introduces a multi-stage strategy to perform contextual relationship modeling. Firstly, we use the densely residual extraction block (DREB) to guide feature extraction. Then, a multi-scale context aggregation block (MCAB) is designed to utilize the long-distance dependencies and multiple scale features, which can fuse features of different levels to fully exploit contextual information. Finally, we develop a parallel attention block (PAB) to capture the channel and spatial information and only pass effective feature representation. Experimental results demonstrate that our method outperforms several state-of-the-art methods, based on both synthetic datasets and real-world rainy images.

DBSwin: Transformer based dual branch network for single image deraining

Rain streaks severely affect the perception of the content and structure of an image so high-performance deraining algorithms are needed in order to eliminate the effects of various rain streaks for high-level computer vision tasks. Although much progress has been made with existing deraining methods, the task of single image deraining remains challenging. In this paper, we first point out that existing Transformers lack sufficient ability to capture channel attention which restricted the ability of models in deraining. To improve the performance of deraining model, we propose a dual branch deraining network based on Transformer. One branch uses dense connections to connect Transformer modules which embed the attention of a composite channel. This branch captures channel attention more finely to learn the representation of rain streaks features. The other branch first obtains features at different scales by gradually expanding the receptive field, then uses these features to obtain attention for regional features, and finally uses the attention to guide the model to focus on areas of high rain streaks density and large scales. By fusing these two branches, the model is able to capture channel attention more finely and to focus on regions of high rain streaks density and large scales. The extensive experimental results on synthetic and real datasets demonstrate that the proposed method outperforms most advanced deraining methods.

Coarse-to-fine multiscale fusion network for single image deraining

Cameras convert optical signals into electronic information to enable image acquisition. However, cameras are susceptible to environmental factors such as light and weather conditions. The information captured by cameras in rainy condition tends to degrade, resulting in reduced contrast and visibility of the images. Deraining aims to restore the image degraded by raindrops and rain accumulation in rainy weather. Most state-of-the-art, end-to-end neural network-based deraining methods have achieved satisfactory results. By considering the features from different scales, we proposed a further improved multiscale information exchange module to extract features from different scales. As a result, the structure achieves a competitive effect on rain mask detection. Moreover, experiments have found that the rain layer-based prediction models tend to leave rain streak residuals in specific image patches. Therefore, we further developed the Feature Attention Compensation module (FAC) to better utilize the deraining image and rain layer information obtained from the multiscale model, which is demonstrated to boost deraining performance. To summarize, we design a coarse-to-fine rain removal model, starting with a rain detection network to get a coarse rain-free image, which is further refined with the FAC module to eventually create a fine rain-free image. We conduct our experiments on both synthetic and real-world datasets. Quantitative and qualitative experimental results demonstrate that the proposed method outperforms the state-of-the-art deraining methods. Source code will be available at https://github.com/Jstar-s/CFMFN

Single image deraining using contrastive perceptual regularization

Rain streaks pollute the image captured from outdoor vision system, and single image deraining approach based on data-driven has witnessed the continuously growing and achieved great success. Here, an end-to-end network for single image deraining is proposed. Firstly, to address the limit of convolution neural network (CNN) which can only extract local feature, a graph based basic block is proposed to extract global feature. The basic block consists of graph convolutional network (GCN) and CNN. The GCN module which combines spatial coherence computing and channel correlation computing is introduced to extract non-local information. While the CNN module, which combines the channel attention and pixel attention, is used to earn more weight from important local features. Secondly, a contrastive perceptual regularization is adopted to enhance the loss function, and a more natural image is restored by utilizing the information from both positive and negative samples with the regularization. The restored image is pulled closer to the positive clear image and pushed farther away to the negative rainy image. The experiment results on several datasets demonstrate that these methods achieve better results than the previous state-of-art methods.

Contrastive learning-based generative network for single image deraining

The crux of image deraining stems from the challenge of recognizing the diverse rain patterns within the rainy image. Most methods for image deraining remain visible rain residuals in the restored image, which suffers from insufficient modeling of rain streaks. In this work, we propose contrastive learning-based generative network (CLGNet), which follows a coarse-to-fine framework. In the coarse phase, our CLGNet employs the hierarchical encoder–decoder structure to remove obvious rain patterns, and first generates the coarse background image. Then, we introduce a well-designed multiscale feature aggregation module in the refining phase to extract and integrate global information dependencies from different scales. In additon, to facilitate the intra-stage and cross-stage information interaction, we propose the intra-stage feature fusion module and the cross-stage feature fusion module to encode broad contextual information. More importantly, we propose an innovative contrastive learning strategy and apply it to each stage of our proposed CLGNet to enhance the decoupling ability of the encoder and help the model recognize complex rain patterns. Extensive experiments on five benchmark datasets demonstrate the superiority of our proposed CLGNet than other state-of-the-art methods for single image deraining on both the visual quality and quantitative evaluation.

LRP-net: A lightweight recursive pyramid network for single image deraining

Single image deraining, as a low-level computer vision task, has been drawn extensive attention in recent years. Rain streaks can degrade subjective visibility quality, meanwhile, bring significant difficulties to subsequent high-level computer vision tasks such as object detection. Nowadays, deep-learning based methods, specifically Convolutional Neural Networks (CNN) based ones are adopted to remove the rain streaks and become the state-of-the-art. However, existing popular deep-networks have complicated branches and numerous layers, which strengthen the ability of removing rain streaks and result in high memory and computational cost inevitably. This restricts many applications in real-time and limited computation resource scene, especially on mobile or edge devices. To handle this issue, this paper proposes a novel Lightweight Recursive Pyramid network (LRP-Net) with a small number of parameters for single image deraining. To begin with, we propose a novel Lightweight Pyramid Deraining (LPD) block which consists of a multi-scale pyramid convolution for sufficient feature extraction and a pointwise convolution for feature fusion. Meanwhile, we also design a novel group convolution strategy in LPD for the sake of remarkable parameter reduction. Secondly, we combine a recursive deraining mechanism, a critical component that serves as a feature fusion iterator to construct our LRPNet a powerful and lightweight multi-stage model. In the benefit of the combination between the LPD block and recursive mechanism, the total number of parameters in LRP-Net is only 130 k, which is nearly a 40- reduction compared with the latest state-of-the-art models. The extensive experiments demonstrate the superiority of LRP-Net in both quantitative assessments and visual quality.