See clearly on rainy days: Hybrid multiscale loss guided multi-feature fusion network for single image rain removal

Huiyuan Fu,Huadong Ma,Yu Zhang

doi:10.1007/s41095-021-0210-3

Huiyuan Fu, Huadong Ma + Show 1 more

Open Access

https://doi.org/10.1007/s41095-021-0210-3

Copy DOI

Abstract

The quality of photos is highly susceptible to severe weather such as heavy rain; it can also degrade the performance of various visual tasks like object detection. Rain removal is a challenging problem because rain streaks have different appearances even in one image. Regions where rain accumulates appear foggy or misty, while rain streaks can be clearly seen in areas where rain is less heavy. We propose removing various rain effects in pictures using a hybrid multiscale loss guided multiple feature fusion de-raining network (MSGMFFNet). Specially, to deal with rain streaks, our method generates a rain streak attention map, while preprocessing uses gamma correction and contrast enhancement to enhanced images to address the problem of rain accumulation. Using these tools, the model can restore a result with abundant details. Furthermore, a hybrid multiscale loss combining L1 loss and edge loss is used to guide the training process to pay attention to edge and content information. Comprehensive experiments conducted on both synthetic and real-world datasets demonstrate the effectiveness of our method.

Highlights

Images captured by cameras are one of the most important information sources for intelligent transportation systems, video surveillance systems, self-driving systems, etc
We propose a hybrid multiscale loss guided multiple feature fusion de-raining network (MSGMFFNet) to tackle these issues
Considering the variety of rain streaks, we propose a hybrid multiscale loss guided multiple feature fusion de-raining network (MSGMFFNet) to use features from multiple inputs to facilitate rain streak removal

Summary

Introduction

Images captured by cameras are one of the most important information sources for intelligent transportation systems, video surveillance systems, self-driving systems, etc. Traditional optimization based approaches [3, 4, 7, 9, 13] assume that a rainy image is made up of a rain streak layer and a clean background layer and treat it as a decomposition problem These methods find it difficult to select effective features. Most approaches tend to mistake background texture details for rain streaks or retain some rain streaks in rainy regions, as most existing single image de-raining approaches are not designed to fully consider the complexity of rain streaks and lack the ability to handle different types of rain streaks These issues affect the features extracted by high

Objectives

Methods

Conclusion