Semi-supervised Video Deraining Based on Enhanced Spatio- Temporal Interaction Network

Abstract

Video rain removal is an important task in computer vision. Although the video rain removal method based on deep learning has achieved great success in recent years, there are still two main challenges: how to use the large amount of information between consecutive frames to extract cross-space and the powerful spatio-temporal characteristics of the time domain, and how to restore the use of high-speed methods to produce high-quality rain-removing videos. In this article, we propose a new end-to-end video rain removal framework called Enhanced Spatio-temporal Interaction Network (ESTINet), which greatly improves the current state-of-the-art video rain removal quality and speed. The enhanced spatio-temporal interaction network uses the advantages of deep residual networks and convolutional long and short-term memory to capture the spatial characteristics and temporal correlation between consecutive frames at the cost of few computing resources. In addition, Different prior formats are designed for labeled synthetic data and unlabeled real data, and a semi-supervised learning mechanism is designed through different prior formats.