Rain Streaks and Snowflakes Removal for Video Sequences via Motion Compensation and Matrix Completion

Abstract

Image and video deraining tasks aim to reconstruct original scenes, from which human vision and computer vision systems can better identify objects and more details present in images and video sequences. This paper proposes a three-step method to detect and remove rain streaks, even snowflakes from great majority video sequences, using motion compensation and low-rank matrix completion method. Firstly, we adopt the optical flow estimation between consecutive frames to detect the motion of rain streaks. We then employ the online dictionary learning for sparse representation technique, and SVM classifier to eliminate parts that are not rain streaks. Finally, we reconstruct the video sequence by using low-rank matrix completion techniques. In particular, by introducing the image dehazing network(GCANet) to our proposed method, the heavy rain caused dense rain accumulation and blurry phenomenon can be worked out well. The experimental results demonstrate the proposed algorithm and perform qualitatively and quantitatively better in several image quality metrics, boosting the best published PSNR metric by 4.47%, 6.05% on two static video sequences and 12.13% on a more challenging dynamic video sequence. In addition, to demonstrate the generality of the proposed method, we further apply it to two challenge tasks, which also achieves state-of-the-art performance.