Dast-Net: Depth-Aware Spatio-Temporal Network for Video Deblurring

Abstract

Video deblurring is a challenging task due to inevitable blurs caused by depth variation, object motion, and camera shake. Although several video deblurring methods resort to depth maps, they rarely produce visually appealing results since the information in the depth maps is used insufficiently. To address this issue, we propose a Depth-Aware Modulated (DAM) block for efficiently utilizing the depth map characteristics, in which the intensity and variation of depth are exploited according to the depth map value and edges. Based on the DAM block, we develop the Depth-Aware Spatio-Temporal Network (DAST-Net) tailored for video deblurring. Particularly, the Depth-Aware Temporal Alignment module uses the depth cues to guide the alignment of adjacent frames. The Depth-Modulated Spatial Fusion module then warps the aligned frames to maintain spatial invariance with the aligned features. The warped depth features are more effective in video deblurring, since they allow for the aggregation of multiple frames. Extensive quantitative and qualitative evaluations demonstrate that the proposed DAST-Net outperforms other state-of-the-art methods.