Abstract
Dynamic scene video deblurring is a challenging task due to the spatially variant blur inflicted by independently moving objects and camera shakes. Recent deep learning works bypass the ill-posedness of explicitly deriving the blur kernel by learning pixel-to-pixel mappings, which is commonly enhanced by larger region awareness. This is a difficult yet simplified scenario because noise is neglected when it is omnipresent in a wide spectrum of video processing applications. Despite its relevance, the problem of concurrent noise and dynamic blur has not yet been addressed in the deep learning literature. To this end, we analyze existing state-of-the-art deblurring methods and encounter their limitations in handling non-uniform blur under strong noise conditions. Thereafter, we propose a first-to-date work that addresses blur- and noise-free frame recovery by casting the restoration problem into a multi-task learning framework. Our contribution is threefold: a) We propose R2-D4, a multi-scale encoder architecture attached to two cascaded decoders performing the restoration task in two steps. b) We design multi-scale residual dense modules, bolstered by our modulated efficient channel attention, to enhance the encoder representations via augmenting deformable convolutions to capture longer-range and object-specific context that assists blur kernel estimation under strong noise. c) We perform extensive experiments and evaluate state-of-the-art approaches on a publicly available dataset under different noise levels. The proposed method performs favorably under all noise levels while retaining a reasonably low computational and memory footprint.
Highlights
V IDEOS aim at faithfully reflecting the motion in dynamic scenes but concurrent motion blur and noise can severely obscure scene perception
We demonstrate that the sequential utilization of off-the-shelf state-of-the-art video denoising and deblurring algorithms is ineffective
MS-RDM: We propose multiscale residual dense modules to learn coarse-to-fine, dense representations, enhanced by MECA, a novel extension of the efficient channel attention module [20] to further modulate deformable convolutions and increase restoration performance while retaining the number of FLOPs
Summary
V IDEOS aim at faithfully reflecting the motion in dynamic scenes but concurrent motion blur and noise can severely obscure scene perception. The problem of spatially variant motion blur, related to independently moving objects in the presence of noise, has not yet been addressed in the deep learning literature. Is it an intrinsically challenging problem, but relevant research is limited by the difficulty in constructing such labeled datasets [17], [18]. Contributions: To address the aforementioned limitations, we propose R2-D4, the first-to-date deeply learned network that leverages the feature-sharing potential of MTL to increase model efficiency and jointly address dynamic video denoising and deblurring. The recently published ARID [37] is a low-light dataset that motivates the proposed problem, exhibiting both noise and blur, but lacks the respective paired clear frames
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have