Temporal-Spatial Consistency and Sampling Neural Network Based Adaptive Hand-held Video Deblurring

Abstract

Video deblurring for hand-held camera is vital in many high-level video enhancement applications. Although quite a few proposed approaches have achieved remarkable success in recent years, many technical challenges still prevail for video (from hand-held camera) containing wide-range scenes and highly-variable objects. In particular, we are lacking effective and versatile strategies to adaptively handle high-level alignment, parallax diminution, unclear area detection, sharpness enhancement in a consistent fashion. To ameliorate, this paper develops a novel method to successively respect temporal-spatial alignment and precise deblurring. Our aim is to devise a new adaptive video deblurring technique by resorting to new modeling strategies. This paper's key originality is hinged upon the joint utility of both self-adaptive intrinsic mode functions (IMFs) based on empirical mode decomposition (EMD) in the temporal domain for video signal and mesh-structure constraint enforcement in the spatial domain, beside, this paper innovatively jointly utilities the subsampling convolution, upsampling convolution and luckiness threshold value based back propagation, which takes video deblurring as a high-precision adaptive convolution. As a result, our new approach can adaptively improve the clarity of the video, and synchronously optimize the camera trajectory of wobbly video. To validate our optimization approach for adaptive video deblurring, we conduct comprehensive experiments on public benchmarks, and make extensive and quantitative evaluations with available state-of-the-art methods as well as popular commercial software. All of our experiments demonstrate the advantages of the optimization method in terms of versatility, accuracy, and efficiency.