Video Super-Resolution for Wireless Capsule Endoscopy Imaging Sensor

Abstract

As a preferred diagnostic imaging sensor for small bowel diseases, wireless capsule endoscopy (WCE) suffers from poor video quality, which can be mitigated via video super-resolution (VSR) techniques. However, the promise of applying VSR into WCE applications has been offset by two significant challenges. Firstly, high-resolution (HR) counterparts of real endoscopic frames are unavailable for supervised learning. Secondly, utilizing temporal correlations to enhance recovery performance becomes difficult due to the poor video quality. To tackle these two challenges, we develop a novel training dataset and an effective alignment network for VSR on practical WCE applications. Specifically, clean natural videos and synthetic endoscopic videos are collected as the ground truth of the training dataset. Then, a set of complex degradation models is designed to generate their low-resolution (LR) counterparts for supervised learning. Although real HR endoscopic videos remain unknown during training, this complicated dataset enables the trained network to have a strong generalization capacity so that it can make reasonable inferences on real endoscopic videos. To utilize temporal correlations from WCE videos, a block-based temporal attention alignment network (BTAAN) is proposed to build accurate and robust correspondences among consecutive frames under poor video quality. This network exploits the global information from neighboring frames to capture large-scale motions, and suppresses the effect of unreliable points by a temporal attention mechanism. Moreover, its block-wise characteristic reduces the computational complexity significantly, which makes it hardware-friendly. Extensive experiments demonstrate the superiority of the proposed method over other state-of-the-art approaches on practical WCE applications.