Deep learning for registration of region of interest in consecutive wireless capsule endoscopy frames

Abstract

Background and objective: Functional gastrointestinal disorders (FGIDs) are reported as worldwide gastrointestinal (GI) diseases. GI motility assessment can assist the diagnosis of patients with intestine motility dysfunction. Wireless capsule endoscopy (WCE) can acquire images in the gastrointestinal (GI) tract including the small intestine where other conventional endoscopes cannot penetrate, and WCE images can reveal GI motility. To generally analyze WCE frames, the high-precision registration of consecutive WCE frames is an absolute necessity. It is difficult and meaningless to register entire WCE frames on a pixel level due to the unpredictable and massive non-rigid deformation between consecutive frames, the low quality of imaging and the complex intestinal environment. Thus, the registration of region of interest (ROI) functioning in a feature level has more significance than entire frame registration.Methods: In this paper we present Timecylce-WCE, an end-to-end automatic registration approach of ROIs on WCE images. The clinicians can determine a ROI by drawing a bounding box in any WCE frame to be registered. This proposed approach is based on a deep-learning model of time-consistency in recurrent-registering, skip-registering and self-registering cycle, and it is fully unsupervised without any label. We incorporate the global correlation map with the local correlation map in matching the features, and a novel overall loss function is designed to enable the convergence of the model. As the output, a thin-plate spline (TPS) transformed region in the template frame is highly aligned with the query ROI in a finer-grained level. To the best of our knowledge this is the first time that a deep-learning-based registration method is proposed for WCE imaging motion.Results: To highlight the effectiveness of the proposed approach, our proposed method is compared with the existing non-deep-learning methods and tested in a validation dataset with labeled matching points. The presented method resulted in the best PCK@10 (Percentage of Correct Key-points, i.e., the predicted and the true joint is within the threshold - 10 pixels) of 66.49%. We also demonstrate that variants of design improved registration accuracy.Conclusions: From the experimental analysis, it is clear that our proposed method outperforms the other existing methods. This lays the groundwork for subsequent studies, such as GI motility assessment, and WCE image synthesis.