Simultaneous Tracking, 3D Reconstruction and Deforming Point Detection for Stereoscope Guided Surgery

Abstract

AbstractTissue deformation is one of the major difficulties in the registration of pre-operative and intra-operative data. Vision based techniques have shown the potential to simultaneously track the endoscope and recover a sparse 3D structure of the tissue. However, most of such methods either assume a static environment or require the tissue organ to have a periodic motion such as respiration. To deal with the general tissue deformation, a new framework is proposed in this paper with the ability of simultaneous stereoscope tracking, 3D reconstruction and deforming point detection in the Minimally Invasive Surgery (MIS) environment. First, we adopt a Parallel Tracking and Mapping (PTAM) framework and extend it for the use of stereoscope in MIS. Second, this newly extended framework enables the detection of deforming points without restricted periodic motion model assumptions. Our proposed method has been evaluated on a phantom model, and in vivo experiments demonstrate its capability for accurate tracking in nearly real time speed as well as 3D reconstruction with hundreds of 3D points. Those experiments have shown that our method is robust towards tissue deformation and hence have promising potential for information integration by registration with pre-operative data.KeywordsTracking AccuracyStereo CameraBundle AdjustmentStructure From MotionReprojection ErrorThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.