Single camera based motion tracking for minimally invasive surgery

Abstract

In this paper, we present a vision based tracking scheme to capture kinematic data of two laparoscopic tools in a surgical trainer or simulator. Camshift algorithm along with Kalman filter is used for occlusion-free tracking of multiple markers attached to the tools. Tracking of the markers in 3D is achieved, using a single stationary camera, by tracking variations in the size of the markers in the image. This 3D marker position data is used to determine the pose of the tools. A camera rig capable of handling multiple cameras is built for validating the proposed tracking system. Experiments were performed to characterize the errors in tool pose, and estimated marker locations. Results indicate that the proposed scheme can be employed in a box trainer or simulator to track tool movements with sub-cm accuracy. Such tool movement data can be further used for surgical skill assessment, or for providing feedback to the trainee/surgeon during the procedure. Provided independent measurements of the camera movements are available, this method can also be used to track tool movements when the camera is not fixed.