Integration of Vision and Inertial Sensors for a Surgical Tool Tracking

Abstract

This paper represents a hybrid Vision/INS system in a microsurgical tool tracking application. Surgical MEMS devices must not only cope with all of the challenges that conventional MEMS devices have, but also address the integration of electronics and signal processing, calibration, reliability, accuracy and testing. A hybrid Vision/INS system with the integration of the Extended Kalman Filter precisely calculates 6D position-orientation of a microsurgical tool during surgery. This configuration guarantees the real-time tracking of the instrument. Ultimately, the vision system supports the IMU to deal with the drift problem but the position error increases dramatically in the absence of the vision system. In this paper, the tool motion modeling is proposed to bind the error in the acceptable range for a short period of missing data. The motion of the tool is modeled and updated at any time that the instrument is in the camera view field. This model is applied to the estimation algorithm whenever the camera is not in line of site and the optical data is missing.