Hybrid Marker-Based Object Tracking Using Kinect v2

Abstract

Marker-based object tracking can be used in many scenarios, such as surgical instrument tracking. In this article, we propose an accurate and flexible-in-design method of hybrid marker-based object tracking using one time-of-flight (ToF) camera alone. The hybrid markers are composed of real markers and virtual markers on the same plane. The real marker is a flat and circular tag covered by infrared (IR) reflective material that facilitates its detection for its high intensity in the IR image, while the virtual marker is introduced as an arbitrary point as long as it is surrounded by the real markers. The virtual marker’s image position is interpolated with the real markers and its depth is estimated with a weighted average filter accounting for perspective projection distortion, the quality of interpolation, and the noise in the depth map. To reduce the position error, the virtual marker’s image coordinate and depth are temporally smoothed with the Kalman filter sequentially. Then, the virtual marker’s spatial position is reconstructed via backward perspective projection. Finally, the pose and position of the target is estimated by template matching of virtual markers. Dedicated experiments were designed to evaluate the performance of the presented method with the real markers’ diameter 10 mm, the incident angle of IR light on the plane smaller than 30°, and the distance from Kinect v2 less than 2000 mm. The results show that the tracking error was less than 0.45° in orientation and 1% in position. The presented method shows perspective in object tracking using a ToF sensor.