Predictive tracking of an object by a pan–tilt camera of a robot

Abstract

Moving-object tracking using a pan–tilt camera setup is quite a well-known task in robotics. However, the presented research addresses specific properties of the tracked object and introduces novel features to the pan–tilt camera control strategy. Pan–tilt camera control does not operate in an isolated environment. It is a part of the visual servoing system with specific goals. The system has to fulfill certain purposes, which affect its configuration and functionality. The pan–tilt system aims at keeping the visually tracked object within the middle of the image. At the same time, the overall visual servoing efficiently recognizes and tracks the object enabling its grasping by the robot arm. It uses a predictive strategy utilizing specific second-order linear models for pan and tilt joints. Model predictive control (MPC) introduces into the system the ability to predict camera operation over the specific horizon according to the predefined tracking goals. As the system anticipates future positions over the horizon of operation, the setpoint prediction of the future tracked system positions is required. Visual object recognition and tracking system use particular strategies for preparing online tracked object extrapolation over MPC horizon. Therefore, the pan–tilt camera system is intrinsically coupled to camera-based recognition and tracking. Predictive pan–tilt positioning keeps the tracked system in the middle of the image, while the visual system extrapolation improves the tracking performance. The proposed approach is thoroughly tested in the dedicated Gazebo-based robot simulator. Finally, the system is implemented and validated on the Velma robot. The results and their comparison with other control strategies confirm the initial assumptions, allowing further visual servoing system development.