Virtual camera-based visual servoing for rotorcraft using monocular camera and gyroscopic feedback

Abstract

In this paper, an image-based visual servoing (IBVS) control strategy with a virtual camera frame is proposed for multirotor vehicles. Compared to previous works, the proposed IBVS controller requires minimal sensors in the position loop: a monocular camera and gyroscope. To achieve this, the visual feature projected onto the virtual image plane is associated with the plane’s normal instead of relying on an additional attitude estimator and prior knowledge of the plane’s inclination. Furthermore, we show that the ratio velocity, when scaled by the image moment, exhibits a similar dynamics to the linear velocity. The finding allows the quantity to be recovered and used for control with a monocular camera without other metric cues. To provide feedback for the controller, an Extended Kalman filter for estimating the ratio velocity, target plane’s inclination, and relative rotation between the current and reference camera frame is developed using only monocular vision and gyroscopic measurements. To validate the proposed controller and estimation strategy, both simulation and real-world flight experiments were carried out. The quadrotor smoothly and robustly tracked both dynamic horizontal and static inclined targets, without prior knowledge of the target’s inclination. Overall, the proposed regime offers a lightweight and robust alternative IBVS solution for rotorcraft.