Bridging the Gap Between Visual Servoing and Visual SLAM: A Novel Integrated Interactive Framework

Abstract

For pose stabilization task of nonholonomic mobile robots, this article proposes a novel integrated interactive framework, bridging the gap between visual servoing and simultaneous localization and mapping (SLAM). The framework consists of two cooperative components, control module for servoing task and SLAM module for feedback signals estimation. In most visual servoing methods, feedback signals for the servoing controller are estimated by means of multiple-view geometry assuming the target scene being always within the camera field of view (FOV). To handle the challenge that the target scene gets out of view during servoing process, the desired image is associated with the initial map by a two-step strategy, and an incremental map is constructed to guarantee available feedback signals estimation. In addition, on the basis of the kinematic model of the mobile robot and velocities designed by the servo controller, the predicted pose is exploited to discard moving objects in the camera FOV, thus making the proposed framework effective in dynamic scenes. Experimental results operated in different scenes without prior information demonstrate the effectiveness of the proposed approach to handle the FOV problem and dynamic scenes. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —Traditional visual servoing stabilization approaches usually require that the feature points in the target scene remain within the FOV of the camera for feedback signals calculation, which is often neglected. Motivated by the requirement of continuous feedback signals to the servo controller, the SLAM technique is introduced to relax the FOV constraint during the servoing process. A novel integrated interactive framework is proposed in this article to further increase the applicability of the servoing system in practice, in which the SLAM module is also redesigned for the flexibility in dynamic scenes. The SLAM module provides feedback signals for the servo controller; meanwhile, velocities designed by the servo controller are utilized for the prediction mechanism in the SLAM module to discard features on moving objects. Experiments validate the applicability of the proposed framework in different scenarios.