Adaptive Image-based Visual Servoing Research Articles

Vision-based UAV adaptive tracking control for moving targets with velocity observation

An adaptive image-based visual servoing (IBVS) controller is designed for a quadrotor unmanned aerial vehicle (UAV) to achieve robust tracking for moving targets, under underactuation and tight coupling constraints of UAV kinematics. Specifically, image features are selected from perspective image moments of a planar target to obtain virtual feature dynamics regarding UAV kinematics and dynamics. By constructing an auxiliary variable, a translational velocity observer for moving target is constructed by using virtual image features. An IBVS tracking controller is designed without target geometric information by combining UAV and visual feature dynamics. Designed controller and observer make the UAV robustly reach desired height and track the moving target, despite uncertainty of target movement. The controller has asymptotical convergence performance, and the target velocity is observed according to Lyapunov stability analysis. Simulation and experimental results show that the proposed method has smoother and more accurate performance in motion tracking and target velocity prediction under system uncertainty.

Immersion-and-Invariance Fuzzy Adaptive Image-Based Visual Servoing of Omnidirectional Mobile Robots Considering Uncertain Feature Point Dynamics

Immersion-and-Invariance Fuzzy Adaptive Image-Based Visual Servoing of Omnidirectional Mobile Robots Considering Uncertain Feature Point Dynamics

Adaptive Visual Servoing Shape Control of a Soft Robot Manipulator Using Bézier Curve Features

Soft continuum robot outperforms its rigid counterpart when executing in complicated environments thanks to its high dexterity of the redundant configuration. Shape control is regarded as a prerequisite to applications in unstructured environments. In this article, we develop an adaptive image-based visual servoing (IBVS) algorithm using an uncalibrated camera to realize visual shape control of the cable-driven soft manipulator. A Bézier curve is selected to be the feature providing shape information of the soft robot. An adaptive law is proposed to simultaneously solve the uncalibrated problem and to analytically model the motion of curve control points. The control algorithm is then designed with third-order Bézier curve being the image features, and theoretically proves its stability. The algorithm has been experimentally validated in the cable-driven soft continuum robot. The experiment results demonstrate rapid converging performance to C-type and S-type reference shapes.

Uncalibrated image-based visual servoing approach for translational trajectory tracking with an uncertain robot manipulator

This work presents an adaptive image-based visual servoing approach for uncertain robot manipulators using an eye-to-hand camera configuration, which does not require any calibration procedure and image velocity measurement. A monocular camera provides visual feedback for the robot’s joint control, allowing successful tracking of the translational trajectory, prescribed in image space, for a given target object attached to the robot end-effector. As our contribution, an indirect adaptive control approach is proposed to deal with the uncertain parameters of the camera–robot system and any camera misalignment concerning the robot frame. The indirect adaptive visual servoing approach is then combined with a direct adaptive motion controller using a cascade control framework to consider the uncertain robot dynamics in our stand-alone robot vision system. Lastly, an adaptive model-based state observer is designed to avoid the need for measuring the image feature velocity in the interconnected control loops. The Lyapunov stability theory and the passivity paradigm are synergized to demonstrate the stability properties of the overall closed-loop system. Experimental results, obtained with a 4-DoF robot manipulator carrying out translational trajectory tracking tasks using a commercial webcam, illustrate the performance and effectiveness of the proposed adaptive control methodology.

Visual Servoing of a Cable-Driven Soft Robot Manipulator With Shape Feature

Soft continuum robot is of superiority in the applications in unstructured environments due to its high flexibility and safe interaction ability. Accurate shape control is regarded as one of the prerequisites to improve its practicability. Considering the situation where the 3D position signals are not available, this letter investigates the vision-based shape control scheme of a soft robot manipulator. The curvature features used to depict the reference image shape of the robot backbone curve are solved through nonlinear optimization with given constraints of two endpoints in pixel coordinates. Thereafter, an adaptive image-based visual servoing controller is designed to track the reference image shape features with an uncalibrated monocular camera. The proposed algorithm was tested on an eight-cable driving soft robot prototype and proved its validity in convergence to the desired image shape.

Adaptive IBVS and Force Control for Uncertain Robotic System with Unknown Dead-zone Inputs

This article introduces a novel control strategy for the uncertain eye-to-hand system, which is considered to work with unknown model of constraint surface and uncalibrated camera model. Besides, the uncertain dynamics and kinematics are also included in the system. In order to be closer to the real robot system, we also consider it with dead-zone inputs situation. So the parameter intervals and slopes of the dead-zone model is also unknown. Hence, a novel adaptive image-based visual servoing (IBVS) and force control approach is put forward. The control method of unknown force and uncalibrated camera model is achieved by adaptive control. The solution of unknown dead-zone inputs is completed by designing a inverse smooth model of dead-zone inputs to offset the nonlinear affect due to the actuator constraint, and the whole system is proved that the force tracking control and image position converge to zero asymptotically. Finally, the MATLAB simulation is set up and the experiment shows the validity of the proposed scheme.

Adaptive Image-Based Leader–Follower Formation Control of Mobile Robots With Visibility Constraints

This article addresses the problem of leader-follower formation control of mobile robots using only onboard monocular cameras that subjected to visibility constraints. An adaptive image-based visual servoing control strategy was proposed following the prescribed performance control methodology. First, the leader-follower visual kinematics in the image plane and an error transformation with predefined performance specifications are presented. Then, an adaptive control law with online estimating the inverse height between the optical center of camera and the single feature point attached to leader is designed to ensure the global stability of the closed-loop system. Finally, the applicability and performance of the proposed control scheme are demonstrated by the numerical simulations and hardware experiments. Compared with other formation control schemes, our solution relies only on onboard visual sensors without communication, since it does not need the relative angle/distance between the robots, or the velocity of the leader. Moreover, it guarantees the prescribed transient and the steady-state performance besides the visibility constraints.

Adaptive Image-Based Visual Servoing Using Reinforcement Learning With Fuzzy State Coding

Image-based visual servoing (IBVS) allows precise control of positioning and motion for relatively stationary targets using visual feedback. For IBVS, a mixture parameter $\beta$ allows better approximation of the image Jacobian matrix, which has a significant effect on the performance of IBVS. However, the setting for the mixture parameter depends on the camera's real-time posture; there is no clear way to define the change rules for most IBVS applications. Using simple model-free reinforcement learning, Q-learning, this article proposes a method to adaptively adjust the image Jacobian matrix for IBVS. If the state-space is discretized, traditional Q-learning encounters problems with the resolution that can cause sudden changes in the action, so the visual servoing system performs poorly. Besides, a robot in a real-world environment also cannot learn on as large a scale as virtual agents, so the efficiency with which agents learn must be increased. This article proposes a method that uses fuzzy state coding to accelerate learning during the training phase and to produce a smooth output in the application phase of the learning experience. A method that compensates for delay also allows more accurate extraction of features in a real environment. The results for simulation and experiment demonstrate that the proposed method performs better than other methods, in terms of learning speed, movement trajectory, and convergence time.

Adaptive Image-Based Visual Servoing for Hovering Control of Quad-Rotor

Image-based visual servoing (IBVS) achieves precise positioning and motion control for a relatively stationary target by visual feedback, but problems persist with convergence and stability. Appropriate servoing gains for the IBVS are critical to the convergence and stability, but this control gain is heuristically a constant for most IBVS applications. This paper proposes an integrated method that allows adaptive adjustment of the servoing gain by reinforcement learning (RL) for IBVS control. The proposed method learns a policy to determine the value of the servoing gain on the fly. To ensure rapid convergence for the RL, truncating ${Q}$ -learning (TQL) with faster convergence is used as learning algorithm, which uses truncated temporal differences (TDs) to update the TD. A nonuniform state space partitioning as a state encoder for RL allows more efficient policy. A strategy that uses the Metropolis derived from the simulated annealing is introduced for selecting the action, in order to balance exploration and exploitation so as to accelerate the learning speed. The integrated IBVS control system is tested using experiments involving a quad-rotor helicopter hovering control. The results of simulation and experiment show that the integrated IBVS method increases stability and ensures more rapid convergence than other methods.

Adaptive Image-Based Visual Servoing With Temporary Loss of the Visual Signal

Image-based visual servoing (IBVS) can reach a desired position for a relatively stationary target using continuous visual feedback. Proper feature extraction and appropriate servoing control laws are essential to performance for IBVS. IBVS control can be interrupted or interfered abruptly if no features are extracted when the observed object is occluded. To address the problem of missing feature points in current images during a visual navigation task, a homography method that uses a priori visual information is proposed to predict all of the missing feature points and to ensure the execution of IBVS. The mixture parameter for the image Jacobian matrix can also affect the control of IBVS. The settings for the mixture parameter are heuristic so there is no a systematic approach for most IBVS applications. An adaptive control approach is proposed to determine the mixture parameter. The proposed method uses a reinforcement learning (RL) method to adaptively adjust the mixture parameter during the robot movement, which allows more efficient control than a constant parameter. A logarithmic interval state-space partition for RL is used to ensure efficient learning. The integrated visual servoing control system is validated by several experiments that involve wheeled mobile robots reaching a target with a desired configuration. The results for simulation and experiment demonstrate that the proposed method has a faster convergence rate than other methods.

Adaptive visual servoing of UAVs using a virtual camera

We propose an adaptive image-based visual servoing controller for an unmanned aerial vehicle (UAV) that is based on a virtual camera. The controller regulates relative position and yaw of the vehicle to a planar target consisting of multiple points. We adopt image moment features, which simplify the controller’s derivation. The control is adaptive to various system parameters. We prove exponential stability of the error dynamics. Experimental results demonstrate controller performance.

Adaptive Image-Based Visual Servoing for an Underactuated Quadrotor System

This paper presents an adaptive image-based visual servoing (IBVS) integratedwith adaptive slidingmode control for a vision-based operation of a quadrotor unmanned aerial vehicle (UAV). For a seamless integration with underactuated quadrotor dynamics, roll and pitch channels are decoupled from the other channels using virtual features. This allows a simple and accurate algorithm for estimating depth information and successful application of the proposed guidance and control algorithm.By employing an adaptive gain in the IBVS controlmethod, the chance of image feature loss is reduced, and performance and stability of the vision-guided UAV control system are improved. The overall setup allows image features to be placed at the desired position in the image plane of a camera mounted on the quadrotor UAV. Stability of the IBVS systemwith the controller is proved using Lyapunov stability analysis. Performance of the overall approach is validated by numerical simulation, vision integrated hardware-inthe-loop simulation, and experiments. The results confirm that the target image is successfully placed at the desired position of the image plane and the quadrotor state variables are properly regulated, showing robustness in the presence of sensor noise, parametric uncertainty, and vibration from motors.