Visual Servoing Of Robot Manipulators Research Articles

Adaptive fault-tolerant control for uncalibrated camera–robot system with multiple uncertainties

In this paper, an adaptive fault-tolerant control scheme is proposed for uncalibrated visual servoing of robot manipulator with multiple uncertainties and possible actuator failures. First, to accommodate the partial-loss-of-effectiveness failures, a parametrization form of fault estimation is derived by applying the adaptive backstepping control technique. Furthermore, an on-line gradient descending minimization method developed based on the Slotine–Li algorithm is designed to handle the uncertainties of uncalibrated camera parameters. By utilizing the Lyapunov analysis method, it can be proved that all signals in the closed-loop system are bounded and the image tracking error converges to zero asymptotically. Simulation based on a 3-DOF manipulator validates the obtained results.

Uncalibrated and Unmodeled Image-Based Visual Servoing of Robot Manipulators Using Zeroing Neural Networks.

Neural networks have been widely investigated for the control of robot manipulators and recurrent neural network (RNN) is accepted as a powerful tool for visual servoing. Different from existing control schemes for robot-camera systems, this article proposes a novel image-based visual servoing (IBVS) control scheme for both the regulation and tracking control of robot manipulators in the framework of a special class of RNN, termed zeroing neural network (ZNN), which does not require prior knowledge about camera configuration and kinematic model parameters. The proposed control scheme is composed of a data-driven mapping estimator and a controller, both of which are designed based on ZNN. To facilitate the deployment of the proposed IBVS control scheme, a discrete-time version of the proposed control scheme is developed. Theoretical analysis for the proposed method is presented in terms of convergence, stability, and robustness. In addition, simulations and experiments are carried out based on different types of robot-camera systems to verify the efficacy and portability of the proposed control scheme for solving regulation and trajectory IBVS problems. Moreover, comparative studies are performed to reveal the merits of the proposed control scheme.

Fuzzy Adaptive Model Predictive Control for Image-based Visual Servoing of Robot Manipulators With Kinematic Constraints

Fuzzy Adaptive Model Predictive Control for Image-based Visual Servoing of Robot Manipulators With Kinematic Constraints

Adaptive event-triggered control for uncalibrated visual servoing of robot manipulators

In the study, an adaptive event-triggered control strategy is proposed for image-based visual servoing of eye-to-hand robot manipulators, where the camera used does not need to be calibrated, and the dynamics behavior of manipulator is considered in design and analysis. To address the uncertainty in camera parameters and the nonlinearity in robot dynamics, and accommodate the errors between the real-time control signals and the piecewise-constant event-triggered control signals, a new and novel robust adaptive estimation approach is developed, and well fused with visual feedback control design so that the boundedness of all closed-loop signals, and the asymptotic convergence of image error towards zero are established simultaneously. Besides rigorous proof in theory, the obtained results are also confirmed by comparative simulation tests.

Constrained Image-Based Visual Servoing of Robot Manipulator with Third-Order Sliding-Mode Observer

A new image-based robot visual servo control strategy based on a third-order sliding-mode observer (TOSM) model predictive control is proposed in this study. This new control strategy solves the problem of robot visual servo control with system constraints and time-varying disturbances when the camera and model of the robot manipulator are uncertain and the joint velocity is unknown. In the proposed method, the joint velocity and system centralized uncertainties are estimated simultaneously based on a third-order sliding-mode observer, and the image-based visual servoing problem is transformed into a nonlinear optimization problem based on a model predictive control method considering both visibility constraints and actuator constraints, which minimizes the predicted trajectory cost function to generate the control signal for each cycle. Simulations were carried out to verify the effectiveness of the proposed control scheme.

Adaptive neural network control for image‐based visual servoing of robot manipulators

This paper presents a novel adaptive neural network control strategy for image-based visual servoing (IBVS) of robotic manipulators with both eye-in-hand and eye-to-hand camera configurations in the presence of unknown dynamics and external disturbances. The IBVS method is combined with the adaptive neural network to construct the proposed adaptive neural network controller to solve the visual servoing control problem of robots. The adaptive neural network based IBVS controller is designed based on the depth-independent interaction matrix, which can be trained on-line to identify the visual servoing robotic system modeling errors. Moreover, the proposed method can approach the unknown nonlinear dynamics for both eye-in-hand and eye-to-hand camera configurations without requiring the robot dynamics to be linearly parameterizable, and the exact knowledge of the robot structure is not needed. On the basis of the nonlinear robot dynamics, the Lyapunov stability analysis is given to prove the asymptotical convergence of the image position and velocity errors. Simulation results for both camera configurations are provided to demonstrate the performance of the proposed adaptive neural network based approach.

Visual servoing of robot manipulator with weak field-of-view constraints

Aiming at the problem of servoing task failure caused by the manipulated object deviating from the camera field-of-view (FOV) during the robot manipulator visual servoing (VS) process, a new VS method based on an improved tracking learning detection (TLD) algorithm is proposed in this article, which allows the manipulated object to deviate from the camera FOV in several continuous frames and maintains the smoothness of the robot manipulator motion during VS. Firstly, to implement the robot manipulator visual object tracking task with strong robustness under the weak FOV constraints, an improved TLD algorithm is proposed. Then, the algorithm is used to extract the image features (object in the camera FOV) or predict image features (object out of the camera FOV) of the manipulated object in the current frame. And then, the position of the manipulated object in the current image is further estimated. Finally, the visual sliding mode control law is designed according to the image feature errors to control the motion of the robot manipulator so as to complete the visual tracking task of the robot manipulator to the manipulated object in complex natural scenes with high robustness. Several robot manipulator VS experiments were conducted on a six-degrees-of-freedom MOTOMANSV3 industrial manipulator under different natural scenes. The experimental results show that the proposed robot manipulator VS method can relax the FOV constraint requirements on real-time visibility of manipulated object and effectively solve the problem of servoing task failure caused by the object deviating from the camera FOV during the VS.

A Gradient-Based Recurrent Neural Network for Visual Servoing of Robot Manipulators with Acceleration Command

Recent decades have witnessed the rapid evolution of robotic applications and their expansion into a variety of spheres with remarkable achievements. This article researches a crucial technique of robot manipulators referred to as visual servoing, which relies on the visual feedback to respond to the external information. In this regard, the visual servoing issue is tactfully transformed into a quadratic programming problem with equality and inequality constraints. Differing from the traditional methods, a gradient-based recurrent neural network (GRNN) for solving the visual servoing issue is newly proposed in this article in the light of the gradient descent method. Then, the stability proof is presented in theory with the pixel error convergent exponentially to zero. Specifically speaking, the proposed method is able to impel the manipulator to approach the desired static point while maintaining physical constraints considered. After that, the feasibility and superiority of the proposed GRNN are verified by simulative experiments. Significantly, the proposed visual servo method can be leveraged to medical robots and rehabilitation robots to further assist doctors in treating patients remotely.

Self-Learning Visual Servoing of Robot Manipulator Using Explanation-Based Fuzzy Neural Networks and Q-Learning

A new self-learning visual servoing system for the robot manipulators is proposed. This system includes two main properties: on-line self training and lifelong learning that are implemented by the Q-Learning algorithm and Explanation-based Fuzzy Neural Networks (EBFNN) respectively. We demonstrate that the number of training samples and the training time for a specific robot positioning accuracy can be reduced using explanation-based fuzzy neural networks and the Q-Learning algorithm. The system uses Q-learning to find the optimal policy in conjunction with the reinforcement learning. This policy is used by a robot to reach an object that has been randomly placed in a static workspace. Background knowledge about the robot and its environment is transferred to the robot agent during the learning process using a set of previously trained neural networks. This system learns the optimal policy in order to select the best action that maximizes the cumulative reward received at each time step. This learning approach does not use either a robot or camera model, or require calibration. Simulation results prove the effectiveness of this methodology to improve the learning process and the performance of the self-learning visual servoing system.

Predictive control architecture for real-time image moments based servoing of robot manipulators

This paper presents a real-time architecture for visual servoing of robot manipulators using nonlinear based predictive control. In order to increase the robustness of the control algorithm, image ...

Fuzzy vs Nonfuzzy in 2D Visual Servoing for Robot Manipulators

In this paper we consider the tracking control problem of planar robots manipulators via visual servoing in the presence of parametric uncertainties associated with the robot dynamics and the camera parameters. An image-based visual servo control is developed using a Takagi-Sugeno (T-S) fuzzy model. The design includes an image coordinate velocity observer. To the best knowledge of the authors such a proposal remains hitherto unpublished. The new scheme is compared experimentally with two different non-fuzzy algorithms showing the good performance and advantages of the complete system.

Adaptive neural controller for visual servoing of robot manipulators with camera-in-hand configuration

In this paper, an adaptive neural controller is proposed for visual servoing of robot manipulators with camera-in-hand configuration. The controller is designed as a combination of a PI kinematic controller and feedforward neural network controller that computes the required torque signals to achieve the tracking. The visual information is provided using the camera mounted on the end-effector and the defined error between the actual image and desired image positions is fed to the PI controller that computes the joint velocity inputs needed to drive errors in the image plane to zero. Then the feedforward neural network controller is designed such that the robot’s joint velocities converges to the given velocity inputs. The stability of combined PI kinematic and feedforward neural network computed torque is proved by Lyapunov theory. It is shown that the neural network can cope with the unknown nonlinearities through the adaptive learning process and requires no preliminary off learning. Simulation results are carried out for a three degrees of freedom microbot robot manipulator to evaluate the controller performance.

Predictive control architecture for real-time image moments based servoing of robot manipulators

Abstract This paper presents a real-time architecture for visual servoing of robot manipulators using nonlinear based predictive control. A 6 d.o.f robot manipulator with an eye-in-configuration is used to design the visual servoing architecture and a real-time implementation using Matlab is done. The results of different experiments conducted with two types of image moments based controllers (proportional and predictive with reference trajectory) are presented and discussed.

Predictive Control Architecture for Visual Servoing of Robot Manipulators

AbstractA novel architecture for integrating reference trajectory and image prediction is proposed to be used in predictive control of visual servoing systems. In the proposed method, a new predictor is developed based on the relation between the camera velocity and the time variation of the visual features given by the interaction matrix. In addition, a reference trajectory is introduced to define the way how to reach the desired features over the prediction horizon starting from the current features. Simulations reveal the efficiency of the proposed architecture to control a 6 degrees of freedom robot manipulator.

A new fuzzy visual servoing with application to robot manipulator

Many stereo vision algorithms require that the images which are very similar in appearance, and the distance between the two cameras should be small enough. Normal visual servoing of robot manipulator needs both image information and joint velocities. In this paper we make two modifications to overcome these problems: 1) a new simple stereo vision model is proposed; 2) fuzzy controller based on only visual information is implemented to a robot manipulator. We successfully apply our new algorithms to real-time visual servoing of an industrial robot. The stereo model proposed in this paper can also be extended to other applications such as, mobile robots, 3D reconstruction, etc.

Adaptive Visual Servoing of Robot Manipulators without Measuring the Image Velocity

The direct adaptive control of planar robot manipulators through visual servoing is considered. A solution is developed for image-based visual systems to allow tracking of a desired trajectory, when both camera calibration and robot dynamics are uncertain. In order to solve the MIMO parameter adaptive problem without using image velocity information, the adaptive camera calibration is formulated as a relative degree two MIMO adaptive control problem. A recently proposed Lyapunov/passivity-based adaptive control for relative degree two MIMO system based on SDU factorization is applied. The resulting adaptive calibration is then combined with an adaptive motion controller for the manipulator, which takes into account its uncertain nonlinear dynamics. The overall stability of the adaptive visual servoing system can be proved thanks to the explicit Lyapunov-like functions of both adaptation schemes.

Global Path-Planning for Constrained and Optimal Visual Servoing

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> Visual servoing consists of steering a robot from an initial to a desired location by exploiting the information provided by visual sensors. This paper deals with the problem of realizing visual servoing for robot manipulators taking into account constraints such as visibility, workspace (that is obstacle avoidance), and joint constraints, while minimizing a cost function such as spanned image area, trajectory length, and curvature. To solve this problem, a new path-planning scheme is proposed. First, a robust object reconstruction is computed from visual measurements which allows one to obtain feasible image trajectories. Second, the rotation path is parameterized through an extension of the Euler parameters that yields an equivalent expression of the rotation matrix as a quadratic function of unconstrained variables, hence, largely simplifying standard parameterizations which involve transcendental functions. Then, polynomials of arbitrary degree are used to complete the parametrization and formulate the desired constraints and costs as a general optimization problem. The optimal trajectory is followed by tracking the image trajectory with an IBVS controller combined with repulsive potential fields in order to fulfill the constraints in real conditions. </para>

Lyapunov/Passivity-Based Adaptive Control of Relative Degree Two MIMO Systems With an Application to Visual Servoing

This note presents a Lyapunov-based design of model-reference adaptive control (MRAC) for multiple-input-multiple-output (MIMO) systems of uniform relative degree two. It consists of an extension of a well known MRAC Lyapunov-based scheme for single-input-single-output (SISO) plants with relative degree one or two. The case of relative degree one, was completely extended to the MIMO case only recently. The corresponding extension of the relative degree two case remained hitherto unpublished. While MIMO MRAC schemes exist which can deal with arbitrary relative degree, the one presented here has the significant advantage of possessing an explicit Lyapunov function which easily leads to a stability and error convergence proof. It can also be regarded as a passivity-based approach. As a result, it becomes easier to guarantee stability when connecting such adaptive system to other adaptive control systems with similar passivity properties, e.g., in adaptive visual servoing of robot manipulators

Robust Visual Servoing of Robot Manipulators with Neuro Compensation

This paper considers the problem of visual servoing of planar robot manipulators in the presence of uncertainty associated with robot dynamics, camera system and Jacobian matrix. By using radial basis function neural networks, these uncertainties can be compensated effectively. Two kinds of robust visual servoing are proposed, one is for image uncertainties, another is for Jacobian matrix uncertainty. By Lyapunov method and input-to-state stability technique, we prove that these robust controllers with neural compensators are stable. Real-time experiments are presented to show the applicability of the approach presented in this paper.

マルチレート制御とむだ時間補償に基づくビジュアルサーボ

This paper presents a visual servoing of robot manipulator based on intersample disturbance rejection with switching scheme and dead-time compensation. First, multirate intersample disturbance rejection algorithm is reviewed, which was proposed by authors for general digital control system with restricted sampling frequency. Second, the novel feedforward scheme is proposed with open-loop estimation and switching function, which enables the disturbance rejection without any sacrifice of the closed-loop characteristics. Third, new precise formulation of delay problems in visual servoing is established as the image processing latency, the difference between sampling period of camera signal and control period of joint servo system, and delay of inner-loop joint servo system. By introducing the workspace controller and nonlinear perspective transformation, the proposed intersample disturbance rejection controller becomes applicable to the visual servoing problem of 2 DOF manipulator with moving object points. Finally, the advantages of the proposed control system are verified through simulations and experiments.