Position-based Visual Servoing Research Articles

Particle-filter-based Pose Estimation from Controlled Motion with Application to Visual Servoing

In this paper, we present a Bayesian algorithm based on particle filters to estimate the camera pose for vision-based control. The state model is represented as a relative camera pose between the current and initial camera frames. The particles in the prior motion model are drawn using the velocity control signal collected from the visual controller of the robot. The pose samples are evaluated using an epipolar geometry measurement model and a suitable weight is associated with each sample. The algorithm takes advantage of the a priori knowledge about motion, i.e., the velocity computed by the visual servo control, to estimate the magnitude of the translation in addition to its direction, hence producing a full camera motion estimate. Its application to position-based visual servoing is demonstrated. Experiments are carried out using a real robot setup. The results show the efficiency of the proposed filter over the motion measurements of the robot. In addition, the filter was able to recover the split performed by the robot joints.

A practical position-based visual servo design and implementation for automated fault insertion test

This paper presents the design and implementation of a practical visual servo. The visual servo has been designed for pose correction of an end-effector that is placed wrongly away from the test point on a printed circuit board (PCB). It is widely acknowledged that the error could be attributed to the connecting joints of robot arms, angular errors, geometric model, and/or camera projection model. In the automated fault insertion test (AFIT), the typical difficulty encountered by a robot is to servo and place the probe tip accurately on the targeted test point, i.e., the conductive pad on the PCB. Conventionally, touch and sense with a probe tip has been utilized. Upon detection of a failure, the visual servo is triggered and the robot literally looks through a camera and re-attempts with the use of visual information from the camera. Currently, no clearly defined specifications in carrying out feature extraction exist and thereby the test point detection as it is not cost-effective to designate part of PCB footprint as the test point separately from the main design to accommodate visual sensing. As a result, various kinds of test points have been implanted into PCB industrial design. This research work requires building of a custom knowledge base of the test point features to support image recognition. Furthermore, the operational factors in industrial manufacturing, e.g., head structure maintenance, the replacement of end-effectors and the changes of projection parameters caused by hardware adjustment, impact the accuracy of the probe placement. These factors cause the original geometric model to shift from the original configuration and thus errors. This research paper proposes the practical design of a closed-looped visual servo to address the issues of precision error, image feature extraction, and manufacturing factors. Besides, the paper details the findings from the design phase to the implementation phase.

Near-Minimum Time Visual Servoing of an Underactuated Robotic Manipulator

Rate of convergence to the desired pose to grasp an object using visual information may be important in some applications, such as a pick and place routine in assembly where the time between two stops of the conveyor is very short. The visually guided robot is required to move fast if vision is to bring the sought benefits to industrial setups. In this paper, the three most famous techniques to visual servoing, mainly the image-based, position-based and hybrid visual servoing are evaluated in terms of their speed of convergence to the grasping pose in a pick and place task of a momentarily motionless target. An alternative open-loop near-minimum time approach is also presented and tested on a 5DOF under-actuated robotic arm. The performance is compared and result shows significant reduction for its time of convergence, to the aforementioned techniques.

Developments in Visual Servoing for Mobile Manipulation

A new trend in mobile robotics is to integrate visual information in feedback control for facilitating autonomous grasping and manipulation. The result is a visual servo system, which is quite beneficial in autonomous mobile manipulation. In view of mobility, it has wider application than the traditional visual servoing in manipulators with fixed base. In this paper, the state of art of vision-guided robotic applications is presented along with the associated hardware. Next, two classical approaches of visual servoing: image-based visual servoing (IBVS) and position-based visual servoing (PBVS) are reviewed; and their advantages and drawbacks in applying to a mobile manipulation system are discussed. A general concept of modeling a visual servo system is demonstrated. Some challenges in developing visual servo systems are discussed. Finally, a practical application of mobile manipulation system which is developed for applications of search and rescue and homecare robotics is introduced.

Particle Filtering for Industrial 6DOF Visual Servoing

Visual servoing allows the introduction of robotic manipulation in dynamic and uncontrolled environments. This paper presents a position-based visual servoing algorithm using particle filtering. The objective is the grasping of objects using the 6 degrees of freedom of the robot manipulator in non-automated industrial environments using monocular vision. A particle filter has been added to the position-based visual servoing algorithm to deal with the different noise sources of those industrial environments. Experiments performed in the real industrial scenario of ROBOFOOT ( http://www.robofoot.eu/ ) project showed accurate grasping and high level of stability in the visual servoing process.

자율주행 모바일 역진자의 비주얼서보잉에 대한 연구

This paper proposes an optimal three-dimensional coordinate implementation of the vision sensor using two CCD cameras. The PBVS (Position based visual servoing) is implemented using the positional information obtained from images. Stereo vision by PBVS method that has enhanced every frame using calibration parameters is effective in the distance calculation. The IBVS (Image based visual servoing) is also implemented using the difference between reference and obtained images. Stereo vision by IBVS method calculates the distance using rotation angle of motors that correspond eyes and neck without enhanced images. The PBVS method is compared with the IBVS method in terms of advantages, disadvantages, computing time, and performances. Finally, the IBVS method is applied for the dual arm manipulator on the mobile inverted pendulum. The autonomous mobile inverted pendulum is successfully demonstrated using the center of the manipulator's mass.

Novel Position-Based Visual Servoing Approach to Robust Global Stability Under Field-of-View Constraint

In this paper, we shed new light on position-based visual servoing (VS) by introducing the concept of a 3-D visible set for position-based VS (PBVS), which can serve to play exactly the role of the 2-D visible set, the image plane, for image-based VS. Our 3-D visible set is convex even in the presence of uncertainties in intrinsic and extrinsic parameters of the camera, just as the image plane. Then, we can ensure the field of view (FOV) constraint simply by controlling the camera pose to follow a smooth straight-line trajectory in our 3-D visible set for PBVS, which simply connects two points determined by the initial and desired poses of the camera. In fact, this is, to our best knowledge, the first result in PBVS that has guaranteed theoretically robust global stability under the FOV constraint with respect to uncertainties in intrinsic and extrinsic parameters of the camera as well as in the distance between the camera and the object at the desired pose. Furthermore, our PBVS scheme is computationally simple compared to the prior works. To demonstrate further the validity and practicality of our PBVS scheme, we also present various experimental results using a six-degree-of-freedom robotic manipulator.

Motion-Estimation-Based Visual Servoing of Nonholonomic Mobile Robots

A 2-1/2-D visual servoing strategy, which is based on a novel motion-estimation technique, is presented for the stabilization of a nonholonomic mobile robot (which is also called the “parking problem”). By taking into account the planar motion constraint of mobile robots, the proposed motion-estimation technique can be applied in both planar and nonplanar scenes. In addition, this approach requires no matrix estimation or decomposition, and it avoids ambiguity and degeneracy problems for the homography or fundamental matrix-based algorithms. Moreover, the field-of-view (FOV) constraint of the onboard camera is largely alleviated because the presented algorithm works well with few feature points. In order to incorporate the advantages of position-based visual servoing and image-based visual servoing, a composite error vector is defined that includes both image signals and the estimated rotational angle. Subsequently, a smooth time-varying feedback controller is adopted to cope with the nonholonomic constraints, which yields global exponential convergent rate for the closed-loop system. On the basis of the perturbed linear system theory, we show that practical exponential stability can be achieved, despite the lack of depth information, which is inherent for monocular camera systems. Both simulation and experiment results are collected to investigate the feasibility of the proposed approach.

혼합 비주얼 서보 제어 기법을 이용한 이동로봇의 목표물 추종

This paper proposes a target tracking algorithm for wheeled mobile robots using in various fields. For the stable tracking, we apply a vision system to a mobile robot which can extract targets through image processing algorithms. Furthermore, this paper presents an algorithm to position the mobile robot at the desired location from the target by estimating its relative position and attitude. We show the problem in the tracking method using the Position-Based Visual Servo(PBVS) control, and propose a tracking method, which can achieve the stable tracking performance by combining the PBVS control with Image-Based Visual Servo(IBVS) control. When the target is located around the outskirt of the camera image, the target can disappear from the field of view. Thus the proposed algorithm combines the control inputs with of the hyperbolic form the switching function to solve this problem. Through both simulations and experiments for the mobile robot we have confirmed that the proposed visual servo control method is able to enhance the stability compared to of the method using only either PBVS or IBVS control method.

Kinematics, pattern recognition, and motion control of mask–panel alignment system

Kinematics, pattern recognition, and motion control methods are investigated as an integrated approach for a mask–panel visual alignment system, which consists of the vision system to extract the mask–panel misalignment and the stage control system to compensate for it. First, a 4PPR alignment mechanism is considered and the inverse kinematic solution is found out to define the relationship between the mask–panel misalignment and the displacements of active joints. Then, a fast alignment mark recognition algorithm is proposed in terms of the geometric template matching (GTM), which is specific to simply shaped patterns but computationally much efficient compared with general correlation-based matchings. Finally, the kinematic solution and the developed vision algorithm are incorporated to implement a two-stage position-based visual servo, where both original fine images and reduced coarse ones are utilized together and the GTM plays a crucial role in achieving a near real-time visual feedback. Experimental results are shown to demonstrate the effectiveness of the GTM-based two-stage alignment control.

An improved self-calibration approach based on adaptive genetic algorithm for position-based visual servo

An improved self-calibrating algorithm for visual servo based on adaptive genetic algorithm is proposed in this paper. Our approach introduces an extension of Mendonca-Cipolla and G. Chesi’s self-calibration for the position-based visual servo technique which exploits the singular value property of the essential matrix. Specifically, a suitable dynamic online cost function is generated according to the property of the three singular values. The visual servo process is carried out simultaneous to the dynamic self-calibration, and then the cost function is minimized using the adaptive genetic algorithm instead of the gradient descent method in G. Chesi’s approach. Moreover, this method overcomes the limitation that the initial parameters must be selected close to the true value, which is not constant in many cases. It is not necessary to know exactly the camera intrinsic parameters when using our approach, instead, coarse coding bounds of the five parameters are enough for the algorithm, which can be done once and for all off-line. Besides, this algorithm does not require knowledge of the 3D model of the object. Simulation experiments are carried out and the results demonstrate that the proposed approach provides a fast convergence speed and robustness against unpredictable perturbations of camera parameters, and it is an effective and efficient visual servo algorithm.

An Improved Self-calibration Approach Based on Enhanced Mutative Scale Chaos Optimization Algorithm for Position-based Visual Servo

An Improved Self-calibration Approach Based on Enhanced Mutative Scale Chaos Optimization Algorithm for Position-based Visual Servo

Simulation and experiments for a vision-based control of a 6R robot

In this paper, we present both the simulation and experimental results for position-based visual servoing control of a 6R robot using two stationary cameras . This method can deal with real-time changes in the relative position of the target with respect to the robot, while providing greater accuracy. In addition, the servo control structure is independent of the target pose coordinates. Direct and inverse kinematics equations, in addition to dynamic equations of the 6R robot, have been derived and simulated. Then we prescribe simulation of image processing, object recognition and pose estimation for the end effector and target-object in 3D Cartesian space and visual control of the robot. Afterwards, performance tests of 6R robot using two cameras have been implemented. Finally, experimental results obtained from actual implementation of visual control and tests of 6R robot in lab are presented. Analysis of error and test data has been carried out according to ISO9283, ANSI-RIA R15.05–2 standards and the MATLAB statistical toolbox. An efficient algorithm is designed for collision detection and contact determination between the 6R robot links and its environment during their rotation in controlling and performance tests. In this technique, bounding spheres on different parts of 6R robot are used to detect collisions among them.

Robotic execution of everyday tasks by means of external vision/force control

In this article, we present an integrated manipulation framework for a service robot, that allows to interact with articulated objects at home environments through the coupling of vision and force modalities. We consider a robot which is observing simultaneously his hand and the object to manipulate, by using an external camera (i.e. robot head). Task-oriented grasping algorithms (Proc of IEEE Int Conf on robotics and automation, pp 1794–1799, 2007) are used in order to plan a suitable grasp on the object according to the task to perform. A new vision/force coupling approach (Int Conf on advanced robotics, 2007), based on external control, is used in order to, first, guide the robot hand towards the grasp position and, second, perform the task taking into account external forces. The coupling between these two complementary sensor modalities provides the robot with robustness against uncertainties in models and positioning. A position-based visual servoing control law has been designed in order to continuously align the robot hand with respect to the object that is being manipulated, independently of camera position. This allows to freely move the camera while the task is being executed and makes this approach amenable to be integrated in current humanoid robots without the need of hand-eye calibration. Experimental results on a real robot interacting with different kind of doors are presented.

이동로봇의 자동충전을 위한 영상기반 비쥬얼 서보잉 방법

This study deals with image-based visual servoing for automatic recharging of mobile robot. Because mobile robot must be recharged periodically, it is necessary to detect and move to docking station. Generally, laser scanner is used for detect of position of docking station. CCD Camera is also used for this purpose. In case of using cameras, the position-based visual servoing method is widely used. But position-based visual servoing method requires the accurate calibration and it is hard and complex work. Another method using cameras is image-based visual servoing. Recently, image based visual servoing is widely used for robotic application. But it has a problem that cannot have linear trajectory in the 3-dimensional space. Because of this weak point, image-based visual servoing has a limit for real application. In case of 2-dimensional movement on the plane, it has also similar problem. In order to solve this problem, we point out the main reason of the problem of the resolved rate control method that has been generally used in the image-based visual servoing and we propose an image-based visual servoing method that can reduce the curved trajectory of mobile robot in the cartesian space.

Position-Based Visual Servoing in Industrial Multirobot Cells Using a Hybrid Camera Configuration

This paper deals with the problem of position-based visual servoing in a multiarm robotic cell equipped with a hybrid eye-in-hand/eye-to-hand multicamera system. The proposed approach is based on the real-time estimation of the pose of a target object by using the extended Kalman filter. The data provided by all the cameras are selected by a suitable algorithm on the basis of the prediction of the object self-occlusions, as well as of the mutual occlusions caused by the robot links and tools. Only an optimal subset of image features is considered for feature extraction, thus ensuring high estimation accuracy with a computational cost independent of the number of cameras. A salient feature of the paper is the implementation of the proposed approach to the case of a robotic cell composed of two industrial robot manipulators. Two different case studies are presented to test the effectiveness of the hybrid camera configuration and the robustness of the visual servoing algorithm with respect to the occurrence of occlusions

Visual servo control. I. Basic approaches

This paper is the first of a two-part series on the topic of visual servo control using computer vision data in the servo loop to control the motion of a robot. In this paper, we describe the basic techniques that are by now well established in the field. We first give a general overview of the formulation of the visual servo control problem. We then describe the two archetypal visual servo control schemes: image-based and position-based visual servo control. Finally, we discuss performance and stability issues that pertain to these two schemes, motivating the second article in the series, in which we consider advanced techniques

Autonomous Acquisition of Seam Coordinates for Arc Welding Robot Based on Visual Servoing

Autonomous acquisition of seam coordinates is a key technology for developing advanced welding robot. This paper describes a position-based visual servo system for robotic seam tracking, which is able to autonomously acquire the seam coordinates of the planar butt joint in the robot base frame and plan the optimal camera angle before welding. A six-axis industrial robot is used in this system, which has an interface for communicating with the master computer. The developed visual sensor device is briefly presented that allows the charge-coupled device (CCD) cameras to rotate about the torch. A set of robust image processing algorithms are proposed so that no special requirements of light source are needed in this system. The feedback errors of this servo system are defined according to the characteristics of the seam image, and the robust tracking controller is developed. Both the image processing program and tracking control program run on the master computer. The experimental results on straight line seam and curve seam show that the accuracy of the seam coordinates acquired with this method is more adequate for high quality welding process.

POSITION-BASED VISUAL SERVOING IN INDUSTRIAL MULTI-ARM ROBOTIC CELLS USING MULTIPLE CAMERAS

This paper deals with position-based visual servoing in a multi-arm robotic cell equipped with a hybrid eye-in-hand/eye-to-hand multi-camera system. The proposed approach is based on the real-time estimation of the pose of a target object using the extended Kalman filter. The data provided by all the cameras are selected by a suitable algorithm on the basis of the prediction of the object self occlusions and of the mutual occlusions caused by the robot links and tools. Only an optimal subset of image features is considered for feature extraction, thus ensuring high estimation accuracy with a computational cost independent of the number of cameras. An experimental case study is presented for the case of two industrial robots performing a vision-guided grasping task.

Hybrid Motion Control and Planning Strategies for Visual Servoing

This paper presents two hybrid strategies for robot visual servoing. Two specific image constraints, the image singularities and image local minima, are considered in both strategies. The hybrid motion control strategy consists of a local switching control between the image-based and position-based visual servoing for direct avoidance of image singularities and image local minima. The hybrid motion planning strategy consists of an artificial potential field-based global hybrid trajectory planner, where a complete set of Cartesian, image, and robot joint constraints under a complex visual servoing scenario are considered. In this strategy, the image singularities are resolved using the damped-least-square-based joint trajectory planning, while the image local minima are evaluated only along the planned image trajectories and automatically avoided in the image-based trajectory tracking. Two global planning methods are considered. In the first method, the end-effector trajectory is directly planned with respect to the stationary target object frame, which provides a much shorter translational path compared with the local planning method. In the second method, the target trajectory is planned with respect to the current end-effector frame, which minimizes the chances of image trajectories leaving the camera field of view. Simulation and experimental results are given to demonstrate the efficiency of the two hybrid strategies.