Feature-based Visual Servoing Research Articles

3D Feature-Based Sampled-Data Visual Tracking*

3D feature-based Visual Servoing (VS) on the one hand shows attractive peculiarities, on the other hand it suffers from drawbacks related to the existence of local minima, which may affect the convergence character of the VS control loop. Furthermore, the performance of the visual tracking module may constitute a bottleneck enforcing severe constraints on the workspace and visual task execution speed. In this paper we introduce a novel sampled-data model of the 3D feature-based VS, and, in order to avoid drawbacks due to local minima, we plan the target reference trajectory in the feature space with the aim to constraint the feature error dynamics to remain close to the desired equilibrium point. Then, we propose a novel feature generation based on the homography provided by a template matching algorithm based on the Zero mean Normalized Cross Correlation (ZNCC) and the design of a visual tracking scheme by resorting to the Extended Kalman Filter (EKF) and Lyapunov direct method, which explicitly takes into account the camera velocity limits, while ensuring stability.

Stability and Convergence Analysis of 3D Feature-Based Visual Servoing

Visual control based on image features has received much attention for its inherent robustness against camera calibration errors, modelling uncertainties and the capability to keep the object in the Field of View (FoV) of the camera. Nevertheless, some drawbacks related to the basin of convergence and the existence of local minima, which make the camera to get stuck in undesired equilibrium points, are still worth being investigated. Nowadays, the availability of cheap and lightweight RGB-D cameras makes the use of three-dimensional features natural. By using an RGB-D camera in an eye-in-hand configuration, this letter proposes an in-depth stability and convergence analysis of 3D feature-based visual servoing. It will be proved that the visual control system is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">almost globally asymptotically stable</i> in the sense that the only trajectories not converging to the desired equilibrium point are those belonging to a zero Lebesgue measure set in the feature space. Moreover, a sufficient condition guaranteeing that the feature trajectories remain in the camera FoV is derived and an algorithm to prevent feature loss caused by violation of the camera FoV constraint is proposed.

Towards Fully Autonomous Ultrasound Scanning Robot With Imitation Learning Based on Clinical Protocols

Ultrasound scanning plays an important role in modern clinical examinations. Thanks to its small footprint, low cost, and popularity, it has been widely used in annual physical examinations and many other diagnosis and intervention procedures. However, the scanning results depend heavily on the clinician operators' skills, causing inconsistency and even false detection. A fully autonomous ultrasound scanning robot can be a promising solution, enabling consistent and accurate operation. In this letter, we propose a procedure-specified imitation learning framework based on clinical protocols to implement autonomous scanning tasks. Based on the targeted carotid artery examination procedure, we design a feature-based visual servo controller for both in-plane and out-of-plane feature tasks. A one-step-exploring (OSE) method is also designed to improve the robustness and transition smoothness in between tasks. The proposed methods are tested and validated with experiments on both medical phantom and human subjects. The results show that the proposed appraoch is able to significantly improve the success rate of procedure completion from 22.2% to 84.6% for the out-of-plane feature-based operation.

Natural Feature-based Visual Servoing for Grasping Target with an Aerial Manipulator

Aerial transportation and manipulation have attracted increasing attention in the unmanned aerial vehicle field, and visual servoing methodology is widely used to achieve the autonomous aerial grasping of a target object. However, the existing marker-based solutions pose a challenge to the practical application of target grasping owing to the difficulty in attaching markers on targets. To address this problem, this study proposes a novel image-based visual servoing controller based on natural features instead of artificial markers. The natural features are extracted from the target images and further processed to provide servoing feature points. A six degree-of-freedom (6-DoF) aerial manipulator system is proposed with differential kinematics deduced to achieve aerial grasping. Furthermore, a controller is designed when the target object is outside a manipulator’s workspace by utilizing both the degrees-of-freedom of unmanned aerial vehicle and manipulator joints. Thereafter, a weight matrix is used as basis to develop a multi-tasking visual servoing framework to integrate the controllers inside and outside the manipulator’s workspace. Lastly, experimental results are provided to verify the effectiveness of the proposed approach.

Direct Visual Servoing Using Wavelet Coefficients

This paper focuses on the design of a 6 degrees of freedom visual servoing control law. Instead of using geometric visual features in standard vision-based approaches, the proposed controller makes use of wavelet coefficients as control signal inputs. It uses the multiple resolution coefficients representing the wavelet transform of an image in the spatial domain. The main contributions are the definition of the multiple resolution wavelet interaction model that links the time-variation of wavelet coefficients to the robot spatial velocity and the associated task function controller. The proposed control law was tested and validated experimentally using a commercial micromanipulator in an eye-to-hand configuration. To be able to judge the efficiency of the control law, several validation tests were carried out under different conditions of use i.e., large illumination variations, noisy images, partial occlusions, and using unknown three-dimensional scenes. It is also demonstrated experimentally that the proposed approach outperforms, the well-known photometry visual servoing as well as a feature-based visual servoing, namely in unfavorable conditions of use.

Adaptive Visual Servoing of Contour Features

In vision-based robotic manipulation, it is usually required that the object should present a desired shape or be viewed at a specific angle to facilitate subsequent operations such as object crawling and component inspection. To implement this kind of visual servoing tasks, image features that describe shape information of objects need to be designed. This paper proposes methods based on the general contour of an object. The Bezier curve and the nonuniform rational B-spline curve are, respectively, used to fit the contour of the object. Based on contour curve parameters, image contour features are designed. Visual measurements of the image features are used to provide feedback on the shape information of the object to the eye-in-hand visual servoing system. Based on the visual feedback, the motion of the robot is controlled to make the image features converge to the desired values. In this way, it is implemented that the object presents the desired shape or is viewed at the desired angle. Moreover, by using adaptive laws to estimate curve parameters online, the proposed method does not require a priori knowledge of the 3-D position of the object. The stability of the proposed controller is analyzed by Lyapunov theory. This method has substantially expanded the application domain of not only the image feature-based visual servoing, but also the B-spline curves. The feasibility of the proposed method is validated by experiments.

Feature-based visual servo control preserving wide-angle properties of super wide-angle lens

Feature-based visual servo control methods determine camera motion by comparing a current feature image and the goal feature image. Because these methods do not require 3D information for the object, no geometric model of the object is needed, and it is therefore robust against various disturbances. Several feature-based visual servo control methods have been proposed, such as using a wide-angle lens allows the camera to capture the object in a wide angular range. However, when using a wide-angle lens with a conventional control scheme the camera undergoes wasted motion due to the peripheral distortion of wide-angle lenses. In addition, correction of the image distortion results in the loss of the wide-angle properties of the lens. This paper proposes a new feature-based visual servo control method based on a super wide-angle lens that preserves wide-angle properties.

Feature-based visual servo control preserving wide-angle properties of super wide-angle lens

Feature-based visual servo control preserving wide-angle properties of super wide-angle lens

21105 超広角レンズの広角特性を維持した特徴ベーストビジュアルサーボ(OS10-4【ロボティクス・メカトロニクス(4)】)

Abstract: Feature-based visual servo determines camera movement by comparing a current feature image and the goal feature image. This method does not require a geometric model of the object because it does not need a 3D information of the object. Therefore, it is robust to various disturbances. Various methods have been proposed. For example using a wide-angle lens enables the camera to capture the object in a wide range However, using a wide-angle lens in the conventional control law, the camera would conduct waste motion because of the peripheral distortion of the wide-angle lens. In addition, if the distortion is corrected, the wide-angle properties will be lost. This paper proposes a new Feature-based visual servoing control law by preserving wide-angle properties of super wide-angle lens.

特徴ベースト視覚サーボにおける首ふり・後退量のファジィ決定

One drawback of the traditional feature-based visual servoing is that undesired set back motion occurs if the rotation around the optical axis is required to move to the goal position and orientation. Another drawback is that some feature points may go off the screen in the case that goal positions for the feature points are near the edges of the screen, unless the gain coefficients are adjusted appropriately. To overcome these drawbacks, this paper introduces a fuzzy control to the feature based visual servoing that uses a middle image made from a current image and the goal image. Input variables to the fuzzy system are: distance to the current and goal position of the feature points from the center of the screen, the distance between the current and goal position of the feature points, rotational angle required to reach the goal position of the feature points, and the distance of the mean position of the current feature points from the center of the screen. An output from the fuzzy system is a heading or setting back motion that prevent the feature points from going off the screen and make the feature points approach the goal position efficiently. Effectiveness of the proposed method is demonstrated experimentally, compared to the other two traditional methods.

Feature-Based Visual Servo Using Middle Images

Feature-based visual servo determines camera movement by comparing a current feature image and the goal feature image. If optical axis rotation is required to move to the goal position and orientation, the traditional feature-based visual servo makes the camera set back. This set back motion is unnecessary. In addition, if the optical axis rotation approximately amounts to π, the camera cannot reach the goal position because of the set back motion. This paper proposes a new feature-based visual servoing which uses a middle image mabe from a current image and the goal image to improve the optical axis rotation. The validity of the proposed method is verified through numerical simulations.

2P2-C13 ヘリコプタの特徴ベースビジュアルサーボ

2P2-C13 ヘリコプタの特徴ベースビジュアルサーボ

A unified approach to visual tracking and servoing

In this paper, we present a generic and flexible system for vision-based robot control. The system integrates visual tracking and visual servoing approaches in a unifying framework. In this framework, the generality is obtained using a template matching algorithm based on an efficient second-order minimization. Contrarily to feature-based visual servoing schemes, we avoid the design of feature-dependent visual tracking algorithms. By integrating the visual tracking process with the visual servoing techniques, we can easily deal with constrained tasks. This reduces the computation cost and improves the precision of the system. The experimental results prove the efficiency of the unified system in real conditions.

20722 中間画像を利用した特徴ベースト法((III),OS12 インテリジェントシステムと制御)

Feature-based visual servo determines camera movement by comparing a current feature image and the goal feature image. If optical axis rotation for moving to the goal image is required, the traditional feature-based visual servo makes the camera set back. This set back motion is unnecessary. In addition, if the optical axis rotation amounts near π, the camera can't reach the goal position because of the set back motion. In this paper, to improve the optical axis rotation, a new feature-based visual servoing method is proposed, which uses a middle image made from a current image and the goal image. The validity of the proposed method is verified through numerical simulations.

Vision Based Simulation and Experiment for Performance Tests of Robot

The feature-based visual servoing approach has been used to control robot through vision. In order to find the position of the end effector by vision and through robot performance tests, computational kinematic approach has been used. The software carries out the duty of environment simulation and operation of an industrial robot. The disputes related to image capturing, image processing, target recognition, and how to control robot by vision system ability have been carried out in the simulation tests. The vision based program has been defined in such a way that it can be carried out by a real robot with the least changes.

回転運動対象物に対する予測補正特徴ベースト法(ビジュアルサーボ)

回転運動対象物に対する予測補正特徴ベースト法(ビジュアルサーボ)

A study on feature-based visual servoing control of robot system by utilizing redundant feature

This paper presents how effective it is to use many features for improving the speed and accuracy of visual servo systems. Some rank conditions which relate the image Jacobian to the control performance are derived. The focus is to describe that the accuracy of the camera position control in the world coordinate system is increased by utilizing redundant features in this paper. It is also proven that the accuracy is improved by increasing the number of features involved. Effectiveness of the redundant features is evaluated by the smallest singular value of the image Jacobian which is closely related to the accuracy with respect to the world coordinate system. Usefulness of the redundant features is verified by the real time experiments on a Dual Arm Robot manipulator made by Samsung Electronic Co. Ltd..

Application of feature-based multi-view servoing for lamp filament alignment

This application of feature-based visual servoing was designed to achieve accurate and robust three-dimensional filament alignment sufficient to meet the exacting photometric specifications imposed by the US Federal Aviation Authority on certain lighting applications. The industrial setting dictated a simple and intuitive uncoupled control approach which minimizes the use of embedded kinematic or dynamic models or the need for calibration.

冗長な特徴量に基づく視覚サーボ

This paper presents how the control performance of the feature-based visual servo system is improved by utilizing the redundant features. It also proves the effectiveness of the redundant features to increase the accuracy of the visual servo system by using the smallest singular value of a matrix called image Jacobian. Real time experiments on a PUMA 560 manipulator show usefulness of the redundant features to improve the performance.

Visual servoing with hand-eye manipulator-optimal control approach

This paper proposes a control theoretic formulation and a controller design method for the feature-based visual servoing with redundant features. The linear time-invariant (LTI) formulation copes with the redundant features and provides a simple framework for controller design. The proposed linear quadratic (LQ) method can deal with the redundant features, which is important because the previous LQ methods are not applicable to redundant systems. Moreover, this LQ method gives flexibility for performance improvement instead of the very limited design parameters provided by the generalized inverse and task function controllers. Validity of the LTI model and effectiveness and flexibility of the LQ optimal controller are evaluated by real-time experiments on a PUMA 560 manipulator.