Visual Control Law Research Articles

Visual Control Law Tuning Using the JADE Algorithm Applied to Leaf Detection and Cutting

In the propagation of plant tissue in vitro it is necessary to obtain from the leaf a circular section cut. In order to achieve this type of cuts the visual feedback is used, allowing us to analyze the RGB planes to detect the green color and find the image centroid. In order to fulfill the cut objective a PD+ control is used with self-tuning based on the adaptive differential evolutionary algorithm JADE.

Collision Avoidance for Redundant Robots in Position-Based Visual Servoing

To tackle the problem on trajectory planning or the design of control law, this paper introduces a visual servoing system for a manipulator with redundant joints that the trajectory of the manipulator approaching the target is determined spontaneously by the visual control law. The proposed method resolves joint solution for visual servoing and obstacle avoidance. The work comprises of two procedures, feature extraction for position-based visual servoing (PBVS) and collision avoidance within the working envelope. In the PBVS control, the target pose must be reconstructed with respect to the robot and this results in a Cartesian motion-planning problem. Once the geometric relationship between the target and the end effector is determined, a secure inverse kinematics method incorporating trajectory planning is used to solve the solution of the redundant manipulator by the virtual repulsive torque method. Therefore, the links of the manipulator can always maintain a safe distance from obstacles while approaching the target smoothly. The proposed method is verified with its applicability in experiments using an eye-in-hand manipulator with seven joints. For reusability and extensibility, the system has been coded and constructed in the framework of the Robot Operating System so as that the developed algorithms can be disseminated to different platforms.

Autonomous Vision-Based Aerial Grasping for Rotorcraft Unmanned Aerial Vehicles.

Autonomous vision-based aerial grasping is an essential and challenging task for aerial manipulation missions. In this paper, we propose a vision-based aerial grasping system for a Rotorcraft Unmanned Aerial Vehicle (UAV) to grasp a target object. The UAV system is equipped with a monocular camera, a 3-DOF robotic arm with a gripper and a Jetson TK1 computer. Efficient and reliable visual detectors and control laws are crucial for autonomous aerial grasping using limited onboard sensing and computational capabilities. To detect and track the target object in real time, an efficient proposal algorithm is presented to reliably estimate the region of interest (ROI), then a correlation filter-based classifier is developed to track the detected object. Moreover, a support vector regression (SVR)-based grasping position detector is proposed to improve the grasp success rate with high computational efficiency. Using the estimated grasping position and the UAV?Äôs states, novel control laws of the UAV and the robotic arm are proposed to perform aerial grasping. Extensive simulations and outdoor flight experiments have been implemented. The experimental results illustrate that the proposed vision-based aerial grasping system can autonomously and reliably grasp the target object while working entirely onboard.

Robotic assembly of smartphone back shells with eye-in-hand visual servoing

Abstract Research concerning smartphone assembly has attracted increasing attention from the manufacturing industry. This paper presents an automatic back shell assembly system using real-time visual servoing in an eye-in-hand configuration. Look-then-move and look-and-move control approaches are sequentially proposed to handle distal and proximal assembly tasks, respectively. During distal assembly, binocular vision is employed to assist a robotic manipulator to locate and reconstruct the positions and orientations of a cellular phone and its back shell in the work space. After lifting the back shell up and then reaching the top of the cellular phone, there is no binocular field of view. A seemingly novel way of 3-D reconstruction by two uniocular fields of view of the two cameras is proposed. With the pair of monocular vision sensors, two monocular visual servo control laws are employed sequentially to perform the proximal assembly task autonomously. Based on the experimental results, the robotic manipulator can be driven to hold the back shell by a vacuum absorption device and position it precisely onto the smartphone to accomplish the automatic assembly task even if the vision system is only roughly calibrated. The proposed control approach for distal and proximal assembly appears to be efficient, flexible, and reliable for potential applications in industrial manufacturing.

3D ultrasound registration-based visual servoing for neurosurgical navigation.

We present a fully image-based visual servoing framework for neurosurgical navigation and needle guidance. The proposed servo-control scheme allows for compensation of target anatomy movements, maintaining high navigational accuracy over time, and automatic needle guide alignment for accurate manual insertions. Our system comprises a motorized 3D ultrasound (US) transducer mounted on a robotic arm and equipped with a needle guide. It continuously registers US sweeps in real time with a pre-interventional plan based on CT or MR images and annotations. While a visual control law maintains anatomy visibility and alignment of the needle guide, a force controller is employed for acoustic coupling and tissue pressure. We validate the servoing capabilities of our method on a geometric gel phantom and real human anatomy, and the needle targeting accuracy using CT images on a lumbar spine gel phantom under neurosurgery conditions. Despite the varying resolution of the acquired 3D sweeps, we achieved direction-independent positioning errors of [Formula: see text]mm and [Formula: see text], respectively. Our method is capable of compensating movements of around 25mm/s and works reliably on human anatomy with errors of [Formula: see text]mm. In all four manual insertions by an expert surgeon, a needle could be successfully inserted into the facet joint, with an estimated targeting accuracy of [Formula: see text]mm, superior to the gold standard. The experiments demonstrated the feasibility of robotic ultrasound-based navigation and needle guidance for neurosurgical applications such as lumbar spine injections.

CLFs-based optimization control for a class of constrained visual servoing systems

In this paper, we use the control Lyapunov function (CLF) technique to present an optimized visual servo control method for constrained eye-in-hand robot visual servoing systems. With the knowledge of camera intrinsic parameters and depth of target changes, visual servo control laws (i.e. translation speed) with adjustable parameters are derived by image point features and some known CLF of the visual servoing system. The Fibonacci method is employed to online compute the optimal value of those adjustable parameters, which yields an optimized control law to satisfy constraints of the visual servoing system. The Lyapunov's theorem and the properties of CLF are used to establish stability of the constrained visual servoing system in the closed-loop with the optimized control law. One merit of the presented method is that there is no requirement of online calculating the pseudo-inverse of the image Jacobian's matrix and the homography matrix. Simulation and experimental results illustrated the effectiveness of the method proposed here.

Visual coupling to multiple neighbors in a crowd influences walking speed and direction

Pedestrians in a crowd are visually coupled to nearby neighbors, yielding a common speed and direction of travel (heading). Previously, we derived a visual control law for walking together with one neighbor (Rio & Warren, VSS 2011; Page & Warren, VSS 2013). Here, we extend this framework to investigate how multiple neighbors combine to influence a pedestrian. We collected data from a participant (N=10) walking with 3 confederates in a diamond configuration across a 12x14m room, while recording head position with an ultrasonic tracking system. On some trials, confederates received covert instructions to change their direction (turn left, turn right) or speed (speed up, slow down), and the effect on participant's final lateral position or speed, respectively, was measured. The number of manipulated confederates (1, 2, or 3), their positions (left, center, right), and their initial distance from the participant (1 or 2 m) were varied. For both direction and speed manipulations, there was a main effect of the number of confederates manipulated (direction: F(8,64)=20.17, p<0.001; speed: F(8,64)=2.29, p<0.05), with more confederates eliciting larger participant responses. For the speed manipulation, there was also a main effect of neighbor distance (F(8,64)=2.29, p<0.05), although this was not the case for direction (F(1,8)=0.033, p>0.05). Multiple linear regression on participants' final position reveals that the confederate directly ahead [center] is weighted more heavily than confederates to the left or right (Beta=0.32, 0.17, 0.26, respectively; R2=0.47); the same holds for multiple regression on participant speed (Beta=0.40, 0.26, 0.20, respectively; R2 = 0.46). The results indicate that the influences of multiple neighbors are linearly combined, weighted by their position and, in the case of speed, their distance. We extend our previous control law to model this multiple visual coupling, which may be applied in simulations of human crowd behavior. Meeting abstract presented at VSS 2014

Follow the leader: visual control of speed in pedestrian following.

When people walk together in groups or crowds they must coordinate their walking speed and direction with their neighbors. This paper investigates how a pedestrian visually controls speed when following a leader on a straight path (one-dimensional following). To model the behavioral dynamics of following, participants in Experiment 1 walked behind a confederate who randomly increased or decreased his walking speed. The data were used to test six models of speed control that used the leader's speed, distance, or combinations of both to regulate the follower's acceleration. To test the optical information used to control speed, participants in Experiment 2 walked behind a virtual moving pole, whose visual angle and binocular disparity were independently manipulated. The results indicate the followers match the speed of the leader, and do so using a visual control law that primarily nulls the leader's optical expansion (change in visual angle), with little influence of change in disparity. This finding has direct applications to understanding the coordination among neighbors in human crowds.

Visual Feedback PID Control for 2DOF Parallel Link Manipulator

This paper investigates the visual feedback control for 2DOF(degrees of freedom) parallel link manipulator with zero steady-state error. Firstly we present the model of visual feedback system of the eye-in-hand configuration. Secondly we consider a novel desired joint angle defined by visual information and joint angles based on the structure of the parallel link, because a desired joint angle cannot be given as a standard robot motion control system. Next we propose the PID control law for the visual feedback system in order to overcome steady-state error with respect to the uncertainty of the position and the orientation for the base frame. For the proposed control law, stability analysis of the closed loop system is discussed in the sense of Lyapunov method with a sufficient condition for the feedback gains. Finally the experimental results are shown in order to confirm the validity of the proposed method by comparing with a previous visual feedback control law.

마커인식과 혼합 비주얼 서보잉 기법을 통한 이동로봇의 자세 안정화 제어

This paper proposes a posture stabilization control algorithm for a wheeled mobile robot using hybrid visual servo control method with a position based and an image based visual servoing (PBVS and IBVS). To overcome chattering phenomena which were shown in the previous researches using a simple switching function based on a threshold, the proposed hybrid visual servo control law introduces the fusion function based on a blending function. Then, the chattering problem and rapid motion of the mobile robot can be eliminated. Also, we consider the nonlinearity of the wheeled mobile robot unlike the previous visual servo control laws using linear control methods to improve the performances of the visual servo control law. The proposed posture stabilization control law using hybrid visual servoing is verified by a theoretical analysis and simulation and experimental results.

Task-specific constraints shape the visual feedback control law used to control hand movements

Task-specific constraints shape the visual feedback control law used to control hand movements

Robotic micromanipulation for microassembly: modelling by sequencial function chart and achievement by multiple scale visual servoings

The paper investigates robotic assembly by focusing on the manipulation of microparts. This task is formalized through the notion of basic tasks which are organized in a logical sequence represented by a function chart and interpreted as the model of the behavior of the experimental setup. The latter includes a robotic system, a gripping system, an imaging system, and a clean environment. The imaging system is a photon videomicroscope able to work at multiple scales. It is modelled by a linear projective model where the relation between the scale factor and the magnification or zoom is explicitly established. So, the usual visual control law is modified in order to take into account this relation. The manipulation of some silicon microparts (400 μm × 400 μm × 100 μm) by means of a distributed robotic system (xyθ system, φz system), a two-finger gripping system and a controllable zoom and focus videomicroscope shows the relevance of the concepts. The 30% of failure rate comes mainly from the physical phenomena (electrostatic and capillary forces) instead of the accuracy of control or the occultations of microparts.

Stereo Tracking and Servoing for Space Applications

This paper proposes a real-time, robust and efficient three-dimensional (3-D) model-based tracking algorithm for visual servoing. A virtual visual servoing approach is used for 3-D tracking. This method is similar to more classical nonlinear pose computation techniques. Robustness is obtained by integrating an M-estimator into the virtual visual control law via an iteratively reweighted least-squares implementation. The presented approach is also extended to the use of multiple cameras. Results show the method to be robust to occlusion, changes in illumination and mistracking.

Real-time markerless tracking for augmented reality: the virtual visual servoing framework

Tracking is a very important research subject in a real-time augmented reality context. The main requirements for trackers are high accuracy and little latency at a reasonable cost. In order to address these issues, a real-time, robust, and efficient 3D model-based tracking algorithm is proposed for a "video see through" monocular vision system. The tracking of objects in the scene amounts to calculating the pose between the camera and the objects. Virtual objects can then be projected into the scene using the pose. Here, nonlinear pose estimation is formulated by means of a virtual visual servoing approach. In this context, the derivation of point-to-curves interaction matrices are given for different 3D geometrical primitives including straight lines, circles, cylinders, and spheres. A local moving edges tracker is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating an M-estimator into the visual control law via an iteratively reweighted least squares implementation. This approach is then extended to address the 3D model-free augmented reality problem. The method presented in this paper has been validated on several complex image sequences including outdoor environments. Results show the method to be robust to occlusion, changes in illumination, and mistracking.

VISUAL CONTROL OF ROBOTIC MANIPULATORS: DESIGNING A SIMPLIFIED STABILIZING CONTROLLER

An Image Based Visual Servoing controller for the tip trajectory tracking of rigid manipulators is considered in this paper. Inspired by the task space robot control, an approach to design a visual control law, that takes into account the manipulator dynamics, is proposed. Moreover, to avoid an excessive computational burden, two simplified visual control laws are introduced supported by rigorous proofs of stability based on Lyapunov theory. Simulation results show the effectiveness of these simplified controllers.

Efficient model-based tracking for robot vision

This paper proposes a real-time, robust and efficient three-dimensional (3D) model-based tracking algorithm. A virtual visual servoing approach is used for monocular 3D tracking. This method is similar to more classical non-linear pose computation techniques. A concise method for derivation of efficient distance-to-contour interaction matrices is described. An oriented edge detector is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating a M-estimator into the virtual visual control law via an iteratively re-weighted least-squares implementation. The method presented in this paper has been validated on several visual servoing experiments considering various objects. Results show the method to be robust to occlusion, changes in illumination and mis-tracking.

Observer-based Dynamic Visual Feedback Control of Robotic Systems

This paper considers vision-based motion control with the manipulator dynamics using position measurements and visual information, which we term dynamic visual feedback control. Firstly the visual feedback system of rigid body motion is described in order to derive the dynamic visual feedback system. Secondly we propose a dynamic visual feedback control law which guarantees local asymptotic stability of the overall closed-loop system using a Lyapunov function. L2-gain performance analysis for the proposed control law has been discussed using the energy function which plays the role of a storage function. Next, we show that the control law is based on passivity and the dynamic visual feedback system is constructed from two passive systems. Finally simulation results confirm the effectiveness of the dynamic visual feedback control law.

Optic flow is used to control human walking.

How is human locomotion visually controlled? Fifty years ago, it was proposed that we steer to a goal using optic flow, the pattern of motion at the eye that specifies the direction of locomotion. However, we might also simply walk in the perceived direction of a goal. These two hypotheses normally predict the same behavior, but we tested them in an immersive virtual environment by displacing the optic flow from the direction of walking, violating the laws of optics. We found that people walked in the visual direction of a lone target, but increasingly relied on optic flow as it was added to the display. The visual control law for steering toward a goal is a linear combination of these two variables weighted by the magnitude of flow, thereby allowing humans to have robust locomotor control under varying environmental conditions.