Virtual Visual Servoing Research Articles

Multi-Log Grasping Using Reinforcement Learning and Virtual Visual Servoing

We explore multi-log grasping using reinforcement learning and virtual visual servoing for automated forwarding in a simulated environment. Automation of forest processes is a major challenge, and many techniques regarding robot control pose different challenges due to the unstructured and harsh outdoor environment. Grasping multiple logs involves various problems of dynamics and path planning, where understanding the interaction between the grapple, logs, terrain, and obstacles requires visual information. To address these challenges, we separate image segmentation from crane control and utilise a virtual camera to provide an image stream from reconstructed 3D data. We use Cartesian control to simplify domain transfer to real-world applications. Because log piles are static, visual servoing using a 3D reconstruction of the pile and its surroundings is equivalent to using real camera data until the point of grasping. This relaxes the limits on computational resources and time for the challenge of image segmentation, and allows for data collection in situations where the log piles are not occluded. The disadvantage is the lack of information during grasping. We demonstrate that this problem is manageable and present an agent that is 95% successful in picking one or several logs from challenging piles of 2–5 logs.

Interlinked Visual Tracking and Robotic Manipulation of Articulated Objects

Robotic manipulation tasks require the knowledge on the configuration of the object in use. Since most objects are generally not equipped with any sensor, an estimator is required. Furthermore, if an object is articulated, i.e., includes passive joints, the estimation process has to reconstruct the pose of the object floating base and its joints variables, concurrently with the manipulation control. We address the estimation problem with an online virtual visual servoing paradigm written as a quadratic program. Our estimator is integrated in closed-loop with the manipulation control governing the robot, which is also a quadratic program. Tracking and manipulation experiments, using a humanoid, show the effectiveness of our approach.

Virtual Visual Servoing for Multicamera Pose Estimation

Exploiting multiple cameras for object pose estimation contributes toward achieving enhanced accuracy and robustness to measurement faultiness. Nonlinear optimization frameworks, such as the Gauss-Newton, were traditionally employed for this purpose. This paper proposes a novel algorithm for multicamera pose estimation by using the concept of virtual visual servoing. Two fusion structures, namely centralized and decentralized fusion, are considered. The former offers higher accuracy at the price of increased computation. The latter improves the computation speed, partially sacrificing the system accuracy. The experimental results imply the proposed configurations to be faster than the Gauss-Newton pose estimation method, while offering the same or even higher level of accuracy.

Direct Visual Servoing to Track Trajectories in Human-Robot Cooperation

This paper describes a dynamic image-based control system to guide two coupled robots. The first robot is a Mitsubishi PA-10 robotic manipulator which has a second mini-robot with 3 degrees of freedom (DOF) attached at its end-effector. The vision system used for the guidance of both robots is composed of a camera at the end-effector of the mini-robot. The paper presents a new method to perform the mini-robot guidance using dynamic control to track a previous generated image trajectory. The mini-robot performs the tracking in a workspace in cooperation with a human operator. Therefore, the proposed visual control is combined with virtual visual servoing to perform a safety behavior.

Virtual Visual Servoing for Real-Time Robot Pose Estimation

AbstractWe propose a system for markerless pose estimation and tracking of a robot manipulator. By tracking the manipulator, we can obtain an accurate estimate of its position and orientation necessary in many object grasping and manipulation tasks. Tracking the manipulator allows also for better collision avoidance. The method is based on the notion of virtual visual servoing. We also propose the use of distance transform in the control loop, which makes the performance independent of the feature search window.

Stereo Tracking and Servoing for Space Applications

This paper proposes a real-time, robust and efficient three-dimensional (3-D) model-based tracking algorithm for visual servoing. A virtual visual servoing approach is used for 3-D tracking. This method is similar to more classical nonlinear pose computation techniques. Robustness is obtained by integrating an M-estimator into the virtual visual control law via an iteratively reweighted least-squares implementation. The presented approach is also extended to the use of multiple cameras. Results show the method to be robust to occlusion, changes in illumination and mistracking.

Classification-based automatic initialization VVS tracking algorithm

Automatic initialization becomes the hot issue in the tracking of augmented reality.The paper based on Virtual Visual Servoing(VVS) tracking algorithm used randomized trees as the classification technique and treated the feature point matching as a classification problem.Then,a linear pose computation method was used to estimate the initial camera position.Finally,the fast veracity was verified and the method had a better performance.

Real-time markerless tracking for augmented reality: the virtual visual servoing framework

Tracking is a very important research subject in a real-time augmented reality context. The main requirements for trackers are high accuracy and little latency at a reasonable cost. In order to address these issues, a real-time, robust, and efficient 3D model-based tracking algorithm is proposed for a "video see through" monocular vision system. The tracking of objects in the scene amounts to calculating the pose between the camera and the objects. Virtual objects can then be projected into the scene using the pose. Here, nonlinear pose estimation is formulated by means of a virtual visual servoing approach. In this context, the derivation of point-to-curves interaction matrices are given for different 3D geometrical primitives including straight lines, circles, cylinders, and spheres. A local moving edges tracker is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating an M-estimator into the visual control law via an iteratively reweighted least squares implementation. This approach is then extended to address the 3D model-free augmented reality problem. The method presented in this paper has been validated on several complex image sequences including outdoor environments. Results show the method to be robust to occlusion, changes in illumination, and mistracking.

Efficient model-based tracking for robot vision

This paper proposes a real-time, robust and efficient three-dimensional (3D) model-based tracking algorithm. A virtual visual servoing approach is used for monocular 3D tracking. This method is similar to more classical non-linear pose computation techniques. A concise method for derivation of efficient distance-to-contour interaction matrices is described. An oriented edge detector is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating a M-estimator into the virtual visual control law via an iteratively re-weighted least-squares implementation. The method presented in this paper has been validated on several visual servoing experiments considering various objects. Results show the method to be robust to occlusion, changes in illumination and mis-tracking.

Virtual Visual Servoing: a framework for real-time augmented reality

Virtual Visual Servoing: a framework for real-time augmented reality

Virtual Visual Servoing: a framework for real-time augmented reality

This paper presents a framework to achieve real-time augmented reality applications. We propose a framework based on the visual servoing approach well known in robotics. We consider pose or viewpoint computation as a similar problem to visual servoing. It allows one to take advantage of all the research that has been carried out in this domain in the past. The proposed method features simplicity, accuracy, efficiency, and scalability wrt. to the camera model as well as wrt. the features extracted from the image. We illustrate the efficiency of our approach on augmented reality applications with various real image sequences.