Vision-based Control Approach Research Articles

Vision-Based Control of a Mobile Manipulator With an Adaptable-Passive Suspension for Unstructured Environments

Abstract The kinematic design, development, and navigation control of a new autonomous mobile manipulator for unstructured terrain is presented in this work. An innovative suspension system is designed based on the kinematic synthesis of an adaptable, passive mechanism. This novel suspension can compensate for irregularities in the terrain by using two pairs of bogies joined by a crank-slider mechanism and facilitates the control of the robotic platform using video cameras. The mobile robot is also equipped with a robotic manipulator, of which a synthesis, simulation, and experimental validation are presented. Additionally, manipulation is accomplished during motion on rough terrain. The proposed mobile robot has been fabricated using additive manufacturing (AM) techniques. A vision-based control approach, from here on named mobile linear-camera space manipulation (MLCSM), for mobile manipulators has been synthesized and implemented to conduct experimental tests. This mobile manipulator has been designed to traverse uneven terrain so that the loading platform is kept close to horizontal while crossing obstacles up to one-third of the size of its wheels. This feature allows for the onboard cameras to stay oriented toward the target; it also allows for any device mounted on the payload platform to remain nearly horizontal during the task. The developed control approach allows us to estimate the position and orientation of the manipulator’s end effector and update its trajectory along the path toward the target. The experiments show a final precision for engagement of a pallet within +/−2.5 mm in position and +/−2 deg in orientation

Calibration-Free Image-Based Trajectory Tracking Control of Mobile Robots With an Overhead Camera

To make the controller implementation easier and to enhance the system robustness and control performance in the presence of the camera parameter uncertainties, it is very desired to develop vision-based control approaches without any offline or online camera calibration. In this article, we propose a new calibration-free image-based trajectory tracking control scheme for nonholonomic mobile robots with a truly uncalibrated fixed camera. By developing a novel camera-parameter-independent kinematic model, both offline and online camera calibration can be avoided in the proposed scheme, and any knowledge of the camera is not needed in the controller design. The proposed trajectory tracking control scheme can guarantee exponential convergence of the image position and velocity tracking errors. To illustrate the performance of the proposed scheme, experimental results are provided in this article. Note to Practitioners —This article was motivated by the vision-based motion control problem of mobile robots in uncalibrated environments. Existing vision-based motion control approaches for nonholonomic mobile robots generally depend on offline precise/coarse or online numerical/adaptive calibration of the camera intrinsic and extrinsic parameters and require precise or coarse knowledge of the camera in their implementation. This article presents a novel calibration-free image-based trajectory tracking control scheme, which can be implemented easily in real environments without any offline or online calibration of the camera parameters and can be used to efficiently control the motion of nonholonomic mobile robots with an arbitrarily placed and truly unknown overhead camera. Experimental results show that the proposed scheme can achieve satisfactory trajectory tracking control performance despite the lack of any knowledge about the camera intrinsic and extrinsic parameters and the presence of unknown camera lens distortions, and hence, can provide a simple but efficient solution to the vision-based motion control problem of nonholonomic mobile robots.

Image-Based Position Control of Mobile Robots With a Completely Unknown Fixed Camera

In this paper, we consider the position control problem of nonholonomic mobile robots, i.e., the regulation of mobile robots to a desired position specified by the image position of a feature point onboard the mobile robot, by using visual feedback information from an overhead fixed camera. Many vision-based control approaches have been proposed for motion control of nonholonomic mobile robots, however, they usually assume that exact or approximate knowledge of full/partial intrinsic and/or extrinsic parameters of the camera can be available, and large or even very small errors in these parameters can deteriorate the control performance, reduce the control accuracy, and even affect the system stability. Hence, it is interesting and highly desirable to develop vision-based controllers without depending on the camera parameters, from both control performance and controller implementation perspectives. With this in mind, we propose novel image-based position control schemes for nonholonomic mobile robots completely without knowing the camera parameters. In the proposed schemes, neither accurate nor approximate knowledge about the camera intrinsic and extrinsic parameters is required, and the totally unknown camera can be mounted on the ceiling with an arbitrary pose, which can make the controller implementation very simple and flexible. Experimental results are provided to show the feasibility and effectiveness of the proposed schemes.

Contribution to generic modeling and vision-based control of a broad class of fully parallel robots

SUMMARYThis paper deals with a generic modeling and vision-based control approach for a broad class of parallel mechanisms. First, a generic architecture representing several families is proposed. Second, inspired by the geometry of lines, a generic differential inverse kinematic model according to the proposed generic structure is introduced. Finally, based on the image projection of cylindrical legs, a kinematic vision-based control using legs observation is presented. The approach is illustrated and validated on the Gough–Stewart and Par4 parallel robots.

Hand-Eye Coordination for Robotic Assembly Tasks

This paper addresses issues related to the design and implementation of robotic assembly tasks. Specifically, we consider automatic assembly systems with real-time visual sensing for the back shells of cellular phones. Typically, industrial assembly tasks are accomplished using either the look-then-move open-loop or the look-and-move closed-loop control approach. For either approach, successful assembly requires that issues concerned with task accuracy must be considered based on camera calibration parameters. For the hand-eye robotic system to operate in real-time, one must adopt an appropriate control structure to maximize task efficiency. Simple and repetitive assembly tasks can be performed quickly through look-then-move open-loop controls. However, relatively slower look-and-move closed-loop control approaches are better suited for complex tasks or those requiring greater flexibility. To accomplish automatic assembly tasks with real-time visual sensing, either eye-to-hand or eye-in-hand vision must be employed. The proposed vision-based control approaches for back shell assembly tasks are likely to have real potential in industrial manufacturing applications.

New hybrid vision-based control approach for automated guided vehicles

Automated guided vehicles (AGVs) are a common choice made by many companies for material handling (MH) in manufacturing systems. AGV-based internal transport of raw materials, goods, and parts is becoming improved with advances in technology. Demands for fast, efficient, and reliable transport imply the usage of the flexible AGVs with onboard sensing and special kinds of algorithms needed for daily operations. So far, the majority of these transport solutions have not considered the modern techniques for visual servoing, monocular SLAM, and consequently, the usage of camera as onboard sensor for AGVs. In this research, a new hybrid control of AGV is proposed. The main control algorithm consists of two independent control loops: position-based control (PBC) for global navigation and image based visual seroving (IBVS) for fine motions needed for accurate steering towards loading/unloading point. By separating the initial transportation task into two parts (global navigation towards the goal pose near the loading/unloading point and fine motion from the goal pose to the loading/unloading point), the proposed hybrid control bypasses the need for artificial landmarks or accurate map of the environment. The state estimation of the robot pose is determined in terms of monocular SLAM, via extended Kalman filter coupled with feedforward neural network—the neural extended Kalman filter (NEKF). NEKF is used to model unknown disturbances and to improve the robot state transition model. The integration of the new hybrid control and NEKF has been tested in laboratory with the mobile robot and simple camera. Experimental results present the effectiveness of the proposed hybrid control approach.

Fluid Flow Control: A Vision-Based Approach

This paper proposes a new approach to control a flow. Controlling a flow consists either to change its state to another state or to maintain its current state whatever external disturbances. Here the control of the laminar plane Poiseuille flow is considered. To estimate the state of this flow, existing control methods rely on a set of limited wall shear stress measurements. These existing methods suffer from limited observations, from noisy measurements and from the initialization involved in the observer required to estimate the flow state. To deal with these issues, this paper proposes a vision-based control approach. More precisely, by visualizing a fluid flow, dense flow velocity maps can be computed via optical flow techniques and subsequently used to build an observer-free closed-loop control law. This approach is formally proven to be of great improvements for the control of this flow in comparison with existing control approaches.

Visual control through the trifocal tensor for nonholonomic robots

We present a new vision-based control approach which drives autonomously a nonholonomic vehicle to a target location. The vision system is a camera fixed on the vehicle and the target location is defined by an image taken previously in that location. The control scheme is based on the trifocal tensor model, which is computed from feature correspondences in calibrated retina across three views: initial, current and target images. The contribution is a trifocal-based control law defined by an exact input–output linearization of the trifocal tensor model. The desired evolution of the system towards the target is directly defined in terms of the trifocal tensor elements by means of sinusoidal functions without needing metric or additional information from the environment. The trifocal tensor presents important advantages for visual control purposes, because it is more robust than two-view geometry as it includes the information of a third view and, contrary to the epipolar geometry, short baseline is not a problem. Simulations show the performance of the approach, which has been tested with image noise and calibration errors.

Grasping of Static and Moving Objects Using a Vision-Based Control Approach

Robotic systems require the use of sensing to enable flexible operation in uncalibrated or partially calibrated environments. Recent work combining robotics with vision has emphasized an active vision paradigm where the system changes the pose of the camera to improve environmental knowledge or to establish and preserve a desired relationship between the robot and objects in the environment. Much of this work has concentrated upon the active observation of objects by the robotic agent. We address the problem of robotic visual grasping (eye-in-hand configuration) of static and moving rigid targets. The objective is to move the image projections of certain feature points of the target to effect a vision-guided reach and grasp. An adaptive control algorithm for repositioning a camera compensates for the servoing errors and the computational delays that are introduced by the vision algorithms. Stability issues along with issues concerning the minimum number of required feature points are discussed. Experimental results are presented to verify the validity and the efficacy of the proposed control algorithms. We then address an adaptation to the control paradigm that focuses upon the autonomous grasping of a static or moving object in the manipulator’s workspace. Our work extends the capabilities of an eye-in-hand system beyond those as a ‘pointer’ or a ‘camera orienter’ to provide the flexibility required to robustly interact with the environment in the presence of uncertainty. The proposed work is experimentally verified using the Minnesota Robotic Visual Tracker (MRVT) [7] to automatically select object features, to derive estimates of unknown environmental parameters, and to supply a control vector based upon these estimates to guide the manipulator in the grasping of a static or moving object.