Vision-based Path Planning Research Articles

Performance evaluation of vision based path planning for dynamic real-time scenarios of mobile robot

Performance evaluation of vision based path planning for dynamic real-time scenarios of mobile robot

Computer Vision-Based Path Planning with Indoor Low-Cost Autonomous Drones: An Educational Surrogate Project for Autonomous Wind Farm Navigation

The application of computer vision in conjunction with GPS is essential for autonomous wind turbine inspection, particularly when the drone navigates through a wind farm to detect the turbine of interest. Although drones for such inspections use GPS, our study only focuses on the computer vision aspect of navigation that can be combined with GPS information for better navigation in a wind farm. Here, we employ an affordable, non-GPS-equipped drone within an indoor setting to serve educational needs, enhancing its accessibility. To address navigation without GPS, our solution leverages visual data captured by the drone’s front-facing and bottom-facing cameras. We utilize Hough transform, object detection, and QR codes to control drone positioning and calibration. This approach facilitates accurate navigation in a traveling salesman experiment, where the drone visits each wind turbine and returns to a designated launching point without relying on GPS. To perform experiments and investigate the performance of the proposed computer vision technique, the DJI Tello EDU drone and pedestal fans are used to represent commercial drones and wind turbines, respectively. Our detailed and timely experiments demonstrate the effectiveness of computer vision-based path planning in guiding the drone through a small-scale surrogate wind farm, ensuring energy-efficient paths, collision avoidance, and real-time adaptability. Although our efforts do not replicate the actual scenario of wind turbine inspection using drone technology, they provide valuable educational contributions for those willing to work in this area and educational institutions who are seeking to integrate projects like this into their courses, such as autonomous systems.

A novel methodology for vision-based path planning and obstacle avoidance in mobile robot applications

ABSTRACT The availability of overhead views of the environments around mobile robots always present useful information provided that these views are correctly perceived. In this paper, a novel approach for vision-based path planning and obstacle avoidance for mobile robots is presented. The approach uses visual perception integrated with a proposed field-based path-planning algorithm to overcome some common path-planning issues such as local minima, problematic destinations, and following long paths around obstacles. An exponential angle field is generated around each obstacle that deviates from the robot’s orientation as it directs towards the destination. Obstacle field is activated or deactivated based on a proposed collision prediction algorithm. To satisfy robot convergence to the goal point, the obstacle field parameters are determined using the Lyapunov stability criterion. The method also proposes a search algorithm for proper exit in the case of robot and/or goal point trapping. Both simulation and experiments are used in validating the algorithm. The results show the effectiveness of the proposed algorithm in converging towards target points, avoiding collision with stationary obstacles and overcoming famous path-planning problems.

A Vision-Based Bio-Inspired Reinforcement Learning Algorithms for Manipulator Obstacle Avoidance

Path planning for robotic manipulators has proven to be a challenging issue in industrial applications. Despite providing precise waypoints, the traditional path planning algorithm requires a predefined map and is ineffective in complex, unknown environments. Reinforcement learning techniques can be used in cases where there is a no environmental map. For vision-based path planning and obstacle avoidance in assembly line operations, this study introduces various Reinforcement Learning (RL) algorithms based on discrete state-action space, such as Q-Learning, Deep Q Network (DQN), State-Action-Reward- State-Action (SARSA), and Double Deep Q Network (DDQN). By positioning the camera in an eye-to-hand position, this work used color-based segmentation to identify the locations of obstacles, start, and goal points. The homogeneous transformation technique was used to further convert the pixel values into robot coordinates. Furthermore, by adjusting the number of episodes, steps per episode, learning rate, and discount factor, a performance study of several RL algorithms was carried out. To further tune the training hyperparameters, genetic algorithms (GA) and particle swarm optimization (PSO) were employed. The length of the path travelled, the average reward, the average number of steps, and the time required to reach the objective point were all measured and compared for each of the test cases. Finally, the suggested methodology was evaluated using a live camera that recorded the robot workspace in real-time. The ideal path was then drawn using a TAL BRABO 5 DOF manipulator. It was concluded that waypoints obtained via Double DQN showed an improved performance and were able to avoid the obstacles and reach the goal point smoothly and efficiently.

Computer Vision-Based Path Planning for Robot Arms in Three-Dimensional Workspaces Using Q-Learning and Neural Networks.

Computer vision-based path planning can play a crucial role in numerous technologically driven smart applications. Although various path planning methods have been proposed, limitations, such as unreliable three-dimensional (3D) localization of objects in a workspace, time-consuming computational processes, and limited two-dimensional workspaces, remain. Studies to address these problems have achieved some success, but many of these problems persist. Therefore, in this study, which is an extension of our previous paper, a novel path planning approach that combined computer vision, Q-learning, and neural networks was developed to overcome these limitations. The proposed computer vision-neural network algorithm was fed by two images from two views to obtain accurate spatial coordinates of objects in real time. Next, Q-learning was used to determine a sequence of simple actions: up, down, left, right, backward, and forward, from the start point to the target point in a 3D workspace. Finally, a trained neural network was used to determine a sequence of joint angles according to the identified actions. Simulation and experimental test results revealed that the proposed combination of 3D object detection, an agent-environment interaction in the Q-learning phase, and simple joint angle computation by trained neural networks considerably alleviated the limitations of previous studies.

A Vision-Based Path Planning and Object Tracking Framework for 6-DOF Robotic Manipulator

Industrial robots are widely used for repetitive, humanly unmanageable, and hazardous tasks. Hence, an improvement in the production efficiency of industrial robot manipulators is of prime concern. This can be achieved through machine vision and path planning techniques with a focus on localization and shortest path calculation. In particular, this is important for manufacturing and bottle filling industries which extensively use robotic manipulators to place/displace bottles during production and post refill placements. This is even more challenging when soft, fragile, or opaque objects have to be detected, since it is significantly difficult for robot vision to focus on their indistinguishable features. To this end, we present an ensemble robot framework with a stereo vision system for tracking colored objects which are sensed using blob analysis. An ensemble robotic framework with neural networks is proposed for predicting and thereby overcoming the inbuilt geometric error present in stereo vision systems. Moreover, we have simplified 2-D correspondence problem to 1-D by using a non-rectified stereo camera model and object tracking by applying the triangulation technique in 3D stereo vision coordinate system (SVCS). Subsequently, the SVCS is transformed into robot stereo vision coordinate system for tracking the object centroid by using an RGB marker placed on the object. Finally, in the learning model we have combined color region tracking with machine learning to achieve high accuracy. The outcomes are in accordance with the designed model and successfully achieve path prediction with up to 91.8% accuracy.

Investigating Reduced Path Planning Strategy for Differential Wheeled Mobile Robot

SummaryThis paper presents a vision-based path planning strategy that aims to reduce the computational time required by a robot to find a feasible path from a starting point to the goal point. The proposed algorithm presents a novel strategy that can be implemented on any well-known path planning algorithm such as A*, D* and probabilistic roadmap (PRM), to improve the swiftness of these algorithms. This path planning algorithm is suitable for real-time scenarios since it reduces the computational time compared to the basis and traditional algorithms. To test the proposed path planning strategy, a tracking control strategy is implemented on a mobile platform. This control strategy consists of three major stages. The first stage deals with gathering information about the surrounding environment using vision techniques. In the second stage, a free-obstacle path is generated using the proposed reduced scheme. In the final stage, a Lyapunov kinematic tracking controller and two Artificial Neural Network (ANN) based-controllers are implemented to track the proposed path by adjusting the rotational and linear velocity of the robot. The proposed path planning strategy is tested on a Pioneer P3-DX differential wheeled mobile robot and an Xtion PRO depth camera. Experimental results prove the efficiency of the proposed path planning scheme, which was able to reduce the computational time by a large percentage which reached up to 88% of the time needed by the basis and traditional scheme, without significant adverse effect on the workability of the basis algorithm. Moreover, the proposed path planning algorithm has improved the path efficiency, in terms of the path length and trackability, challenging the traditional trade-off between swiftness and path efficiency.

Vision based Path Planning and Tracking control using Mobile Robot

Vision based Path Planning and Tracking control using Mobile Robot

Vision Based Path Planning and Tracking Control Using Mobile Robot

Vision Based Path Planning and Tracking Control Using Mobile Robot

A PSO–Lyapunov Hybrid Stable Adaptive Fuzzy Tracking Control Approach for Vision-Based Robot Navigation

This paper proposes a novel methodology for autonomous mobile robot navigation utilizing the concept of tracking control. Vision-based path planning and subsequent tracking are performed by utilizing proposed stable adaptive state feedback fuzzy tracking controllers designed using the Lyapunov theory and particle-swarm-optimization (PSO)-based hybrid approaches. The objective is to design two self-adaptive fuzzy controllers, for <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$x$</tex></formula> -direction and <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$y$</tex></formula> -direction movements, optimizing both its structures and free parameters, such that the designed controllers can guarantee desired stability and, simultaneously, can provide satisfactory tracking performance for the vision-based navigation of mobile robot. The design methodology for the controllers simultaneously utilizes the global search capability of PSO and Lyapunov-theory-based local search method, thus providing a high degree of automation. Two different variants of hybrid approaches have been employed in this work. The proposed schemes have been implemented in both simulation and experimentations with a real robot, and the results demonstrate the usefulness of the proposed concept.

Machine vision based path planning for a robotic golf club head welding system

A path planning method based on machine vision techniques is constructed for a golf-club head robotic welding system. This system uses 3D machine vision techniques to recognize the weldseam and generates a welding path for the robot. The location of the weldseam is discovered by applying a Sobel mask to the captured data. A Laplace mask is also useful to filter out the noise points due to the scatter light refraction of tack-welding spots. The weldseam is then replenished and smoothed out by a B-spline curve fitting. The task frame of the weldseam is computed by finding the tangent, normal, and bi-normal of the curve. The robotic welding path is obtained by further rotations and translation along the axes of the task frame according to the requirement of the welding attitude. The developed machine vision technique and the mathematic framework pertaining to the generation of a welding task frame can readily be used for various three-dimensional welding tasks.

휴머노이드 로봇을 위한 비전기반 장애물 회피 시스템 개발

This paper addresses the vision based autonomous walking control system. To handle the obstacles which exist beyond the field of view(FOV), we used the 3d panoramic depth image. Moreover, to decide the avoidance direction and walking motion of a humanoid robot for the obstacle avoidance by itself, we proposed the vision based path planning using 3d panoramic depth image. In the vision based path planning, the path and walking motion are decided under environment condition such as the size of obstacle and available avoidance space. The vision based path planning is applied to a humanoid robot, URIA. The results from these evaluations show that the proposed method can be effectively applied to decide the avoidance direction and the walking motion of a practical humanoid robot.

A two-layered subgoal based mobile robot navigation algorithm with vision system and IR sensors

The present work describes the real-life implementation of a mobile robot navigation scheme where vision sensing is employed as primary sensor for path planning and IR sensors are employed as secondary sensors for actual navigation of the mobile robot with obstacle avoidance capability in a static or dynamic indoor environment. This two-layer based, goal-driven architecture utilizes a wireless camera in the first layer to acquire image and perform image processing, online, to determine subgoal, employing a shortest path algorithm, online. The subgoal information is then utilized in the second layer to navigate the robot utilizing IR sensors. Once the subgoal is reached, vision based path planning and IR guided navigation is reactivated. This sequential process is continued in an iterative fashion until the robot reaches the goal. The algorithm has been effectively tested for several real-life environments created in our laboratory and the results are found to be satisfactory.