Autonomous Motion Planning Research Articles

Advancements in UAV Path Planning: A Deep Reinforcement Learning Approach with Soft Actor-Critic for Enhanced Navigation

This paper tackles the intricate challenge of autonomous navigation for Unmanned Aerial Vehicles (UAVs) through dynamically changing environments. We focus on a sophisticated Deep Reinforcement Learning (DRL) approach using the Soft Actor-Critic (SAC) algorithm, optimized for UAV path planning within a continuous action space. This methodology leverages environmental image data to enhance the precision of flight maneuvers and effective obstacle avoidance. Our approach, validated through extensive simulations in Gazebo and field tests, demonstrates the algorithm’s efficacy in enabling UAVs to adeptly navigate obstacles using depth maps. The study further explores the robustness of the SAC algorithm by comparing it with traditional DRL methods, emphasizing its superior performance in real-world applications. This research contributes significantly to advancing UAV technology, particularly in autonomous motion planning, by integrating cutting-edge machine learning techniques. The video link is: https://www.youtube.com/watch?v=Nd_aMzejNXY .

Robots for Interventions: Proof-of-Concept Method for High-Speed Filling Line Interventions Towards the Removal of Gloves

PurposeAlthough pharmaceutical high-speed filling lines are fully automated, interventions are still performed by hand via glove ports. Bringing robots into the high-speed production lines for the handling of interventions could increase both the thoughput and the sterility of the system. The work presented in this manuscript proposes a semi-autonomous intervention strategy for the removal of erroneous plungers.MethodsA YOLOv8-based computer vision algorithm continuously monitors the state of the plunger feed. Autonomous motion planning is used for large, general movements, while human-operated teleoperation with system guidance allows for small, precise motions. The combination of both creates a intuitive, gloveless intervention method.ResultsA user test with a group of 22 volunteers shows that an untrained operator can perform the teleoperated intervention via this system in an average of ±30\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\pm 30$$\\end{document}s and ±21\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\pm 21$$\\end{document}s when the plungers are static and dynamic respectively. Via a Likert-scale based questionnaire, it was found that the test group experienced the haptic system as very intuitive. Additionally, the system is capable of running completely remotely via a two-laptop setup, which has been tested at distances up to 100km.ConclusionAn intuitive, haptic semi-autonomous teleoperation system is created for the execution of a plunger removal intervention. Both in a static and dynamic scenario, an improvement of the operating time is observed compared to the current glove-based method (2 to 3 min). The system can run fully remotely and was found to be very intuitive by the user group.

Space and time joint optimization for vertical takeoff and landing aircraft in dense obstacles and interference environments

ABSTRACT The Vertical Takeoff and Landing Aircraft (VTOL) is capable of both low-speed hovering and high-speed flight. This paper proposes an autonomous motion planning and anti-wind disturbance control algorithm for the VTOL, implemented spatial and temporal joint motion planning for VTOL, under the limitation of maximum flight speed of the aircraft, VTOL can achieve fully autonomous flight in complex environments, which can realize the autonomous flight of a single aircraft in a complex and dense environment, achieve real-time perception and avoidance of static and dynamic obstacles. The information generated by motion planning includes the position, velocity, and acceleration information of the aircraft, and proposes a wind disturbance estimation and active control strategy in a wind disturbance environment, accurate estimation of wind disturbance is achieved through a state observer, and the wind disturbance was suppressed through nonlinear error feedback control law, implemented wind disturbance estimation and suppression for aircraft in wind disturbance environments, and achieved complete autonomous motion planning and trajectory tracking of VTOL in a complex and wind disturbance environment. Our work can provide a basis for the collaborative control of multiple UAVs.

Autonomous overtaking decision and motion planning of intelligent vehicles based on graph convolutional network and conditional imitation learning

To ensure safe overtaking of intelligent vehicles in dynamic interactive environments, this paper proposes an end-to-end learning method for autonomous overtaking based on Graph Convolutional Network (GCN) and Conditional Imitation Learning (CIL). This method completes the autonomous overtaking behavior by directly mapping the environmental perception data to the underlying vehicle control actions (e.g. throttle and steer angle). This method fully considers the influence of other vehicles’ driving behavior on the overtaking behavior of the ego vehicle. Firstly, the dynamic interactive environments information around the ego vehicle is aggregated in the form of graph-structured data, and the aggregated global features are used as the input of the GCN to output the optimal action instructions that the ego vehicle should take. Secondly, combined with CIL, the action instructions output by the GCN are used as high-level commands to guide CIL. Finally, combined with other perception data, the underlying control actions of the vehicle will be output by CIL to complete safe overtaking in dynamic interactive environments. The method proposed in this paper can effectively extract the global information of the driving scene and complete the collision-free autonomous overtaking behavior, which greatly improves the intelligence of the driving system. The feasibility of the method has been verified by experiments on the CARLA simulation platform. The experimental results prove that the performance of this method is better than that of the conventional end-to-end learning framework, and it has better success rate and generalization performance.

Assistance in Teleoperation of Redundant Robots through Predictive Joint Maneuvering

In teleoperation of redundant robotic manipulators, translating an operator’s end effector motion command to joint space can be a tool for maintaining feasible and precise robot motion. Through optimizing redundancy resolution, the control system can ensure the end effector maintains maneuverability by avoiding joint limits and kinematic singularities. In autonomous motion planning, this optimization can be done over an entire trajectory to improve performance over local optimization. However, teleoperation involves a human-in-the-loop who determines the trajectory to be executed through a dynamic sequence of motion commands. We present two systems, PrediKCT and PrediKCS, for utilizing a predictive model of operator commands in order to accomplish this redundancy resolution in a manner that considers future expected motion during teleoperation. Using a probabilistic model of operator commands allows optimization over an expected trajectory of future motion rather than consideration of local motion alone. Evaluation through a user study demonstrates improved control outcomes from this predictive redundancy resolution over minimum joint velocity solutions and inverse kinematics-based motion controllers.

MTP-GO: Graph-Based Probabilistic Multi-Agent Trajectory Prediction With Neural ODEs

Enabling resilient autonomous motion planning requires robust predictions of surrounding road users' future behavior. In response to this need and the associated challenges, we introduce our model titled MTP-GO. The model encodes the scene using temporal graph neural networks to produce the inputs to an underlying motion model. The motion model is implemented using neural ordinary differential equations where the state-transition functions are learned with the rest of the model. Multimodal probabilistic predictions are obtained by combining the concept of mixture density networks and Kalman filtering. The results illustrate the predictive capabilities of the proposed model across various data sets, outperforming several state-of-the-art methods on a number of metrics.

Event-triggered reconfigurable reinforcement learning motion-planning approach for mobile robot in unknown dynamic environments

Deep reinforcement learning (DRL) is an essential technique for autonomous motion planning of mobile robots in dynamic and uncertain environments. In attempting to acquire a satisfactory DRL-based motion planning strategy, the mobile robots encountered several difficulties, including poor convergence, insufficient sample information, and low learning efficiency. These problems not only consume plenty of training time, but also bring a negative impact on motion planning performance. One promising research direction is to provide a more effective network framework for DRL-based policies. Along this line of thinking, our paper presents a novel DRL-based motion planning approach called Reconfigurable Structure of Deep Deterministic Policy Gradient (RS-DDPG) for mobile robots. To account for the poor convergence, the proposed approach first introduces an event-triggered reconfigurable actor–critic network framework for motion policy that adaptively changes its network structure to suppress the overestimation of action value. Then, the time convergence of the motion policy can be enhanced based on the value actions with minor valuation deviation. Afterwards, an adaptive reward mechanism is designed for reconfigurable networks to compensate for the lack of sample information. To deal with the problem of low learning efficiency, we developed a sample pretreatment method for the experience samples, which employs three novel techniques to improve the sample utilization, including a double experience memory buffer, a variable proportional sampling principle, and a similarity judgment mechanism. In extensive experiments, the proposed method outperforms the compared approaches.

RDA: An Accelerated Collision Free Motion Planner for Autonomous Navigation in Cluttered Environments

Autonomous motion planning is challenging in multi-obstacle environments due to nonconvex collision avoidance constraints. Directly applying numerical solvers to these nonconvex formulations fails to exploit the constraint structures, resulting in excessive computation time. In this paper, we present an accelerated collision-free motion planner, namely regularized dual alternating direction method of multipliers (RDADMM or RDA for short), for the model predictive control (MPC) based motion planning problem. The proposed RDA addresses nonconvex motion planning via solving a smooth biconvex reformulation via duality and allows the collision avoidance constraints to be computed in parallel for each obstacle to reduce computation time significantly. We validate the performance of the RDA planner through path-tracking experiments with car-like robots in both simulation and real-world settings. Experimental results show that the proposed method generates smooth collision-free trajectories with less computation time compared with other benchmarks and performs robustly in cluttered environments. The source code is available at https://github.com/hanruihua/RDA_planner.

Motion Planning for Autonomous Vehicles Based on Dynamic Bio-inspired Neural Network Model

Background: Nowadays, with the increase of the number of cars, traffic accidents occur frequently, improving the active safety of cars, and traffic safety has become an inevitable requirement of the development of intelligent transportation. Objective: Motion planning for autonomous driving is a challenging problem due to the uncertainty of the unknown complex environment. Conventional methods may fail in some cases on account of poor real-time performance and instability. The objective is to propose a motion planning method to plan a reasonable path autonomously for an autonomous vehicle in unknown environments and ensure obstacle avoidance. Methods: The solution is to use the improved Glasius Bio-inspired Neural Network (GBNN) to plan a feasible route for the autonomous vehicle, which adaptively changes the direction of movement according to the changes of neuron values. Neuron connection weight in the neural network is reasonably improved to increase the update rate of activity value, so that this model can more adapt to the changes of dynamic environment. Meanwhile, the planned trajectory is fitted to meet vehicle kinematics model. Results: Unstructured and structured roads are tested in static and dynamic environments, respectively. During the test, static obstacles and surrounding vehicles’ intentions are all generated randomly. In this way, the generated route can be optimal and adapt to the stochastic as well. Conclusion: Simulation results in unstructed road and structed road show the effectiveness of the proposed algorithm in autonomous motion planning. In addition, the performance of path length, smoothness and turning angle has been improved.

EfferDeepNet: An Efficient Semantic Segmentation Method for Outdoor Terrain

The recognition of terrain and outdoor complex environments based on vision sensors is a key technology in practical robotics applications, and forms the basis of autonomous navigation and motion planning. While traditional machine learning methods can be applied to outdoor terrain recognition, their recognition accuracy is low. In order to improve the accuracy of outdoor terrain recognition, methods based on deep learning are widely used. However, the network structure of deep learning methods is very complex, and the number of parameters is large, which cannot meet the actual operating requirements of of unmanned systems. Therefore, in order to solve the problems of poor real-time performance and low accuracy of deep learning algorithms for terrain recognition, this paper proposes the efficient EfferDeepNet network for pixel level terrain recognition in order to realize global perception of outdoor environment. First, this method uses convolution kernels with different sizes in the depthwise separable convolution (DSC) stage to extract more semantic feature information. Then, an attention mechanism is introduced to weight the acquired features, focusing on the key local feature areas. Finally, in order to avoid redundancy due to a large number of features and parameters in the model, this method uses a ghost module to make the network more lightweight. In addition, to solve the problem of pixel level terrain recognition having a negative effect on image boundary segmentation, the proposed method integrates an enhanced feature extraction network. Experimental results show that the proposed EfferDeepNet network can quickly and accurately perform global recognition and semantic segmentation of terrain in complex environments.

UVMS task-priority planning framework for underwater task goal classification optimization.

This paper presents a task prioritization strategy based on a generic underwater task goal classification transformation for multitasking underwater operational tasks: attitude control, floating manipulation, collision-free motion, especially optimizing trajectory of the end-effector of an underwater vehicle manipulator system (UVMS) in a complex marine environment. The design framework aims to divide the complex underwater operational tasks into UVMS executable generic task combinations and optimize the resource consumption during the whole task. In order to achieve the corresponding underwater task settings, the system needs to satisfy different task scheduling structures. We consider the actual application scenarios of the operational goals and prioritize and define each category of task hierarchy accordingly. Multiple tasks simultaneously enable fast adaptation to UVMS movements and planning to complete UVMS autonomous movements. Finally, an underwater vehicle manipulator system implements the task prioritization planning framework for a practical scenario with different constraints on different goals. We quickly and precisely realize the interconversion of different tasks under goal constraints. The autonomous motion planning and real-time performance of UVMS are improved to cope with the increasing operational task requirements and the complex and changing practical engineering application environments.

Autonomous live working robot navigation with real‐time detection and motion planning system on distribution line

In this study, an autonomous robot navigation system is designed for live working on distribution line. The developed system features a real-time detection and motion planning system, incorporating a manipulator capable of grasping power components. In order to accurately identify targets, the authors propose an object detection method based on the Larger Scale ‘You Only Look Once’ Version 4 (LS-YOLOv4) algorithm for detecting the insulators and drop fuses. The LS-YOLOv4 extracts features of power components by Convolutional Neural Network (CNN), and then performs feature fusion. Then the authors develop a motion planning method based on the Node Control Optimal Rapidly Exploring Random Trees (NC-RRT*), which can drive the robot to realise the autonomous robot motion planning and obstacle avoidance. On the grasping function, the authors present a reliable Lightweight-based Convolutional Neural Network (L-CNN) grasping point detection method. Finally, the authors evaluate fully autonomous robotic system in both simulated and real-world experiments. The experimental results demonstrate that the proposed system can effectively identify the target and complete the grasping task in an efficient way. Notably, the proposed motion planning method can take into account both planning efficiency and accuracy to manipulation tasks.

Asynchronous Multithreading Reinforcement-Learning-Based Path Planning and Tracking for Unmanned Underwater Vehicle

The underwater unmanned vehicle (UUV) is widely used in various marine operations, in which path planning and trajectory tracking are the critical technologies to achieve autonomous motion planning. Unlike previous research methods, this article proposes the asynchronous multithreading proximal policy optimization-based path planning (AMPPO-PP) and trajectory tracking (AMPPO-TT) algorithms and applies these two methods to different task scenarios of UUVs. Taking advantage of the AMPPO, the expensive online computational procedure is converted to an offline training process. The proposed algorithms enable the UUV to learn autonomous planning, tracking, and emergency obstacle avoiding. Besides, the algorithm architecture of the AMPPO-PP and the AMPPO-TT is described in detail. By refining the reward in each timestep and utilizing the reward-shaping trick, the reward sparsity is avoided. The goal-distance heuristic reward function is used to make the UUV explore more directionally. Various simulation environments are developed from simple to complex, along with multiple comparative experiments to verify the effectiveness of the proposed algorithms.

Review of Robotics Systems Available to aid Visually Impaired Persons

The need for assistance robots has risen dramatically in this time of age for persons with disabilities particularity for visually impaired persons as their number continues to rise. For them independent living can be performed by an aid robot which is one of the key assistive technology devices that can help to regain dignity and self-confidence. This paper reviews the existing design and development of a personal assistant robots that uses a certain scientific algorithm to detect and estimate the relative location of an objects in an indoor environment using voice instructions. These semi-humanoid robots have built in multiple HD cameras located on different parts of the robots. Autonomous movement, object detection, distance measuring, and motion planning are all done with the cameras. Furthermore, the robot’s utility is increased by keeping the user informed about the results of its actions.

Drone-Based Autonomous Motion Planning System for Outdoor Environments under Object Detection Uncertainty

Recent advances in autonomy of unmanned aerial vehicles (UAVs) have increased their use in remote sensing applications, such as precision agriculture, biosecurity, disaster monitoring, and surveillance. However, onboard UAV cognition capabilities for understanding and interacting in environments with imprecise or partial observations, for objects of interest within complex scenes, are limited, and have not yet been fully investigated. This limitation of onboard decision-making under uncertainty has delegated the motion planning strategy in complex environments to human pilots, which rely on communication subsystems and real-time telemetry from ground control stations. This paper presents a UAV-based autonomous motion planning and object finding system under uncertainty and partial observability in outdoor environments. The proposed system architecture follows a modular design, which allocates most of the computationally intensive tasks to a companion computer onboard the UAV to achieve high-fidelity results in simulated environments. We demonstrate the system with a search and rescue (SAR) case study, where a lost person (victim) in bushland needs to be found using a sub-2 kg quadrotor UAV. The navigation problem is mathematically formulated as a partially observable Markov decision process (POMDP). A motion strategy (or policy) is obtained once a POMDP is solved mid-flight and in real time using augmented belief trees (ABT) and the TAPIR toolkit. The system’s performance was assessed using three flight modes: (1) mission mode, which follows a survey plan and used here as the baseline motion planner; (2) offboard mode, which runs the POMDP-based planner across the flying area; and (3) hybrid mode, which combines mission and offboard modes for improved coverage in outdoor scenarios. Results suggest the increased cognitive power added by the proposed motion planner and flight modes allow UAVs to collect more accurate victim coordinates compared to the baseline planner. Adding the proposed system to UAVs results in improved robustness against potential false positive readings of detected objects caused by data noise, inaccurate detections, and elevated complexity to navigate in time-critical applications, such as SAR.

Modular Robotic Limbs for Astronaut Activities Assistance.

In order to meet the assist requirements of extravehicular activity (EVA) for astronauts, such as moving outside the international space station (ISS) or performing on-orbit tasks by a single astronaut, this paper proposes an astronaut robotic limbs system (AstroLimbs) for extravehicular activities assistance. This system has two robotic limbs that can be fixed on the backpack of the astronaut. Each limb is composed of several basic module units with identical structure and function, which makes it modularized and reconfigurable. The robotic limbs can work as extra arms of the astronaut to assist them outside the space station cabin. In this paper, the robotic limbs are designed and developed. The reinforcement learning method is introduced to achieve autonomous motion planning capacity for the robot, which makes the robot intelligent enough to assist the astronaut in unstructured environment. In the meantime, the movement of the robot is also planned to make it move smoothly. The structure scene of the ISS for extravehicular activities is modeled in a simulation environment, which verified the effectiveness of the proposed method.

Relevant experience learning: A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles (UAVs) play a vital role in military warfare. In a variety of battlefield mission scenarios, UAVs are required to safely fly to designated locations without human intervention. Therefore, finding a suitable method to solve the UAV Autonomous Motion Planning (AMP) problem can improve the success rate of UAV missions to a certain extent. In recent years, many studies have used Deep Reinforcement Learning (DRL) methods to address the AMP problem and have achieved good results. From the perspective of sampling, this paper designs a sampling method with double-screening, combines it with the Deep Deterministic Policy Gradient (DDPG) algorithm, and proposes the Relevant Experience Learning-DDPG (REL-DDPG) algorithm. The REL-DDPG algorithm uses a Prioritized Experience Replay (PER) mechanism to break the correlation of continuous experiences in the experience pool, finds the experiences most similar to the current state to learn according to the theory in human education, and expands the influence of the learning process on action selection at the current state. All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV. The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm, while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV's Autonomous Motion Planning in Complex Unknown Environments.

Autonomous motion planning (AMP) of unmanned aerial vehicles (UAVs) is aimed at enabling a UAV to safely fly to the target without human intervention. Recently, several emerging deep reinforcement learning (DRL) methods have been employed to address the AMP problem in some simplified environments, and these methods have yielded good results. This paper proposes a multiple experience pools (MEPs) framework leveraging human expert experiences for DRL to speed up the learning process. Based on the deep deterministic policy gradient (DDPG) algorithm, a MEP–DDPG algorithm was designed using model predictive control and simulated annealing to generate expert experiences. On applying this algorithm to a complex unknown simulation environment constructed based on the parameters of the real UAV, the training experiment results showed that the novel DRL algorithm resulted in a performance improvement exceeding 20% as compared with the state-of-the-art DDPG. The results of the experimental testing indicate that UAVs trained using MEP–DDPG can stably complete a variety of tasks in complex, unknown environments.

A RDA-Based Deep Reinforcement Learning Approach for Autonomous Motion Planning of UAV in Dynamic Unknown Environments

Autonomous motion planning (AMP) in dynamic unknown environments emerges as an urgent requirement with the prosperity of unmanned aerial vehicle (UAV). In this paper, we present a DRL-based planning framework to address the AMP problem, which is applicable in both military and civilian fields. To maintain learning efficiency, a novel reward difference amplifying (RDA) scheme is proposed to reshape the conventional reward functions and is introduced into state-of-the-art DRLs to constructs novel DRL algorithms for the planner’s learning. Different from conventional motion planning approaches, our DRL-based methods provide an end-to-end control for UAV, which directly maps the raw sensory measurements into high-level control signals. The training and testing experiments demonstrate that our RDA scheme makes great contributions to the performance improvement and provides the UAV good adaptability to dynamic environments.

Human skill integrated motion planning of assembly manipulation for 6R industrial robot

PurposeThe purpose of this paper is to propose a new method based on three-dimensional (3D) vision technologies and human skill integrated deep learning to solve assembly positioning task such as peg-in-hole.Design/methodology/approachHybrid camera configuration was used to provide the global and local views. Eye-in-hand mode guided the peg to be in contact with the hole plate using 3D vision in global view. When the peg was in contact with the workpiece surface, eye-to-hand mode provided the local view to accomplish peg-hole positioning based on trained CNN.FindingsThe results of assembly positioning experiments proved that the proposed method successfully distinguished the target hole from the other same size holes according to the CNN. The robot planned the motion according to the depth images and human skill guide line. The final positioning precision was good enough for the robot to carry out force controlled assembly.Practical implicationsThe developed framework can have an important impact on robotic assembly positioning process, which combine with the existing force-guidance assembly technology as to build a whole set of autonomous assembly technology.Originality/valueThis paper proposed a new approach to the robotic assembly positioning based on 3D visual technologies and human skill integrated deep learning. Dual cameras swapping mode was used to provide visual feedback for the entire assembly motion planning process. The proposed workpiece positioning method provided an effective disturbance rejection, autonomous motion planning and increased overall performance with depth images feedback. The proposed peg-hole positioning method with human skill integrated provided the capability of target perceptual aliasing avoiding and successive motion decision for the robotic assembly manipulation.