Mobile Robots In Dynamic Environments Research Articles

Adaptive Deep Ant Colony Optimization–Asymmetric Strategy Network Twin Delayed Deep Deterministic Policy Gradient Algorithm: Path Planning for Mobile Robots in Dynamic Environments

Path planning is one of the main focal points and challenges in mobile robotics research. Traditional ant colony optimization (ACO) algorithms encounter issues such as low efficiency, slow convergence, and a tendency to become stuck in local optima and search stagnation when applied to complex dynamic environments. Addressing these challenges, this study introduces an adaptive deep ant colony optimization (ADACO) algorithm, which significantly improves efficiency and convergence speed through enhanced pheromone diffusion mechanisms and updating strategies, applied to global path planning. To adapt to dynamically changing environments and achieve more precise local path planning, an asymmetric strategy network TD3 algorithm (ATD3) is further proposed, which utilizes global path planning information within the strategy network only, creating a new hierarchical path planning algorithm—ADACO-ATD3. Simulation experiments demonstrate that the proposed algorithm significantly outperforms in terms of path length and number of iterations, effectively enhancing the mobile robot’s path planning performance in complex dynamic environments.

Real-Time Submap Trimming-Based Map Updating for Long-Term Localization of Mobile Robot in Dynamic Environments

Real-Time Submap Trimming-Based Map Updating for Long-Term Localization of Mobile Robot in Dynamic Environments

Simulation of Dynamic Path Planning of Symmetrical Trajectory of Mobile Robots Based on Improved A* and Artificial Potential Field Fusion for Natural Resource Exploration

With the rapid development of new-generation artificial intelligence and Internet of Things technology, mobile robot technology has been widely used in various fields. Among them, the autonomous path-planning technology of mobile robots is one of the cores for realizing their autonomous driving and obstacle avoidance. This study conducts an in-depth discussion on the real-time and dynamic obstacle avoidance capabilities of mobile robot path planning. First, we proposed a preprocessing method for obstacles in the grid map, focusing on the closed processing of the internal space of concave obstacles to ensure the feasibility of the path while effectively reducing the number of grid nodes searched by the A* algorithm, thereby improving path search efficiency. Secondly, in order to achieve static global path planning, this study adopts the A algorithm. However, in practice, algorithm A has problems such as a large number of node traversals, low search efficiency, redundant path nodes, and uneven turning angles. To solve these problems, we optimized the A* algorithm, focusing on optimizing the heuristic function and weight coefficient to reduce the number of node traversals and improve search efficiency. In addition, we use the Bezier curve method to smooth the path and remove redundant nodes, thereby reducing the turning angle. Then, in order to achieve dynamic local path planning, this study adopts the artificial potential field method. However, the artificial potential field method has the problems of unreachable target points and local minima. In order to solve these problems, we optimized the repulsion field so that the target point is at the lowest point of the global energy of the gravitational field and the repulsive field and eliminated the local optimal point. Finally, for the path-planning problem of mobile robots in dynamic environments, this study proposes a hybrid path-planning method based on a combination of the improved A* algorithm and the artificial potential field method. In this study, we not only focus on the efficiency of mobile robot path planning and real-time dynamic obstacle avoidance capabilities but also pay special attention to the symmetry of the final path. By introducing symmetry, we can more intuitively judge whether the path is close to the optimal state. Symmetry is an important criterion for us to evaluate the performance of the final path.

Autonomous Driving of Mobile Robots in Dynamic Environments Based on Deep Deterministic Policy Gradient: Reward Shaping and Hindsight Experience Replay.

In this paper, we propose a reinforcement learning-based end-to-end learning method for the autonomous driving of a mobile robot in a dynamic environment with obstacles. Applying two additional techniques for reinforcement learning simultaneously helps the mobile robot in finding an optimal policy to reach the destination without collisions. First, the multifunctional reward-shaping technique guides the agent toward the goal by utilizing information about the destination and obstacles. Next, employing the hindsight experience replay technique to address the experience imbalance caused by the sparse reward problem assists the agent in finding the optimal policy. We validated the proposed technique in both simulation and real-world environments. To assess the effectiveness of the proposed method, we compared experiments for five different cases.

A new path planning strategy integrating improved ACO and DWA algorithms for mobile robots in dynamic environments.

This article is concerned with the path planning of mobile robots in dynamic environments. A new path planning strategy is proposed by integrating the improved ant colony optimization (ACO) and dynamic window approach (DWA) algorithms. An improved ACO is developed to produce a globally optimal path for mobile robots in static environments. Through improvements in the initialization of pheromones, heuristic function, and updating of pheromones, the improved ACO can lead to a shorter path with fewer turning points in fewer iterations. Based on the globally optimal path, a modified DWA is presented for the path planning of mobile robots in dynamic environments. By deleting the redundant nodes, optimizing the initial orientation, and improving the evaluation function, the modified DWA can result in a more efficient path for mobile robots to avoid moving obstacles. Some simulations are conducted in different environments, which confirm the effectiveness and superiority of the proposed path planning algorithms.

Stochastic Model Predictive Control with Optimal Linear Feedback for Mobile Robots in Dynamic Environments

Robot navigation around humans can be a challenging problem since human movements are hard to predict. Stochastic model predictive control (MPC) can account for such uncertainties and approximately bound the probability of a collision to take place. In this paper, to counteract the rapidly growing human motion uncertainty over time, we incorporate state feedback in the stochastic MPC. This allows the robot to more closely track reference trajectories. To this end the feedback policy is left as a degree of freedom in the optimal control problem. The stochastic MPC with feedback is validated in simulation experiments and is compared against nominal MPC and stochastic MPC without feedback. The added computation time can be limited by reducing the number of additional variables for the feedback law with a small compromise in control performance.

Model-Free Safety Critical Model Predictive Control for Mobile Robot in Dynamic Environments

Model-Free Safety Critical Model Predictive Control for Mobile Robot in Dynamic Environments

Hierarchical Control Framework for Path Planning of Mobile Robots in Dynamic Environments Through Global Guidance and Reinforcement Learning

Hierarchical Control Framework for Path Planning of Mobile Robots in Dynamic Environments Through Global Guidance and Reinforcement Learning

Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm.

This paper proposes an improved Soft Actor-Critic Long Short-Term Memory (SAC-LSTM) algorithm for fast path planning of mobile robots in dynamic environments. To achieve continuous motion and better decision making by incorporating historical and current states, a long short-term memory network (LSTM) with memory was integrated into the SAC algorithm. To mitigate the memory depreciation issue caused by resetting the LSTM's hidden states to zero during training, a burn-in training method was adopted to boost the performance. Moreover, a prioritized experience replay mechanism was implemented to enhance sampling efficiency and speed up convergence. Based on the SAC-LSTM framework, a motion model for the Turtlebot3 mobile robot was established by designing the state space, action space, reward function, and overall planning process. Three simulation experiments were conducted in obstacle-free, static obstacle, and dynamic obstacle environments using the ROS platform and Gazebo9 software. The results were compared with the SAC algorithm. In all scenarios, the SAC-LSTM algorithm demonstrated a faster convergence rate and a higher path planning success rate, registering a significant 10.5 percentage point improvement in the success rate of reaching the target point in the dynamic obstacle environment. Additionally, the time taken for path planning was shorter, and the planned paths were more concise.

Developing a dynamic obstacle avoidance system for autonomous mobile robots using Bayesian optimization and object tracking: Implementation and testing

This study addresses the challenges faced by autonomous mobile robots in dynamic environments where moving obstacles such as people, cars, and other mobile robots are prevalent. Current motion planning methods for mobile robots focus on stationary obstacles or equivalent static ones at each control moment, resulting in collisions with moving objects or the need for longer distances to avoid them. To address this issue, we propose a system that incorporates dynamic information about moving obstacles into the motion planning decision-making process. We developed an object tracking technique using 2D LiDARs to gather this information and optimized the relevant parameters using Bayesian optimization. The dynamic information of moving obstacles is saved in a costmap representation at each instant, allowing the robot to yield or ignore confronting obstacles based on the prediction of who shall pass first. Simulation and real-world testing demonstrate that our approach saves paths for mobile robots in dynamic environments and prevents them from colliding with dynamic obstacles, improving the practicality of mobile robots working alongside moving humans.

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

AbstractWhen deploying mobile robots in real‐world scenarios, such as airports, train stations, hospitals, and schools, collisions with pedestrians are intolerable and catastrophic. Motion safety becomes one of the most fundamental requirements for mobile robots. However, until now, efficient and safe robot navigation in such dynamic environments is still an open problem. The critical reason is that the inconsistency between navigation efficiency and motion safety is greatly intensified by the high dynamics and uncertainties of pedestrians. To face the challenge, this paper proposes a safe deep reinforcement learning algorithm named Conflict‐Averse Safe Reinforcement Learning (CASRL) for autonomous robot navigation in dynamic environments. Specifically, it first separates the collision avoidance sub‐task from the overall navigation task and maintains a safety critic to evaluate the safety/risk of actions. Later, it constructs two task‐specific but model‐agnostic policy gradients for goal‐reaching and collision avoidance sub‐tasks to eliminate their mutual interference. Then, it further performs a conflict‐averse gradient manipulation to address the inconsistency between two sub‐tasks. Finally, extensive experiments are performed to evaluate the superiority of CASRL. Simulation results show an average 8.2% performance improvement over the vanilla baseline in eight groups of dynamic environments, which is further extended to 13.4% in the most challenging group. Besides, forty real‐world experiments fully illustrated that the CASRL could be successfully deployed on a real robot.

Smooth Autonomous Patrolling for a Differential-Drive Mobile Robot in Dynamic Environments.

Today, mobile robots have a wide range of real-world applications where they can replace or assist humans in many tasks, such as search and rescue, surveillance, patrolling, inspection, environmental monitoring, etc. These tasks usually require a robot to navigate through a dynamic environment with smooth, efficient, and safe motion. In this paper, we propose an online smooth-motion-planning method that generates a smooth, collision-free patrolling trajectory based on clothoid curves. Moreover, the proposed method combines global and local planning methods, which are suitable for changing large environments and enabling efficient path replanning with an arbitrary robot orientation. We propose a method for planning a smoothed path based on the golden ratio wherein a robot's orientation is aligned with a new path that avoids unknown obstacles. The simulation results show that the proposed algorithm reduces the patrolling execution time, path length, and deviation of the tracked trajectory from the patrolling route compared to the original patrolling method without smoothing. Furthermore, the proposed algorithm is suitable for real-time operation due to its computational simplicity, and its performance was validated through the results of an experiment employing a differential-drive mobile robot.

Optimal predictive neuro-navigator design for mobile robot navigation with moving obstacles.

Introduction: The challenge of navigating a Mobile robot in dynamic environments has grasped significant attention in recent years. Despite the available techniques, there is still a need for efficient and reliable approaches that can address the challenges of real-time near optimal navigation and collision avoidance. Methods: This paper proposes a novel Log-concave Model Predictive Controller (MPC) algorithm that addresses these challenges by utilizing a unique formulation of cost functions and dynamic constraints, as well as a convergence criterion based on Lyapunov stability theory. The proposed approach is mapped onto a novel recurrent neural network (RNN) structure and compared with the CVXOPT optimization tool. The key contribution of this study is the combination of neural networks with model predictive controller to solve optimal control problems locally near the robot, which offers several advantages, including computational efficiency and the ability to handle nonlinear and complex systems. Results: The major findings of this study include the successful implementation and evaluation of the proposed algorithm, which outperforms other methods such as RRT, A-Star, and LQ-MPC in terms of reliability and speed. This approach has the potential to facilitate real-time navigation of mobile robots in dynamic environments and ensure a feasible solution for the proposed constrained-optimization problem.

Path planning of mobile robots in dynamic environment based on analytic geometry and cubic Bézier curve with three shape parameters

With the improvement of technology, autonomous mobile robots have become an indispensable part of human life. Since path planning is the crucial part of mobile robots, these paths should be feasible, smooth and collision-free in a certain environment. However, forming desirable paths in a dynamic environment with static and moving obstacles is still a problem in the literature. In this paper, we present a new approach based on the analytic geometry for collision avoidance and the cubic Bézier curve with three shape parameters to obtain global and local paths in a dynamic environment. In this study, the environment for the mobile robot is considered to be a square plane without a grid consisting of several static obstacles in different geometric shapes, a mobile robot, and several dynamic obstacles in the shape of a circle with the same size. Moreover, we expressed the influence of the shape parameters on the optimal path using the objective function defined by the path length. Also, the paths generated by the cubic Bézier curve with three shape parameters and the classical cubic Bézier curve are compared in terms of their path length.

Semantic SLAM for mobile robots in dynamic environments based on visual camera sensors

Visual simultaneous localization and mapping (vSLAM) is inherently constrained by the static world assumption, which renders success in the presence of dynamic objects rather challenging. In this paper, we propose a real-time semantic vSLAM system designed for both indoor and outdoor dynamic environments. By employing object detection, we identify 80 categories and utilize motion consistency checks to pinpoint outliers in each image. Distinct methods are presented for examining the motion states of humans and other objects. For detected humans, an algorithm is introduced to assess whether an individual is seated, subsequently dividing the bounding boxes of seated individuals into two parts based on human body proportions. We then use the same threshold values for standing individuals to determine the states of the two boxes belonging to seated individuals. For non-human objects, we propose an algorithm capable of automatically adjusting the threshold values for different bounding boxes, thereby ensuring consistent detection performance across various objects. Ultimately, we retain points within static boxes contained in dynamic boxes while eliminating other points in dynamic boxes to benefit from a larger number of detected categories. Our SLAM is evaluated on indoor TUM and Bonn RGB-D datasets, with further testing conducted on the outdoor stereo KITTI dataset. The results reveal that our SLAM outperforms most SLAM systems in dynamic environments. Moreover, we test our system in real-world environments with a monocular camera, demonstrating its robustness and universality across diverse settings.

Flexible Behavioral Decision Making of Mobile Robot in Dynamic Environment

How to achieve flexible behavioral decision making in a dynamic environment is an important prerequisite for a mobile robot to execute various tasks. Traditional methods suffer from discontinuous and unstable learning ability. This article brings the mobile robot continuous and stable learning ability in the dynamic environment, through improving the binding manner between the cerebellum and basal ganglia in the neuromodulatory system proposed in our previous work, thus to enhance its flexible behavioral decision-making performance. Moreover, a more biological significance index, i.e., the curiosity index, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Cur$ </tex-math></inline-formula> , is designed to mimic the activity switch of the locus coeruleus between the tonic and phasic mode, to modulate the exploration-exploitation tradeoff in the reinforcement learning (RL) of the basal ganglia. Influence of varying learning rate and varying discount rate on the performance of the RL is also investigated. The experiment results in static and dynamic environments, as well as the comparative experiment in the complex maze environment and the complex dynamic environment, demonstrate the potential of the proposed neuromodulatory system.

D$$^2$$WA: “dynamic” DWA for motion planning of mobile robots in dynamic environments

D$$^2$$WA: “dynamic” DWA for motion planning of mobile robots in dynamic environments

Motion Saliency-Based Collision Avoidance for Mobile Robots in Dynamic Environments

Obstacle avoidance is a sizable challenge for robots working in a multiple dynamic obstacle environment. Conventional obstacle avoidance methods often require complex calculations to process all dynamic obstacles detected in the scene. Avoiding dangerous moving objects in time is often difficult when obstacles are many. In this article, we propose a robot obstacle avoidance method based on motion saliency for dynamic environments. First, we use segmented dynamic objects to calculate the saliency of dynamic objects and segment dangerous dynamic objects. Then, we use the B-spline curve to predict the movement of dangerous dynamic objects and combine it with the nonlinear model predictive control method to avoid dangerous obstacles in the dynamic environment of the robot. Considering the motion behavior of different dynamic objects, our obstacle avoidance strategy is to generate obstacle-free paths by adjusting the speed of the robot or connecting the centers of the rolling variable-size circles. Finally, we conduct a series of experiments to verify the effectiveness of our method.

Deep Learning-Based NMPC for Local Motion Planning of Last-Mile Delivery Robot.

Feasible local motion planning for autonomous mobile robots in dynamic environments requires predicting how the scene evolves. Conventional navigation stakes rely on a local map to represent how a dynamic scene changes over time. However, these navigation stakes depend highly on the accuracy of the environmental map and the number of obstacles. This study uses semantic segmentation-based drivable area estimation as an alternative representation to assist with local motion planning. Notably, a realistic 3D simulator based on an Unreal Engine was created to generate a synthetic dataset under different weather conditions. A transfer learning technique was used to train the encoder-decoder model to segment free space from the occupied sidewalk environment. The local planner uses a nonlinear model predictive control (NMPC) scheme that inputs the estimated drivable space, the state of the robot, and a global plan to produce safe velocity commands that minimize the tracking cost and actuator effort while avoiding collisions with dynamic and static obstacles. The proposed approach achieves zero-shot transfer from a simulation to real-world environments that have never been experienced during training. Several intensive experiments were conducted and compared with the dynamic window approach (DWA) to demonstrate the effectiveness of our system in dynamic sidewalk environments.

Infusing Model Predictive Control Into Meta-Reinforcement Learning for Mobile Robots in Dynamic Environments

The successful operation of mobile robots requires them to adapt rapidly to environmental changes. To develop an adaptive decision-making tool for mobile robots, we propose a novel algorithm that combines meta-reinforcement learning (meta-RL) with model predictive control (MPC). Our method employs an off-policy meta-RL algorithm as a baseline to train a policy using transition samples generated by MPC when the robot detects certain events that can be effectively handled by MPC, with its explicit use of robot dynamics. The key idea of our method is to switch between the meta-learned policy and the MPC controller in a randomized and event-triggered fashion to make up for suboptimal MPC actions caused by the limited prediction horizon. During meta-testing, the MPC module is deactivated to significantly reduce computation time in motion control. We further propose an online adaptation scheme that enables the robot to infer and adapt to a new task within a single trajectory. The performance of our method has been demonstrated through simulations using a nonlinear car-like vehicle model with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(i)$</tex-math></inline-formula> synthetic movements of obstacles, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(ii)$</tex-math></inline-formula> real-world pedestrian motion data. The simulation results indicate that our method outperforms other algorithms in terms of learning efficiency and navigation quality.