Mobile Robot In Unknown Environment Research Articles

Design and Experimental Validation of Deep Reinforcement Learning-Based Fast Trajectory Planning and Control for Mobile Robot in Unknown Environment.

This article is concerned with the problem of planning optimal maneuver trajectories and guiding the mobile robot toward target positions in uncertain environments for exploration purposes. A hierarchical deep learning-based control framework is proposed which consists of an upper level motion planning layer and a lower level waypoint tracking layer. In the motion planning phase, a recurrent deep neural network (RDNN)-based algorithm is adopted to predict the optimal maneuver profiles for the mobile robot. This approach is built upon a recently proposed idea of using deep neural networks (DNNs) to approximate the optimal motion trajectories, which has been validated that a fast approximation performance can be achieved. To further enhance the network prediction performance, a recurrent network model capable of fully exploiting the inherent relationship between preoptimized system state and control pairs is advocated. In the lower level, a deep reinforcement learning (DRL)-based collision-free control algorithm is established to achieve the waypoint tracking task in an uncertain environment (e.g., the existence of unexpected obstacles). Since this approach allows the control policy to directly learn from human demonstration data, the time required by the training process can be significantly reduced. Moreover, a noisy prioritized experience replay (PER) algorithm is proposed to improve the exploring rate of control policy. The effectiveness of applying the proposed deep learning-based control is validated by executing a number of simulation and experimental case studies. The simulation result shows that the proposed DRL method outperforms the vanilla PER algorithm in terms of training speed. Experimental videos are also uploaded, and the corresponding results confirm that the proposed strategy is able to fulfill the autonomous exploration mission with improved motion planning performance, enhanced collision avoidance ability, and less training time.

Autonomous navigation of mobile robots in unknown environments using off-policy reinforcement learning with curriculum learning

Reinforcement learning (RL) is effective for autonomous navigation tasks without prior knowledge of the environment. However, traditional mobile robot navigation algorithms, based on off-policy RL, often face challenges such as low sample efficiency during training and lack of adequate safety mechanisms. In this paper, we present an off-policy RL navigation model named Soft Actor-Critic with Curriculum Prioritization and Fuzzy Logic (SCF). The model uses energy as a prioritized evaluation metric for experience replay. And through task-level curriculum, the agent’s learning sequence is formulated, thereby enhancing sampling efficiency and safety. We propose a Curriculum-based Energy Prioritization (CEP) approach. It selects a replay trajectory that matches the current agent’s capability based on trajectory energy. Our results show that robots using off-policy RL often have limitations in dynamic obstacle avoidance. To rectify this, our model uses a fuzzy logic controller to enhance real-time obstacle avoidance. The SCF approach enables mobile robots to navigate adeptly in unpredictable and dynamic environments, ensuring optimal planning control while being safe and robust. Experiments in Gazebo simulation environment and real world confirm the effectiveness of our proposed method. The comparison results show the superior performance of this method, especially in unknown and dynamic environments.

A Novel Approach for Target Attraction and Obstacle Avoidance of a Mobile Robot in Unknown Environments Using a Customized Spiking Neural Network

In recent years, implementing reinforcement learning in autonomous mobile robots (AMRs) has become challenging. Traditional methods face complex trials, long convergence times, and high computational requirements. This paper introduces an innovative strategy using a customized spiking neural network (SNN) for autonomous learning and control of mobile robots (AMR) in unknown environments. The model combines spike-timing-dependent plasticity (STDP) with dopamine modulation for learning. It utilizes the Izhikevich neuron model, leading to biologically inspired and computationally efficient control systems that adapt to changing environments. The performance of the model is evaluated in a simulated environment, replicating real-world scenarios with obstacles. In the initial training phase, the model faces significant challenges. Integrating brain-inspired learning, dopamine, and the Izhikevich neuron model adds complexity. The model achieves an accuracy rate of 33% in reaching its target during this phase. Collisions with obstacles occur 67% of the time, indicating the struggle of the model to adapt to complex obstacles. However, the model’s performance improves as the study progresses to the testing phase after the robot has learned. Its accuracy surges to 94% when reaching the target, and collisions with obstacles reduce it to 6%. This shift demonstrates the adaptability and problem-solving capabilities of the model in the simulated environment, making it more competent for real-world applications.

A YOLO-GGCNN based grasping framework for mobile robots in unknown environments

A YOLO-GGCNN based grasping framework for mobile robots in unknown environments

Immune deep reinforcement learning-based path planning for mobile robot in unknown environment

A new deep deterministic policy gradient (DDPG) integrating kinematics analysis and immune optimization (KAI-DDPG) is proposed to address the drawbacks of DDPG in path planning. An orientation angle reward component, linear velocity reward factor, and safety performance reward factor are added to the DDPG reward function based on kinematic modeling and analysis of mobile robots. A multi-objective performance index turns the path planning problem into one of multi-objective optimization. We propose KA-DDPG, which uses the orientation angle, linear speed, and safety degree as evaluation indices, and information entropy to alter the influence coefficient of the multi-objective function in the reward function. KAI-DDPG is proposed to address the low learning and training efficiency of KA-DDPG, using immune optimization to optimize the experience samples in the experience buffer pool. Performance indices of traditional path planning and the proposed techniques are compared on a gazebo simulation platform, and the results suggest that KAI-DDPG can mitigate the drawbacks of DDPG, such as a protracted training cycle and poor path planning technique, and can broaden the range of application.

Online path planning of cooperative mobile robots in unknown environments using improved Q‐Learning and adaptive artificial potential field

Online path planning in unknown environments with obstacles is a challenging issue in mobile robot systems. Cooperative mobile robots have received much attention in recent years due to their robustness and ability to perform more complex tasks and operations faster. In this article, by unifying the Dyna Q-learning and a novel concept (introduced as a feature matrix) an improved Q-learning is proposed that accelerates the learning process even with poor choice of parameters. In addition, to overcoming the local minima, an adaptive artificial potential field method is developed, so that two parameters of the new concept named the virtual obstacle are controlled using the massachusetts institute of technology (MIT) method to navigate the mobile robot in the proper direction. Then, with proper utilizing of the improved Q-learning and adaptive artificial potential field method, the path planning is performed online effectively while target tracking and collision avoidance are guaranteed. Finally, the performance of the proposed method is tested on decentralized cooperative mobile robots. The simulation results showed optimal path planning in terms of the distance in an unknown environment and collision avoidance with the obstacles. In addition, getting out of the local minima (i.e. immobility) is guaranteed in all situations.

Radio SLAM: A Review on Radio-Based Simultaneous Localization and Mapping

Simultaneous localization and mapping (SLAM) algorithm has enabled the automation of mobile robots in unknown environments. It enables the robot to navigate through an unknown trajectory by employing sensors that provide measurements to infer the surrounding environment and use this information to localize the robot. Sensor technology plays a key role in defining the quality of measurements as it affects the overall performance of SLAM. While visual sensors, like cameras, can capture rich features of the environment, they, however, fail to work in low-light conditions. On the other hand, radio frequency sensors are invariant to light conditions, however, high-frequency signals such as millimeter wave (mm-wave) are prone to severe channel attenuation, therefore, they are suitable for short-range indoor applications. Despite the high localization accuracy that the mm-wave frequency band has to offer, its shortcomings have limited the amount of research work carried out to enhance the performance of SLAM. Therefore, this paper aims to provide an overview of the recent developments in radio SLAM, with a specific focus on mm-wave enabled localization and SLAM methods. However, some notable research work based on other radio frequency sensors has also been discussed. In addition, we highlight the role of deep learning-based methods for localization and identify some of the key challenges in data-driven implementation.

Deep deterministic policy gradient reinforcement learning for collision-free navigation of mobile robots in unknown environments

Deep deterministic policy gradient reinforcement learning for collision-free navigation of mobile robots in unknown environments

Position Detection of Doors and Windows Based on DSPP-YOLO

Autonomous exploration of autonomous mobile robots in unknown environments is a hot topic at present. Object detection is an important research direction in improving the autonomous capability of autonomous mobile robots in unknown environments. In object detection, doors and windows have many similar features and are difficult to distinguish. Therefore, improving the detection accuracy of doors and windows is helpful to improve the autonomous ability of autonomous mobile robots. Aiming at the problem of insufficient doors and windows detection accuracy caused by the large difference between the receptive fields of doors and windows, this paper proposes DSPP-YOLO (DenseNet SPP) algorithm. Firstly, on the basis of deepening the network addition, to prevent the loss of shallow location feature information, some residual blocks in YOLOV3 are improved to dense blocks by using the idea of DenseNet. Secondly, the spatial pyramid pooling (SPP) structure is fused into the YOLOV3 feature extraction network to realize multiscale receptive field fusion. Finally, K-means ++ algorithm is used to re-cluster the size of candidate boxes to reduce the error caused by candidate boxes. DSPP-YOLO realizes the position detection of doors and windows by an autonomous robot in an unknown, complex environment. This method is tested. Under the condition of the same data set, the detection accuracy of the DSPP-YOLO algorithm is 77.4% for doors and 38.1% for windows. Compared with YOLOV3 algorithm, the calculation consumption time of the DSPP-YOLO algorithm does not increase, and the detection accuracy of doors is improved by 3.3%, the detection accuracy of windows is improved by 8.8%, and the average accuracy of various types is improved by 6.05%.

An improved beetle antennae search path planning algorithm for vehicles.

With the development of society, the application of mobile robots in industry and life is increasingly extensive, and the local path planning of mobile robots in unknown environments is a problem that needs to be solved. Aiming at the problem that the traditional beetle antennae search (BAS) algorithm can easily fall into local optimum and the optimization accuracy is low, we propose an improved beetle antennae search. It introduces a map safety threshold, the addition of virtual target points, and the smoothing of the path. Map safety threshold means extra space with obstacles at all times, improving path reliability by avoiding collisions. Adding virtual target points reduces situations where the vehicle gets stuck in local optima. The B-spline smoothing path reduces the original path’s straight turns to improve the path’s robustness. The effectiveness and superiority of the algorithm are verified by comparing and testing the existing path planning algorithms through simulation in different environments.

Autonomous Exploration of Unknown Indoor Environments for High-Quality Mapping Using Feature-Based RGB-D SLAM

Simultaneous localization and mapping (SLAM) system-based indoor mapping using autonomous mobile robots in unknown environments is crucial for many applications, such as rescue scenarios, utility tunnel monitoring, and indoor 3D modeling. Researchers have proposed various strategies to obtain full coverage while minimizing exploration time; however, mapping quality factors have not been considered. In fact, mapping quality plays a pivotal role in 3D modeling, especially when using low-cost sensors in challenging indoor scenarios. This study proposes a novel exploration algorithm to simultaneously optimize exploration time and mapping quality using a low-cost RGB-D camera. Feature-based RGB-D SLAM is utilized due to its various advantages, such as low computational cost and dense real-time reconstruction ability. Subsequently, our novel exploration strategies consider the mapping quality factors of the RGB-D SLAM system. Exploration time optimization factors are also considered to set a new optimum goal. Furthermore, a Voronoi path planner is adopted for reliable, maximal obstacle clearance and fixed paths. According to the texture level, three exploration strategies are evaluated in three real-world environments. We achieve a significant enhancement in mapping quality and exploration time using our proposed exploration strategies compared to the baseline frontier-based exploration, particularly in a low-texture environment.

A Path-Planning Approach Based on Potential and Dynamic Q-Learning for Mobile Robots in Unknown Environment.

The path-planning approach plays an important role in determining how long the mobile robots can travel. To solve the path-planning problem of mobile robots in an unknown environment, a potential and dynamic Q-learning (PDQL) approach is proposed, which combines Q-learning with the artificial potential field and dynamic reward function to generate a feasible path. The proposed algorithm has a significant improvement in computing time and convergence speed compared to its classical counterpart. Experiments undertaken on simulated maps confirm that the PDQL when used for the path-planning problem of mobile robots in an unknown environment outperforms the state-of-the-art algorithms with respect to two metrics: path length and turning angle. The simulation results show the effectiveness and practicality of the proposal for mobile robot path planning.

An Adaptive Threshold Line Segment Feature Extraction Algorithm for Laser Radar Scanning Environments

An accurate map is needed for the autonomous navigation of mobile robots in unknown environments. The application of laser radars has the advantages of high ranging accuracy and long ranging distances. Due to the small amount of data on laser radars and the influence of noise on the sensor itself, these amount to causing problems such as low accuracies of map construction and large positioning errors. Currently, the feature extraction of environmental line segments based on radar scanning data generally adopts the idea of recursion. However, the amount of calculations for applying recursion is large, and the threshold of extracted feature points needs to be set manually. Moreover, the fixed segmentation threshold will cause under-segmentation or over-segmentation. In this paper, an adaptive threshold-based feature extraction method for environmental line segments is proposed. The method denoises the original data first, and then an adaptive threshold of the nearest neighbor algorithm is provided to improve the accuracy of breakpoint judgment; next, the slope difference between adjacent line segments is evaluated according to the line segment fitting error in order to obtain the optimal corner feature. Finally, the point set is segmented to fit line-segment features. Based on actual environment tests, the environmental similarity of the line segment features extracted by the new algorithm in this paper increases by 8.3% compared with the IEPF (Iterative End Point Fit) algorithm. The algorithm avoids recursive operations, improves the efficiency by four times, and meets the real-time requirements of line segment fitting.

AEGQ: A Novel Autonomous Exploration Algorithm for Mobile Robot in Unknown Environments Using Deep Learning

AEGQ: A Novel Autonomous Exploration Algorithm for Mobile Robot in Unknown Environments Using Deep Learning

Immune Deep Reinforcement Learning Based Path Planning for Mobile Robot in Unknown Environment

Immune Deep Reinforcement Learning Based Path Planning for Mobile Robot in Unknown Environment

Research on obstacle avoidance path planning of electrical control robot in unknown environment

Aiming at the problems of robot path planning with artificial potential field method, such as target unreachable and local minimum points, an improved artificial potential field method model is proposed. The obstacles in unknown environment are designed in the form of grid map, so that the robot can avoid obstacles and move towards the target through perception. Matlab GUI is used to set up a two-dimensional coordinate system environment including obstacles and target points, and the optimal trajectory movement simulation experiment is carried out from the initial position to the desired position in the plane map. Experiments show that the improved artificial potential field method can make the mobile robot navigate in unknown environment, avoid obstacles and find the appropriate path, so as to achieve the requirement of collision free. The simulation results are close to the expected results, which shows that this method can effectively improve the feasibility of path planning and the effectiveness of obstacle avoidance of mobile robot in unknown environment.

Behavioral Decision-Making of Mobile Robot in Unknown Environment with the Cognitive Transfer

Behavioral Decision-Making of Mobile Robot in Unknown Environment with the Cognitive Transfer

Reactive Navigation Framework for Mobile Robots by Heuristically Evaluated Pre-sampled Trajectories

This paper describes and analyzes a reactive navigation framework for mobile robots in unknown environments. The approach does not rely on a global map and only considers the local occupancy in its robot-centered 3D grid structure. The proposed algorithm enables fast navigation by heuristic evaluations of pre-sampled trajectories on-the-fly. At each cycle, these paths are evaluated by a weighted cost function, based on heuristic features such as closeness to the goal, previously selected trajectories, and nearby obstacles. This paper introduces a systematic method to calculate a feasible pose on the selected trajectory, before sending it to the controller for the motion execution. Defining the structures in the framework and providing the implementation details, the paper also explains how to adjust its offline and online parameters. To demonstrate the versatility and adaptability of the algorithm in unknown environments, physics-based simulations on various maps are presented. Benchmark tests show the superior performance of the proposed algorithm over its previous iteration and another state-of-art method. The open-source implementation of the algorithm and the benchmark data can be found at https://github.com/RIVeR-Lab/tentabot.

A Behavior-Based Mobile Robot Navigation Method with Deep Reinforcement Learning

Deep reinforcement learning-based mobile robot navigation has attracted some recent interest. In the single-agent case, a robot can learn to navigate autonomously without a map of the environment. In the multi-agent case, robots can learn to avoid collisions with each other. In this work, we propose a behavior-based mobile robot navigation method which directly maps the raw sensor data and goal information to the control command. The learned navigation policy can be applied in both single-agent and multi-agent scenarios. Two basic navigation behaviors are considered in our method, which are goal reaching and collision avoidance. The two behaviors are fused based on the risk-level estimation of the current state. The navigation task is decomposed using the behavior-based framework, which is capable of reducing the complexity of the learning process. The simulations and real-world experiments demonstrate that the proposed method can enable the collision-free autonomous navigation of multiple mobile robots in unknown environments.

An adaptive behavior decision model of mobile robot based on the neuromodulation

How to effectively improve the behavior decision ability in unknown environment is a great challenge for mobile robots. Traditional methods suffer from the low efficiency and large computation load. Motivated by the regulation effect of serotonin and dopamine on human behavior, a novel adaptive behavior decision model for mobile robot is proposed in this work. This model integrates two kinds of adjustment factors that mimic the effects of two neuromodulators, serotonin and dopamine, to regulate the behavior decision of a mobile robot in unknown environment. Adjustment factor $$\mathbf {p}$$ is designed to simulate the function of the serotonin and make the robot avoid obstacles effectively, while the adjustment factor $$\mathbf {r}$$ is designed to simulate the function of the dopamine and make the robot approach the target quickly. Both of the adjustment functions can be adaptively regulated with the robot’s movement. Static and dynamic simulation results show that this neuromodulatory model has excellent collision avoidance capacity which can avoid the obstacles fluently, and the trajectory obtained tends to be optimal. Moreover, it is relatively simple, thus can effectively reduce the computation time and the load capacity, and greatly improve the running efficiency of the mobile robot in unknown environment.