Algorithms In Dynamic Environments Research Articles

Path planning of mobile robot based on improved TD3 algorithm in dynamic environment

This paper proposes an improved TD3 (Twin Delayed Deep Deterministic Policy Gradient) algorithm to address the flaws of low success rate and slow training speed, when using the original TD3 algorithm in mobile robot path planning in dynamic environment. Firstly, prioritized experience replay and transfer learning are introduced to enhance the learning efficiency, where the probability of beneficial experiences being sampled in the experience pool is increased, and the pre-trained model is applied in an obstacle-free environment as the initial model for training in a dynamic environment. Secondly, dynamic delay update strategy is devised and OU noise is added to improve the success rate of path planning, where the probability of missing high-quality value estimate is reduced through changing the delay update interval dynamically, and the correlated exploration of the mobile robot inertial navigation system in the dynamic environment is temporally improved. The algorithm is tested by simulation where the Turtlebot3 robot model as a training object, the ROS melodic operating system and Gazebo simulation software as an experimental environment. Meanwhile, the result shows that the improved TD3 algorithm has a 16.6 % increase in success rate and a 23.5 % reduction in algorithm training time. A generalization experiment was designed finally, and it indicates that superior generation performance has been acquired in mobile robot path planning with continuous action spaces through the improved TD3 algorithm.

DSOMF: A Dynamic Environment Simultaneous Localization and Mapping Technique Based on Machine Learning.

To address the challenges of reduced localization accuracy and incomplete map construction demonstrated using classical semantic simultaneous localization and mapping (SLAM) algorithms in dynamic environments, this study introduces a dynamic scene SLAM technique that builds upon direct sparse odometry (DSO) and incorporates instance segmentation and video completion algorithms. While prioritizing the algorithm's real-time performance, we leverage the rapid matching capabilities of Direct Sparse Odometry (DSO) to link identical dynamic objects in consecutive frames. This association is achieved through merging semantic and geometric data, thereby enhancing the matching accuracy during image tracking through the inclusion of semantic probability. Furthermore, we incorporate a loop closure module based on video inpainting algorithms into our mapping thread. This allows our algorithm to rely on the completed static background for loop closure detection, further enhancing the localization accuracy of our algorithm. The efficacy of this approach is validated using the TUM and KITTI public datasets and the unmanned platform experiment. Experimental results show that, in various dynamic scenes, our method achieves an improvement exceeding 85% in terms of localization accuracy compared with the DSO system.

An improved ant colony algorithm based on Q-Learning for route planning of autonomous vehicle

In view of the problems existing in the path planning algorithms of unmanned vehicles, such as low search efficiency, slow convergence speed and easy to fall into the local optimal. Based on the characteristics of route planning for unmanned vehicles, this paper introduces Q-Learning into the traditional ant colony algorithm to enhance the learning ability of the algorithm in dynamic environment, so as to improve the overall efficiency of route search. By mapping pheromones into Q values in Q-learning, rapid search in complex environments is realized, and a collection-free path satisfying constraints is quickly found. The results of case analysis show that compared with the traditional ant colony algorithm and the improved ant colony algorithm considering reward and punishment factors, the improved ant colony algorithm based on Q-Learning can effectively reduce the number of iterations, shorten the path optimization time and path length and other performance indicators, and has many advantages in jumping out of the local optimal, improving the global search ability and improving the convergence speed, and has good adaptability and robustness in complex environments. It ensures the safety and stability of unmanned vehicles in complex environments.

Bi-AM-RRT*: A Fast and Efficient Sampling-Based Motion Planning Algorithm in Dynamic Environments

Bi-AM-RRT*: A Fast and Efficient Sampling-Based Motion Planning Algorithm in Dynamic Environments

Incremental updating fuzzy tolerance rough set approach in intuitionistic fuzzy information systems with fuzzy decision

As information technology develops rapidly and data is constantly updated, efficient mining knowledge from dynamic intuitionistic fuzzy information systems with fuzzy decision (IFFD) is a significant topic. In the dynamic data environment, the corresponding IFFD is always changing with time when object sets of datasets may evolve in time. Fuzzy tolerance rough set (FTRS), as one of extended rough set models, has a strong capacity for the expression of information and better representation of uncertainty. Therefore, it is very effective and necessary to use FTRS to acquire valuable knowledge from dynamic IFFD. In this paper, we investigate the dynamic fuzzy tolerance rough set approach for IFFD. Firstly, a new fuzzy tolerance similarity is defined to describe fuzzy tolerance relation between objects in the IFFD. Second, we construct a fuzzy tolerance rough set model in the IFFD, discuss some properties, and propose the corresponding static algorithm. Subsequently, incremental approaches which update fuzzy tolerance rough approximations with the insertion and deletion of objects in the IFFD are investigated and the corresponding dynamic algorithms are also designed. Finally, the feasibility of the FTRS model and the effectiveness and efficiency of the dynamic algorithm in dynamic environments are validated through a series of comparative experiments on nine datasets.

YDD-SLAM: Indoor Dynamic Visual SLAM Fusing YOLOv5 with Depth Information

Simultaneous location and mapping (SLAM) technology is key in robot autonomous navigation. Most visual SLAM (VSLAM) algorithms for dynamic environments cannot achieve sufficient positioning accuracy and real-time performance simultaneously. When the dynamic object proportion is too high, the VSLAM algorithm will collapse. To solve the above problems, this paper proposes an indoor dynamic VSLAM algorithm called YDD-SLAM based on ORB-SLAM3, which introduces the YOLOv5 object detection algorithm and integrates deep information. Firstly, the objects detected by YOLOv5 are divided into eight subcategories according to their motion characteristics and depth values. Secondly, the depth ranges of the dynamic object and potentially dynamic object in the moving state in the scene are calculated. Simultaneously, the depth value of the feature point in the detection box is compared with that of the feature point in the detection box to determine whether the point is a dynamic feature point; if it is, the dynamic feature point is eliminated. Further, multiple feature point optimization strategies were developed for VSLAM in dynamic environments. A public data set and an actual dynamic scenario were used for testing. The accuracy of the proposed algorithm was significantly improved compared to that of ORB-SLAM3. This work provides a theoretical foundation for the practical application of a dynamic VSLAM algorithm.

Finding Robotic Wide Free Space Path in Dynamic Environment by Improving MAX–MIN Ant System Algorithm

— This paper proposes a new obstacle avoidance method for finding an optimal path passing through wide free space based on the optimized MAX-MIN Ant System (MMAS) algorithm in dynamic environments. The proposed improvement of the MMAS algorithm occurs in two stages. The first stage analyses the environment with two modifications by proposing a new relation of pheromone trail updating to construct the consequence modified (deposited) pheromone trail update in each iteration based on adding a new parameter (clean) to calculate the possible wide empty locations. Another alteration at this level involves checking and identifying the four horizontal and vertical states. Suppose the two nodes on either side of the four diagonal nodes are barriers in the next neighbor locations. In that case, the Tight Tunnel state is then attained, and it is managed by sealing this location to prevent passing through it before the robot movement stage. Another alteration is made to the robot-moving stage by recommending a new relation of tour building probabilities to determine the best option for moving the robot from the start node through a dynamic environment that incorporates dynamic obstacles moving through free space by locating and exhibiting the ideal path via broad free space. The main outcome of the experiment simulation of the new method involves that the robot avoids going through narrow tunnels and chooses the best course through open areas to reach its destination without running into any impediments in various dynamic environments conditions. In addition, the comparison of average time occupied to find the paths by the new method reduces to 54.32% as compared with that required with the method of hybrid the basic D* algorithm with the Lbest PSO algorithm, and to 11.95% in comparison to the time occupied recorded when applying the method of the improved MAX–MIN ACO algorithm.

A Novel Optimistic Local Path Planner: Agoraphilic Navigation Algorithm in Dynamic Environment

This paper presents a novel local path planning algorithm developed based on the new free space attraction (Agoraphilic) concept. The proposed algorithm is capable of navigating robots in unknown static, as well as dynamically cluttered environments. Unlike the other navigation algorithms, the proposed algorithm takes the optimistic approach of the navigation problem. It does not look for problems to avoid, but rather for solutions to follow. This human-like decision-making behaviour distinguishes the new algorithm from all the other navigation algorithms. Furthermore, the new algorithm utilises newly developed tracking and prediction algorithms, to safely navigate mobile robots. This is further supported by a fuzzy logic controller designed to efficiently account for the inherent high uncertainties in the robot’s operational environment at a reduced computational cost. This paper also includes physical experimental results combined with bench-marking against other recent methods. The reported results verify the algorithm’s successful advantages in navigating robots in both static and dynamic environments.

A robust RGB-D SLAM based on multiple geometric features and semantic segmentation in dynamic environments

In indoor dynamic scenes, traditional visual simultaneous localization and mapping (SLAM) algorithms based on RGB-D cameras incorrectly use dynamic features to estimate the poses of the cameras, and do not fully utilize the geometric information in the scenes, resulting in the low positioning accuracy and robustness of SLAM systems. To solve this problem, this paper proposes an RGB-D SLAM algorithm based on multiple geometric features and semantic segmentation. The core of our SLAM system is the proposed robust exclusion method of dynamic point and line features. The method consists of the following three steps: (a) identify potential dynamic point features using motion consistency checks; (b) obtain potential motion regions via semantic segmentation, and then determine dynamic regions by combining these with dynamic point features; (c) remove point and line features in dynamic regions. The exclusion method of dynamic point and line features can be easily integrated into RGB-D SLAM systems for improving the accuracy and robustness of SLAM systems in dynamic scenes. Experimental results on the Technische Universität München (TUM) dataset demonstrate that the proposed algorithm has better positioning accuracy and stability than the original dynamic semantic (DS-SLAM) algorithm in dynamic environments. The effectiveness of the proposed algorithm is verified by comparison with other classical visual SLAM algorithms. Better mapping performance is achieved by this proposed algorithm in actual indoor scenes.

Ensemble methods based on characterization of dynamism for dynamic multi-objective optimization

Dynamic multi-objective optimization problems involve multiple conflicting and time-dependent objectives that change continuously depending on the environment. Therefore, effectively tracking the movement of the Pareto-optimal front and the Pareto-optimal set (PS) using a single strategy is difficult. In this study, we propose an ensemble method called EMCD, which is based on the characterization of dynamism. EMCD selects different strategies to generate new populations depending on the characteristics of environmental changes. In our proposed algorithm, historical information is used to locate new populations when the current and historical environmental changes are similar. Otherwise, EMCD establishes whether the PS has changed. If the PS changes, individuals from the population of the new environment are generated by using a set of knee points based on the two previous consecutive populations. If not, new populations are generated by using the perturbation-based approach. In addition, we use an adaptive diversity introduction strategy to ensure the solvability of our proposed algorithm in dynamic environments. To verify the effectiveness of EMCD, we compare its performance to that of three state-of-the-art algorithms on fourteen benchmark problems. The experimental results show that EMCD outperforms the comparison algorithms on most of the benchmark problems.

Reinforcement-Learning-Aided Safe Planning for Aerial Robots to Collect Data in Dynamic Environments

We study the data collection problem in an Internet of Things (IoT) network where an unmanned aerial vehicle (UAV) is utilized to aggregate data from a set of IoT devices. We formulate the scheduling and path planning problems for the UAV. The goal of the scheduling problem is to find the sequence of nodes that the UAV will visit to complete the data collection task in the shortest possible time, ensuring that it does not run out of energy during its mission. We express this problem as a mixed-integer nonlinear problem and propose an efficient algorithm to solve the aforementioned NP-hard problem in polynomial time. Path planning problem aims to find a collision-free path for the UAV. While the state-of-the-art schemes have focused on solving the path planning problem in static environments, we study the problem in a dynamic environment with moving obstacles. We develop an algorithm that works on both static and dynamic environments. Our method combines deep reinforcement learning (RL) with graph-based global path planning algorithms to find a collision-free path for the UAV. One important advantage of our RL-based method over the existing studies is its map independency, which allows us to transform the agent’s learning from one environment to another. Via simulation studies, we show that our method is significantly effective in improving the safety of the path planning algorithms in dynamic environments.

The Adaptive Fingerprint Localization in Dynamic Environment

Indoor localization service is an indispensable part of modern intelligent life, among which Wi-Fi based fingerprint localization system is popular in indoor positioning researches due to its advantages of low cost and widely deployment. However, Wi-Fi based localization system is susceptible to dynamic environment, and fingerprint collection and updating are time-consuming and labor-intensive. To address this problem, we propose a novel positioning framework based on multiple transfer learning fusion using Generalized Policy Iteration (GPI). Firstly, a 1-Dimension Convolutional Autoencoder (1-D CAE) is designed to extract features from one-dimensional fingerprint data; similar to Convolutional Neural Network (CNN), it can not only pay more attention to the information of different dimensions of fingerprints, but also compress redundant information and reduce noise. After that, Domain Adversarial Neural Network (DANN) and Passive Aggressive (PA) algorithm are fused to train localization model based on unlabeled fingerprint of target domain using the theory of GPI in offline stage. Finally, the model is fine-tuned with unlabeled fingerprints and few labeled fingerprints in daily online predictions to improve the performance of the localization system. Various evaluations in five typical scenarios validate the effectiveness of proposed algorithm in dynamic environment, with low tendency, easy recalibration, long-term stabilization high accuracy and so on.

Setting Method of Downsampling Factor and Grid Factor for NDT Relocation Algorithm in Dynamic Environment

Normal Distribution Transform (NDT) algorithm plays the role of detecting the relative pose of the vehicle in high-precision map. In this paper, a method of setting downsampling factor and grid factor for NDT relocation algorithm in dynamic environment is proposed, which can solve the problems of excessive NDT relocation error and location loss caused by dynamic objects accounting for 1 % to 35 % of the volume of scanning point cloud in vehicle environment. To simulate a real dynamic point cloud environment, the single-frame LiDAR point cloud space is voxelized into a mesh. Each grid is assigned a random number evenly distributed between 0 and 1. The threshold value for whether to add a Gaussian noise point is also set. Seven representative dynamic objects on the highway are selected. The threshold value of probability distribution function of Gaussian noise object needs to be set. Then the volume content of the dynamic object in the single frame point cloud space is calculated according to the set threshold value by using the definite integral. By changing the content and volume of dynamic obstacles in a dynamic environment, the effects of the downsampling factor and the grid factor on the accuracy of the repositioning trajectory are obtained. The resampling and mesh coefficients are optimized based on the analysis of the repositioning trajectory accuracy. The results show that When the current sampling factor is fixed, the grid factor of the NDT algorithm is proportional to the RMSE factor. When the NDT grid factor is fixed and the down sampling factor is equal to the side length of the obstacle, the NDT relocation accuracy is the highest and reaches the local optimum.

Any-time randomized kinodynamic path planning algorithm in dynamic environments with application to quadrotor

Kinodynamic path planning is an open challenge in unmanned autonomous vehicles and is considered an NP-Hard problem. Planning a feasible path for vertical take-off and landing quadrotor (VTOL-Q) from an initial state to a target state in 3D space by considering the environmental constraints such as moving obstacles avoidance and non-holonomic constraints such as hard bounds of VTOL-Q is the key motivation of this study. To this end, let us propose the any-time randomized kinodynamic (ATRK) path-planning algorithm applicable in the VTOL-Q. ATRK path-planning algorithm is based on the Rapidly-exploring random trees (RRT) and consists of three main components: high-level, mid-level, and low-level controller. The high-level controller utilizes a randomized sampling-based approach to generate offspring vertices for rapid exploring and expanding in the configuration space. The mid-level controller uses the any-time method to avoid collision with moving obstacles. The low-level controller with a six-DOF dynamic model accounts for the kinodynamic constraints of VTOL-Q in the randomized offspring vertices to plan a feasible path. Simulation results on three different test-scenario demonstrate the kinodynamic constraints of the VTOL-Q are integrated into the randomized offspring vertices. Also, in presence of moving obstacles, the ATRK re-plans the path in the local area as through an any-time approach.

An adaptively reversed diffusion dual-drive evolutionary algorithm in dynamic environments for intelligence prediction

Prediction problems are difficult to be carried out in a dynamic environment, for two key questions: one is how to monitor environmental changes, the other is how to make respond timely after the environment has changed. Multi-population strategy is often adopted to address both two key problems. However, there are two stubborn questions that limit and affect the effectiveness of the strategy, i.e., (1) Search overlaps are easy to occur which lead to lose the ability of local exploit, (2) In the course of evolution, the search range of subpopulations tends to assimilate, so the subpopulation will gradually lose the ability to explore whole search space. Therefore, this paper adopts multi-population strategy, and presents a novel intelligence algorithm based on particle swarm optimization to solve the above problem, called adaptively reversed diffusion dual-drive evolutionary algorithm (ARDDEA). (1) Firstly, ARDDEA monitors the environmental changes by setting the global dynamic sentry in each subgroup. (2) Secondly, in order to avoid searching overlapping of the sub-population, a new exclusion strategy is proposed in this paper. A new distance determination method, i.e., between-swarms average Mahalanobis​ distance, is devised in the exclusion strategy to decide the inter-population distance. If the distance is too small between two sub-populations, furtherly, a Hill–valley decision function is used to determine whether they tracked the same peak or not. If so, the inferior subpopulations will be reinitialized by a reverse diffusion operation (RD) proposed in this paper. Besides, (3) a new dual-drive kinetic updating equation is proposed to enhance the search capability of the population. The new algorithm compared with several state-of-art dynamic optimization algorithms on the moving peak problem. The results show that the ARDDEA algorithm can track the optimal solution more effectively in the dynamic environment, and shows strong robustness and adaptability. It is a hope algorithm applied to prediction problems.

Robot navigation based on improved A* algorithm in dynamic environment

PurposeThe purpose of this paper is to put forward a path planning method in complex environments containing dynamic obstacles, which improves the performance of the traditional A* algorithm, this method can plan the optimal path in a short running time.Design/methodology/approachTo plan an optimal path in a complex environment with dynamic and static obstacles, a novel improved A* algorithm is proposed. First, obstacles are identified by GoogLeNet and classified into static obstacles and dynamic obstacles. Second, the ray tracing algorithm is used for static obstacle avoidance, and a dynamic obstacle avoidance waiting rule based on dilate principle is proposed. Third, the proposed improved A* algorithm includes adaptive step size adjustment, evaluation function improvement and path planning with quadratic B-spline smoothing. Finally, the proposed improved A* algorithm is simulated and validated in real-world environments, and it was compared with traditional A* and improved A* algorithms.FindingsThe experimental results show that the proposed improved A* algorithm is optimal and takes less execution time compared with traditional A* and improved A* algorithms in a complex dynamic environment.Originality/valueThis paper presents a waiting rule for dynamic obstacle avoidance based on dilate principle. In addition, the proposed improved A* algorithm includes adaptive step adjustment, evaluation function improvement and path smoothing operation with quadratic B-spline. The experimental results show that the proposed improved A* algorithm can get a shorter path length and less running time.

Agoraphilic navigation algorithm in dynamic environment with obstacles motion tracking and prediction

AbstractThis paper presents a new algorithm to navigate robots in dynamically cluttered environments. The proposed algorithm uses basic concepts of space attraction (hence the term Agoraphilic) to navigate robots through dynamic obstacles. The new algorithm in this paper is an advanced development of the original Agoraphilic navigation algorithm that was only able to navigate robots in static environments. The Agoraphilic algorithm does not look for obstacles (problems) to avoid but rather for a free space (solutions) to follow. Therefore, it is also described as an optimistic navigation algorithm. This algorithm uses only one attractive force created by the available free space. The free-space concept allows the Agoraphilic algorithm to overcome inherited challenges of general navigation algorithms. However, the original Agoraphilic algorithm has the limitation in navigating robots only in static, not in dynamic environments. The presented algorithm was developed to address this limitation of the original Agoraphilic algorithm. The new algorithm uses a developed object tracking module to identify the time-varying free spaces by tracking moving obstacles. The capacity of the algorithm was further strengthened by the new prediction module. Future space prediction allowed the algorithm to make decisions considering future growing/diminishing free spaces. This paper also includes a bench-marking study of the new algorithm compared with a recently published APF-based algorithm under a similar operating environment. Furthermore, the algorithm was validated based on experimental tests and simulation tests.

Semantic visual SLAM in dynamic environment

Human-computer interaction requires accurate localization and effective mapping, while dynamic objects can influence the accuracy of localization and mapping. State-of-the-art SLAM algorithms assume that the environment is static. This paper proposes a new SLAM method that uses mask R-CNN to detect dynamic ob-jects in the environment and build a map containing semantic information. In our method, the reprojection error, photometric error and depth error are used to assign a robust weight to each keypoint. Thus, the dynamic points and the static points can be separated, and the geometric segmentation of the dynamic objects can be realized by using the dynamic keypoints. Each pixel is assigned a semantic label to rebuild a semantic map. Finally, our proposed method is tested on the TUM RGB-D dataset, and the experimental results show that the proposed method outperforms state-of-the-art SLAM algorithms in dynamic environments.

A dual prediction strategy with inverse model for evolutionary dynamic multiobjective optimization

In practical applications and daily life, dynamic multiobjective optimization problems (DMOPs) are ubiquitous. The purpose of dealing with DMOPs is to track moving Pareto Front (PF) and find a series of Pareto Set (PS) at different times. Prediction-based strategies improve the performance of multiobjective evolutionary algorithms in dynamic environments. However, how to ensure the accuracy of prediction models is always a challenge. In this study, a dual prediction strategy with inverse model (DPIM) is developed, to alleviate the negative impact of inaccurate prediction. When a change is confirmed, DPIM responses to it by predicting the individuals in the objective space. Furthermore, the inverse model is established to connect the decision space with the objective space, which can guide the search for promising decision areas. Specifically, the inverse model is also predicted to minimize the error in the process of mapping the population from the objective space back to the decision space. The effectiveness of the proposed DPIM is proved by comparison with four effective DMOEAs on 14 benchmark problems with various real-word scenarios. The experimental results show that DPIM can obtain high-quality populations with good convergence and distribution in dynamic environments.

Evaluation of Instance-Based Learning and Q-Learning Algorithms in Dynamic Environments

Reinforcement learning is an unsupervised learning algorithm, where learning is based upon feedback from the environment. Prior research has proposed cognitive (e.g., Instance-based Learning or IBL) and statistical (Q-learning) reinforcement learning algorithms. However, an evaluation of these algorithms in a single dynamic environment has not been explored. In this paper, a comparison between the statistical Q-learning algorithm and the cognitive IBL algorithm is presented. A well-known environment, “Frozen Lake,” is used to train, generalize, and scale Q-learning and IBL algorithms. For generalizing, the Q-learning and IBL agents were trained on one version of the Frozen Lake and tested on a permuted version of the same environment. For scaling, the two algorithms were tested on a larger version of the Frozen Lake environment. Results revealed that the IBL algorithm used less training time and generalized better to different environment variants. The IBL algorithm was also able to show scalability by retaining its superior performance in the larger environment. These results indicate that the IBL algorithm could be proposed as an alternative to the standard reinforcement learning algorithms based on dynamic programming such as Q-learning. The inclusion of human factors (such as memory) in the IBL algorithm makes it suitable for robust learning in complex and dynamic environments.