Navigation Problem Research Articles

Survey Paper on Quantum Deep Reinforcement Learning for Robot Navigation Tasks

Abstract: Quantum deep reinforcement learning is an integration of the principles of quantum computing with deep reinforcement learning in an effort to advance robot navigation tasks. Due to the explosion of the state and action spaces, traditionally DRL methods are limited in scalability and efficiency. QDRL, however, uses quantum superposition and entanglement for more efficient processing of big amounts of information-it enables robots to learn optimal policies for navigating complex environments. We introduce here a new QDRL framework in which quantum neural networks encode policy functions and value estimates. We demonstrate how quantum circuits can take advantage of multi-dimensional state representations in order to strengthen the power of a robot in changing its environment over time. Benchmark experiments on navigation tasks reflect phenomenal acceleration in learning performance along with better decision-making accuracy compared to classical DRL methods. We then consider integrating such quantum algorithms, as Grover's search, to improve exploration strategies in sparse reward settings. The experiments show that QDRL speeds up convergence rates and provides robots with the possibility to react more rapidly to unexpected obstacles. It discusses new applications for quantum computing of autonomous systems, potentially applied to robotics, autonomous vehicles, and other environments which require complex decision making. Some lines of work in the future are optimization of the resources needed for quantum computers and hybrid architectures for the resolution of classical problems in navigation

Multi-agent long-distance end-to-end indoor navigation: using imitation learning pre-training and global map

Abstract To address the navigation challenges of mobile robot swarm coordination in complex environments, this paper presents a distributed navigation approach for mobile robot swarm. The study initially formulates the problem of robot swarm navigation as a partially observable Markov decision process, employing a distributed proximal policy optimization algorithm to train decision models mapping observations to actions directly. Furthermore, it adopts imitation learning pre-training to maximize expert decisions in the initial training phase, thereby reducing exploration space. Lastly, an improved path point generation algorithm along with a spatiotemporal reachability calculation method is proposed to guide the Deep Reinforcement Learning policy in accomplishing long-distance indoor navigation. Comprehensive performance evaluations are conducted in simulated environments to compare the proposed method with existing approaches. Results demonstrate that the proposed approach effectively enhances both convergence speed and model performance, while significantly improving the navigation capabilities of robot swarms in complex environments.

Social robot navigation through constrained optimization: A comprehensive study of uncertainty-based objectives and constraints in the simulated and real world

This paper provides an empirical evaluation, in both simulation and real scenarios, of the social navigation problem when considering human motion prediction and its stochastic effects. To this end, we study several different optimization criteria and constraints related to the uncertainty of predicting pedestrians’ motion, embedded into the Model Predictive Control (MPC) scheme.The main research question of this work is the following: what are the most important uncertainty-based criteria for the social MPC both in simulated and real-world environments? In order to achieve a solid answer to this question, we extend the results previously obtained from our work (Akhtyamov et al., 2023) in the simulated environments and provide a real-world setting that mimics similar conditions, for a fair comparison of the qualitative and quantitative results.The main conclusions supported by both of the evaluation environments are the advantages of using adaptive constraints as a clear undisputed enhancement and the problems raised when considering uncertainty-aware criteria. We hope this paper is of interest to the community for deciding and designing uncertainty-aware approaches for social robot navigation.

Mobile robots control with neuromorphic classifires of environment states

The article proposes to implement mobile robots control systems using neuromorphic classifiers, in which biosimi-lar neurons are built on an original neuro-fuzzy model, trained to display the function of the Izhikevich neuron. Us-ing this approach, neuromorphic classifiers of spatial and spatiotemporal patterns were developed. Such a classi-fiers were used in non-contact supervisory control system for mobile robot based on static and dynamic states of the environment. The experiments performed demonstrated the effectiveness of using the neuromorphic classifi-ers, both in a neural interface when recognizing imaginary supervisory commands of the user, and when solving problems of robot navigation in environments with static and dynamic obstacles. At comparative analysis of navi-gation systems implemented using the neuromorphic classifiers, modular fuzzy logic and the adaptive neuro-fuzzy system ANFIS it was shown that the system with the neuromorphic classifiers give the highest average speed of the robot, as well as the shortest travel time. An experiment with the use of the neuromorphic classifiers in the con-trol system based on dynamic states of the environment showed that robot successfully avoids collisions with per-son.

Автономное следование безэкипажного судна по траектории и предотвращение столкновений

In this paper we propose a method to solve the problem of autonomous vessel navigation in highly uncertain conditions. The primary goal was to make the Reinforcement Learning agent to learn the policy allowing the AV to follow a certain path while avoid collisions with any other objects. Maneuvering of AV in the proposed conditions is the principal subject of this paper. Several scenarios were explored during the research with static and dynamic objects. The agent was trained with model free off-policy algorithms. The training process was divided in several parts where we were experimenting with meta-learning approach elements to achieve the robustness of the agent behavior.

Multi-Strategy Improved Harris Hawk Optimization Algorithm and Its Application in Path Planning.

Path planning is a key problem in the autonomous navigation of mobile robots and a research hotspot in the field of robotics. Harris Hawk Optimization (HHO) faces challenges such as low solution accuracy and a slow convergence speed, and it easy falls into local optimization in path planning applications. For this reason, this paper proposes a Multi-strategy Improved Harris Hawk Optimization (MIHHO) algorithm. First, the double adaptive weight strategy is used to enhance the search capability of the algorithm to significantly improve the convergence accuracy and speed of path planning; second, the Dimension Learning-based Hunting (DLH) search strategy is introduced to effectively balance exploration and exploitation while maintaining the diversity of the population; and then, Position update strategy based on Dung Beetle Optimizer algorithm is proposed to reduce the algorithm's possibility of falling into local optimal solutions during path planning. The experimental results of the comparison of the test functions show that the MIHHO algorithm is ranked first in terms of performance, with significant improvements in optimization seeking ability, convergence speed, and stability. Finally, MIHHO is applied to robot path planning, and the test results show that in four environments with different complexities and scales, the average path lengths of MIHHO are improved by 1.99%, 14.45%, 4.52%, and 9.19% compared to HHO, respectively. These results indicate that MIHHO has significant performance advantages in path planning tasks and helps to improve the path planning efficiency and accuracy of mobile robots.

Stable solution to the problem of autonomous navigation of moving objects on analytical trajectory intervals using inertial measurement results

Stable solution to the problem of autonomous navigation of moving objects on analytical trajectory intervals using inertial measurement results

A deep residual reinforcement learning algorithm based on Soft Actor-Critic for autonomous navigation

The problem of autonomous navigation has attracted significant attention from robotics research community in the last few decades. In this paper, we address the problem of low data utilization due to the large amount of episode experience value data. A maximum entropy algorithm based on prioritized experience replay (Learning Good Experience based on Soft Actor-Criti, LGE-SAC) is proposed to quickly reproduce past good experience episodes. As the deep reinforcement learning method is susceptible to failure to plan ahead and explore the target position in a long sequence environment, a deep Residual Soft Actor-Critic (RSAC) is proposed to alleviate this problem. The reinforcement learning policy is fused with the Artificial Potential Field method to improve the generalization ability of the proposed algorithm, thus improving robot adaptation in new test environments. In order to validate the effectiveness of the proposed algorithm, we conducted simulation experiments in Gazebo simulator environment and real experiments on a Turtlebot3 robot equipped with LiDAR sensor. Simulation and experiment results show that the proposed algorithm effectively avoids obstacles and succeeds in reaching the goal compared to other obstacle avoidance algorithms. In comparison with the Artificial Potential Field method, the planning success rate of the proposed RSAC algorithm in the test environment is increased by 30%, and at the same time, the number of planning steps is reduced by half, and the generalization ability is improved.

Fractional Strong Metric Dimension of Convex Polytopes and its applications

The fractional versions of various metric related parameters have recently gained importance due to their applications in the fields of sensor networking, robot navigation and linear optimization problems. Convex polytopes are collection of those polytopes of Euclidean space which are their convex subsets. They have key importance in the field of network designing due to their stable and resilient structure which aids optimal data transfer. The identification and removal of components (nodes) of a communication network causing abruption in its flow is of key importance for optimal data transmission. These components are referred as strong resolving neighbourhood (SRNs) in graph theory and assigning least weight to these components aids the computation of fractional strong metric dimension (FSMD). In this paper, we compute FSMD for certain convex polytopes which include $\mathbb{P}_{n}$, $\mathbb{P}_{n}^{1}$ and $\mathbb{P}_{n}^{2}$. In this regard, it is shown that for $n \geq 3$, FSMD of $\mathbb{P}_{n}$ and $\mathbb{P}_{n}^{2}$ is $n$ and $\frac{3n}{2}$, respectively. Also, FSMD of $\mathbb{P}_{n}^{1}$ is $n$ when $n$ is odd and $\frac{3n}{2}$ when $n$ is even. Finally, an application of FSMD in the context of internet connection networks is furnished.

Why do patients who are triaged as low-acuity visit the emergency department? – A Polish perspective

BackgroundEmergency departments (EDs) worldwide are dealing with overcrowding, system fragmentation, and coordination problems, which impact patient wait times, staff job satisfaction, and patient outcomes. Inappropriate ED visits, particularly those for low acuity conditions, exacerbate these challenges. However, the motivations behind these visits are poorly understood, with limited data from the patient perspective. This study investigated patient-reported motives behind ED visits triaged as low acuity in Poznan, Poland, to propose health care system flow enhancements. Material and methodsA cross-sectional survey and retrospective chart review were conducted in the ED of the Hipolit Cegielski Medical Center in Poznań, Poland, over three months in 2022–23. Patients who were triaged to have low acuity conditions were invited to participate in the survey. The data collected through the questionnaire included patient and ED visit characteristics. Additional information on the visits was extracted from the patient charts. Main resultsThis study involved 293 patients who underwent low-acuity triage. Among them, 58 % were deemed to have conditions that could have been treated in primary care. Most of the patients (74 %) visited the ED of their own volition due to concerns about their health. Other reasons for ED attendance were challenges accessing primary care or a specialist clinic (11 %), system navigation problems (5 %), or a lack of trust in their primary care provider (2 %). ConclusionsThis study showed that of the patients surveyed, the majority had conditions that could have been treated outside of the ED setting. We recommend prioritizing education, particularly among younger adults, to increase awareness about nonurgent care options while improving health care policies.

Wind Finslerian Structures: From Zermelo’s Navigation to the Causality of Spacetimes

The notion of wind Finslerian structure Σ \Sigma is developed; this is a generalization of Finsler metrics (and Kropina ones) where the indicatrices at the tangent spaces may not contain the zero vector. In the particular case that these indicatrices are ellipsoids, called here wind Riemannian structures, they admit a double interpretation which provides: (a) a model for classical Zermelo’s navigation problem even when the trajectories of the moving objects (planes, ships) are influenced by strong winds or streams, and (b) a natural description of the causal structure of relativistic spacetimes endowed with a non-vanishing Killing vector field K K (SSTK splittings), in terms of Finslerian elements. These elements can be regarded as conformally invariant Killing initial data on a partial Cauchy hypersurface. The links between both interpretations as well as the possibility to improve the results on one of them using the other viewpoint are stressed. The wind Finslerian structure Σ \Sigma is described in terms of two (conic, pseudo) Finsler metrics, F F and F l F_l , the former with a convex indicatrix and the latter with a concave one. Notions such as balls and geodesics are extended to Σ \Sigma . Among the applications, we obtain the solution of Zermelo’s navigation with arbitrary time-independent wind, metric-type properties for Σ \Sigma (distance-type arrival function, completeness, existence of minimizing, maximizing or closed geodesics), as well as description of spacetime elements (Cauchy developments, black hole horizons) in terms of Finslerian elements in Killing initial data. A general Fermat’s principle of independent interest for arbitrary spacetimes, as well as its applications to SSTK\xspace spacetimes and Zermelo’s navigation, are also provided.

Research on multi-path optimization problem based on particle swarm optimization algorithm

The problem of finding the optimal path within a certain range exists in various scenarios in our lives, such as the traveling salesman problem, robot automatic path selection problem, vehicle and pedestrian navigation, game path navigation, communication routing, logistics and transportation planning problems, etc. , through the optimization of path problems, we can help people improve resource utilization, improve work efficiency, reduce production costs, or complete the goal of a specific scenario, etc. Therefore, solving the path optimization problem is a very important task in our reality. In order to improve the game experience of the players in the game, this paper studies how to use the particle swarm optimization method to solve the optimal route of NPC.

Determination of occurrence of atmospheric blockings over the Caspian Sea region in winters of 1959–2022, and influence of them on the ice regime of the Northern Caspian Sea

The ice regime of the Caspian Sea has pronounced influence on the heat and moisture exchange of the reservoir with the atmosphere, the state of the ecosystem, as well as human marine activities, including shipping, fishing, construction of hydraulic structures, etc. Consequently, the development of existing ideas about the causes of changes in the characteristics of the ice regime of such water bodies is actual and socially significant problem of limnology, hydrometeorology, ecology, and navigation. This study was aimed at determination the frequency of occurrence of atmospheric blockings over the Caspian region with standings longer 5 days in winter period of 1959–2022, and investigation of influence of them on the ice regime in the Northern area of the Sea. The following information and data were used: changes in hourly mean values of atmospheric pressure at the sea level, geopotential of isobaric surfaces 850, 500 and 300 hPa, presented in the ERA5 reanalysis; and observational data on air temperature and ice cover from hydrometeorological stations located in the Caspian region of Kazakhstan and Russia. It has been established that in the winter during a long standing of atmospheric blockings the mean daily air temperatures noticeably drop. The values of all the studied characteristics of every atmospheric blocking which occurred in 1959–2022 were estimated as well the influence of them on the ice regime in the North of the Caspian Sea was analyzed. The relationship between a decrease in the frequency of atmospheric blockings (AB), sums of negative air temperatures on its coasts, and the ice cover thickness in the corresponding areas of coastal waters has been revealed in the region. The longer the total AB duration, the lower is the air temperature, and the ice thickness in February is larger.

ОБОБЩЕННЫЙ АЛГОРИТМ РАСПРЕДЕЛЕНИЯ РЕСУРСОВ КОМПЬЮТЕРНОГО ГЕНЕРАТОРА ИЗОБРАЖЕНИЯ ПО 3D-МОДЕЛИ ТЕРРИТОРИИ МЕСТНОСТИ

The development of modern software systems that use computer graphics algorithms in real time to create 3D-models of the terrain expands the range of training tasks available to the Customer for training pilots on aviation simulators. Particular attention is paid to training pilots in the skills of solving navigation problems while flying over certain 3D terrain, using visual references in the form of three-dimensional models of reference objects. To achieve this goal, it is necessary to solve several important aspects: the selection of 3D-objects on the terrain as reference points, the development of an information resource algorithm and the optimal distribution of computer image generator resources throughout the 3D-model of the terrain to ensure the possibility of synthesizing a sufficient number of 3D-models reference objects in real time. Solutions are presented for modeling a terrain area of sufficient size for distributing information resources and training the pilot to solve navigation problems.

Autonomous Vision-Based Navigation and Stability Augmentation Control of a Biomimetic Robotic Hammerhead Shark

The application potential of robotic fish embedded with intelligent visual navigation algorithms in underwater autonomous operation is on full display recently. However, the existing visual navigation methods are limited by the underwater visual conditions and the motion characteristics of robotic fish. To this end, this paper proposes a novel autonomous navigation framework integrated with visual stabilization control. In practice, a stereo vision-based navigation network is proposed to generate the guidance law. On this basis, a biomimetic robotic hammerhead shark with a controllable cephalofoil is developed, and a nonlinear model predictive controller for cephalofoil stabilization relying on the dynamic model is elaborately designed. Extensive simulations and underwater experiments are conducted to validate the effectiveness and superiority of the proposed methods, which significantly enhance exploration efficiency and reduce image jitter by 26.02% compared to the traditional methods. The obtained results provide a new idea for underwater robots to autonomously explore the ocean. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by the problem of vision-based underwater autonomous navigation for a biomimetic robotic fish that possesses underwater visual stability and good maneuverability. The existing visual navigation networks usually generate unexpected navigation instructions when dealing with complex or ambiguous underwater scenes. Additionally, image jitter caused by the rhythmic motion of robotic fish can lead to navigation failure. This paper suggests an integrated navigation framework that includes a biomimetic platform design, a visual stabilization controller, and an intelligent underwater navigation network. Specifically, a novel sphyrnidae-inspired robotic shark is designed as a new platform with superior motion performance. To enhance underwater visual stability, a nonlinear model predictive control-based visual stabilization controller is proposed. Furthermore, a deep stereo attention navigation network based on a parallax attention mechanism is proposed to improve the generalization of vision-based autonomous navigation. A series of underwater search experiments on the robotic shark demonstrate the effectiveness and superiority of the proposed navigation framework. Hopefully, our proposed methods can provide valuable guidance and support for universal underwater robot navigation to accomplish practical marine tasks, such as underwater rescue, resource exploitation, biological observation, and so on.

A study on dual quaternion based cooperative relative navigation of multiple UAVs with monocular vision-inertial integration

This paper addresses a cooperative relative navigation problem for multiple aerial agents, relying on visual tracking information between vehicles. The research aims to investigate a sensor fusion architecture and algorithm that leverages partially available absolute navigation knowledge while exploiting collaborative visual interaction between vehicles in mission flight areas, where satellite navigation-denied regions are irregularly located. To achieve this, the paper introduces a new approach to defining the relative poses of cameras and develops a corresponding process to secure the relative pose information. This contrasts with previous research, which simply linearized the relative pose information of aircraft cameras into navigation states defined in an absolute coordinate system. Specifically, the target pose in relative navigation is defined, and the pose of the camera and feature points are directly derived using dual quaternion representation, which compactly represents both translation and rotation. Furthermore, a mathematical model for the relative pose of the camera is derived through the dual quaternion framework, enabling an explicit pose formulation of relative navigation. The study investigates navigation performance in typical mission flight scenarios using an in-house high-fidelity simulator and quantitatively highlights the contributions of the proposed scheme by comparing the navigation error performance. Consequently, the proposed method demonstrates to have navigation accuracy in decimeter level even in GNSS-denied environments and an improved 3D Root Mean Square (RMS) error by 30% smaller than the conventional absolute navigation framework.

Алгоритм планирования маршрута БПЛА в условиях недостатка визуальных ориентиров

Today, unmanned aircraft is used to solve a variety of tasks. One of the problems that arise when using unmanned aerial vehicles (UAVs) is the problem of navigation, in particular, determining their exact location in space. Today, there are various methods to solve this problem. A promising way to solve it in conditions where it is impossible to use a satellite navigation system is video navigation systems that use visual landmarks to determine location. In this paper, an overview of the existing types of UAV video navigation systems was carried out. The peculiarities of the operation of video navigation systems using the comparison of visual landmarks with satellite maps were also highlighted. The task was set to find a route in conditions of a lack of visual landmarks, in which the problem of losing reference points may arise. The graph data format used is described, which stores the values of the metric of the diversity of the underlying surface, which characterizes the magnitude of the probability of the problem of losing reference points. The existing algorithms for finding a path in a graph are considered and the need to develop a new algorithm is justified. A theoretical description of the algorithm was proposed, in which the path search process is divided into two stages: first, a solution option is searched for with the maximum value of the metric of diversity of the underlying surface, and then a route with the minimum length is selected. Several options of the implementation of the proposed algorithm were also presented and their asymptotic complexity was estimated. Based on the theoretical description, a software implementation of the proposed algorithm was developed and an experimental verification of the correctness of its operation on a test card with various flight tasks was performed. According to the test results, the proposed algorithm has shown its effectiveness by constructing routes correctly in all test examples. In the future, this algorithm can be used as part of the UAV video navigation system. The main advantages and disadvantages of the proposed algorithm were also described and ways for its further improvement were proposed.

Autonomous UAV Navigation with Adaptive Control Based on Deep Reinforcement Learning

Unmanned aerial vehicle (UAV) navigation plays a crucial role in its ability to perform autonomous missions in complex environments. Most of the existing reinforcement learning methods to solve the UAV navigation problem fix the flight altitude and velocity, which largely reduces the difficulty of the algorithm. But the methods without adaptive control are not suitable in low-altitude environments with complex situations, generally suffering from weak obstacle avoidance. Some UAV navigation studies with adaptive flight only have weak obstacle avoidance capabilities. To address the problem of UAV navigation in low-altitude environments, we construct autonomous UAV navigation in 3D environments with adaptive control as a Markov decision process and propose a deep reinforcement learning algorithm. To solve the problem of weak obstacle avoidance, we creatively propose the guide attention method to make a UAV’s decision focus shift between the navigation task and obstacle avoidance task according to changes in the obstacle. We raise a novel velocity-constrained loss function and add it to the original actor loss to improve the UAV’s velocity control capability. Simulation experiment results demonstrate that our algorithm outperforms some of the state-of-the-art deep reinforcement learning algorithms performing UAV navigation tasks in a 3D environment and has outstanding performance in algorithm effectiveness, with the average reward increasing by 9.35, the success rate of navigation tasks increasing by 14%, and the collision rate decreasing by 14%.

Stable solution to the problem of autonomous navigation of moving objects on analytical trajectory intervals based on the results of inertial measurements

Provision of solution of the problem of autonomous navigation on a long time interval is considered. The main problem when using inertial navigation systems is caused by the accumulation of positioning and angular orientation errors. It is shown that this problem can be solved by combining methods of nonlinear filtering in processing the results of inertial measurements of platform-free inertial navigation systems and methods of reducing the dimensionality of the vector of navigation parameters on trajectory sections described by analytical models. Orthodromic (shortest) and circular intervals, most typical for program trajectories of transport objects – highways, railroads, airlines, etc., are considered as such trajectory sections. For circular intervals, analytical dependencies of the spatial coordinates of strapdown inertial navigation systems were obtained for the first time. These dependencies make it possible to reduce the dimension of the vector of navigation parameters and, as a consequence, to build a model of an autonomous observer of navigation parameters to use equations excluded from the general system of these parameters. It is shown that using stochastic nonlinear filtering methods it is possible to solve the problem of noise-resistant autonomous navigation on trajectory intervals described by analytical models. The proposed approach to solving the problem of autonomous navigation of moving objects makes it possible to significantly reduce computational costs in the practical implementation of algorithms for nonlinear filtering of current navigation parameters in comparison with traditional models of strapdown inertial navigation systems. The effectiveness of the proposed approach is illustrated with a numerical example. The results obtained are useful in developing navigation support for various transport objects moving along program trajectories, for example, aircraft for various purposes, railway, road, river transport, etc.

Research and Optimization of the Descent Phase of Civil Aviation Aircraft in the Vertical Navigation Problem

Research and Optimization of the Descent Phase of Civil Aviation Aircraft in the Vertical Navigation Problem