Unknown Environment Research Articles

Federated deep reinforcement learning for mobile robot navigation

Navigation of a mobile robot in an unknown environment ensuring the safety of the robot and its surroundings is of utmost importance. Traditional methods, such as pathplanning algorithms, simultaneous localization and mapping, computer vision, and fuzzy techniques, have been employed to address this challenge. However, to achieve better generalization and self-improvement capabilities, reinforcement learning has gained significant attention. The concern of privacy issues in sharing data is also rising in various domains. In this study, a deep reinforcement learning strategy is applied to the mobile robot to move from its initial position to a destination. Specifically, the Deep Q-Learning algorithm has been used for this purpose. This strategy is trained using a federated learning approach to overcome privacy issues and to set a foundation for further analysis of distributed learning. The application scenario considered in this work involves the navigation of a mobile robot to a charging point within a greenhouse environment. The results obtained indicate that both the traditional deep reinforcement learning and federated deep reinforcement learning frameworks are providing 100% success rate. However federated deep reinforcement learning could be a better alternate since it overcomes the privacy issue along with other advantages discussed in this paper.

Autonomous Exploration Method for Fast Unknown Environment Mapping by Using UAV Equipped With Limited FOV Sensor

Autonomous exploration is crucial and highly challenging for various intelligent operations of unmanned aerial vehicles (UAVs). However, low-efficiency problems caused by low-quality trajectory or back-and-forth maneuvers (BFM) still exist. To improve the exploration efficiency in unknown environments, a fast autonomous exploration planner (FAEP) is proposed in this paper. We firstly design a novel frontiers exploration sequence generation method to significantly reduce BFM during the exploration by considering multi-level factors in asymmetric traveling salesman problem (ATSP). Then, according to the exploration sequence and the distribution of frontiers, an adaptive yaw planning method is proposed to cover more frontiers by yaw change during an exploration journey. In addition, to increase the fluency of flight, a dynamic replanning strategy is also adopted. The main technical contributions of the proposed approach are to provide a comprehensive global coverage path with less BFM, and use adaptive yaw planning to generate more reasonable yaw trajectory. We present extensive comparison and real-world experiments to validate the effectiveness and correctness of our method. Experimental results show the proposed approach has better performance compared to typical and state-of-the-art methods in these flat scenarios without lots of scattered obstacles. We release our implementation as an open-source package <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{1}$</tex-math></inline-formula> for the community.

Environmental-structure-perception-based adaptive pose fusion method for LiDAR-visual-inertial odometry

Environmental-structure-perception-based adaptive pose fusion method for LiDAR-visual-inertial odometry

Safe Maneuvering Planning for Flights in Complex Environments

Safe trajectory planning of unmanned aerial vehicles (UAVs) has drawn significant attention. For avoiding obstacles, the two prevailing methods of distance penalty and convex polyhedra confinement ignore the flight velocity, aggressive maneuvering increases the risk of collision. Other approaches ensure safety by limiting the flight speed, leading to conservative and slow trajectories. This paper presents a motion planning method that ensures flexible maneuverability and a high level of UAV safety in unknown complex environments. Firstly, an improved path planning method considering safety and search efficiency is proposed to generate discrete primitives. By representing the confident flight area with ball-shaped areas expected no obstacles included, the safe region constraint is designed and incorporated into the first-stage optimization with the low-order kinematic model. Subsequently, an agile maneuvering approach with environmental adaptability is integrated into the second-stage optimization with the high-order kinematic model. This approach can effectively adjust the agile flight according to the obstacle distribution and motion velocity. Simulations and real-world experiments are conducted to demonstrate the robustness and effectiveness of the presented approach. Moreover, benchmark comparison manifests that the proposed method outperforms the cutting-edge methods from the aspects of the perception uncertainty adaptation and flight safety.

Star-Searcher: A Complete and Efficient Aerial System for Autonomous Target Search in Complex Unknown Environments

Star-Searcher: A Complete and Efficient Aerial System for Autonomous Target Search in Complex Unknown Environments

Development, Simulation and Visualization of Autonomous Farm Vehicles

Abstract: A Research in Autonomous Agricultural Vehicle Technology is presented. With the coming of the Smart Farming Revolution the development of more complicated machines has begun. Agricultural machines are made with the capability of driving themselves, using GPS maps and electronic sensors, an approach that will lead to the development of futuristic Precise Autonomous Farming Systems. The basic issues to be addressed are autonomous path planning, obstacle detection and avoidance for which various sensors will be employed such as GPS, laser-based sensors, ultrasonic sensors, IMU sensors and Camera sensors whose data has to be constantly monitored using Dead Reckoning algorithm and constant Pose Estimation. In these sensors, Camera, Ultrasonic, Laser-Based Sensors are used as Detection sensors while the IMU and GPS sensors are used as Localization sensors. Global Avoidance Subsystem which examines the whole environment of an autonomous vehicle mission-level path planner that pre-plans paths around all known obstacles. While Local Avoidance Subsystem/Reactive planner consists of an obstacle filter and an obstacle avoidance algorithm and re-plans a short way for the vehicle to navigate. With the ROS framework and all sensors data, a vehicle can guide itself through an unknown environment. This will help the farmer to undercut the labour shortage and improve the precision in farming which in turn improves the output to meet the growing demands

Reinforcement learning based robot navigation using illegal actions for autonomous docking of surface vehicles in unknown environments

Autonomous Surface Vehicles (ASVs) are bound to play a fundamental role in the maintenance of offshore wind farms. Robust navigation for inspection vehicles should take into account the operation of docking within a harbouring structure, which is a critical and still unexplored maneuver. This work proposes an end-to-end docking approach for ASVs, based on Reinforcement Learning (RL), which teaches an agent to tackle collision-free navigation towards a target pose that allows the berthing of the vessel. The developed research presents a methodology that introduces the concept of illegal actions to facilitate the vessel’s exploration during the learning process. This method improves the adopted Actor-Critic (AC) framework by accelerating the agent’s optimization by approximately 38.02%. A set of comprehensive experiments demonstrate the accuracy and robustness of the presented method in scenarios with simulated environmental constraints (Beaufort Scale and Douglas Sea Scale), and a diversity of docking structures. Validation with two different real ASVs in both controlled and real environments demonstrates the ability of this method to enable safe docking maneuvers without prior knowledge of the scenario.

Research on obstacle avoidance of multi-AUV cluster formation based on virtual structure and artificial potential field method★

The formation of autonomous underwater vehicle (AUV) in the three-dimensional environment is very important for carrying out complex ocean tasks. During formation sailing, it is necessary to consider avoiding obstacles while maintaining the formation. In this paper, an AUV formation obstacle avoidance method based on virtual structure and artificial potential field method is proposed. Firstly, a complete mathematical modeling of AUV is carried out, and a sliding mode position tracker based on virtual position points is designed based on this model. Then the tracking points of each AUV in the formation are calculated according to the expected trajectory and formation feature structure to ensure that the spacing between AUVs meets the requirements of the formation. Finally, the artificial potential field method is introduced into the position tracker to avoid obstacles between the obstacle and the AUV. Five groups of simulation experiments of parallel formation, vertical formation, cross formation, triangular formation and unknown complex environment are designed. The formation scalability of the algorithm is simulated and compared with the virtual leader method in related references. Finally, KKSwarm platform is used to conduct a real simulation in 2D environment to verify the effectiveness of the proposed algorithm.The experimental results show that the proposed method can effectively realize the formation navigation of multiple underwater vehicles and successfully avoid obstacles, which provides an effective method to solve the problem of the formation obstacle avoidance of underactuated AUV.

U-ATSS: A lightweight and accurate one-stage underwater object detection network

Due to the harsh and unknown marine environment and the limited diving ability of human beings, underwater robots become an important role in ocean exploration and development. However, the performance of underwater robots is limited by blurred images, low contrast and color deviation, which are resulted from complex underwater imaging environments. The existing mainstream object detection networks perform poorly when applied directly to underwater tasks. Although using a cascaded detector network can get high accuracy, the inference speed is too slow to apply to actual tasks. To address the above problems, this paper proposes a lightweight and accurate one-stage underwater object detection network, called U-ATSS. Firstly, we compressed the backbone of ATSS to significantly reduce the number of network parameters and improve the inference speed without losing the detection accuracy, to achieve lightweight and real-time performance of the underwater object detection network. Then, we propose a plug-and-play receptive field module F-ASPP, which can obtain larger receptive fields and richer spatial information, and optimize the learning rate strategy as well as classification loss function to significantly improve the detection accuracy and convergence speed. We evaluated and compared U-ATSS with other methods on the Kesci Underwater Object Detection Algorithm Competition dataset containing a variety of marine organisms. The experimental results show that U-ATSS not only has obvious lightweight characteristics, but also shows excellent performance and competitiveness in terms of detection accuracy.

Autonomous underwater vehicle link alignment control in unknown environments using reinforcement learning

AbstractHigh‐speed underwater wireless optical communication holds immense promise in ocean monitoring and surveys, providing crucial support for the real‐time sharing of observational data collected by autonomous underwater vehicles (AUVs). However, due to inaccurate target information and external interference in unknown environments, link alignment is challenging and needs to be addressed. In response to these challenges, we propose a reinforcement learning‐based alignment method to control the AUV to establish an optical link and maintain alignment. Our alignment control system utilizes a combination of sensors, including a depth sensor, Doppler velocity log (DVL), gyroscope, ultra‐short baseline device, and acoustic modem. These sensors are used in conjunction with a particle filter to observe the environment and estimate the AUV's state accurately. The soft actor‐critic algorithm is used to train a reinforcement learning‐based controller in a simulated environment to reduce pointing errors and energy consumption in alignment. After experimental validation in simulation, we deployed the controller on an actual AUV called Tri‐TON. In experiments at sea, Tri‐TON maintained the link and angular pointing errors within 1 m and , respectively. Experimental results demonstrate that the proposed alignment control method can establish underwater optical communication between AUV fleets, thus improving the efficiency of marine surveys.

Sequential sampling for functional estimation via Sieve

AbstractSequential sampling methods are often used to estimate functions describing models subjected to time‐intensive simulations or expensive experiments. These methods provide guidelines for point selection in the domain to capture maximum information about the function. However, in most sequential sampling methods, determining a new point is a time‐consuming process. In this paper, we propose a new method, named Sieve, to sequentially select points of an initially unknown function based on the definition of proper intervals. In contrast with existing methods, Sieve does not involve function estimation at each iteration. Therefore, it presents a greater computational efficiency for achieving a given accuracy in estimation. Sieve brings in tools from computational geometry to subdivide regions of the domain efficiently. Further, we validate our proposed method through numerical simulations and two case studies on the calibration of internal combustion engines and the optimal exploration of an unknown environment by a mobile robot.

A new method for unmanned aerial vehicle path planning in complex environments

To solve the problems of UAV path planning, such as low search efficiency, uneven path, and inability to adapt to unknown environments, this paper proposes A double-layer optimization A* and dynamic window method for UAV path planning. Firstly, the neighboring node clip-off rule is defined to optimize the node expansion mode of the A* algorithm, and the obstacle coverage model is designed to dynamically adjust the heurizing function of the A* algorithm to improve the path search efficiency. Then, the Bresenham algorithm is adopted for collision detection and critical path nodes are extracted to significantly reduce the number of path turning points. Secondly, a new tracking index is proposed to optimize the evaluation function of the dynamic window method to make the local path fit the global path further. By detecting the dangerous distance, the dynamic adaptive method of evaluation function weight is designed to improve the fixed weight of the dynamic window method. Finally, the key turning point of optimizing the A* algorithm is taken as the temporary target point to improve the DWA algorithm, and the local part follows the global part, and the fusion of the two algorithms is realized. Simulation results show that the proposed method can significantly improve the efficiency and smoothness of mobile robot path planning, enhance the real-time obstacle avoidance and adaptive ability of unknown environments, and better meet the requirements of complex planning tasks.

Trajectory Planning for UAV-Assisted Data Collection in IoT Network: A Double Deep Q Network Approach

Unmanned aerial vehicles (UAVs) are becoming increasingly valuable as a new type of mobile communication device and autonomous decision-making device in many application areas, including the Internet of Things (IoT). UAVs have advantages over other stationary devices in terms of high flexibility. However, a UAV, as a mobile device, still faces some challenges in optimizing its trajectory for data collection. Firstly, the high complexity of the movement action and state space of the UAV’s 3D trajectory is not negligible. Secondly, in unknown urban environments, a UAV must avoid obstacles accurately in order to ensure a safe flight. Furthermore, without a priori wireless channel characterization and ground device locations, a UAV must reliably and safely complete the data collection from the ground devices under the threat of unknown interference. All of these require the proposing of intelligent and automatic onboard trajectory optimization techniques. This paper transforms the trajectory optimization problem into a Markov decision process (MDP), and deep reinforcement learning (DRL) is applied to the data collection scenario. Specifically, the double deep Q-network (DDQN) algorithm is designed to address intelligent UAV trajectory planning that enables energy-efficient and safe data collection. Compared with the traditional algorithm, the DDQN algorithm is much better than the traditional Q-Learning algorithm, and the training time of the network is shorter than that of the deep Q-network (DQN) algorithm.

Analysis of the behavior of a mobile robot with precise and fuzzy pid controller in an uncertain environment

The aim of this study was to develop methods for building intelligent systems that ensure safe movement on a planned and specified trajectory in an environment with unknown obstacles for planning the movement of a mobile robot. To achieve this goal, modern concepts and methods of planning systems development were analyzed. By moving a mobile robot in an unknown environment, it is proposed to develop an intelligent system for planning its modeling in an uncertain static environment, to realize an intelligent system for planning the movement of a mobile robot in an unknown dynamic environment and modeling the resulting system. As a result, analytical studies were conducted on a computer using mathematical modeling, analytical geometry, kinematic and dynamic analysis, fuzzy logic, neural networks, robotics, mechatronics, discrete integration and applied programming methods, and experimental studies were conducted using a mobile robot model.

Probabilistic reach-avoid for Bayesian neural networks

Model-based reinforcement learning seeks to simultaneously learn the dynamics of an unknown stochastic environment and synthesise an optimal policy for acting in it. Ensuring the safety and robustness of sequential decisions made through a policy in such an environment is a key challenge for policies intended for safety-critical scenarios. In this work, we investigate two complementary problems: first, computing reach-avoid probabilities for iterative predictions made with dynamical models, with dynamics described by Bayesian neural network (BNN); second, synthesising control policies that are optimal with respect to a given reach-avoid specification (reaching a “target” state, while avoiding a set of “unsafe” states) and a learned BNN model. Our solution leverages interval propagation and backward recursion techniques to compute lower bounds for the probability that a policy's sequence of actions leads to satisfying the reach-avoid specification. Such computed lower bounds provide safety certification for the given policy and BNN model. We then introduce control synthesis algorithms to derive policies maximizing said lower bounds on the safety probability. We demonstrate the effectiveness of our method on a series of control benchmarks characterized by learned BNN dynamics models. On our most challenging benchmark, compared to purely data-driven policies the optimal synthesis algorithm is able to provide more than a four-fold increase in the number of certifiable states and more than a three-fold increase in the average guaranteed reach-avoid probability.

E-DQN-Based Path Planning Method for Drones in Airsim Simulator under Unknown Environment.

To improve the rapidity of path planning for drones in unknown environments, a new bio-inspired path planning method using E-DQN (event-based deep Q-network), referring to introducing event stream to reinforcement learning network, is proposed. Firstly, event data are collected through an airsim simulator for environmental perception, and an auto-encoder is presented to extract data features and generate event weights. Then, event weights are input into DQN (deep Q-network) to choose the action of the next step. Finally, simulation and verification experiments are conducted in a virtual obstacle environment built with an unreal engine and airsim. The experiment results show that the proposed algorithm is adaptable for drones to find the goal in unknown environments and can improve the rapidity of path planning compared with that of commonly used methods.

Adaptive positioning method for ball head based on impedance control

PurposeComponent positioning is an important part of aircraft assembly, aiming at the problem that it is difficult to accurately fall into the corresponding ball socket for the ball head connected with aircraft component. This study aims to propose a ball head adaptive positioning method based on impedance control.Design/methodology/approachFirst, a target impedance model for ball head positioning is constructed, and a reference positioning trajectory is generated online based on the contact force between the ball head and the ball socket. Second, the target impedance parameters were optimized based on the artificial fish swarm algorithm. Third, to improve the robustness of the impedance controller in unknown environments, a controller is designed based on model reference adaptive control (MRAC) theory and an adaptive impedance control model is built in the Simulink environment. Finally, a series of ball head positioning experiments are carried out.FindingsDuring the positioning of the ball head, the contact force between the ball head and the ball socket is maintained at a low level. After the positioning, the horizontal contact force between the ball head and the socket is less than 2 N. When the position of the contact environment has the same change during ball head positioning, the contact force between the ball head and the ball socket under standard impedance control will increase to 44 N, while the contact force of the ball head and the ball socket under adaptive impedance control will only increase to 19 N.Originality/valueIn this paper, impedance control is used to decouple the force-position relationship of the ball head during positioning, which makes the entire process of ball head positioning complete under low stress conditions. At the same time, by constructing an adaptive impedance controller based on MRAC, the robustness of the positioning system under changes in the contact environment position is greatly improved.

A Reflective Object Reconstruction Method Based on Neural Radiance Field

We propose a neural rendering-based 3D reconstruction method for reconstructing the geometry and BRDF of reflective objects from multi-view images captured in unknown environments. Multi-view reconstruction of reflective objects is extremely challenging because specular reflections are view-dependent and therefore violate multi-view consistency, which is the cornerstone of most multi-view reconstruction methods. Recent neural rendering techniques can model the interaction between ambient light and object surfaces to adapt to view-dependent reflections, making it possible to reconstruct reflective objects from multi-view images. However, accurately modeling ambient light in neural rendering is difficult. We propose a two-step approach to solve this problem. First, by introducing view-dependent photometric losses, our method accurately reconstructs the geometry of reflective objects. Then, with the object geometry fixed, we use more accurate sampling to recover the BRDF of the ambient light and object. Experiments show that our method is able to accurately reconstruct the geometry and BRDF of reflective objects from only RGB images without knowing the ambient light and object masks.

Dynamic State Estimation for Distribution Networks Based on Adaptive Set Membership Filter Under Unknown But Bounded Noise Environments

Dynamic State Estimation for Distribution Networks Based on Adaptive Set Membership Filter Under Unknown But Bounded Noise Environments

A framework for collaborative multi-robot mapping using spectral graph wavelets

The exploration of large-scale unknown environments can benefit from the deployment of multiple robots for collaborative mapping. Each robot explores a section of the environment and communicates onboard pose estimates and maps to a central server to build an optimized global multi-robot map. Naturally, inconsistencies can arise between onboard and server estimates due to onboard odometry drift, failures, or degeneracies. The mapping server can correct and overcome such failure cases using computationally expensive operations such as inter-robot loop closure detection and multi-modal mapping. However, the individual robots do not benefit from the collaborative map if the mapping server provides no feedback. Although server updates from the multi-robot map can greatly alleviate the robotic mission strategically, most existing work lacks them, due to their associated computational and bandwidth-related costs. Motivated by this challenge, this paper proposes a novel collaborative mapping framework that enables global mapping consistency among robots and the mapping server. In particular, we propose graph spectral analysis, at different spatial scales, to detect structural differences between robot and server graphs, and to generate necessary constraints for the individual robot pose graphs. Our approach specifically finds the nodes that correspond to the drift’s origin rather than the nodes where the error becomes too large. We thoroughly analyze and validate our proposed framework using several real-world multi-robot field deployments where we show improvements of the onboard system up to 90% and can recover the onboard estimation from localization failures and even from the degeneracies within its estimation.