Exploration Of Unknown Environments Research Articles

3D Maps Integration based on Overlapping Regions Matching


 This paper presents a method of 3D maps integration based on overlapping regions detection and matching.
 The algorithm works without an initial guess about transformation. The process of finding transformation between maps can be divided into two steps.
 The first one is the estimation of an initial transformation based on feature extraction, description, and matching.
 The assumption is that the maps have an overlapping area that can be used during feature based processing.
 Then the found initial solution is corrected using local methods, for example, Iterative Closest Point (ICP) algorithm.
 The maps are stored in the octree based representation (octomaps) but during transformation
 estimation, a point cloud representation is used as well. In addition, the presented method was verified in various experiments: in a simulation, with wheeled robots, and with publicly available datasets.
 Eventually, the solution can be applied to many robotic applications related to the exploration of unknown environments.
 Nevertheless, so far the implemented method was validated with a group of wheeled robots.

Contributions of expected learning progress and perceptual novelty to curiosity-driven exploration

Exploration is curiosity-driven when it relies on the intrinsic motivation to know rather than on extrinsic rewards. Recent evidence shows that artificial agents perform better on a variety of tasks when their learning is curiosity-driven, and humans often engage in curiosity-driven learning when sampling information from the environment. However, which mechanisms underlie curiosity is still unclear. Here, we let participants freely explore different unknown environments that contained learnable sequences of events with varying degrees of noise and volatility. A hierarchical reinforcement learning model captured how participants were learning in these different kinds of unknown environments, and it also tracked the errors they expected to make and the learning opportunities they were planning to seek. With this computational approach, we show that participants' exploratory behavior is guided by learning progress and perceptual novelty. Moreover, we demonstrate an overall tendency of participants to avoid extreme forms of uncertainty. These findings elucidate the cognitive mechanisms that underlie curiosity-driven exploration of unknown environments. Implications of this novel way of quantifying curiosity within a reinforcement learning framework are discussed.

H2GNN: Hierarchical-Hops Graph Neural Networks for Multi-Robot Exploration in Unknown Environments

Multi-robot coarse-to-fine exploration in unknown environments makes great sense in many application fields like search and rescue. For different stages of the task, robots need to extract information from the environment discriminately, which can improve their decision-making capability. To this end, we present the Hierarchical-Hops Graph Neural Networks (H2GNN) to enable robots to targetedly integrate the key information of the graph-represented environment, which distinguishes the importance of information from different hops around robots based on the multi-head attention mechanism. And in order to improve the efficiency of cooperation, we utilize multi-agent reinforcement learning (MARL) to help robots to learn collaborative strategies implicitly. We conduct experiments to verify our proposed method in a simulation environment, and the experimental results demonstrate that the H2GNN significantly improves the multi-robot exploration performance in unknown environments.

Information-Driven Fast Marching Autonomous Exploration With Aerial Robots

Autonomous exploration in unknown environments is a fundamental task of Unmanned Aerial Vehicles (UAVs). To choose exploration goals wisely, we propose an information-driven exploration strategy by applying the fast marching method to exploration of UAVs. A frontier point detection algorithm is designed to obtain Candidate Goals (CGs) by utilizing the structural characteristics of the octree-based map. With the sum of the information gain during the exploration journey as an evaluation indicator, we present a novel utility function to evaluate CGs by considering the trade-off between information gain and travel consumption. Given the effect of the environment on UAVs, UAVs are required to march aggressively in the Euclidean Symbol Distance Field (ESDF) to calculate the flight time, which is defined as the travel consumption. The uncertainty of the environment is minimized gradually by maximizing the utility function during each exploration journey. To take full advantage of the mobility of UAVs, we perform B-spline trajectory optimization and yaw angle planning based on the fast marching paths. We conduct sufficient comparison and evaluation experiments in simulation environments. Experimental results show the superiority of the proposed exploration strategy. The code related to the experiments will be published at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/BoLeiChen/fastmarching-exploration</uri> .

A Shadowcasting-Based Next-Best-View Planner for Autonomous 3D Exploration

In this letter, we address the problem of autonomous exploration of unknown environments with an aerial robot equipped with a sensory set that produces large point clouds, such as LiDARs. The main goal is to gradually explore an area while planning paths and calculating information gain in short computation time, suitable for implementation on an on-board computer. To this end, we present a planner that randomly samples viewpoints in the environment map. It relies on a novel and efficient gain calculation based on the Recursive Shadowcasting algorithm. To determine the Next-Best-View (NBV), our planner uses a cuboid-based evaluation method that results in an enviably short computation time. To reduce the overall exploration time, we also use a dead end resolving strategy that allows us to quickly recover from dead ends in a challenging environment. Comparative experiments in simulation have shown that our approach outperforms the current state-of-the-art in terms of computational efficiency and total exploration time. The video of our approach can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://www.youtube.com/playlist?list =PLC0C6uwoEQ8ZDhny1VdmFXLeTQOSBibQl</uri> .

Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning

In this letter, we present an autonomous navigation system for goal-driven exploration of unknown environments through deep reinforcement learning (DRL). Points of interest (POI) for possible navigation directions are obtained from the environment and an optimal waypoint is selected, based on the available data. Following the waypoints, the robot is guided towards the global goal and the local optimum problem of reactive navigation is mitigated. Then, a motion policy for local navigation is learned through a DRL framework in a simulation. We develop a navigation system where this learned policy is integrated into a motion planning stack as the local navigation layer to move the robot between waypoints towards a global goal. The fully autonomous navigation is performed without any prior knowledge while a map is recorded as the robot moves through the environment. Experiments show that the proposed method has an advantage over similar exploration methods, without reliance on a map or prior information in complex static as well as dynamic environments.

Multi-Agent Autonomy: Advancements and Challenges in Subterranean Exploration

Artificial intelligence has undergone immense growth and maturation in recent years, though autonomous systems have traditionally struggled when fielded in diverse and previously unknown environments. DARPA is seeking to change that with the Subterranean Challenge, by providing roboticists the opportunity to support civilian and military first responders in complex and high-risk underground scenarios. The subterranean domain presents a handful of challenges, such as limited communication, diverse topology and terrain, and degraded sensing. Team MARBLE proposes a solution for autonomous exploration of unknown subterranean environments in which coordinated agents search for artifacts of interest. The team presents two navigation algorithms in the form of a metric-topological graph-based planner and a continuous frontier-based planner. To facilitate multi-agent coordination, agents share and merge new map information and candidate goal points. Agents deploy communication beacons at different points in the environment, extending the range at which maps and other information can be shared. Onboard autonomy reduces the load on human supervisors, allowing agents to detect and localize artifacts and explore autonomously outside established communication networks. Given the scale, complexity, and tempo of this challenge, a range of lessons was learned, most importantly, that frequent and comprehensive field testing in representative environments is key to rapidly refining system performance.

A Hybrid and Hierarchical Approach for Spatial Exploration in Dynamic Environments

Exploration in unknown dynamic environments is a challenging problem in an AI system, and current techniques tend to produce irrational exploratory behaviours and fail in obstacle avoidance. To this end, we present a three-tiered hierarchical and modular spatial exploration model that combines the intrinsic motivation integrated deep reinforcement learning (DRL) and rule-based real-time obstacle avoidance approach. We address the spatial exploration problem in two levels on the whole. On the higher level, a DRL based global module learns to determine a distant but easily reachable target that maximizes the current exploration progress. On the lower level, another two-level hierarchical movement controller is used to produce locally smooth and safe movements between targets based on the information of known areas and free space assumption. Experimental results on diverse and challenging 2D dynamic maps show that the proposed model achieves almost 90% coverage and generates smoother trajectories compared with a state-of-the-art IM based DRL and some other heuristic methods on the basis of avoiding obstacles in real time.

DGS-SLAM: A Fast and Robust RGBD SLAM in Dynamic Environments Combined by Geometric and Semantic Information

Visual Simultaneous Localization and Mapping (VSLAM) is a prerequisite for robots to accomplish fully autonomous movement and exploration in unknown environments. At present, many impressive VSLAM systems have emerged, but most of them rely on the static world assumption, which limits their application in real dynamic scenarios. To improve the robustness and efficiency of the system in dynamic environments, this paper proposes a dynamic RGBD SLAM based on a combination of geometric and semantic information (DGS-SLAM). First, a dynamic object detection module based on the multinomial residual model is proposed, which executes the motion segmentation of the scene by combining the motion residual information of adjacent frames and the potential motion information of the semantic segmentation module. Second, a camera pose tracking strategy using feature point classification results is designed to achieve robust system tracking. Finally, according to the results of dynamic segmentation and camera tracking, a semantic segmentation module based on a semantic frame selection strategy is designed for extracting potential moving targets in the scene. Extensive evaluation in public TUM and Bonn datasets demonstrates that DGS-SLAM has higher robustness and speed than state-of-the-art dynamic RGB-D SLAM systems in dynamic scenes.

Learning Risk-aware Costmaps for Traversability in Challenging Environments.

One of the main challenges in autonomous robotic exploration and navigation in unknown and unstructured environments is determining where the robot can or cannot safely move. A significant source of difficulty in this determination arises from stochasticity and uncertainty, coming from localization error, sensor sparsity and noise, difficult-to-model robot-ground interactions, and disturbances to the motion of the vehicle. Classical approaches to this problem rely on geometric analysis of the surrounding terrain, which can be prone to modeling errors and can be computationally expensive. Moreover, modeling the distribution of uncertain traversability costs is a difficult task, compounded by the various error sources mentioned above. In this work, we take a principled learning approach to this problem. We introduce a neural network architecture for robustly learning the distribution of traversability costs. Because we are motivated by preserving the life of the robot, we tackle this learning problem from the perspective of learning tail-risks, i.e. the conditional value-at-risk (CVaR). We show that this approach reliably learns the expected tail risk given a desired probability risk threshold between 0 and 1, producing a traversability costmap which is more robust to outliers, more accurately captures tail risks, and is more computationally efficient, when compared against baselines. We validate our method on data collected by a legged robot navigating challenging, unstructured environments including an abandoned subway, limestone caves, and lava tube caves.

A New Foot Trajectory Planning Method for Legged Robots and Its Application in Hexapod Robots

Compared with wheeled and tracked robots, legged robots have better movement ability and are more suitable for the exploration of unknown environments. In order to further improve the adaptability of legged robots to complex terrains such as slopes, obstacle environments, and so on, this paper makes a new design of the legged robot’s foot sensing structure that can successfully provide accurate feedback of the landing information. Based on this information, a new foot trajectory planning method named three-element trajectory determination method is proposed. For each leg in one movement period, the three elements are the start point in the support phase, the end point in the support phase, and the joint angle changes in the transfer phase where the first two elements are used to control the height, distance, and direction of the movement, and the third element is used make decisions during the lifting process of the leg. For the support phase, the trajectory is described in Cartesian space, and a spline of linear function with parabolic blends is used. For the transfer phase, the trajectory is described in joint-space, and the joint angle function is designed as the superposition of the joint angle reverse-chronological function and the interpolation function which is obtained based on joint angle changes. As an important legged robot, a hexapod robot that we designed by ourselves with triangle gait is chosen to test the proposed foot trajectory planning method. Experiments show that, while the foot’s landing information can be read and based on the three-element trajectory planning method, the hexapod robot can achieve stable movement even in very complex scenes. Although the experiments are performed on a hexapod robot, our method is applicable to all forms of legged robots.

Internal and external frontier‐based algorithm for autonomous mobile robot exploration in unknown environment

Navigation in the absence of initial environmental information is a situation in which a robot is faced with the difficulty of traversing an unknown area for exploration with obtaining the environmental information simultaneously. Therefore, to complete and optimize the exploration efficiently, the robot needs an autonomous path-planning algorithm. This work proposes a new autonomous path-planning algorithm for exploration in an unknown environment based on paired frontiers, which we call internal and external frontiers algorithm (IEFA), that defines extended area for navigation of the mobile robot. For each exploration round, the robot defines external frontiers using the maximum range of sensors. Then, the robot generates internal frontiers, that is, pairs of external frontiers by varying the range of sensors. According to the size of each pair of frontiers, the algorithm generates the target point for robot navigation. The frontiers of internal layer are utilized as a main parameter for generation of next exploration point. We evaluated the proposed algorithm in simulation environments using the ROS toolbox of MATLAB and compared it with two previous exploration algorithms. From the experimental results, the proposed algorithm showed from 31% to 85% better performance in the path distance than previous algorithms.

Autonomous Exploration of Mobile Robots via Deep Reinforcement Learning Based on Spatiotemporal Information on Graph

In this paper, we address the problem of autonomous exploration in unknown environments for ground mobile robots with deep reinforcement learning (DRL). To effectively explore unknown environments, we construct an exploration graph considering historical trajectories, frontier waypoints, landmarks, and obstacles. Meanwhile, to take full advantage of the spatiotemporal feature and historical information in the autonomous exploration task, we propose a novel network called Spatiotemporal Neural Network on Graph (Graph-STNN). Specifically, the proposed Graph-STNN extracts the spatial feature using graph convolutional network (GCN) and the temporal feature using temporal convolutional network (TCN). Then, gated recurrent unit (GRU) is performed to synthesize the spatial feature, the temporal feature, and the historical state information into the current state feature. Combined with DRL, our Graph-STNN helps estimation of the optimal target point through extracted hybrid features. The simulation experiment shows that our approach is more effective than the GCN-based approach and the information entropy-based approach. Moreover, Graph-STNN also performs better generalization ability than GCN-based, information entropy-based, and random methods. Finally, we validate our approach on the simulation platform Stage with the actual robot model.

Adaptive Coordinated Control Strategy of Multi Manipulator System based on Multi-Agent

With people's increasing awareness of life and the increasing complexity of exploration in unknown environment, a single robot can not meet the increasing demand, including the price, flexibility and efficiency of robots. As a common mechanical control system in industrial production instead of human production, multi manipulator system can be applied in complex environment, multi task and other conditions. In order to settle the coordinated control fault of multi manipulator system, we study adaptive coordinated control strategy with the help of multi-agent research method in this paper, which can simplify the complexity of the problem and design an efficient and feasible system control protocol. The complex items in the multi manipulator system are treated as non affine systems. Using the design idea of non affine algorithm, combined with implicit function theorem and median theorem, the non affine system is transformed into affine systems, the controller is separated, and a distributed adaptive control strategy is designed. The results indicate that manipulator systems can effectively track the active manipulator system in finite time and the significance of the algorithm is proven by MATLAB simulation analysis.

Collision‐free local planner for unknown subterranean navigation

When operating in confined spaces or near obstacles, collision-free path planning is an essential requirement for autonomous exploration in unknown environments. This study presents an autonomous exploration technique using a carefully designed collision-free local planner. Using LiDAR range measurements, a local end-point selection method is designed, and the path is generated from the current position to the selected end-point. The generated path showed the consistent collision-free path in real-time by adopting the Euclidean signed distance field-based grid-search method. The results consistently demonstrated the safety and reliability of the proposed path-planning method. Real-world experiments are conducted in three different mines, demonstrating successful autonomous exploration flights in environment with various structural conditions. The results showed the high capability of the proposed flight autonomy framework for lightweight aerial robot systems. In addition, our drone performed an autonomous mission in the tunnel circuit competition (Phase 1) of the DARPA Subterranean Challenge.

Unified robot task and motion planning with extended planner using ROS simulator

Safe and efficient exploration of unknown and unstructured environments is critical for an autonomous mobile robot in high-dimensional workspace. Autonomous exploration is essential to safely and robustly build a complete and accurate path in an environment map with obstacles. Task planning tends to find a sequence of discrete actions in solving the symbolic problems. Motion planning finds the optimal path by avoiding the geometric obstacles in the environment. Fusion of these two approaches is advantageous in finding a path in unknown environments. This paper presents a hybrid path planning approach, termed RobMAP (Robotic Motion and Action Planning) for achieving efficient navigation by performing desired tasks in a sequence and thereby generate the optimal path in a large- scale indoor environment. In achieving this, action-planning is done using Planning Domain Definition Language (PDDL) planner through ROSPlan framework and collision-free path is generated using sampling-based Rapidly exploring Random Tree (RRT) algorithm. The ROSPlan system is used in less constrained workspace and it estimates the optimized information gain in exploring each action. Further, the integration of these two methods leads to an intelligent motion planner that searches for an optimal path towards a goal based on the PDDL action planner specification. Extensive experimentation is performed using Robot Operating System (ROS) controlled Rviz tool on a synthetic indoor environment by executing a sequence of actions specified using PDDL. The Turtlebot is equipped with a laser sensor and it explores the environment map with higher precision and accuracy in finding the optimal path. The demonstrated results corroborate that the proposed method of integrating RRT with PDDL provides optimal solutions with lesser path length in much lesser run time. The accuracy, precision and coverage area are higher when compared with other exploration strategies used in the existing work.

Collaborative Complete Coverage and Path Planning for Multi-Robot Exploration.

In mobile robotics research, the exploration of unknown environments has always been an important topic due to its practical uses in consumer and military applications. One specific interest of recent investigation is the field of complete coverage and path planning (CCPP) techniques for mobile robot navigation. In this paper, we present a collaborative CCPP algorithms for single robot and multi-robot systems. The incremental coverage from the robot movement is maximized by evaluating a new cost function. A goal selection function is then designed to facilitate the collaborative exploration for a multi-robot system. By considering the local gains from the individual robots as well as the global gain by the goal selection, the proposed method is able to optimize the overall coverage efficiency. In the experiments, our CCPP algorithms are carried out on various unknown and complex environment maps. The simulation results and performance evaluation demonstrate the effectiveness of the proposed collaborative CCPP technique.

An autonomous unmanned aerial vehicle system for fast exploration of large complex indoor environments

AbstractThis paper introduces an autonomous system employing multirotor unmanned aerial vehicles for fast 3D exploration and inspection of vast, unknown, dynamic, and complex environments containing large open spaces as well as narrow passages. The system exploits the advantage of small‐size aerial vehicles capable of carrying all necessary sensors and computational power while providing full autonomy and mobility in constrained unknown environments. Particular emphasis is put on the robustness of the algorithms with respect to challenging real‐world conditions and the real‐time performance of all algorithms that enable fast reactions to changes in environment and thus also provide effective use of limited flight time. The system presented here was employed as a part of a heterogeneous ground and aerial system in the modeled Search &amp; Rescue scenario in an unfinished nuclear power plant during the Urban Circuit of the Subterranean Challenge (SubT Challenge) organized by the Defense Advanced Research Projects Agency. The main goal of this simulated disastrous scenario is to autonomously explore and precisely localize specified objects in a completely unknown environment and to report their position before the end of the mission. The proposed system was part of the multirobot team that finished in third place overall and in first place among the self‐funded teams. The proposed unmanned aerial vehicle system outperformed all aerial systems participating in the SubT Challenge with respect to versatility, and it was also the self‐deployable autonomous aerial system that explored the largest part of the environment.

Exploration of Unknown Environments Using Deep Reinforcement Learning

My research presents a method for efficient exploration of an outdoor, unknown area, which aims to achieve precise coverage of regions of interest within that area. While this method for autonomous exploration was designed for autonomous controllers in unmanned aerial vehicles (UAVs), the concepts apply to any vehicle which uses autonomous navigation. We consider an environment with areas of interest of various sizes littered throughout, and a reinforcement learning agent which is tasked with discovering and mapping these areas in an efficient manner.

PLGRIM: Hierarchical Value Learning for Large-scale Exploration in Unknown Environments

In order for an autonomous robot to efficiently explore an unknown environment, it must account for uncertainty in sensor measurements, hazard assessment, localization, and motion execution. Making decisions for maximal reward in a stochastic setting requires value learning and policy construction over a belief space, i.e., probability distribution over all possible robot-world states. However, belief space planning in a large spatial environment over long temporal horizons suffers from severe computational challenges. Moreover, constructed policies must safely adapt to unexpected changes in the belief at runtime. This work proposes a scalable value learning framework, PLGRIM (Probabilistic Local and Global Reasoning on Information roadMaps), that bridges the gap between (i) local, risk-aware resiliency and (ii) global, reward-seeking mission objectives. Leveraging hierarchical belief space planners with information-rich graph structures, PLGRIM addresses large-scale exploration problems while providing locally near-optimal coverage plans. We validate our proposed framework with high-fidelity dynamic simulations in diverse environments and on physical robots in Martian-analog lava tubes.