Informative Path Planning Research Articles

A Reinforcement Learning Framework for Efficient Informative Sensing

Large-scale spatial data can be collected using mobile robots with sensing and navigation capabilities. Due to limited battery lifetime and scarcity of charging stations, it is important to plan informative paths so as to maximize the utility of data given a limited travel budget, which is known as the informative path planning (IPP) problem. IPP is NP-hard, and existing solutions suffer from high complexity or low optimality. In this paper, we present a novel IPP solution based on reinforcement learning (RL). The basic idea is to learn the structural characteristics of informative paths, so informative paths can be predicted. As such, when budgets change, we avoid solving the problem from scratch and thus path planning efficiency can be improved dramatically. Among the 20 path planning experiments in two areas, the proposed RL based solution achieves the best path utility in 15 experiments, compared with state-of-the-art algorithms. More importantly, the inference complexity is linear with respect to the budget (equivalently, the maximum number of steps in RL), which is lower than other solutions. Despite the NP-hardness, the path planning process can be finished within a few seconds in our experiments on two graphs of different sizes.

Orienteering-based informative path planning for environmental monitoring

The use of robotic mobile sensors for environmental monitoring applications has gained increasing attention in recent years. In this context, a common application is to determine the region of space where the analyzed phenomena is above or below a given threshold level — this problem is known as level set estimation. One example is the analysis of water in a lake, where the operators might want to determine where the dissolved oxygen level is above a critical threshold value. Recent research proposes to model the spatial phenomena of interest using Gaussian Processes, and then use an informative path planning procedure to determine where to gather data. In this paper, in contrast to previous works, we consider the case where a mobile platform with low computational power can continuously acquire measurements with a negligible energy cost. This scenario imposes a change in the perspective, since now efficiency is achieved by reducing the distance traveled by the mobile platform and the computation required by this path selection process. In this paper we propose two active learning algorithms aimed at facing this issue: specifically, (i) SBOLSE casts informative path planning into an orienteering problem and (ii) PULSE that exploits a less accurate but computationally faster path selection procedure. Evaluation of our algorithms, both on a real world and a synthetic dataset show that our approaches can compute informative paths that achieve a high quality classification, while significantly reducing the travel distance and the computation time.

Data-driven planning via imitation learning

Robot planning is the process of selecting a sequence of actions that optimize for a task=specific objective. For instance, the objective for a navigation task would be to find collision-free paths, whereas the objective for an exploration task would be to map unknown areas. The optimal solutions to such tasks are heavily influenced by the implicit structure in the environment, i.e. the configuration of objects in the world. State-of-the-art planning approaches, however, do not exploit this structure, thereby expending valuable effort searching the action space instead of focusing on potentially good actions. In this paper, we address the problem of enabling planners to adapt their search strategies by inferring such good actions in an efficient manner using only the information uncovered by the search up until that time. We formulate this as a problem of sequential decision making under uncertainty where at a given iteration a planning policy must map the state of the search to a planning action. Unfortunately, the training process for such partial-information-based policies is slow to converge and susceptible to poor local minima. Our key insight is that if we could fully observe the underlying world map, we would easily be able to disambiguate between good and bad actions. We hence present a novel data-driven imitation learning framework to efficiently train planning policies by imitating a clairvoyant oracle: an oracle that at train time has full knowledge about the world map and can compute optimal decisions. We leverage the fact that for planning problems, such oracles can be efficiently computed and derive performance guarantees for the learnt policy. We examine two important domains that rely on partial-information-based policies: informative path planning and search-based motion planning. We validate the approach on a spectrum of environments for both problem domains, including experiments on a real UAV, and show that the learnt policy consistently outperforms state-of-the-art algorithms. Our framework is able to train policies that achieve up to more reward than state-of-the art information-gathering heuristics and a speedup as compared with A* on search-based planning problems. Our approach paves the way forward for applying data-driven techniques to other such problem domains under the umbrella of robot planning.

Real-time path planning to dispatch a mobile sensor into an operational area

This paper addresses problems of large planning time and cost uncertainty for informative path planning of a mobile sensor where the location of sensor deployment is different of that of an operational area. The first problem is that the cost has no term dependent on sensor state before arriving at the operational area and it causes large planning time. The information of the state of interest dissipates over time during the planning time and it degrades performance of sensing operation. The other problem is that the cost is dependent on the parameters to be estimated. To assess the cost, the target state in the future should be predicted by integrating the system model based on noisy initial estimate. The limitation of the informative path planning has a greater impact on performance in this specific problem. A strategy to cope with these problems is to devise a real-time path planning algorithm by using online optimization. The proposed algorithm is divided into two phases; determining the path to the boundary of the operational area and guiding the sensor by an informative potential field in the area. Detailed analysis on performance of the proposed algorithm compared to an optimal solution by nonlinear programming is given. The simulation results have demonstrated that the proposed algorithm can cope with performance degradation observed in the optimal solution.

The Team Surviving Orienteers problem: routing teams of robots in uncertain environments with survival constraints

We study the following multi-robot coordination problem: given a graph, where each edge is weighted by the probability of surviving while traversing it, find a set of paths for K robots that maximizes the expected number of nodes collectively visited, subject to constraints on the probabilities that each robot survives to its destination. We call this the Team Surviving Orienteers (TSO) problem, which is motivated by scenarios where a team of robots must traverse a dangerous environment, such as aid delivery after disasters. We present the TSO problem formally along with several variants, which represent "survivability-aware" counterparts for a wide range of multi-robot coordination problems such as vehicle routing, patrolling, and informative path planning. We propose an approximate greedy approach for selecting paths, and prove that the value of its output is within a factor $$1-e^{-p_s/\lambda }$$1-e-ps/ź of the optimum where $$p_s$$ps is the per-robot survival probability threshold, and $$1/\lambda \le 1$$1/ź≤1 is the approximation factor of an oracle routine for the well-known orienteering problem. We also formalize an on-line update version of the TSO problem, and a generalization to heterogeneous teams where both robot types and paths are selected. We provide numerical simulations which verify our theoretical findings, apply our approach to real-world scenarios, and demonstrate its effectiveness in large-scale problems with the aid of a heuristic for the orienteering problem.

Adaptive continuous‐space informative path planning for online environmental monitoring

AbstractAutonomous mobile robots are increasingly employed to take measurements for environmental monitoring, but planning informative, measurement‐rich paths through large three‐dimensional environments is still challenging. Designing such paths, known as the informative path planning (IPP) problem, has been shown to be NP‐hard. Existing algorithms focus on providing guarantees on suboptimal solutions, but do not scale well to large problems. In this paper, we introduce a novel IPP algorithm that uses an evolutionary strategy to optimize a parameterized path in continuous space, which is subject to various constraints regarding path budgets and motion capabilities of an autonomous mobile robot. Moreover, we introduce a replanning scheme to adapt the planned paths according to the measurements taken in situ during data collection. When compared to two state‐of‐the‐art solutions, our method provides competitive results at significantly lower computation times and memory requirements. The proposed replanning scheme enables to build models with up to 25% lower uncertainty within an initially unknown area of interest. Besides presenting theoretical results, we tailored the proposed algorithms for data collection using an autonomous surface vessel for an ecological study, during which the method was validated through three field deployments on Lake Zurich, Switzerland. Spatiotemporal variations are shown over a period of three months and in an area of 350 m × 350 m × 13 m. Whereas our theoretical solution can be applied to multiple applications, our field results specifically highlight the effectiveness of our planner for monitoring toxic microorganisms in a pre‐alpine lake, and for identifying hot‐spots within their distribution.

Physiopsychological research about the effect of playing in interpersonal relationship

This paper addresses problems of large planning time and cost uncertainty for informative path planning of a mobile sensor where the location of sensor deployment is different of that of an operational area. The first problem is that the cost has no term dependent on sensor state before arriving at the operational area and it causes large planning time. The information of the state of interest dissipates over time during the planning time and it degrades performance of sensing operation. The other problem is that the cost is dependent on the parameters to be estimated. To assess the cost, the target state in the future should be predicted by integrating the system model based on noisy initial estimate. The limitation of the informative path planning has a greater impact on performance in this specific problem. A strategy to cope with these problems is to devise a real-time path planning algorithm by using online optimization. The proposed algorithm is divided into two phases; determining the path to the boundary of the operational area and guiding the sensor by an informative potential field in the area. Detailed analysis on performance of the proposed algorithm compared to an optimal solution by nonlinear programming is given. The simulation results have demonstrated that the proposed algorithm can cope with performance degradation observed in the optimal solution.

Guest Editorial Special Section on the 2014 Workshop on the Algorithmic Foundations of Robotics

The papers in this special section were presented at the 11th Workshop on the Algorithmic Foundation of Robotics (WAFR), which was held at Bogazici University, Istanbul, Turkey, during August 3–5, 2014. WAFR is a prestigious biennial single-track workshop on algorithms for robotics and automation. It features cutting-edge research in a broad range of planning problems (such as manipulation, motion, path, multi-robot, and kynodynamic planning), geometric and topological computation, and novel applications like surgical planning, active sensing, and informative path planning.

Adaptive informative path planning in metric spaces

In contrast to classic geometric motion planning, informative path planning (IPP) seeks a path for a robot to sense the world and gain information. In adaptive IPP, the robot chooses the next sensing location conditioned on all information acquired so far, and the robot’s goal is to minimize the travel cost required for identifying a true hypothesis. Adaptive IPP is NP-hard, because the robot must trade-off information gain and travel cost optimally. In this paper we present Recursive Adaptive Identification (RAId), a new polynomial-time approximation algorithm for adaptive IPP. We prove a polylogarithmic approximation bound when the robot travels in a metric space. Furthermore, our experiments suggest that RAId is practical and provides good approximate solutions for two distinct robot planning tasks. Although RAId is designed primarily for noiseless observations, a simple extension allows it to handle some tasks with noisy observations.

Informative path planning as a maximum traveling salesman problem with submodular rewards

In this paper we extend the classic problem of finding the maximum weight Hamiltonian cycle in a graph to the case where the objective is a submodular function of the edges. We consider a greedy algorithm and a 2-matching based algorithm, and we show that they have approximation factors of 12+κ and max{23(2+κ),23(1−κ)} respectively, where κ is the curvature of the submodular function. Both algorithms require a number of calls to the submodular function that is cubic to the number of vertices in the graph. We then present a method to solve a multi-objective optimization consisting of both additive edge costs and submodular edge rewards. We provide simulation results to empirically evaluate the performance of the algorithms. Finally, we demonstrate an application in monitoring an environment using an autonomous mobile sensor, where the sensing reward is related to the entropy reduction of a given set of measurements.

Learning the Covariance Dynamics of a Large-Scale Environment for Informative Path Planning of Unmanned Aerial Vehicle Sensors

This work addresses problems regarding trajectory planning for unmanned aerial vehicle sensors. Such sensors are used for taking measurements of large nonlinear systems. The sensor investigations presented here entails methods for improving estimations and predictions of large nonlinear systems. Thoroughly understanding the global system state typically requires probabilistic state estimation. Thus, in order to meet this requirement, the goal is to find trajectories such that the measurements along each trajectory minimize the expected error of the predicted state of the system. The considerable nonlinearity of the dynamics governing these systems necessitates the use of computationally costly Monte-Carlo estimation techniques, which are needed to update the state distribution over time. This computational burden renders planning to be infeasible since the search process must calculate the covariance of the posterior state estimate for each candidate path. To resolve this challenge, this work proposes to replace the computationally intensive numerical prediction process with an approximate covariance dynamics model learned using a nonlinear time-series regression. The use of autoregressive time-series featuring a regularized least squares algorithm facilitates the learning of accurate and efficient parametric models. The learned covariance dynamics are demonstrated to outperform other approximation strategies, such as linearization and partial ensemble propagation, when used for trajectory optimization, in terms of accuracy and speed, with examples of simplified weather forecasting.

Efficient Informative Sensing using Multiple Robots

The need for efficient monitoring of spatio-temporal dynamics in large environmental applications, such as the water quality monitoring in rivers and lakes, motivates the use of robotic sensors in order to achieve sufficient spatial coverage. Typically, these robots have bounded resources, such as limited battery or limited amounts of time to obtain measurements. Thus, careful coordination of their paths is required in order to maximize the amount of information collected, while respecting the resource constraints. In this paper, we present an efficient approach for near-optimally solving the NP-hard optimization problem of planning such informative paths. In particular, we first develop eSIP (efficient Single-robot Informative Path planning), an approximation algorithm for optimizing the path of a single robot. Hereby, we use a Gaussian Process to model the underlying phenomenon, and use the mutual information between the visited locations and remainder of the space to quantify the amount of information collected. We prove that the mutual information collected using paths obtained by using eSIP is close to the information obtained by an optimal solution. We then provide a general technique, sequential allocation, which can be used to extend any single robot planning algorithm, such as eSIP, for the multi-robot problem. This procedure approximately generalizes any guarantees for the single-robot problem to the multi-robot case. We extensively evaluate the effectiveness of our approach on several experiments performed in-field for two important environmental sensing applications, lake and river monitoring, and simulation experiments performed using several real world sensor network data sets.