Adaptive spatial discretization using reinforcement learning

Abstract

AbstractA well-known challenge for deformation monitoring is the spatial discretization, i.e. the choice of monitoring points at which measurements are to be taken. Well-chosen monitoring points employ prior knowledge to yield a significant amount of information about a certain aspect of the monitored object. However, the choice of such a set of points is typically made to be practically expedient or left to the measurement instrument itself. We aim to derive adaptive discretization strategies that implicitly incorporate domain knowledge about the monitored object via a cycle of interaction and learning. In those strategies, previous measurements impact the locations of subsequent ones. We formulate the choice of monitoring points as a decision theoretical problem and review the framework of reinforcement learning which formalizes the problem of deriving optimal sequential decisions under uncertainty. Iterative algorithms produce solution schemes for this optimal control task. We benchmark the performance of reinforcement learning and compare its results to random, pseudorandom, and numerically designed discretization strategies on several geodetically motivated examples. Advantages, disadvantages, and practical feasibility of the approach are evaluated and reveal a significant boost in efficiency of the data collection scheme compared to classical approaches.