JEM: Joint Entropy Minimization for Active State Estimation with Linear POMDP Costs

Abstract

Active state estimation is the problem of controlling a partially observed Markov decision process (POMDP) to minimize the uncertainty associated with its latent states. Selecting meaningful, yet tractable, measures of uncertainty to optimize is a key challenge in active state estimation, with the vast majority of popular uncertainty measures leading to POMDP costs that are nonlinear in the belief state, which makes them difficult (and often impossible) to optimize directly using standard POMDP solvers. To address this challenge, in this paper we propose the joint entropy of the state, observation, and control trajectories of POMDPs as a novel tractable uncertainty measure for active state estimation. By expressing the joint entropy in stage-additive form, we show that joint-entropy-minimization (JEM) problems can be reformulated as standard POMDPs with cost functions that are linear in the belief state. Linearity of the costs is of considerable practical significance since it enables the solution of our JEM problems directly using standard POMDP solvers. We illustrate JEM in simulations where it reduces the probability of error in state trajectory estimates whilst being more computationally efficient than competing active state estimation formulations.