Efficient solutions of interactive dynamic influence diagrams using model identification

Abstract

Interactive dynamic influence diagram (I-DID) is one of the graphical frameworks for sequential decision making in partially observable environment. Subject agent in I-DID maintains beliefs over not only physical states of the environment, but also over models of the other agents. Consequently, solving I-DIDs suffers from the exponential growth of models ascribed to the other agents over time. Previous methods to solve I-DIDs aim at clustering equivalent models by comparing the entire or partial policy trees of the candidate models, which is time-consuming. In this paper, we present a new method for further reducing the model space by identifying the true model of the other agent and pruning the other irrelevant models. Toward this, we use an information-theoretic method—mutual information to measure the relevance between the candidate models and the true model in terms of predicted and observed actions of the other agent. We construct a dynamic Bayesian network to learn the value of parameters needed in the computation of mutual information. This approach bounds the model space by containing only the true model of the other agent. We evaluate our approach on multiple problem domains and empirically demonstrate the efficiency in solving I-DIDs.