Modeling the world by minimizing relative entropy

Abstract

Relative entropy and the principle of minimum relative entropy are proposed as fundamental concepts that can be used as a foundation for designing and building adaptive, generally intelligent agents that build a model of the world. An equation is derived that describes how such an agent can build a model of the world by minimizing relative entropies. With this equation we prove that the agent then continually improves its model of the world. A main advantage of this equation and the derivation is that it clarifies and shows the relation between different uses and interpretations of relative entropy, such as measures of dissimilarity, learning progress or surprise, minimization and maximization of relative entropies for minimal belief updates and how it can select actions that lead to exploratory and curious behavior. As an example we applied our equation and approach to play the game of Mastermind.