Abstract
The expected free energy (EFE) is a central quantity in the theory of active inference. It is the quantity that all active inference agents are mandated to minimize through action, and its decomposition into extrinsic and intrinsic value terms is key to the balance of exploration and exploitation that active inference agents evince. Despite its importance, the mathematical origins of this quantity and its relation to the variational free energy (VFE) remain unclear. In this letter, we investigate the origins of the EFE in detail and show that it is not simply "the free energy in the future." We present a functional that we argue is the natural extension of the VFE but actively discourages exploratory behavior, thus demonstrating that exploration does not directly follow from free energy minimization into the future. We then develop a novel objective, the free energy of the expected future (FEEF), which possesses both the epistemic component of the EFE and an intuitive mathematical grounding as the divergence between predicted and desired futures.
Highlights
The Free-Energy Principle (FEP) (K. Friston, 2010; K. Friston & Ao, 2012a; K. Friston, Kilner, & Harrison, 2006) is an emerging theory from theoretical neuroscience which offers a unifying explanation of the dynamics of selforganising systems (K. Friston, 2019; Parr, Da Costa, & Friston, 2020)
We argue that the Expected Free Energy (EFE) is not the only functional which can quantify the notion of the free energy of policy-conditioned futures, and we propose a different functional The Free Energy of the Future, which we argue is a more natural extension of the Variational Free Energy (VFE) to account for future states
We argue that the natural extension of the free energy into the future must possess direct analogs to the two crucial properties of the VFE: it must be expressible as a KL-divergence between a posterior and a generative model, such that minimizing it causes the variational density to better approximate the true posterior
Summary
The Free-Energy Principle (FEP) (K. Friston, 2010; K. Friston & Ao, 2012a; K. Friston, Kilner, & Harrison, 2006) is an emerging theory from theoretical neuroscience which offers a unifying explanation of the dynamics of selforganising systems (K. Friston, 2019; Parr, Da Costa, & Friston, 2020). Friston, 2019; Parr, Da Costa, & Friston, 2020) It proposes that such systems can be interpreted as embodying a process of variational inference which minimizes a single information-theoretic objective – the Variational Free-Energy (VFE). We propose our own mathematically principled starting point for action-selection under active inference – the divergence between desired and expected futures, from which we obtain a novel functional the Free-Energy of the Expected Future (FEEF), which has close relations to the generalized free energy (Parr & Friston, 2019). This functional has a natural interpretation in terms of the divergence between a veridicial and a biased generative model; it allows use of the same functional for both inference and policy selection, and it naturally decomposes into an extrinsic value term and an epistemic action term, maintaining the attractive exploratory properties of EFE-based active inference while possessing a mathematically principled starting point with an intuitive interpretation
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
