Reference duality and representation duality in information geometry

Abstract

Classical information geometry prescribes, on the parametric family of probability functions Mθ: (i) a Riemannian metric given by the Fisher information; (ii) a pair of dual connections (giving rise to the family of α-connections) that preserve the metric under parallel transport by their joint actions; and (iii) a family of (non-symmetric) divergence functions (α-divergence) defined on Mθ × Mθ, which induce the metric and the dual connections. The role of α parameter, as used in α-connection and in α-embedding, is not commonly differentiated. For instance, the case with α = ±1 may refer either to dually-flat (e- or m-) connections or to exponential and mixture families of density functions. Here we illuminate that there are two distinct types of duality in information geometry, one concerning the referential status of a point (probability function, normalized or denormalized) expressed in the divergence function ("reference duality") and the other concerning the representation of probability functions under an arbitrary monotone scaling ("representation duality"). They correspond to, respectively, using α as a mixture parameter for constructing divergence functions or as a power exponent parameter for monotone embedding of probability functions. These two dualities are coupled into referential-representational biduality for manifolds of denormalized probability functions with α-Hessian structure (i.e, transitively flat α-geometry) and for manifolds induced from homogeneous divergence functions with (α,β)-parameters but one-parameter family of (α ⋅ β)-connections.