Abstract
We show that the predicted probability distributions for any $N$-parameter statistical model taking the form of an exponential family can be explicitly and analytically embedded isometrically in a $N{+}N$-dimensional Minkowski space. That is, the model predictions can be visualized as control parameters are varied, preserving the natural distance between probability distributions. All pairwise distances between model instances are given by the symmetrized Kullback-Leibler divergence. We give formulas for these intensive symmetrized Kullback Leibler (isKL) coordinate embeddings, and illustrate the resulting visualizations with the Bernoulli (coin toss) problem, the ideal gas, $n$ sided die, the nonlinear least squares fit, and the Gaussian fit. We highlight how isKL can be used to determine the minimum number of parameters needed to describe probabilistic data, and conclude by visualizing the prediction space of the two-dimensional Ising model, where we examine the manifold behavior near its critical point.
Highlights
Many features of multiparameter models are best understood by studying the manifold of model predictions [1]
We shall illustrate many times in the rest of this manuscript that this bound no longer holds when one considers embeddings in Minkowski space. These Minkowski space embeddings can be constructed by defining a pairwise distance between probability distributions dS that violates the triangle inequality, which in turn breaks the curse of dimensionality, as noted in [11]
Just as any rotation or translation of an object isometrically embedded in Euclidean space forms another isometric embedding, so there is a family of intensive symmetrized Kullback-Leibler (isKL) embeddings formed by the isometries of Minkowski space
Summary
Many features of multiparameter models are best understood by studying the manifold of model predictions [1]. The Ising model of magnetism and the lambda cold dark matter ( CDM) model of the cosmic microwave background predict the underlying statistics for experimental observation or, more generally, a distribution of possible. Local analysis of parameter sensitivity shows that the Ising model [7] and the CDM model [11] are sloppy, in the sense that they have a hierarchy of sensitivity eigenvalues spanning many decades These local sensitivities are quantitatively measured by the natural distance in the space of probability distributions, the Fisher information metric (FIM) [12]. [11], it was shown that the model manifold of probability distributions can be visualized using Intensive Principle Component Analysis (InPCA) by embedding in a Minkowski space. V how we can use this method to determine the minimum number of parameters needed to describe probabilistic data
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
