Space Of Probability Densities Research Articles

Generalized functional additive mixed models with (functional) compositional covariates for areal Covid-19 incidence curves

Abstract We extend the generalized functional additive mixed model to include compositional and functional compositional (density) covariates carrying relative information of a whole. Relying on the isometric isomorphism of the Bayes Hilbert space of probability densities with a sub-space of the L2, we include functional compositions as transformed functional covariates with constrained yet interpretable effect function. The extended model allows for the estimation of linear, non-linear, and time-varying effects of scalar and functional covariates, as well as (correlated) functional random effects, in addition to the compositional effects. We use the model to estimate the effect of the age, sex, and smoking (functional) composition of the population on regional Covid-19 incidence data for Spain, while accounting for climatological and socio-demographic covariate effects and spatial correlation.

Fisher information and shape-morphing modes for solving the Fokker–Planck equation in higher dimensions

The Fokker–Planck equation describes the evolution of the probability density associated with a stochastic differential equation. As the dimension of the system grows, solving this partial differential equation (PDE) using conventional numerical methods becomes computationally prohibitive. Here, we introduce a fast, scalable, and interpretable method for solving the Fokker–Planck equation which is applicable in higher dimensions. This method approximates the solution as a linear combination of shape-morphing Gaussians with time-dependent means and covariances. These parameters evolve according to the method of reduced-order nonlinear solutions (RONS) which ensures that the approximate solution stays close to the true solution of the PDE for all times. As such, the proposed method approximates the transient dynamics as well as the equilibrium density, when the latter exists. Our approximate solutions can be viewed as an evolution on a finite-dimensional statistical manifold embedded in the space of probability densities. We show that the metric tensor in RONS coincides with the Fisher information matrix on this manifold. We also discuss the interpretation of our method as a shallow neural network with Gaussian activation functions and time-varying parameters. In contrast to existing deep learning methods, our method is interpretable, requires no training, and automatically ensures that the approximate solution satisfies all properties of a probability density.

Geometric thermodynamics for the Fokker–Planck equation: stochastic thermodynamic links between information geometry and optimal transport

We propose a geometric theory of non-equilibrium thermodynamics, namely geometric thermodynamics, using our recent developments of differential-geometric aspects of entropy production rate in non-equilibrium thermodynamics. By revisiting our recent results on geometrical aspects of entropy production rate in stochastic thermodynamics for the Fokker–Planck equation, we introduce a geometric framework of non-equilibrium thermodynamics in terms of information geometry and optimal transport theory. We show that the proposed geometric framework is useful for obtaining several non-equilibrium thermodynamic relations, such as thermodynamic trade-off relations between the thermodynamic cost and the fluctuation of the observable, optimal protocols for the minimum thermodynamic cost and the decomposition of the entropy production rate for the non-equilibrium system. We clarify several stochastic-thermodynamic links between information geometry and optimal transport theory via the excess entropy production rate based on a relation between the gradient flow expression and information geometry in the space of probability densities and a relation between the velocity field in optimal transport and information geometry in the space of path probability densities.

JKO estimates in linear and non-linear Fokker–Planck equations, and Keller–Segel: $L^p$ and Sobolev bounds

We analyze some parabolic PDEs with different drift terms which are gradient flows in the Wasserstein space and consider the corresponding discrete-in-time JKO scheme. We prove with optimal transport techniques how to control the L p and L $\infty$ norms of the iterated solutions in terms of the previous norms, essentially recovering well-known results obtained on the continuous-in-time equations. Then we pass to higher order results, and in particulat to some specific BV and Sobolev estimates, where the JKO scheme together with the so-called five gradients inequality allows to recover some inequalities that can be deduced from the Bakry-Emery theory for diffusion operators, but also to obtain some novel ones, in particular for the Keller-Segel chemiotaxis model. 1 Short introduction The goal of this paper is to present some estimates on evolution PDEs in the space of probability densities which share two important features: they include a linear diffusion term, and they are gradient flows in the Wasserstein space W2. These PDEs will be of the form $\partial$t$\rho$ -- $\Delta$$\rho$ -- $\nabla$ $\times$ ($\rho$$\nabla$u[$\rho$]) = 0, complemented with no-flux boundary conditions and an intial condition on $\rho$0. We will in particular concentrate on the Fokker-Plack case, where u[$\rho$] = V and V is a fixed function (with possible regularity assumptions) independent of $\rho$, on the case where u[$\rho$] = W * $\rho$ is obtained by convolution and models interaction between particles, and on the parabolic-elliptic Keller-Segel case where u[$\rho$] is related to $\rho$ via an elliptic equation. This last case models the evolution of a biological population $\rho$ subject to diffusion but attracted by the concentration of a chemo-attractant, a nutrient which is produced by the population itself, so that its distribution is ruled by a PDE where the density $\rho$ appears as a source term. Under the assumption that the production rate of this nutrient is much faster than the motion of the cells, we can assume that its distribution is ruled by a statical PDE with no explicit time-dependence, and gives rise to a system which is a gradient flow in the variable $\rho$ (the parabolic-parabolic case, where the time scale for the cells and for the nutrient are comparable, is also a gradient flow, in the product space W2 x L 2 , but we will not consider this case). Since we mainly concentrate on the case of bounded domains, in the Keller-Segel case the term u[$\rho$] cannot be expressed as a convoluton and requires ad-hoc computations. In all the paper, the estimates will be studied on a time-discretized version of these PDEs, consisting in the so-called JKO (Jordan-Kinderleherer-Otto) scheme, based on iterated optimization problems involving the Wasserstein distance W2. We will first present 0-order estimates, on the L p and L $\infty$ norms of the solution. This is just a translation into the JKO language of well-known properties of these equations. The main goal of this part is hence to

DYNAMIC PROBABILISTIC FORECASTING WITH UNCERTAINTY

In this paper, we introduce a dynamical model for the time evolution of probability density functions incorporating uncertainty in the parameters. The uncertainty follows stochastic processes, thereby defining a new class of stochastic processes with values in the space of probability densities. The purpose is to quantify uncertainty that can be used for probabilistic forecasting. Starting from a set of traded prices of equity indices, we do some empirical studies. We apply our dynamic probabilistic forecasting to option pricing, where our proposed notion of model uncertainty reduces to uncertainty on future volatility. A distribution of option prices follows, reflecting the uncertainty on the distribution of the underlying prices. We associate measures of model uncertainty of prices in the sense of Cont.

Second-Order Models for Optimal Transport and Cubic Splines on the Wasserstein Space

On the space of probability densities, we extend the Wasserstein geodesics to the case of higher-order interpolation such as cubic spline interpolation. After presenting the natural extension of cubic splines to the Wasserstein space, we propose a simpler approach based on the relaxation of the variational problem on the path space. We explore two different numerical approaches, one based on multimarginal optimal transport and entropic regularization and the other based on semi-discrete optimal transport.

Dynamic Probabilistic Forecasting with Uncertainty

We introduce a dynamical model for the time evolution of probability density functions incorporating uncertainty in the parameters. The uncertainty follows stochastic processes, thereby defining a new class of stochastic processes with values in the space of probability densities. The purpose is to quantify uncertainty that can be used for probabilistic forecasting. Starting from a set of traded prices of equity indices such as FTSEMIB, FTSE100 and S&P500 we do some empirical studies. We apply our dynamic probabilistic forecasting to option pricing, where our proposed notion of model uncertainty reduces to uncertainty on future volatility. A distribution of option prices follows, reflecting the uncertainty on the distribution of the underlying prices. We associate measures of model uncertainty of prices in the context of Cont (2006). As a further application we look at the Sharpe ratio and the VaR measure of market risk as well, proposing some decision rules for investors, regulators and risk managers.

Information Geometric Approach to Multisensor Estimation Fusion

Distributed estimation fusion is concerned with the combination of local estimates from multiple distributed sensors to produce a fused result. In this paper, we characterize local estimates as posterior probability densities, and assume that they all belong to a parametric family. Our starting point is to consider this family as a Riemannian manifold by introducing the Fisher information metric. From the perspective of information geometry, the fused density is formulated as an informative barycenter in the space of probability densities and sought by minimizing the sum of its squared geodesic distances from the local posterior densities. Under Gaussian assumptions, a geodesic projection (GP) method and a Siegel distance (SD) method in the information-geometric framework are proposed to tackle the problem. The GP method gives a fusion result in accord with the covariance intersection estimate but under an information-geometric criterion, while the SD method appears to achieve a better approximation of the informative barycenter. Numerical examples are provided to demonstrate the performance of the proposed estimation fusion algorithms.

Entropy Dissipation Semi-Discretization Schemes for Fokker–Planck Equations

We propose a new semi-discretization scheme to approximate nonlinear Fokker–Planck equations, by exploiting the gradient flow structures with respect to the 2-Wasserstein metric in the space of probability densities. We discretize the underlying state by a finite graph and define a discrete 2-Wasserstein metric in the discrete probability space. Based on such metric, we introduce a gradient flow of the discrete free energy as semi discretization scheme. We prove that the scheme maintains dissipativity of the free energy and converges to a discrete Gibbs measure at exponential dissipation rate. We exhibit these properties on several numerical examples.

Well-posedness of evolution equations with time-dependent nonlinear mobility: A modified minimizing movement scheme

AbstractWe prove the existence of nonnegative weak solutions to a class of second- and fourth-order nonautonomous nonlinear evolution equations with an explicitly time-dependent mobility function posed on the whole space {{{\mathbb{R}}^{d}}}, for arbitrary {d\geq 1}. Exploiting a very formal gradient flow structure, the cornerstone of our proof is a modified version of the classical minimizing movement scheme for gradient flows. The mobility function is required to satisfy – at each time point separately – the conditions by which one can define a modified Wasserstein distance on the space of probability densities with finite second moment. The explicit dependency on the time variable is assumed to be at least of Lipschitz regularity. We also sketch possible extensions of our result to the case of bounded spatial domains and more general mobility functions.

The Camassa–Holm equation as an incompressible Euler equation: A geometric point of view

The group of diffeomorphisms of a compact manifold endowed with the L2 metric acting on the space of probability densities gives a unifying framework for the incompressible Euler equation and the theory of optimal mass transport. Recently, several authors have extended optimal transport to the space of positive Radon measures where the Wasserstein–Fisher–Rao distance is a natural extension of the classical L2-Wasserstein distance. In this paper, we show a similar relation between this unbalanced optimal transport problem and the Hdiv right-invariant metric on the group of diffeomorphisms, which corresponds to the Camassa–Holm (CH) equation in one dimension. Geometrically, we present an isometric embedding of the group of diffeomorphisms endowed with this right-invariant metric in the automorphisms group of the fiber bundle of half densities endowed with an L2 type of cone metric. This leads to a new formulation of the (generalized) CH equation as a geodesic equation on an isotropy subgroup of this automorphisms group; On S1, solutions to the standard CH thus give radially 1-homogeneous solutions of the incompressible Euler equation on R2 which preserves a radial density that has a singularity at 0. An other application consists in proving that smooth solutions of the Euler–Arnold equation for the Hdiv right-invariant metric are length minimizing geodesics for sufficiently short times.

On geodesic completeness for Riemannian metrics on smooth probability densities

The geometric approach to optimal transport and information theory has triggered the interpretation of probability densities as an infinite-dimensional Riemannian manifold. The most studied Riemannian structures are Otto's metric, yielding the $L^2$-Wasserstein distance of optimal mass transport, and the Fisher--Rao metric, predominant in the theory of information geometry. On the space of smooth probability densities, none of these Riemannian metrics are geodesically complete---a property desirable for example in imaging applications. That is, the existence interval for solutions to the geodesic flow equations cannot be extended to the whole real line. Here we study a class of Hamilton--Jacobi-like partial differential equations arising as geodesic flow equations for higher-order Sobolev type metrics on the space of smooth probability densities. We give order conditions for global existence and uniqueness, thereby providing geodesic completeness. The system we study is an interesting example of a flow equation with loss of derivatives, which is well-posed in the smooth category, yet non-parabolic and fully non-linear. On a more general note, the paper establishes a link between geometric analysis on the space of probability densities and analysis of Euler-Arnold equations in topological hydrodynamics.

Geometry of matrix decompositions seen through optimal transport and information geometry

The space of probability densities is an infinite-dimensional Riemannian manifold, with Riemannian metrics in two flavors: Wasserstein and Fisher--Rao. The former is pivotal in optimal mass transport (OMT), whereas the latter occurs in information geometry---the differential geometric approach to statistics. The Riemannian structures restrict to the submanifold of multivariate Gaussian distributions, where they induce Riemannian metrics on the space of covariance matrices. Here we give a systematic description of classical matrix decompositions (or factorizations) in terms of Riemannian geometry and compatible principal bundle structures. Both Wasserstein and Fisher--Rao geometries are discussed. The link to matrices is obtained by considering OMT and information geometry in the category of linear transformations and multivariate Gaussian distributions. This way, OMT is directly related to the polar decomposition of matrices, whereas information geometry is directly related to the $QR$, Cholesky, spectral, and singular value decompositions. We also give a coherent description of gradient flow equations for the various decompositions; most flows are illustrated in numerical examples. The paper is a combination of previously known and original results. As a survey it covers the Riemannian geometry of OMT and polar decompositions (smooth and linear category), entropy gradient flows, and the Fisher--Rao metric and its geodesics on the statistical manifold of multivariate Gaussian distributions. The original contributions include new gradient flows associated with various matrix decompositions, new geometric interpretations of previously studied isospectral flows, and a new proof of the polar decomposition of matrices based an entropy gradient flow.

Diffeomorphic Density Matching by Optimal Information Transport

We address the following problem: given two smooth densities on a manifold, find an optimal diffeomorphism that transforms one density into the other. Our framework builds on connections between the Fisher-Rao information metric on the space of probability densities and right-invariant metrics on the infinite-dimensional manifold of diffeomorphisms. This optimal information transport, and modifications thereof, allows us to construct numerical algorithms for density matching. The algorithms are inherently more efficient than those based on optimal mass transport or diffeomorphic registration. Our methods have applications in medical image registration, texture mapping, image morphing, non-uniform random sampling, and mesh adaptivity. Some of these applications are illustrated in examples.

Poisson's Equation in Nonlinear Filtering

The aim of this paper is to provide a variational interpretation of the nonlinear filter in continuous time. A time-stepping procedure is introduced, consisting of successive minimization problems in the space of probability densities. The weak form of the nonlinear filter is derived via analysis of the first-order optimality conditions for these problems. The derivation shows the nonlinear filter dynamics may be regarded as a gradient flow, or a steepest descent, for a certain energy functional with respect to the Kullback--Leibler divergence. The second part of the paper is concerned with derivation of the feedback particle filter algorithm, based again on the analysis of the first variation. The algorithm is shown to be exact. That is, the posterior distribution of the particle matches exactly the true posterior, provided the filter is initialized with the true prior.

The Immediate Exchange model: an analytical investigation

We study the Immediate Exchange model, recently introduced by Heinsalu and Patriarca [Eur. Phys. J. B 87: 170 (2014)], who showed by simulations that the wealth distribution in this model converges to a Gamma distribution with shape parameter $2$. Here we justify this conclusion analytically, in the infinite-population limit. An infinite-population version of the model is derived, describing the evolution of the wealth distribution in terms of iterations of a nonlinear operator on the space of probability densities. It is proved that the Gamma distributions with shape parameter $2$ are fixed points of this operator, and that, starting with an arbitrary wealth distribution, the process converges to one of these fixed points. We also discuss the mixed model introduced in the same paper, in which exchanges are either bidirectional or unidirectional with fixed probability. We prove that, although, as found by Heinsalu and Patriarca, the equilibrium distribution can be closely fit by Gamma distributions, the equilibrium distribution for this model is {\it{not}} a Gamma distribution.

An Application of the Renormalization Group Method to Stable Limit Laws

Let \((X_n)_{n=1}^{\infty }\) be a sequence of independent identically distributed random variables. We study the normalized partial sums and the corresponding renormalization group flow in the space of probability densities. We prove the convergence to stable limit laws under suitable assumptions on the initial density.

Extreme-value distributions and renormalization group

In the classical theorems of extreme value theory the limits of suitably rescaled maxima of sequences of independent, identically distributed random variables are studied. The vast majority of the literature on the subject deals with affine normalization. We argue that more general normalizations are natural from a mathematical and physical point of view and work them out. The problem is approached using the language of renormalization-group transformations in the space of probability densities. The limit distributions are fixed points of the transformation and the study of its differential around them allows a local analysis of the domains of attraction and the computation of finite-size corrections.

THE MATHER MEASURE AND A LARGE DEVIATION PRINCIPLE FOR THE ENTROPY PENALIZED METHOD

We present the rate function and a large deviation principle for the entropy penalized Mather problem when the Lagrangian is generic (it is known that in this case the Mather measure μ is unique and the support of μ is the Aubry set). We assume the Lagrangian L(x, v), with x in the torus 𝕋Nand v∈ℝN, satisfies certain natural hypotheses, such as superlinearity and convexity in v, as well as some technical estimates. Consider, for each value of ϵ and h, the entropy penalized Mather problem [Formula: see text] where the entropy S is given by [Formula: see text], and the minimization is performed over the space of probability densities μ(x, v) on 𝕋N×ℝNthat satisfy the discrete holonomy constraint ∫𝕋N×ℝNφ(x + hv) - φ(x) dμ = 0. It is known [17] that there exists a unique minimizing measure μϵ, hwhich converges to a Mather measure μ, as ϵ, h→0. In the case in which the Mather measure μ is unique we prove a Large Deviation Principle for the limit limϵ, h→0ϵ ln μϵ, h(A), where A ⊂ 𝕋N×ℝN. In particular, we prove that the deviation function I can be written as [Formula: see text], where ϕ0is the unique viscosity solution of the Hamilton – Jacobi equation, [Formula: see text]. We also prove a large deviation principle for the limit ϵ→ 0 with fixed h.Finally, in the last section, we study some dynamical properties of the discrete time Aubry–Mather problem, and present a proof of the existence of a separating subaction.

Generalized Central Limit Theorem and Renormalization Group

We introduce a simple instance of the renormalization group transformation in the Banach space of probability densities. By changing the scaling of the renormalized variables we obtain, as fixed points of the transformation, the L\'evy strictly stable laws. We also investigate the behavior of the transformation around these fixed points and the domain of attraction for different values of the scaling parameter. The physical interest of a renormalization group approach to the generalized central limit theorem is discussed.