Optimal Transport Map Research Articles

The Singularity Set of Optimal Transportation Maps

The Singularity Set of Optimal Transportation Maps

The flow map of the Fokker–Planck equation does not provide optimal transport

In Khrulkov and Oseledets (2022) the authors conjecture that, by integrating the flow of the ODE given by the Wasserstein velocity in a Fokker–Planck equation, one obtains an optimal transport map. On the other hand this result was thought to be false in Kim and Milman (2012) but no proof was provided. In this note we show that the result claimed by Khrulkov and Oseledets cannot hold. This strengthens a counterexample which was built in Tanana (2021).

Express Construction for GANs from Latent Representation to Data Distribution

Generative Adversarial Networks (GANs) are powerful generative models for numerous tasks and datasets. However, most of the existing models suffer from mode collapse. The most recent research indicates that the reason for it is that the optimal transportation map from random noise to the data distribution is discontinuous, but deep neural networks (DNNs) can only approximate continuous ones. Instead, the latent representation is a better raw material used to construct a transportation map point to the data distribution than random noise. Because it is a low-dimensional mapping related to the data distribution, the construction procedure seems more like expansion rather than starting all over. Besides, we can also search for more transportation maps in this way with smoother transformation. Thus, we have proposed a new training methodology for GANs in this paper to search for more transportation maps and speed the training up, named Express Construction. The key idea is to train GANs with two independent phases for successively yielding latent representation and data distribution. To this end, an Auto-Encoder is trained to map the real data into the latent space, and two couples of generators and discriminators are used to produce them. To the best of our knowledge, we are the first to decompose the training procedure of GAN models into two more uncomplicated phases, thus tackling the mode collapse problem without much more computational cost. We also provide theoretical steps toward understanding the training dynamics of this procedure and prove assumptions. No extra hyper-parameters have been used in the proposed method, which indicates that Express Construction can be used to train any GAN models. Extensive experiments are conducted to verify the performance of realistic image generation and the resistance to mode collapse. The results show that the proposed method is lightweight, effective, and less prone to mode collapse.

Generalizations of Talagrand Inequality for Sinkhorn Distance Using Entropy Power Inequality.

The distance that compares the difference between two probability distributions plays a fundamental role in statistics and machine learning. Optimal transport (OT) theory provides a theoretical framework to study such distances. Recent advances in OT theory include a generalization of classical OT with an extra entropic constraint or regularization, called entropic OT. Despite its convenience in computation, entropic OT still lacks sufficient theoretical support. In this paper, we show that the quadratic cost in entropic OT can be upper-bounded using entropy power inequality (EPI)-type bounds. First, we prove an HWI-type inequality by making use of the infinitesimal displacement convexity of the OT map. Second, we derive two Talagrand-type inequalities using the saturation of EPI that corresponds to a numerical term in our expressions. These two new inequalities are shown to generalize two previous results obtained by Bolley et al. and Bai et al. Using the new Talagrand-type inequalities, we also show that the geometry observed by Sinkhorn distance is smoothed in the sense of measure concentration. Finally, we corroborate our results with various simulation studies.

Distribution-on-distribution regression via optimal transport maps

Summary We present a framework for performing regression when both covariate and response are probability distributions on a compact interval. Our regression model is based on the theory of optimal transportation, and links the conditional Fréchet mean of the response to the covariate via an optimal transport map. We define a Fréchet-least-squares estimator of this regression map, and establish its consistency and rate of convergence to the true map, under both full and partial observations of the regression pairs. Computation of the estimator is shown to reduce to a standard convex optimization problem, and thus our regression model can be implemented with ease. We illustrate our methodology using real and simulated data.

Robust and accurate optimal transportation map by self-adaptive sampling

Robust and accurate optimal transportation map by self-adaptive sampling

Variational Approach to Regularity of Optimal Transport Maps: General Cost Functions

We extend the variational approach to regularity for optimal transport maps initiated by Goldman and the first author to the case of general cost functions. Our main result is an epsilon -regularity result for optimal transport maps between Hölder continuous densities slightly more quantitative than the result by De Philippis–Figalli. One of the new contributions is the use of almost-minimality: if the cost is quantitatively close to the Euclidean cost function, a minimizer for the optimal transport problem with general cost is an almost-minimizer for the one with quadratic cost. This further highlights the connection between our variational approach and De Giorgi’s strategy for epsilon -regularity of minimal surfaces.

Existence and uniqueness of optimal transport maps obtained by the secondary variational method

Existence and uniqueness of optimal transport maps obtained by the secondary variational method

Pseudo-Riemannian geometry encodes information geometry in optimal transport

Optimal transport and information geometry both study geometric structures on spaces of probability distributions. Optimal transport characterizes the cost-minimizing movement from one distribution to another, while information geometry originates from coordinate invariant properties of statistical inference. Their relations and applications in statistics and machine learning have started to gain more attention. In this paper we give a new differential-geometric relation between the two fields. Namely, the pseudo-Riemannian framework of Kim and McCann, which provides a geometric perspective on the fundamental Ma–Trudinger–Wang (MTW) condition in the regularity theory of optimal transport maps, encodes the dualistic structure of statistical manifold. This general relation is described using the framework of c-divergence under which divergences are defined by optimal transport maps. As a by-product, we obtain a new information-geometric interpretation of the MTW tensor on the graph of the transport map. This relation sheds light on old and new aspects of information geometry. The dually flat geometry of Bregman divergence corresponds to the quadratic cost and the pseudo-Euclidean space, and the logarithmic L^{(alpha )}-divergence introduced by Pal and the first author has constant sectional curvature in a sense to be made precise. In these cases we give a geometric interpretation of the information-geometric curvature in terms of the divergence between a primal-dual pair of geodesics.

Regularity of optimal transport maps on locally nearly spherical manifolds

Given a compact connected n-dimensional Riemannian manifold, we investigate the smoothness of the optimal transport map between the smooth densities with respect to the squared Riemannian distance cost. The optimal map is characterized by exp(gradu), where the potential function u satisfies a Monge–Ampère type equation. Delanoë [7] showed the smoothness of u on the Riemannian surfaces when the scalar curvature is close to 1 in C 2 norm. In this work, we study the regularity issue on Riemannian manifolds with curvature sufficiently close to curvature of round sphere in C 2 norm in all dimensions and prove that the 𝒞-curvature on such Riemannian manifolds satisfies an improved Ma-Trudinger-Wang condition and the Jacobian of the exponential map is positive. As a consequence, we imply the smoothness of the optimal transport map by the continuity method.

Minimax estimation of smooth optimal transport maps

Brenier’s theorem is a cornerstone of optimal transport that guarantees the existence of an optimal transport map T between two probability distributions P and Q over Rd under certain regularity conditions. The main goal of this work is to establish the minimax estimation rates for such a transport map from data sampled from P and Q under additional smoothness assumptions on T. To achieve this goal, we develop an estimator based on the minimization of an empirical version of the semidual optimal transport problem, restricted to truncated wavelet expansions. This estimator is shown to achieve near minimax optimality using new stability arguments for the semidual and a complementary minimax lower bound. Furthermore, we provide numerical experiments on synthetic data supporting our theoretical findings and highlighting the practical benefits of smoothness regularization. These are the first minimax estimation rates for transport maps in general dimension.

Sharp boundary ε-regularity of optimal transport maps

In this paper we develop a boundary ε-regularity theory for optimal transport maps between bounded open sets with C1,α-boundary. Our main result asserts sharp C1,α-regularity of transport maps at the boundary in form of a linear estimate under certain assumptions: The main quantitative assumptions are that the local nondimensionalized transport cost is small and that the boundaries are locally almost flat in C1,α. Our method is completely variational and builds on the recently developed interior regularity theory.

Fast hybrid tempered ensemble transform filter formulation for Bayesian elliptical problems via Sinkhorn approximation

Abstract. Identification of unknown parameters on the basis of partial and noisy data is a challenging task, in particular in high dimensional and non-linear settings. Gaussian approximations to the problem, such as ensemble Kalman inversion, tend to be robust and computationally cheap and often produce astonishingly accurate estimations despite the simplifying underlying assumptions. Yet there is a lot of room for improvement, specifically regarding a correct approximation of a non-Gaussian posterior distribution. The tempered ensemble transform particle filter is an adaptive Sequential Monte Carlo (SMC) method, whereby resampling is based on optimal transport mapping. Unlike ensemble Kalman inversion, it does not require any assumptions regarding the posterior distribution and hence has shown to provide promising results for non-linear non-Gaussian inverse problems. However, the improved accuracy comes with the price of much higher computational complexity, and the method is not as robust as ensemble Kalman inversion in high dimensional problems. In this work, we add an entropy-inspired regularisation factor to the underlying optimal transport problem that allows the high computational cost to be considerably reduced via Sinkhorn iterations. Further, the robustness of the method is increased via an ensemble Kalman inversion proposal step before each update of the samples, which is also referred to as a hybrid approach. The promising performance of the introduced method is numerically verified by testing it on a steady-state single-phase Darcy flow model with two different permeability configurations. The results are compared to the output of ensemble Kalman inversion, and Markov chain Monte Carlo methods results are computed as a benchmark.

Cortical Morphometry Analysis based on Worst Transportation Theory.

Biomarkers play an important role in early detection and intervention in Alzheimer's disease (AD). However, obtaining effective biomarkers for AD is still a big challenge. In this work, we propose to use the worst transportation cost as a univariate biomarker to index cortical morphometry for tracking AD progression. The worst transportation (WT) aims to find the least economical way to transport one measure to the other, which contrasts to the optimal transportation (OT) that finds the most economical way between measures. To compute the WT cost, we generalize the Brenier theorem for the OT map to the WT map, and show that the WT map is the gradient of a concave function satisfying the Monge-Ampere equation. We also develop an efficient algorithm to compute the WT map based on computational geometry. We apply the algorithm to analyze cortical shape difference between dementia due to AD and normal aging individuals. The experimental results reveal the effectiveness of our proposed method which yields better statistical performance than other competiting methods including the OT.

Conditional Density Estimation, Latent Variable Discovery, and Optimal Transport

AbstractA framework is proposed that addresses both conditional density estimation and latent variable discovery. The objective function maximizes explanation of variability in the data, achieved through the optimal transport barycenter generalized to a collection of conditional distributions indexed by a covariate—either given or latent—in any suitable space. Theoretical results establish the existence of barycenters, a minimax formulation of optimal transport maps, and a general characterization of variability via the optimal transport cost. This framework leads to a family of nonparametric neural network‐based algorithms, the BaryNet, with a supervised version that estimates conditional distributions and an unsupervised version that assigns latent variables. The efficacy of BaryNets is demonstrated by tests on both artificial and real‐world data sets. A parallel drawn between autoencoders and the barycenter framework leads to the Barycentric autoencoder algorithm (BAE). © 2020 Wiley Periodicals LLC.

Convergence Rates for Discretized Monge\u2013Amp\xe8re Equations and Quantitative Stability of Optimal Transport

In recent works—both experimental and theoretical—it has been shown how to use computational geometry to efficiently construct approximations to the optimal transport map between two given probability measures on Euclidean space, by discretizing one of the measures. Here we provide a quantitative convergence analysis for the solutions of the corresponding discretized Monge–Ampère equations. This yields H^{1}-converge rates, in terms of the corresponding spatial resolution h, of the discrete approximations of the optimal transport map, when the source measure is discretized and the target measure has bounded convex support. Periodic variants of the results are also established. The proofs are based on new quantitative stability results for optimal transport maps, shown using complex geometry.

On one-dimensionality of metric measure spaces

In this paper, we prove that a metric measure space which has at least one open set isometric to an interval, and for which the (possibly non-unique) optimal transport map exists from any absolutely continuous measure to an arbitrary measure, is a one-dimensional manifold (possibly with boundary). As an immediate corollary we obtain that if a metric measure space is a very strict C D ( K , N ) CD(K,N) -space or an essentially non-branching M C P ( K , N ) MCP(K,N) -space with some open set isometric to an interval, then it is a one-dimensional manifold. We also obtain the same conclusion for a metric measure space which has a point in which the Gromov-Hausdorff tangent is unique and isometric to the real line, and for which the optimal transport maps not only exist but are unique. Again, we obtain an analogous corollary in the setting of essentially non-branching M C P ( K , N ) MCP(K,N) -spaces.

A fast approach to optimal transport: the back-and-forth method

We present a method to efficiently solve the optimal transportation problem for a general class of strictly convex costs. Given two probability measures supported on a discrete grid with n points we compute the optimal transport map between them in $$O(n\log (n))$$ operations and O(n) storage space. Our approach allows us to solve optimal transportation problems on spatial grids as large as $$4096\times 4096$$ and $$384\times 384\times 384$$ in a matter of minutes.

Bounds on optimal transport maps onto log-concave measures

We consider strictly log-concave measures, whose bounds degenerate at infinity. We prove that the optimal transport map from the Gaussian onto such a measure is locally Lipschitz, and that the eigenvalues of its Jacobian have controlled growth at infinity.

Equivalence of ray monotonicity properties and classification of optimal transport maps for strictly convex norms

Abstract In this paper, we first define ray increasing and decreasing monotonicity of maps. If 𝑇 is an optimal transport map for the Monge problem with cost function ∥ y - x ∥ sc \lVert y-x\rVert_{\mathrm{sc}} in R n R^{n} or 𝑇 is an optimal transport map for the Monge problem with cost function d ⁢ ( x , y ) d(x,y) , the geodesic distance, in more general, non-branching geodesic spaces 𝑋, we show respectively equivalence of some previously introduced monotonicity properties and the property of ray increasing as well as ray decreasing monotonicity which we define in this paper. Then, by solving secondary variational problems associated with strictly convex and concave functions respectively, we show that there exist ray increasing and decreasing optimal transport maps for the Monge problem with cost function ∥ y - x ∥ sc \lVert y-x\rVert_{\mathrm{sc}} . Finally, we give the classification of optimal transport maps for the Monge problem such that the cost function ∥ y - x ∥ sc \lVert y-x\rVert_{\mathrm{sc}} further satisfies the uniform smoothness and convexity estimates. That is, all of the optimal transport maps for such Monge problem can be divided into three different classes: the ray increasing map, the ray decreasing map and others.