Multi-marginal Optimal Transport Research Articles

Sufficiency of -cyclical monotonicity in a class of multi-marginal optimal transport problems

Sufficiency of -cyclical monotonicity in a class of multi-marginal optimal transport problems

Hierarchical Multi-Marginal Optimal Transport for Network Alignment

Finding node correspondence across networks, namely multi-network alignment, is an essential prerequisite for joint learning on multiple networks. Despite great success in aligning networks in pairs, the literature on multi-network alignment is sparse due to the exponentially growing solution space and lack of high-order discrepancy measures. To fill this gap, we propose a hierarchical multi-marginal optimal transport framework named HOT for multi-network alignment. To handle the large solution space, multiple networks are decomposed into smaller aligned clusters via the fused Gromov-Wasserstein (FGW) barycenter. To depict high-order relationships across multiple networks, the FGW distance is generalized to the multi-marginal setting, based on which networks can be aligned jointly. A fast proximal point method is further developed with guaranteed convergence to a local optimum. Extensive experiments and analysis show that our proposed HOT achieves significant improvements over the state-of-the-art in both effectiveness and scalability.

Convergence rate of entropy-regularized multi-marginal optimal transport costs

Abstract We investigate the convergence rate of multi-marginal optimal transport costs that are regularized with the Boltzmann–Shannon entropy, as the noise parameter $\varepsilon $ tends to $0$ . We establish lower and upper bounds on the difference with the unregularized cost of the form $C\varepsilon \log (1/\varepsilon )+O(\varepsilon )$ for some explicit dimensional constants C depending on the marginals and on the ground cost, but not on the optimal transport plans themselves. Upper bounds are obtained for Lipschitz costs or locally semiconcave costs for a finer estimate, and lower bounds for $\mathscr {C}^2$ costs satisfying some signature condition on the mixed second derivatives that may include degenerate costs, thus generalizing results previously in the two marginals case and for nondegenerate costs. We obtain in particular matching bounds in some typical situations where the optimal plan is deterministic.

An external dual charge approach to the multimarginal optimal transport with coulomb cost

In this paper, we study the multimarginal optimal transport with Coulomb cost, also known in the physics literature as the Strictly-Correlated Electrons functional. We prove that the dual Kantorovich potential is an electrostatic potential induced by an external charge density, which we call the dual charge . We study its properties and use it to discretize the potential in one and three space dimensions.

A general framework for multi-marginal optimal transport

A general framework for multi-marginal optimal transport

Relaxed many-body optimal transport and related asymptotics

Abstract Optimization problems on probability measures in ℝ d {\mathbb{R}^{d}} are considered where the cost functional involves multi-marginal optimal transport. In a model of N interacting particles, for example in Density Functional Theory, the interaction cost is repulsive and described by a two-point function c ⁢ ( x , y ) = ℓ ⁢ ( | x - y | ) {c(x,y)=\ell(\lvert x-y\rvert)} where ℓ : ℝ + → [ 0 , ∞ ] {\ell:\mathbb{R}_{+}\to[0,\infty]} is decreasing to zero at infinity. Due to a possible loss of mass at infinity, non-existence may occur and relaxing the initial problem over sub-probabilities becomes necessary. In this paper, we characterize the relaxed functional generalizing the results of [4] and present a duality method which allows to compute the Γ-limit as N → ∞ {N\to\infty} under very general assumptions on the cost ℓ ⁢ ( r ) {\ell(r)} . We show that this limit coincides with the convex hull of the so-called direct energy. Then we study the limit optimization problem when a continuous external potential is applied. Conditions are given with explicit examples under which minimizers are probabilities or have a mass < 1 {<1} . In a last part, we study the case of a small range interaction ℓ N ⁢ ( r ) = ℓ ⁢ ( r / ε ) {\ell_{N}(r)=\ell(r/\varepsilon)} ( ε ≪ 1 {\varepsilon\ll 1} ) and we show how the duality approach can also be used to determine the limit energy as ε → 0 {\varepsilon\to 0} of a very large number N ε {N_{\varepsilon}} of particles.

Estimating pollution spread in water networks as a Schrödinger bridge problem with partial information

Incidents where water networks are contaminated with microorganisms or pollutants can result in a large number of infected or ill persons, and it is therefore important to quickly detect, localize and estimate the spread and source of the contamination. In many of today’s water networks only limited measurements are available, but with the current internet of things trend the number of sensors is increasing and there is a need for methods that can utilize this information. Motivated by this fact, we address the problem of estimating the spread of pollution in a water network given measurements from a set of sensors. We model the water flow as a Markov chain, representing the system as a set of states where each state represents the amount of water in a specific part of the network, e.g., a pipe or a part of a pipe. Then we seek the most likely flow of the pollution given the expected water flow and the sensors observations. This is a large-scale optimization problem that can be formulated as a Schrödinger bridge problem with partial information, and we address this by exploiting the connection with the entropy regularized multimarginal optimal transport problem. The software EPANET is used to simulate the spread of pollution in the water network and will be used for testing the performance of the methodology.

Scalable Computation of Dynamic Flow Problems via Multimarginal Graph-Structured Optimal Transport

In this work, we develop a new framework for dynamic network flow problems based on optimal transport theory. We show that the dynamic multicommodity minimum-cost network flow problem can be formulated as a multimarginal optimal transport problem, where the cost function and the constraints on the marginals are associated with a graph structure. By exploiting these structures and building on recent advances in optimal transport theory, we develop an efficient method for such entropy-regularized optimal transport problems. In particular, the graph structure is utilized to efficiently compute the projections needed in the corresponding Sinkhorn iterations, and we arrive at a scheme that is both highly computationally efficient and easy to implement. To illustrate the performance of our algorithm, we compare it with a state-of-the-art linear programming (LP) solver. We achieve good approximations to the solution at least one order of magnitude faster than the LP solver. Finally, we showcase the methodology on a traffic routing problem with a large number of commodities. Funding: This work was supported by KTH Digital Futures, Knut och Alice Wallenbergs Stiftelse [Grants KAW 2018.0349, KAW 2021.0274, the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation], Vetenskapsrådet [Grant 2020-03454], and the National Science Foundation [Grants 1942523 and 2206576].

Scalable Computation of Dynamic Flow Problems via Multimarginal Graph-Structured Optimal Transport

In this work, we develop a new framework for dynamic network flow problems based on optimal transport theory. We show that the dynamic multicommodity minimum-cost network flow problem can be formulated as a multimarginal optimal transport problem, where the cost function and the constraints on the marginals are associated with a graph structure. By exploiting these structures and building on recent advances in optimal transport theory, we develop an efficient method for such entropy-regularized optimal transport problems. In particular, the graph structure is utilized to efficiently compute the projections needed in the corresponding Sinkhorn iterations, and we arrive at a scheme that is both highly computationally efficient and easy to implement. To illustrate the performance of our algorithm, we compare it with a state-of-the-art linear programming (LP) solver. We achieve good approximations to the solution at least one order of magnitude faster than the LP solver. Finally, we showcase the methodology on a traffic routing problem with a large number of commodities. Funding: This work was supported by KTH Digital Futures, Knut och Alice Wallenbergs Stiftelse [Grants KAW 2018.0349, KAW 2021.0274, the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation], Vetenskapsrådet [Grant 2020-03454], and the National Science Foundation [Grants 1942523 and 2206576].

Monge solutions and uniqueness in multi-marginal optimal transport via graph theory

We study a multi-marginal optimal transport problem with surplus b(x1,…,xm)=∑{i,j}∈Pxi⋅xj, where P⊆Q:={{i,j}:i,j∈{1,2,...m},i≠j}. We reformulate this problem by associating each surplus of this type with a graph with m vertices whose set of edges is indexed by P. We then establish uniqueness and Monge solution results for two general classes of surplus functions. Among many other examples, these classes encapsulate the Gangbo and Świȩch surplus [15] and the surplus ∑i=1m−1xi⋅xi+1+xm⋅x1 studied in an earlier work by the present authors [26].

A Global Optimization Approach for Multimarginal Optimal Transport Problems with Coulomb Cost

In this work, we construct a novel numerical method for solving the multi-marginal optimal transport problems with Coulomb cost. This type of optimal transport problems arises in quantum physics and plays an important role in understanding the strongly correlated quantum systems. With a Monge-like ansatz, the orginal high-dimensional problems are transferred into mathematical programmings with generalized complementarity constraints, and thus the curse of dimensionality is surmounted. However, the latter ones are themselves hard to deal with from both theoretical and practical perspective. Moreover in the presence of nonconvexity, brute-force searching for global solutions becomes prohibitive as the problem size grows large. To this end, we propose a global optimization approach for solving the nonconvex optimization problems, by exploiting an efficient proximal block coordinate descent local solver and an initialization subroutine based on hierarchical grid refinements. We provide numerical simulations on some typical physical systems to show the efficiency of our approach. The results match well with both theoretical predictions and physical intuitions, and give the first visualization of optimal transport maps for some two dimensional systems.

The Exactness of the [formula omitted] Penalty Function for a Class of Mathematical Programs with Generalized Complementarity Constraints

In a Mathematical Program with Generalized Complementarity Constraints (MPGCC), complementarity relationships are imposed between each pair of variable blocks. MPGCC includes the traditional Mathematical Program with Complementarity Constraints (MPCC) as a special case. On account of the disjunctive feasible region, MPCC and MPGCC are generally difficult to handle. The ℓ1 penalty method, often adopted in computation, opens a way of circumventing the difficulty. Yet it remains unclear about the exactness of the ℓ1 penalty function, namely, whether there exists a sufficiently large penalty parameter so that the penalty problem shares the optimal solution set with the original one. In this paper, we consider a class of MPGCCs that are of multi-affine objective functions. This problem class finds applications in various fields, e.g., the multi-marginal optimal transport problems in many-body quantum physics and the pricing problems in network transportation. We first provide an instance from this class, the exactness of whose ℓ1 penalty function cannot be derived by existing tools. We then establish the exactness results under rather mild conditions. Our results cover those existing ones for MPCC and apply to multi-block contexts.

An efficient implementable inexact entropic proximal point algorithm for a class of linear programming problems

We introduce a class of specially structured linear programming (LP) problems, which has favorable modeling capability for important application problems in different areas such as optimal transport, discrete tomography, and economics. To solve these generally large-scale LP problems efficiently, we design an implementable inexact entropic proximal point algorithm (iEPPA) combined with an easy-to-implement dual block coordinate descent method as a subsolver. Unlike existing entropy-type proximal point algorithms, our iEPPA employs a more practically checkable stopping condition for solving the associated subproblems while achieving provable convergence. Moreover, when solving the capacity constrained multi-marginal optimal transport (CMOT) problem (a special case of our LP problem), our iEPPA is able to bypass the underlying numerical instability issues that often appear in the popular entropic regularization approach, since our algorithm does not require the proximal parameter to be very small in order to obtain an accurate approximate solution. Numerous numerical experiments show that our iEPPA is efficient and robust for solving some large-scale CMOT problems on synthetic data. The preliminary experiments on the discrete tomography problem also highlight the potential modeling capability of our model.

Low-Rank Tensor Approximations for Solving Multimarginal Optimal Transport Problems

By adding entropic regularization, multi-marginal optimal transport problems can be transformed into tensor scaling problems, which can be solved numerically using the multi-marginal Sinkhorn algorithm. The main computational bottleneck of this algorithm is the repeated evaluation of marginals. Recently, it has been suggested that this evaluation can be accelerated when the application features an underlying graphical model. In this work, we accelerate the computation further by combining the tensor network dual of the graphical model with additional low-rank approximations. We provide an example for the color transfer between several images, in which these additional low-rank approximations save more than 96% of the computation time.

Transport Type Metrics on the Space of Probability Measures Involving Singular Base Measures

We develop the theory of a metric, which we call the $$\nu $$ -based Wasserstein metric and denote by $$W_\nu $$ , on the set of probability measures $${\mathcal {P}}(X)$$ on a domain $$X \subseteq \mathbb {R}^m$$ . This metric is based on a slight refinement of the notion of generalized geodesics with respect to a base measure $$\nu $$ and is relevant in particular for the case when $$\nu $$ is singular with respect to m-dimensional Lebesgue measure; it is also closely related to the concept of linearized optimal transport. The $$\nu $$ -based Wasserstein metric is defined in terms of an iterated variational problem involving optimal transport to $$\nu $$ ; we also characterize it in terms of integrations of classical Wasserstein metric between the conditional probabilities when measures are disintegrated with respect to optimal transport to $$\nu $$ , and through limits of certain multi-marginal optimal transport problems. We also introduce a class of metrics which are dual in a certain sense to $$W_\nu $$ , defined relative to a fixed based measure $$\mu $$ , on the set of measures which are absolutely continuous with respect to a second fixed based measure $$\sigma $$ . As we vary the base measure $$\nu $$ , the $$\nu $$ -based Wasserstein metric interpolates between the usual quadratic Wasserstein metric (obtained when $$\nu $$ is a Dirac mass) and a metric associated with the uniquely defined generalized geodesics obtained when $$\nu $$ is sufficiently regular (eg, absolutely continuous with respect to Lebesgue). When $$\nu $$ concentrates on a lower dimensional submanifold of $$\mathbb {R}^m$$ , we prove that the variational problem in the definition of the $$\nu $$ -based Wasserstein metric has a unique solution. We also establish geodesic convexity of the usual class of functionals, and of the set of source measures $$\mu $$ such that optimal transport between $$\mu $$ and $$\nu $$ satisfies a strengthening of the generalized nestedness condition introduced in McCann and Pass (Arch Ration Mech Anal 238(3):1475–1520, 2020). We finally introduce a slight variant of the dual metric mentioned above in order to prove convergence of an iterative scheme to solve a variational problem arising in game theory.

A family of pairwise multi-marginal optimal transports that define a generalized metric

The Optimal transport (OT) problem is rapidly finding its way into machine learning. Favoring its use are its metric properties. Many problems admit solutions with guarantees only for objects embedded in metric spaces, and the use of non-metrics can complicate solving them. Multi-marginal OT (MMOT) generalizes OT to simultaneously transporting multiple distributions. It captures important relations that are missed if the transport only involves two distributions. Research on MMOT, however, has been focused on its existence, uniqueness, practical algorithms, and the choice of cost functions. There is a lack of discussion on the metric properties of MMOT, which limits its theoretical and practical use. Here, we prove new generalized metric properties for a family of pairwise MMOTs. We first explain the difficulty of proving this via two negative results. Afterward, we prove the MMOTs’ metric properties. Finally, we show that the generalized triangle inequality of this family of MMOTs cannot be improved. We illustrate the superiority of our MMOTs over other generalized metrics, and over non-metrics in both synthetic and real tasks.

Unbalanced Multi-marginal Optimal Transport

Entropy-regularized optimal transport and its multi-marginal generalization have attracted increasing attention in various applications, in particular due to efficient Sinkhorn-like algorithms for computing optimal transport plans. However, it is often desirable that the marginals of the optimal transport plan do not match the given measures exactly, which led to the introduction of the so-called unbalanced optimal transport. Since unbalanced methods were not examined for the multi-marginal setting so far, we address this topic in the present paper. More precisely, we introduce the unbalanced multi-marginal optimal transport problem and its dual and show that a unique optimal transport plan exists under mild assumptions. Furthermore, we generalize the Sinkhorn algorithm for regularized unbalanced optimal transport to the multi-marginal setting and prove its convergence. For cost functions decoupling according to a tree, the iterates can be computed efficiently. At the end, we discuss three applications of our framework, namely two barycenter problems and a transfer operator approach, where we establish a relation between the barycenter problem and the multi-marginal optimal transport with an appropriate tree-structured cost function.

Inference With Aggregate Data in Probabilistic Graphical Models: An Optimal Transport Approach

We consider inference (filtering) problems over probabilistic graphical models with aggregate data generated by a large population of individuals. We propose a new efficient belief propagation type algorithm over tree graphs with polynomial computational complexity as well as a global convergence guarantee. This is in contrast to previous methods that either exhibit prohibitive complexity as the population grows or do not guarantee convergence. Our method is based on optimal transport, or more specifically, multimarginal optimal transport theory. In particular, we consider an inference problem with aggregate observations, that can be seen as a structured multimarginal optimal transport problem where the cost function decomposes according to the underlying graph. Consequently, the celebrated Sinkhorn/iterative scaling algorithm for multi-marginal optimal transport can be leveraged together with the standard belief propagation algorithm to establish an efficient inference scheme which we call Sinkhorn belief propagation (SBP). We further specialize the SBP algorithm to cases associated with hidden Markov models due to their significance in control and estimation. We demonstrate the performance of our algorithm on applications such as inferring population flow from aggregate observations. We also show that in the special case where the aggregate observations are in Dirac form, our algorithm naturally reduces to the standard belief propagation algorithm.

Accelerating the Sinkhorn Algorithm for Sparse Multi-Marginal Optimal Transport via Fast Fourier Transforms

We consider the numerical solution of the discrete multi-marginal optimal transport (MOT) by means of the Sinkhorn algorithm. In general, the Sinkhorn algorithm suffers from the curse of dimensionality with respect to the number of marginals. If the MOT cost function decouples according to a tree or circle, its complexity is linear in the number of marginal measures. In this case, we speed up the convolution with the radial kernel required in the Sinkhorn algorithm via non-uniform fast Fourier methods. Each step of the proposed accelerated Sinkhorn algorithm with a tree-structured cost function has a complexity of O(KN) instead of the classical O(KN2) for straightforward matrix–vector operations, where K is the number of marginals and each marginal measure is supported on, at most, N points. In the case of a circle-structured cost function, the complexity improves from O(KN3) to O(KN2). This is confirmed through numerical experiments.

Polynomial-time algorithms for multimarginal optimal transport problems with structure

Multimarginal Optimal Transport (MOT) has attracted significant interest due to applications in machine learning, statistics, and the sciences. However, in most applications, the success of MOT is severely limited by a lack of efficient algorithms. Indeed, MOT in general requires exponential time in the number of marginals k and their support sizes n. This paper develops a general theory about what “structure” makes MOT solvable in mathrm {poly}(n,k) time. We develop a unified algorithmic framework for solving MOT in mathrm {poly}(n,k) time by characterizing the structure that different algorithms require in terms of simple variants of the dual feasibility oracle. This framework has several benefits. First, it enables us to show that the Sinkhorn algorithm, which is currently the most popular MOT algorithm, requires strictly more structure than other algorithms do to solve MOT in mathrm {poly}(n,k) time. Second, our framework makes it much simpler to develop mathrm {poly}(n,k) time algorithms for a given MOT problem. In particular, it is necessary and sufficient to (approximately) solve the dual feasibility oracle—which is much more amenable to standard algorithmic techniques. We illustrate this ease-of-use by developing mathrm {poly}(n,k)-time algorithms for three general classes of MOT cost structures: (1) graphical structure; (2) set-optimization structure; and (3) low-rank plus sparse structure. For structure (1), we recover the known result that Sinkhorn has mathrm {poly}(n,k) runtime; moreover, we provide the first mathrm {poly}(n,k) time algorithms for computing solutions that are exact and sparse. For structures (2)-(3), we give the first mathrm {poly}(n,k) time algorithms, even for approximate computation. Together, these three structures encompass many—if not most—current applications of MOT.