Optimal Transport Theory Research Articles

Cross-hospital Sepsis Early Detection via Semi-supervised Optimal Transport with Self-paced Ensemble.

Leveraging machine learning techniques for Sepsis early detection and diagnosis has attracted increasing interest in recent years. However, most existing methods require a large amount of labeled training data, which may not be available for a target hospital that deploys a new Sepsis detection system. More seriously, as treated patients are diversified between hospitals, directly applying a model trained on other hospitals may not achieve good performance for the target hospital. To address this issue, we propose a novel semi-supervised transfer learning framework based on optimal transport theory and self-paced ensemble for Sepsis early detection, called SPSSOT, which can efficiently transfer knowledge from the source hospital (with rich labeled data) to the target hospital (with scarce labeled data). Specifically, SPSSOT incorporates a new optimal transport-based semi-supervised domain adaptation component that can effectively exploit all the unlabeled data in the target hospital. Moreover, self-paced ensemble is adapted in SPSSOT to alleviate the class imbalance issue during transfer learning. In a nutshell, SPSSOT is an end-to-end transfer learning method that automatically selects suitable samples from two domains (hospitals) respectively and aligns their feature spaces. Extensive experiments on two open clinical datasets, MIMIC-III and Challenge, demonstrate that SPSSOT outperforms state-of-the-art transfer learning methods by improving 1-3% of AUC.

An optimal transport theory based approach for efficient dispatch of transactions in energy markets

AbstractNowadays, transactive energy markets (TEMs) are emerging as interesting frameworks in deregulated power markets to control the balance of supply and demand in the entire electrical network. Due to wide deployment of renewable energy resources, grid connected micro‐grids, and open access transmission and distribution networks, the planning and operation of TEMs become complex. So, an efficient optimal dispatch model for TEMs should be developed to achieve the objectives of TEMs, such as feasible sizes of transactions and optimal dispatch of these transactions with minimal operating costs. The transactive dispatch problem is similar to the resource allocation/matching problem. Recently, optimal transport (OT) has received significant attention in various fields including optimization theory and resource matching problems due to its potency and relevance in modeling and optimization. An OT‐based approach is proposed here for optimal dispatch of transactions in energy markets while minimizing the cost of transactions considering the operating constraints of the system. The proposed approach can efficiently determine the feasible sizes of transactions without any security issues. The optimal solutions of the OT‐based approach are obtained using a Sinkhorn iterative technique. Also, the load uncertainties are considered in this work to analyse the impacts of load uncertainties on the optimal dispatch of transactions. The numerical simulation results on the modified IEEE 9‐bus system, modified IEEE 57‐bus system, modified IEEE 118‐bus system, and Indian Northern Regional Power Grid (NRPG) system illustrate the efficacy of the proposed OT‐based framework.

Joint learning of text alignment and abstractive summarization for long documents via unbalanced optimal transport

Abstract Recently, neural abstractive text summarization (NATS) models based on sequence-to-sequence architecture have drawn a lot of attention. Real-world texts that need to be summarized range from short news with dozens of words to long reports with thousands of words. However, most existing NATS models are not good at summarizing long documents, due to the inherent limitations of their underlying neural architectures. In this paper, we focus on the task of long document summarization (LDS). Based on the inherent section structures of source documents, we divide an abstractive LDS problem into several smaller-sized problems. In this circumstance, how to provide a less-biased target summary as the supervision for each section is vital for the model’s performance. As a preliminary, we formally describe the section-to-summary-sentence (S2SS) alignment for LDS. Based on this, we propose a novel NATS framework for the LDS task. Our framework is built based on the theory of unbalanced optimal transport (UOT), and it is named as UOTSumm. It jointly learns three targets in a unified training objective, including the optimal S2SS alignment, a section-level NATS summarizer, and the number of aligned summary sentences for each section. In this way, UOTSumm directly learns the text alignment from summarization data, without resorting to any biased tool such as ROUGE. UOTSumm can be easily adapted to most existing NATS models. And we implement two versions of UOTSumm, with and without the pretrain-finetune technique. We evaluate UOTSumm on three publicly available LDS benchmarks: PubMed, arXiv, and GovReport. UOTSumm obviously outperforms its counterparts that use ROUGE for the text alignment. When combined with UOTSumm, the performance of two vanilla NATS models improves by a large margin. Besides, UOTSumm achieves better or comparable performance when compared with some recent strong baselines.

Transport map Bayesian parameter estimation for dynamical systems

AbstractAccurate online state and parameter estimation of uncertain non‐linear dynamical systems is a demanding task that has been traditionally handled by adopting non‐linear Kalman Filters or particle filters. However, in case of Kalman filters the system needs to be linearised and for particle filters the computational demand can be high. Recent advances in optimal transport theory and the application to Bayesian model updating pave the way for other approaches to system and parameter identification. They also provide a way of formulating the problem in such a way that efficient online estimation for complex systems is possible. In this work, we investigate the properties of the transport map approach when compared to standard Markov Chain Monte Carlo in an off‐line setting as a first step towards on‐line parameter estimation. We apply both approaches to an analytical exponential model and a dynamical system with seven unknown parameters subjected to ground displacement. Details on the theory of transport maps and on the used MCMC algorithm are also given.

A survey of Optimal Transport for Computer Graphics and Computer Vision

AbstractOptimal transport is a long‐standing theory that has been studied in depth from both theoretical and numerical point of views. Starting from the 50s this theory has also found a lot of applications in operational research. Over the last 30 years it has spread to computer vision and computer graphics and is now becoming hard to ignore. Still, its mathematical complexity can make it difficult to comprehend, and as such, computer vision and computer graphics researchers may find it hard to follow recent developments in their field related to optimal transport. This survey first briefly introduces the theory of optimal transport in layman's terms as well as most common numerical techniques to solve it. More importantly, it presents applications of these numerical techniques to solve various computer graphics and vision related problems. This involves applications ranging from image processing, geometry processing, rendering, fluid simulation, to computational optics, and many more. It is aimed at computer graphics researchers desiring to follow optimal transport research in their field as well as optimal transport researchers willing to find applications for their numerical algorithms.

Ollivier–Ricci Curvature Based Spatio-Temporal Graph Neural Networks for Traffic Flow Forecasting

Traffic flow forecasting is a basic function of intelligent transportation systems, and the accuracy of prediction is of great significance for traffic management and urban planning. The main difficulty of traffic flow predictions is that there is complex underlying spatiotemporal dependence in traffic flow; thus, the existing spatiotemporal graph neural network (STGNN) models need to model both temporal dependence and spatial dependence. Graph neural networks (GNNs) are adopted to capture the spatial dependence in traffic flow, which can model the symmetric or asymmetric spatial relations between nodes in the traffic network. The transmission process of traffic features in GNNs is guided by the node-to-node relationship (e.g., adjacency or spatial distance) between nodes, ignoring the spatial dependence caused by local topological constraints in the road network. To further consider the influence of local topology on the spatial dependence of road networks, in this paper, we introduce Ollivier–Ricci curvature information between connected edges in the road network, which is based on optimal transport theory and makes comprehensive use of the neighborhood-to-neighborhood relationship to guide the transmission process of traffic features between nodes in STGNNs. Experiments on real-world traffic datasets show that the models with Ollivier–Ricci curvature information outperforms those based on only node-to-node relationships between nodes by ten percent on average in the RMSE metric. This study indicates that by utilizing complex topological features in road networks, spatial dependence can be captured more sufficiently, further improving the predictive ability of traffic forecasting models.

OPTIMAL TRANSPORT GUIDED UNSUPERVISED LEARNING FOR ENHANCING LOW-QUALITY RETINAL IMAGES.

Real-world non-mydriatic retinal fundus photography is prone to artifacts, imperfections, and low-quality when certain ocular or systemic co-morbidities exist. Artifacts may result in inaccuracy or ambiguity in clinical diagnoses. In this paper, we proposed a simple but effective end-to-end framework for enhancing poor-quality retinal fundus images. Leveraging the optimal transport theory, we proposed an unpaired image-to-image translation scheme for transporting low-quality images to their high-quality counterparts. We theoretically proved that a Generative Adversarial Networks (GAN) model with a generator and discriminator is sufficient for this task. Furthermore, to mitigate the inconsistency of information between the low-quality images and their enhancements, an information consistency mechanism was proposed to maximally maintain structural consistency (optical discs, blood vessels, lesions) between the source and enhanced domains. Extensive experiments were conducted on the EyeQ dataset to demonstrate the superiority of our proposed method perceptually and quantitatively.

3d Virtual Histology Reveals Pathological Alterations of Cerebellar Granule Cells in Multiple Sclerosis

We investigate structural properties of neurons in the granular layer of human cerebellum with respect to their involvement in multiple sclerosis (MS). To this end we analyze data recorded by X-ray phase contrast tomography from tissue samples collected post mortem from a MS and a healthy control group. Using automated segmentation and histogram analysis based on optimal transport theory (OT) we find that the distributions representing nuclear structure in the granular layer move to a more compact nuclear state, i.e. smaller, denser and more heterogeneous nuclei in MS. We have previously made a similar observation for neurons of the dentate gyrus in Alzheimer’s disease, suggesting that more compact structure of neuronal nuclei which we attributed to increased levels of heterochromatin, may possibly represent a more general phenomenon of cellular senescence associated with neurodegeneration.

Entropy martingale optimal transport and nonlinear pricing–hedging duality

The objective of this paper is to develop a duality between a novel entropy martingale optimal transport (EMOT) problem and an associated optimisation problem. In EMOT, we follow the approach taken in the entropy optimal transport (EOT) problem developed in Liero et al. (Invent. Math. 211:969–1117, 2018), but we add the constraint, typical of martingale optimal transport (MOT) theory, that the infimum of the cost functional is taken over martingale probability measures. In the associated problem, the objective functional, related via Fenchel conjugacy to the entropic term in EMOT, is no longer linear as in (martingale) optimal transport. This leads to a novel optimisation problem which also has a clear financial interpretation as a nonlinear subhedging problem. Our theory allows us to establish a nonlinear robust pricing–hedging duality which also covers a wide range of known robust results. We also focus on Wasserstein-induced penalisations and study how the duality is affected by variations in the penalty terms, with a special focus on the convergence of EMOT to the extreme case of MOT.

Taming hyperparameter tuning in continuous normalizing flows using the JKO scheme

A normalizing flow (NF) is a mapping that transforms a chosen probability distribution to a normal distribution. Such flows are a common technique used for data generation and density estimation in machine learning and data science. The density estimate obtained with a NF requires a change of variables formula that involves the computation of the Jacobian determinant of the NF transformation. In order to tractably compute this determinant, continuous normalizing flows (CNF) estimate the mapping and its Jacobian determinant using a neural ODE. Optimal transport (OT) theory has been successfully used to assist in finding CNFs by formulating them as OT problems with a soft penalty for enforcing the standard normal distribution as a target measure. A drawback of OT-based CNFs is the addition of a hyperparameter, alpha , that controls the strength of the soft penalty and requires significant tuning. We present JKO-Flow, an algorithm to solve OT-based CNF without the need of tuning alpha . This is achieved by integrating the OT CNF framework into a Wasserstein gradient flow framework, also known as the JKO scheme. Instead of tuning alpha , we repeatedly solve the optimization problem for a fixed alpha effectively performing a JKO update with a time-step alpha . Hence we obtain a ”divide and conquer” algorithm by repeatedly solving simpler problems instead of solving a potentially harder problem with large alpha .

GAT–GMM: Generative Adversarial Training for Gaussian Mixture Models

Generative adversarial networks (GANs) learn the distribution of observed samples through a zero-sum game between two machine players, a generator and a discriminator. While GANs achieve great success in learning the complex distribution of image, sound, and text data, they perform suboptimally in learning multimodal distribution-learning benchmarks such as Gaussian mixture models (GMMs). In this paper, we propose Generative Adversarial Training for Gaussian Mixture Models (GAT-GMM), a minimax GAN framework for learning GMMs. Motivated by optimal transport theory, we design the zero-sum game in GAT-GMM using a random linear generator and a softmax-based quadratic discriminator architecture, which leads to a nonconvex concave minimax optimization problem. We show that a gradient descent ascent (GDA) method converges to an approximate stationary minimax point of the GAT-GMM optimization problem. In the benchmark case of a mixture of two symmetric, well-separated Gaussians, we further show that this stationary point recovers the true parameters of the underlying GMM. We discuss the application of the proposed GAT-GMM framework for learning GMMs in the distributed federated learning setting, where the widely used expectation-maximization (EM) algorithm can incur great computational and communication costs. On the other hand, we show that GAT-GMM provides a scalable learning approach and a distributed GDA algorithm can still solve the GAT-GMM minimax problem without incurring extra computation costs. We numerically support our theoretical results by performing experiments which show that our minimax framework is successful in centralized learning tasks and can outperform standard EM-type algorithms in the federated setting.

Geometric thermodynamics for the Fokker–Planck equation: stochastic thermodynamic links between information geometry and optimal transport

We propose a geometric theory of non-equilibrium thermodynamics, namely geometric thermodynamics, using our recent developments of differential-geometric aspects of entropy production rate in non-equilibrium thermodynamics. By revisiting our recent results on geometrical aspects of entropy production rate in stochastic thermodynamics for the Fokker–Planck equation, we introduce a geometric framework of non-equilibrium thermodynamics in terms of information geometry and optimal transport theory. We show that the proposed geometric framework is useful for obtaining several non-equilibrium thermodynamic relations, such as thermodynamic trade-off relations between the thermodynamic cost and the fluctuation of the observable, optimal protocols for the minimum thermodynamic cost and the decomposition of the entropy production rate for the non-equilibrium system. We clarify several stochastic-thermodynamic links between information geometry and optimal transport theory via the excess entropy production rate based on a relation between the gradient flow expression and information geometry in the space of probability densities and a relation between the velocity field in optimal transport and information geometry in the space of path probability densities.

The five gradients inequality for non quadratic costs

We give a proof of the “five gradients inequality” of Optimal Transportation Theory for general costs of the form c(x,y)=h(x-y) where h is a C 1 strictly convex radially symmetric function.

Multi‐marginal Approximation of the Linear Gromov–Wasserstein Distance

AbstractRecently, two concepts from optimal transport theory have successfully been brought to the Gromov–Wasserstein (GW) setting. This introduces a linear version of the GW distance and multi‐marginal GW transport. The former can reduce the computational complexity when computing all GW distances of a large set of inputs. The latter allows for a simultaneous matching of more than two marginals, which can for example be used to compute GW barycenters. The aim of this paper is to show an approximation result which characterizes the linear version as a limit of a multi‐marginal GW formulation.

Optimal control in stochastic thermodynamics

We review recent progress in optimal control in stochastic thermodynamics. Theoretical advances provide in-depth insight into minimum-dissipation control with either full or limited (parametric) control, and spanning the limits from slow to fast driving and from weak to strong driving. Known exact solutions give a window into the properties of minimum-dissipation control, which are reproduced by approximate methods in the relevant limits. Connections between optimal-transport theory and minimum-dissipation protocols under full control give deep insight into the properties of optimal control and place bounds on the dissipation of thermodynamic processes. Since minimum-dissipation protocols are relatively well understood and advanced approximation methods and numerical techniques for estimating minimum-dissipation protocols have been developed, now is an opportune time for application to chemical and biological systems.

Entropy-regularized Wasserstein distributionally robust shape and topology optimization

This brief note aims to introduce the recent paradigm of distributional robustness in the field of shape and topology optimization. Acknowledging that the probability law of uncertain physical data is rarely known beyond a rough approximation constructed from observed samples, we optimize the worst-case value of the expected cost of a design when the probability law of the uncertainty is “close” to the estimated one up to a prescribed threshold. The “proximity” between probability laws is quantified by the Wasserstein distance, a notion pertaining to optimal transport theory. The combination of the classical entropic regularization technique in this field with recent results from convex duality theory allows to reformulate the distributionally robust optimization problem in a way which is tractable for computations. Two numerical examples are presented, in the different settings of density-based topology optimization and geometric shape optimization. They exemplify the relevance and applicability of the proposed formulation regardless of the selected optimal design framework.

A hybrid particle-stochastic map filter

Filtering in nonlinear state-space models is known to be a challenging task due to the posterior distribution being either intractable or expressed in a complex form. One of the most successful methods, particle filtering (PF), although generally outperforming traditional filters, suffers from sample degeneracy. Drawing from optimal transport theory, the stochastic map filter (SMF) accommodates a solution to this problem, but its performance is influenced by the limited flexibility of nonlinear map parameterisation. To alleviate these drawbacks, we propose a hybrid filter which combines the PF and SMF, and hence call it PSMF. Specifically, the PSMF splits the likelihood into two parts, which are then updated by PF and SMF, respectively. The proposed approach adopts systematic resampling and smoothing to break the particle degeneracy caused by the PF. To investigate the influence of the nonlinearity of transport maps, we introduce two variants of the proposed filter, the PSMF-L and PSMF-NL, which are based on linear and nonlinear maps, respectively. The PSMF is tested on various nonlinear state-space models and a nonlinear non-Gaussian target tracking model. The proposed linear PSMF-L outperforms all the reference models for medium-to-large numbers of particles, whilst the PSMF-NL shows better resilience to parameter changes.

Optimal Transport for Free Energy Estimation

Optimal transport theory is a growing field of mathematics, which has recently found many applications. Here we take advantage of optimal transport for computational free energy estimation. We show analytically, and then via simulation, that this approach is effective in terms of optimizing the barriers of an alchemical transformation.

Thermodynamic Unification of Optimal Transport: Thermodynamic Uncertainty Relation, Minimum Dissipation, and Thermodynamic Speed Limits

Thermodynamics serves as a universal means for studying physical systems from an energy perspective. In recent years, with the establishment of the field of stochastic and quantum thermodynamics, the ideas of thermodynamics have been generalized to small fluctuating systems. Independently developed in mathematics and statistics, the optimal transport theory concerns the means by which one can optimally transport a source distribution to a target distribution, deriving a useful metric between probability distributions, called the Wasserstein distance. Despite their seemingly unrelated nature, an intimate connection between these fields has been unveiled in the context of continuous-state Langevin dynamics, providing several important implications for nonequilibrium systems. In this study, we elucidate an analogous connection for discrete cases by developing a thermodynamic framework for discrete optimal transport. We first introduce a novel quantity called dynamical state mobility, which significantly improves the thermodynamic uncertainty relation and provides insights into the precision of currents in nonequilibrium Markov jump processes. We then derive variational formulas that connect the discrete Wasserstein distances to stochastic and quantum thermodynamics of discrete Markovian dynamics described by master equations. Specifically, we rigorously prove that the Wasserstein distance equals the minimum product of irreversible entropy production and dynamical state mobility over all admissible Markovian dynamics. These formulas not only unify the relationship between thermodynamics and the optimal transport theory for discrete and continuous cases, but also generalize it to the quantum case. In addition, we demonstrate that the obtained variational formulas lead to remarkable applications in stochastic and quantum thermodynamics, such as stringent thermodynamic speed limits and the finite-time Landauer principle. These bounds are tight and can be saturated for arbitrary temperatures, even in the zero-temperature limit. Notably, the finite-time Landauer principle can explain finite dissipation even at extremely low temperatures, which cannot be explained by the conventional Landauer principle.1 MoreReceived 26 July 2022Revised 1 December 2022Accepted 13 December 2022DOI:https://doi.org/10.1103/PhysRevX.13.011013Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasEntropy productionNonequilibrium & irreversible thermodynamicsNonequilibrium statistical mechanicsQuantum thermodynamicsStochastic processesStochastic thermodynamicsThermodynamics of computationTechniquesGraph theoryStatistical PhysicsInterdisciplinary PhysicsGeneral Physics

Optimal Transport for Unsupervised Denoising Learning.

Recently, much progress has been made in unsupervised denoising learning. However, existing methods more or less rely on some assumptions on the signal and/or degradation model, which limits their practical performance. How to construct an optimal criterion for unsupervised denoising learning without any prior knowledge on the degradation model is still an open question. Toward answering this question, this work proposes a criterion for unsupervised denoising learning based on the optimal transport theory. This criterion has favorable properties, e.g., approximately maximal preservation of the information of the signal, whilst achieving perceptual reconstruction. Furthermore, though a relaxed unconstrained formulation is used in practical implementation, we prove that the relaxed formulation in theory has the same solution as the original constrained formulation. Experiments on synthetic and real-world data, including realistic photographic, microscopy, depth, and raw depth images, demonstrate that the proposed method even compares favorably with supervised methods, e.g., approaching the PSNR of supervised methods while having better perceptual quality. Particularly, for spatially correlated noise and realistic microscopy images, the proposed method not only achieves better perceptual quality but also has higher PSNR than supervised methods. Besides, it shows remarkable superiority in harsh practical conditions with complex noise, e.g., raw depth images. Code is available at https://github.com/wangweiSJTU/OTUR.