Mean-field Optimal Control Problem Research Articles

The turnpike property for mean-field optimal control problems

Abstract We study the turnpike phenomenon for optimal control problems with mean-field dynamics that are obtained as the limit $N\rightarrow \infty$ of systems governed by a large number $N$ of ordinary differential equations. We show that the optimal control problems with large time horizons give rise to a turnpike structure of the optimal state and the optimal control. For the proof, we use the fact that the turnpike structure for the problems on the level of ordinary differential equations is preserved under the corresponding mean-field limit.

Stochastic filtering under model ambiguity

In this paper, we study a non-linear filtering problem in the presence of signal model uncertainty. The model ambiguity is characterized by a class of probability measures from which the true one is taken. After interchanging the order of extremum problems by using the min-max theorem, we find that the uncertain filtering problem can be converted to a weighted conditional mean-field optimal control problem. Further, we characterize the ambiguity filter and prove its unique existence.

Stochastic Mitra–Wan forestry models analyzed as a mean field optimal control problem

Stochastic Mitra–Wan forestry models analyzed as a mean field optimal control problem

Regularity of the value function and quantitative propagation of chaos for mean field control problems

We investigate a mean field optimal control problem obtained in the limit of the optimal control of large particle systems with forcing and terminal data which are not assumed to be convex. We prove that the value function, which is known to be Lipschitz continuous but not of class \(C^1\), in general, without convexity, is actually smooth in an open and dense subset of the space of times and probability measures. As a consequence, we prove a new quantitative propagation of chaos-type result for the optimal solutions of the particle system starting from this open and dense set.

A measure theoretical approach to the mean-field maximum principle for training NeurODEs

In this paper we consider a measure-theoretical formulation of the training of NeurODEs in the form of a mean-field optimal control with L2-regularization of the control. We derive first order optimality conditions for the NeurODE training problem in the form of a mean-field maximum principle, and show that it admits a unique control solution, which is Lipschitz continuous in time. As a consequence of this uniqueness property, the mean-field maximum principle also provides a strong quantitative generalization error for finite sample approximations, yielding a rigorous justification of a phenomenon that we call coupled descent, indicating the simultaneous decrease of generalization and training errors. We consider two approaches to the derivation of the mean-field maximum principle, including one that is based on a generalized Lagrange multiplier theorem on convex sets of spaces of measures, which is arguably much simpler than those currently available in the literature for mean-field optimal control problems. The latter is also new, and can be considered as a result of independent interest.

On the regularized risk of distributionally robust learning over deep neural networks

In this paper, we explore the relation between distributionally robust learning and different forms of regularization to enforce robustness of deep neural networks. In particular, starting from a concrete min-max distributionally robust problem, and using tools from optimal transport theory, we derive first-order and second-order approximations to the distributionally robust problem in terms of appropriate regularized risk minimization problems. In the context of deep ResNet models, we identify the structure of the resulting regularization problems as mean-field optimal control problems where the number and dimension of state variables are within a dimension-free factor of the dimension of the original unrobust problem. Using the Pontryagin maximum principles associated with these problems, we motivate a family of scalable algorithms for the training of robust neural networks. Our analysis recovers some results and algorithms known in the literature (in settings explained throughout the paper) and provides many other theoretical and algorithmic insights that to our knowledge are novel. In our analysis, we employ tools that we deem useful for a future analysis of more general adversarial learning problems.

Semiconcavity and sensitivity analysis in mean-field optimal control and applications

In this article, we investigate some of the fine properties of the value function associated with an optimal control problem in the Wasserstein space of probability measures. Building on new interpolation and linearisation formulas for non-local flows, we prove semiconcavity estimates for the value function, and establish several variants of the so-called sensitivity relations which provide connections between its superdifferential and the adjoint curves stemming from the maximum principle. We subsequently make use of these results to study the propagation of regularity for the value function along optimal trajectories, as well as to investigate sufficient optimality conditions and optimal feedbacks for mean-field optimal control problems.

Disordered high-dimensional optimal control

Mean field optimal control problems are a class of optimization problems that arise from optimal control when applied to the many body setting. In the noisy case one has a set of controllable stochastic processes and a cost function that is a functional of their trajectories. The goal of the optimization is to minimize this cost over the control variables. Here we consider the case in which we have N stochastic processes, or agents, with the associated control variables, which interact in a disordered way so that the resulting cost function is random. The goal is to find the average minimal cost for N → ∞, when a typical realization of the quenched random interactions is considered. We introduce a simple model and show how to perform a dimensional reduction from the infinite dimensional case to a set of one dimensional stochastic partial differential equations of the Hamilton–Jacobi–Bellman and Fokker–Planck type. The statistical properties of the corresponding stochastic terms must be computed self-consistently, as we show explicitly.

One-dimensional multi-agent optimal control with aggregation and distance constraints: qualitative properties and mean-field limit

In this paper we consider an optimal control problem for a large population of interacting agents with deterministic dynamics, aggregating potential and constraints on reciprocal distances, in dimension 1. We study existence and qualitative properties of periodic in time optimal trajectories of the finite agents optimal control problem, with particular interest on the compactness of the solutions’ support and on the saturation of the distance constraint. Moreover, we prove, through a Γ-convergence result, the consistency of the mean-field optimal control problem with density constraints with the corresponding underlying finite agent one and we deduce some qualitative results for the time periodic equilibria of the limit problem.

Mean-field optimal control for biological pattern formation

We propose a mean-field optimal control problem for the parameter identification of a given pattern. The cost functional is based on the Wasserstein distance between the probability measures of the modeled and the desired patterns. The first-order optimality conditions corresponding to the optimal control problem are derived using a Lagrangian approach on the mean-field level. Based on these conditions we propose a gradient descent method to identify relevant parameters such as angle of rotation and force scaling which may be spatially inhomogeneous. We discretize the first-order optimality conditions in order to employ the algorithm on the particle level. Moreover, we prove a rate for the convergence of the controls as the number of particles used for the discretization tends to infinity. Numerical results for the spatially homogeneous case demonstrate the feasibility of the approach.

Deep Learning and Mean-Field Games: A Stochastic Optimal Control Perspective

We provide a rigorous mathematical formulation of Deep Learning (DL) methodologies through an in-depth analysis of the learning procedures characterizing Neural Network (NN) models within the theoretical frameworks of Stochastic Optimal Control (SOC) and Mean-Field Games (MFGs). In particular, we show how the supervised learning approach can be translated in terms of a (stochastic) mean-field optimal control problem by applying the Hamilton–Jacobi–Bellman (HJB) approach and the mean-field Pontryagin maximum principle. Our contribution sheds new light on a possible theoretical connection between mean-field problems and DL, melting heterogeneous approaches and reporting the state-of-the-art within such fields to show how the latter different perspectives can be indeed fruitfully unified.

Differential inclusions in Wasserstein spaces: The Cauchy-Lipschitz framework

In this article, we propose a general framework for the study of differential inclusions in the Wasserstein space of probability measures. Based on earlier geometric insights on the structure of continuity equations, we define solutions of differential inclusions as absolutely continuous curves whose driving velocity fields are measurable selections of multifunction taking their values in the space of vector fields. In this general setting, we prove three of the founding results of the theory of differential inclusions: Filippov's theorem, the Relaxation theorem, and the compactness of the solution sets. These contributions – which are based on novel estimates on solutions of continuity equations – are then applied to derive a new existence result for fully non-linear mean-field optimal control problems with closed-loop controls.

On a mean field optimal control problem

In this paper we consider a mean field optimal control problem with an aggregation–diffusion constraint, where agents interact through a potential, in the presence of a Gaussian noise term. Our analysis focuses on a PDE system coupling a Hamilton–Jacobi and a Fokker–Planck equation, describing the optimal control aspect of the problem and the evolution of the population of agents, respectively. The main contribution of the paper is a result on the existence of solutions for the aforementioned system. We notice this model is in close connection with the theory of mean-field games systems. However, a distinctive feature concerns the nonlocal character of the interaction; it affects the drift term in the Fokker–Planck equation as well as the Hamiltonian of the system, leading to new difficulties to be addressed.

Nonexponential Sanov and Schilder theorems on Wiener space: BSDEs, Schrödinger problems and control

We derive new limit theorems for Brownian motion, which can be seen as nonexponential analogues of the large deviation theorems of Sanov and Schilder in their Laplace principle forms. As a first application, we obtain novel scaling limits of backward stochastic differential equations and their related partial differential equations. As a second application, we extend prior results on the small-noise limit of the Schrödinger problem as an optimal transport cost, unifying the control-theoretic and probabilistic approaches initiated respectively by T. Mikami and C. Léonard. Lastly, our results suggest a new scheme for the computation of mean field optimal control problems, distinct from the conventional particle approximation. A key ingredient in our analysis is an extension of the classical variational formula (often attributed to Borell or Boué–Dupuis) for the Laplace transform of Wiener measure.

Mean-field optimal control problem of SDDES driven by fractional Brownian Motion

We consider a mean-field optimal control problem for stochastic differential equations with delay driven by fractional Brownian motion with Hurst parameter greater than one half. Stochastic optimal ...

Mean-field optimal control as Gamma-limit of finite agent controls

This paper focuses on the role of a government of a large population of interacting agents as a meanfield optimal control problem derived from deterministic finite agent dynamics. The control problems are constrained by a Partial Differential Equation of continuity-type without diffusion, governing the dynamics of the probability distribution of the agent population. We derive existence of optimal controls in a measure-theoretical setting as natural limits of finite agent optimal controls without any assumption on the regularity of control competitors. In particular, we prove the consistency of mean-field optimal controls with corresponding underlying finite agent ones. The results follow from a Γ -convergence argument constructed over the mean-field limit, which stems from leveraging the superposition principle.

A mean-field optimal control formulation of deep learning

Recent work linking deep neural networks and dynamical systems opened up new avenues to analyze deep learning. In particular, it is observed that new insights can be obtained by recasting deep learning as an optimal control problem on difference or differential equations. However, the mathematical aspects of such a formulation have not been systematically explored. This paper introduces the mathematical formulation of the population risk minimization problem in deep learning as a mean-field optimal control problem. Mirroring the development of classical optimal control, we state and prove optimality conditions of both the Hamilton-Jacobi-Bellman type and the Pontryagin type. These mean-field results reflect the probabilistic nature of the learning problem. In addition, by appealing to the mean-field Pontryagin's maximum principle, we establish some quantitative relationships between population and empirical learning problems. This serves to establish a mathematical foundation for investigating the algorithmic and theoretical connections between optimal control and deep learning.

Mean Field Control Hierarchy

In this paper we model the role of a government of a large population as a mean field optimal control problem. Such control problems are constrained by a PDE of continuity-type, governing the dynamics of the probability distribution of the agent population. We show the existence of mean field optimal controls both in the stochastic and deterministic setting. We derive rigorously the first order optimality conditions useful for numerical computation of mean field optimal controls. We introduce a novel approximating hierarchy of sub-optimal controls based on a Boltzmann approach, whose computation requires a very moderate numerical complexity with respect to the one of the optimal control. We provide numerical experiments for models in opinion formation comparing the behavior of the control hierarchy.

Necessary and Sufficient Solution to Optimal Control for Linear Continuous Time Mean-field System

Different from previous works, this paper will provide the necessary and sufficient (essential) solution to linear quadratic (LQ) optimal control problem for continuous-time mean-field systems. Firstly by applying the Maximum Principle developed in this paper, the necessary and sufficient solvability condition of the finite horizon mean-field optimal control problem is presented in an explicit expression form (in analytical form) for the first time, which is completely different from the operator-type conditions obtained in previous works. Secondly, we explore the analytical solution to the forward and backward stochastic differential equation (FBSDE) of the mean-field optimal control problem. Accordingly, the optimal controller, in terms of system state and its expectation, is designed via a coupled Riccati equation. It is very interesting to show that the coupled Riccati equation is derived from the solution to the FBSDE, which has the similarity with the case of standard optimal control.

A stochastic maximum principle in mean-field optimal control problems for jump diffusions

This paper is concerned with the study of a stochastic control problem, where the controlled system is described by a stochastic differential equation (SDE) driven by a Poisson random measure and an independent Brownian motion. The cost functional involves the mean of certain nonlinear functions of the state variable. The inclusion of this mean terms in the running and the final cost functions introduces a major difficulty when applying the dynamic programming principle. A key idea of solving the problem is to use the stochastic maximum principle method (SMP). In the first part of the paper, we focus on necessary optimality conditions while the control set is assumed to be convex. Then we prove that these conditions are in fact sufficient provided some convexity conditions are fulfilled. In the second part, the results are applied to solve the mean-variance portfolio selection problem in a jump setting.