Mean-field Systems Research Articles

Optimal control for unknown mean-field discrete-time system based on Q-Learning

Solving the optimal mean-field control problem usually requires complete system information. In this paper, a Q-learning algorithm is discussed to solve the optimal control problem of the unknown mean-field discrete-time stochastic system. First, through the corresponding transformation, we turn the stochastic mean-field control problem into a deterministic problem. Second, the H matrix is obtained through Q-function, and the control strategy relies only on the H matrix. Therefore, solving H matrix is equivalent to solving the mean-field optimal control. The proposed Q-learning method iteratively solves H matrix and gain matrix according to input system state information, without the need for system parameter knowledge. Next, it is proved that the control matrix sequence obtained by Q-learning converge to the optimal control, which shows theoretical feasibility of the Q-learning. Finally, two simulation cases verify the effectiveness of Q-learning algorithm.

Linear-quadratic non-zero sum differential game for mean-field stochastic systems with asymmetric information

In this study, we consider a class of asymmetric information linear-quadratic non-zero sum stochastic differential game problems. The system dynamics are governed by a forward linear mean-field stochastic differential equation and the cost functional is quadratic. In order to facilitate some practical applications, the mean-field terms for the state process and control process are considered in both the system dynamics and cost functional. Using the classical calculus of variation and dual methods, the open-loop Nash equilibrium point can be expressed by introducing an auxiliary mean-field forward backward stochastic differential equation, which comprises one forward and two backward components. Due to some Riccati equations and ordinary differential equations that possess unique solutions, the Nash equilibrium point can also be represented in feedback form for several special cases under asymmetric information. The corresponding filtering equations are derived, and the existence and uniqueness of the solutions are proved. An investment problem in finance is discussed to demonstrate the good performance of the theoretical results.

Information upper bound for McKean-Vlasov stochastic differential equations.

We develop an information-theoretic framework to quantify information upper bound for the probability distributions of the solutions to the McKean-Vlasov stochastic differential equations. More precisely, we derive the information upper bound in terms of Kullback-Leibler divergence, which characterizes the entropy of the probability distributions of the solutions to McKean-Vlasov stochastic differential equations relative to the joint distributions of mean-field particle systems. The order of information upper bound is also figured out.

On the Diffusive-Mean Field Limit for Weakly Interacting Diffusions Exhibiting Phase Transitions

The objective of this article is to analyse the statistical behaviour of a large number of weakly interacting diffusion processes evolving under the influence of a periodic interaction potential. We focus our attention on the combined mean field and diffusive (homogenisation) limits. In particular, we show that these two limits do not commute if the mean field system constrained to the torus undergoes a phase transition, that is to say, if it admits more than one steady state. A typical example of such a system on the torus is given by the noisy Kuramoto model of mean field plane rotators. As a by-product of our main results, we also analyse the energetic consequences of the central limit theorem for fluctuations around the mean field limit and derive optimal rates of convergence in relative entropy of the Gibbs measure to the (unique) limit of the mean field energy below the critical temperature.

Linear-quadratic mean-field type stackelberg differential games for stochastic jump-diffusion systems

<p style='text-indent:20px;'>In this paper, we consider linear-quadratic (LQ) leader-follower Stackelberg differential games for mean-field type stochastic systems with jump diffusions, where the system includes mean-field variables, i.e., the expected value of state and control variables. We first solve the LQ mean-field type control problem of the follower using the stochastic maximum principle and obtain the state-feedback representation of the open-loop optimal solution in terms of the coupled integro-Riccati differential equations (CIRDEs) via the Four-Step Scheme. Next, we solve the problem of the leader, which is the LQ control problem subject to the mean-field type forward-backward stochastic system with jump diffusions, where the constraint characterizes the rational behavior of the follower. Using the variational approach, we obtain the (mean-field type) stochastic maximum principle. However, to obtain the state-feedback representation of the open-loop optimal solution of the leader, there is a technical challenge due to the jump process. We consider two different cases, in which the state-feedback type control in terms of the CIRDEs can be characterized by generalizing the Four-Step Scheme. We finally show that the state-feedback type controls of the open-loop optimal solutions for the leader and the follower constitute the Stackelberg equilibrium.</p>

Phase transitions on a class of generalized Vicsek-like models of collective motion.

Systems composed of interacting self-propelled particles (SPPs) display different forms of order-disorder phase transitions relevant to collective motion. In this paper, we propose a generalization of the Vicsek model characterized by an angular noise term following an arbitrary probability density function, which might depend on the state of the system and thus have a multiplicative character. We show that the well established vectorial Vicsek model can be expressed in this general formalism by deriving the corresponding angular probability density function, as well as we propose two new multiplicative models consisting of bivariate Gaussian and wrapped Gaussian distributions. With the proposed formalism, the mean-field system can be solved using the mean resultant length of the angular stochastic term. Accordingly, when the SPPs interact globally, the character of the phase transition depends on the choice of the noise distribution, being first order with a hybrid scaling for the vectorial and wrapped Gaussian distributions, and second order for the bivariate Gaussian distribution. Numerical simulations reveal that this scenario also holds when the interactions among SPPs are given by a static complex network. On the other hand, using spatial short-range interactions displays, in all the considered instances, a discontinuous transition with a coexistence region, consistent with the original formulation of the Vicsek model.

Effect of Self Feedback on Mean-Field Coupled Oscillators: Revival and Quenching of Oscillations

Interaction among different units in a network of oscillators may often lead to quenching of oscillations and the importance of oscillation quenching can be found in controlling the dynamics of many real world systems. But there are also many real life phenomena where suppression of oscillation should be avoided for maintaining the sustained evolution of the system. In this work, we propose a self-feedback control scheme through which one is able to either achieve quenching or to retrieve the rhythmic behavior in a network of mean-field diffusively coupled systems. It is found that for proper choice of the strength of the control signal, the system converges to an oscillatory state from oscillation quenched state and for further increase of the strength the system enters in chaotic region. Moreover, reversal of the phase of the control signal can induce suppression of oscillation. Thus, the proposed modification offers a better control on the dynamics of the mean-field coupled system. In addition to this, a new transition phenomenon from inhomogeneous limit cycle (IHLC) to homogeneous limit cycle (HLC) through chaotic route has also been found in the modified system.

Stability of Synchronous Slowly Oscillating Periodic Solutions for Systems of Delay Differential Equations with Coupled Nonlinearity

We study stability of so-called synchronous slowly oscillating periodic solutions (SOPSs) for a system of identical delay differential equations (DDEs) with linear decay and nonlinear delayed negative feedback that are coupled through their nonlinear term. Under a row sum condition on the coupling matrix, existence of a unique SOPS for the corresponding scalar DDE implies existence of a unique synchronous SOPS for the coupled DDEs. However, stability of the SOPS for the scalar DDE does not generally imply stability of the synchronous SOPS for the coupled DDEs. We obtain an explicit formula, depending only on the spectrum of the coupling matrix, the strength of the linear decay and the values of the nonlinear negative feedback function near plus/minus infinity, that determines the stability of the synchronous SOPS in the asymptotic regime where the nonlinear term is heavily weighted. We also treat the special cases of so-called weakly coupled systems, near uniformly coupled systems, and doubly nonnegative coupled systems, in the aforementioned asymptotic regime. Our approach is to estimate the characteristic (Floquet) multipliers for the synchronous SOPS. We first reduce the analysis of the multidimensional variational equation to the analysis of a family of scalar variational-type equations, and then establish limits for an associated family of monodromy-type operators. We illustrate our results with examples of systems of DDEs with mean-field coupling and systems of DDEs arranged in a ring.

Homogenization of the backward-forward mean-field games systems in periodic environments

We study the homogenization properties in the small viscosity limit and in periodic environments of the (viscous) backward-forward mean-field games system. We consider separated Hamiltonians and provide results for systems with (i) ‘‘smoothing’’ coupling and general initial and terminal data, and (ii) with ‘‘local coupling’’ but well-prepared data. The limit is a first-order forward-backward system. In the nonlocal coupling case, the averaged system is of mean field games-type, which is well-posed in some cases. For the problems with local coupling, the homogenization result is proved assuming that the formally obtained limit system has smooth solutions with well prepared initial and terminal data. It is also shown, using a very general example (potential mean field games), that the limit system is not necessarily of mean field games-type. A complete theory for such systems was developed by the authors in [28].

Stochastic [formula omitted] control for discrete-time mean-field systems with Poisson jump

In this paper we consider the problem of stochastic H2/H∞ control for Poisson jump-diffusion system driven by Brownian motion and Poisson process. Firstly, a mean-field stochastic bounded real lemma(SBRL) is derived in this paper. Secondly, a sufficient condition for the solvability of Poisson jump-diffusion linear quadratic (LQ) optimal control of discrete-time mean-field type is presented. Thirdly, based on the results of SBRL and LQ control, the sufficient conditions for the existence of mean-field stochastic H2/H∞ control of Poisson jump-diffusion system are established by the solvability of the coupled matrix value equation. Finally, an example of recursive algorithm is presented to demonstrate the effectiveness of the proposed theory.

Partial derivative with respect to the measure and its application to general controlled mean-field systems

Let (E,E) be an arbitrary measurable space. The paper first focuses on studying the partial derivative of a function f:P2,0(Rd×E)→R defined on the space of probability measures μ over (Rd×E,B(Rd)⊗E) whose first marginal μ1≔μ(⋅×E) has a finite second order moment. This partial derivative is taken with respect to q(dx,z), where μ has the disintegration μ(dxdz)=q(dx,z)μ2(dz) with respect to its second marginal μ2(⋅)=μ(Rd×⋅). Simplifying the language, we will speak of the derivative with respect to the law μ conditioned to its second marginal. Our results extend those of the derivative of a function g:P2(Rd)→R over the space of probability measures with finite second order moment by P.L. Lions (see Lions (2013)) but cover also as a particular case recent approaches considering E=Rk and supposing the differentiability of f over P2(Rd×Rk), in order to use the derivative ∂μf to define the partial derivative (∂μf)1. The second part of the paper focuses on investigating a stochastic maximum principle, where the controlled state process is driven by a general mean-field stochastic differential equation with partial information. The control set is just supposed to be a measurable space, and the coefficients of the controlled system, i.e., those of the dynamics as well as of the cost functional, depend on the controlled state process X, the control v, a partial information on X, as well as on the joint law of (X,v). Through considering a new second-order variational equation and the corresponding second-order adjoint equation, and a totally new method to prove the estimate for the solution of the first-order variational equation, the optimal principle is proved through spike variation of an optimal control and with the help of the tailor-made form of second-order expansion. We emphasize that in our assumptions we do not need any regularity of the coefficients neither in the control variable nor with respect to the law of the control process.

Zeroth order phase transition induced by ergodicity breaking in a mean-field system

A spin chain model with mean-field interactions is studied within the microcanonical ensemble. Under local microcanonical dynamics, gaps in the space of extensive variables can be open up leading to the ergodicity breaking of this system. As a consequence, our mean-field system can exhibit the zero-order phase transition accompanied with an entropy jump at the transition point. Moreover, the global phase diagram of the system is constructed.

Controllability Gramian and Kalman rank condition for mean-field control systems

This paper is concerned with the exact controllability of linear mean-field stochastic systems with deterministic coefficients. With the help of the theory of mean-field backward stochastic differential equations (MF-BSDEs, for short) and some delicate analysis, we obtain a mean-field version of the Gramian matrix criterion for the general time-variant case, and a mean-field version of the Kalman rank condition for the special time-invariant case.

Semi-linear Poisson-mediated Flocking in a Cucker-Smale Model

We propose a family of compactly supported parametric interaction functions in the general Cucker-Smale flocking dynamics such that the mean-field macroscopic system of mass and momentum balance equations with non-local damping terms can be converted from a system of partial integro-differential equations to an augmented system of partial differential equations in a compact set. We treat the interaction functions as Green’s functions for an operator corresponding to a semi-linear Poisson equation and compute the density and momentum in a translating reference frame, i.e. one that is taken in reference to the flock’s centroid. This allows us to consider the dynamics in a fixed, flock-centered compact set without loss of generality. We approach the computation of the non-local damping using the standard finite difference treatment of the chosen differential operator, resulting in a tridiagonal system which can be solved quickly.

At the mercy of the common noise: blow-ups in a conditional McKean–Vlasov Problem

We extend a model of positive feedback and contagion in large mean-field systems, by introducing a common source of noise driven by Brownian motion. Although the driving dynamics are continuous, the positive feedback effect can lead to ‘blow-up’ phenomena whereby solutions develop jump-discontinuities. Our main results are twofold and concern the conditional McKean–Vlasov formulation of the model. First and foremost, we show that there are global solutions to this McKean–Vlasov problem, which can be realised as limit points of a motivating particle system with common noise. Furthermore, we derive results on the occurrence of blow-ups, thereby showing how these events can be triggered or prevented by the pathwise realisations of the common noise.

The classical limit of mean-field quantum spin systems

The theory of strict deformation quantization of the two-sphere S2⊂R3 is used to prove the existence of the classical limit of mean-field quantum spin chains, whose ensuing Hamiltonians are denoted by HN, where N indicates the number of sites. Indeed, since the fibers A1/N=MN+1(C) and A0 = C(S2) form a continuous bundle of C*-algebras over the base space I={0}∪1/N*⊂[0,1], one can define a strict deformation quantization of A0 where quantization is specified by certain quantization maps Q1/N:Ã0→A1/N, with Ã0 being a dense Poisson subalgebra of A0. Given now a sequence of such HN, we show that under some assumptions, a sequence of eigenvectors ψN of HN has a classical limit in the sense that ω0(f) ≔ limN→∞⟨ψN, Q1/N(f)ψN⟩ exists as a state on A0 given by ω0(f)=1n∑i=1nf(Ωi), where n is some natural number. We give an application regarding spontaneous symmetry breaking, and moreover, we show that the spectrum of such a mean-field quantum spin system converges to the range of some polynomial in three real variables restricted to the sphere S2.

Extended mean-field games

We introduce a new class of coupled forward-backward in time systems consisting of a forward Hamilton–Jacobi and a backward quasilinear transport equation, which we call extended mean-field games system. This new class of equations strictly contains the classical mean-field games system with no common noise and its homogenization limit, and optimal transportation-type control problems. We also identify a new and meaningful ‘‘monotonicity’’-type condition that yields well-posedeness. The same condition yields uniqueness in the Hilbertian setting for the master equation without common noise as well as the hyperbolic system describing finite-state mean-field games.

Bio-Inspired Dynamic Collective Choice in Large-Population Systems: A Robust Mean-Field Game Perspective.

Inspired by the collective decision making in biological systems, such as honeybee swarm searching for a new colony, we study a dynamic collective choice problem for large-population systems with the purpose of realizing certain advantageous features observed in biology. This problem focuses on the situation where a large number of heterogeneous agents subject to adversarial disturbances move from initial positions toward one of the destinations in a finite time while trying to remain close to the average trajectory of all agents. To overcome the complexity of this problem resulting from the large population and the heterogeneity of agents, and also to enforce some specific choices by individuals, we formulate the problem under consideration as a robust mean-field game with non-convex and non-smooth cost functions. Through Nash equivalence principle, we first deal with a single-player H∞ tracking problem by taking the population behavior as a fixed trajectory, and then establish a mean-field system to estimate the population behavior. Optimal control strategies and worst disturbances, independent of the population size, are designed, which give a way to realize the collective decision-making behavior emerged in biological systems. We further prove that the designed strategies constitute ϵN -Nash equilibrium, where ϵN goes toward zero as the number of agents increases to infinity. The effectiveness of the proposed results are illustrated through two simulation examples.

Pareto-Optimal Strategy for Linear Mean-Field Stochastic Systems With H∞ Constraint.

This article presents results on designing the Pareto-optimal strategy under H∞ constraint for the linear mean-field stochastic systems disturbed by external disturbances. First, combining the stochastic H∞ control theory with the stochastic mean-field theory, we derive the stochastic bounded real lemma (SBRL) of our considered linear mean-field stochastic systems with the stochastic initial condition. Second, we use the mean-field forward-backward stochastic differential equation to solve the mean-field linear quadratic Pareto-optimal problem with indefinite cost functionals. It is proved that the existence of a closed-loop Pareto-optimal strategy is equivalent to the solvability of the coupled generalized differential Riccati equations when some conditions are satisfied. Finally, a necessary and sufficient condition for the Pareto-optimal strategy under the H∞ constraint is researched by four-coupled matrix-valued equations. Besides, we also obtain the Pareto frontier for the mean-field stochastic system with only state-dependent noise. A practical example is presented to show the effectiveness of our main results.

An elementary approach to uniform in time propagation of chaos

Based on a coupling approach, we prove uniform in time propagation of chaos for weakly interacting mean-field particle systems with possibly non-convex confinement and interaction potentials. The approach is based on a combination of reflection and synchronous couplings applied to the individual particles. It provides explicit quantitative bounds that significantly extend previous results for the convex case.