Minor Agents Research Articles

Mean field games with major and minor agents: the limiting problem and Nash equilibrium

In this paper, we consider a mean field game (MFG) with a major and N minor agents. We first consider the limiting problem and allow the coefficients to vary with the conditional distribution in a nonlinear way. We use the stochastic maximum principle to transform the limiting control problem into a system of two coupled conditional distribution dependent forward–backward stochastic differential equations (FBSDEs), and prove the existence and uniqueness result of the FBSDEs when the dependence between major agent and minor agents is sufficiently weak. We then use the solution of the limiting problem to construct an O ( N − 1 2 ) -Nash equilibrium for the MFG with a major and N minor agents.

Decentralized Linear Quadratic Systems With Major and Minor Agents and Non-Gaussian Noise

A decentralized linear quadratic system with a major agent and a collection of minor agents is considered. The major agent affects the minor agents, but not vice versa. The state of the major agent is observed by all agents. In addition, the minor agents have a noisy observation of their local state. The noise processes is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">not</i> assumed to be Gaussian. The structures of the optimal strategy and the best linear strategy are characterized. It is shown that major agent's optimal control action is a linear function of the major agent's MMSE (minimum mean squared error) estimate of the system state while the minor agent's optimal control action is a linear function of the major agent's MMSE estimate of the system state and a “correction term” which depends on the difference of the minor agent's MMSE estimate of its local state and the major agent's MMSE estimate of the minor agent's local state. Since the noise is non-Gaussian, the minor agent's MMSE estimate is a non-linear function of its observation. It is shown that replacing the minor agent's MMSE estimate by its LLMS (linear least mean square) estimate gives the best linear control strategy. The results are proved using a direct method based on conditional independence, common-information-based splitting of state and control actions, and simplifying the per-step cost based on conditional independence, orthogonality principle, and completion of squares.

Risk‐sensitive large‐population linear‐quadratic‐Gaussian Games with major and minor agents

AbstractThis paper studies large‐population dynamic games involving a linear‐quadratic‐Gaussian (LQG) system with an exponential cost functional. The parameter in the cost functional can describe an investor's risk attitude. In the game, there are a major agent and a population of minor agents where is very large. The agents in the games are coupled via both their individual stochastic dynamics and their individual cost functions. The mean field methodology yields a set of decentralized controls, which are shown to be an ‐Nash equilibrium for a finite population system where . Numerical results are established to illustrate the impact of the population's collective behaviors and the consistency of the mean field estimation.

Discrete‐time mean field social control with a major agent

AbstractThis paper considers a linear‐quadratic mean field control problem involving a major agent and minor agents. We aim to optimize a social cost as a weighted sum of the individual costs under decentralized information. Firstly, the forward‐backward stochastic difference equations (FBSDEs) are obtained for this problem by variational analysis. Then, by decoupling the FBSDEs, we design the decentralized control laws, which are further shown to be asymptotically optimal.

Mixed Social Optima and Nash Equilibrium in Linear-Quadratic-Gaussian Mean-Field System

This article investigates a class of mixed stochastic linear-quadratic-Gaussian social optimization and Nash game in the context of a large-scale system. Two types of interactive agents are involved: a major agent and a large number of weakly coupled minor agents. All minor agents are cooperative to minimize the social cost as the sum of their individual costs, whereas such social cost is conflictive to that of the major agent. Thus, the major agent and all minor agents are further competitive to reach some nonzero-sum Nash equilibrium. Applying the mean-field approximations and person-by-person optimality, we obtain auxiliary control problems for the major agent and minor agents, respectively. The decentralized social strategy is derived by a class of new consistency condition (CC) system, which consists of mean-field forward–backward stochastic differential equations. The well-posedness of CC system is obtained by the discounting method. The related asymptotic social optimality for minor agents and Nash equilibrium for major–minor agents are also verified.

LQG mean field games with a major agent: Nash certainty equivalence versus probabilistic approach

Mean field game (MFG) systems consisting of a major agent and a large number of minor agents were introduced in (Huang, 2010) in an LQG setup. The Nash certainty equivalence was used to obtain a Markovian closed-loop Nash equilibrium for the limiting system when the number of minor agents tends to infinity. In the past years several approaches to major–minor mean field game problems have been developed, principally (i) the Nash certainty equivalence and analytic approach, (ii) master equations, (iii) asymptotic solvability, and (iv) the probabilistic approach. For the LQG case, (Firoozi, Jaimungal, and Caines, 2020) develops a convex analysis approach and retrieves the equilibrium obtained via (i). Moreover, (Huang, 2021) establishes the equivalency of the Markovian closed-loop Nash equilibrium obtained via (i) with those obtained via (ii) and (iii). Finally, in this work we demonstrate that the Markovian closed-loop Nash equilibrium of (i) is equivalent to that of (iv). These three studies answer the long-standing questions about the consistency of the solutions to major-minor LQG MFG systems derived using different approaches.

A class of hybrid LQG mean field games with state-invariant switching and stopping strategies

A novel framework is presented that combines Mean Field Game (MFG) theory and Hybrid Optimal Control (HOC) theory to obtain a unique ɛ-Nash equilibrium for a non-cooperative game with switching and stopping times. We consider the case where there exists one major agent with a significant influence on the system together with a large number of minor agents constituting two subpopulations, each agent with individually asymptotically negligible effect on the whole system. Each agent has stochastic linear dynamics with quadratic costs, and the agents are coupled in their dynamics and costs by the average state of minor agents (i.e. the empirical mean field). It is shown that for a class of Hybrid LQG MFGs, the optimal switching and stopping times are state-invariant and only depend on the dynamical parameters of each agent. Accordingly, a hybrid systems formulation of the game is presented via the indexing by discrete events: (i) the switching of the major agent between alternative dynamics or (ii) the termination of the agents’ trajectories in one or both of the subpopulations of minor agents. Optimal switchings and stopping time strategies together with best response control actions for, respectively, the major agent and all minor agents are established with respect to their individual cost criteria by an application of Hybrid LQG MFG theory.

A comparison of the precision of 3D images of facial tissues from the forensic point of view

The size and shape of the facial skeleton are fundamental determinants of the facial appearance. Small asymmetries in skeletal morphology are noticeable, but changes during forensic identification, orthodontic therapy, or maxillofacial surgical intervention are mostly much more powerful. Soft tissues can particularly eliminate deformities or accentuate them. Orthodontics and maxillofacial surgery use imaging techniques which can directly analyze soft and hard tissues. For that reason, those techniques can be generally used also in forensic anthropology and forensic dentistry settings. The aim of this study is to compare accuracy of 3D facial images acquired by the 3dMD facial scanner and the cone-beam CT unit. Our research group consisted of 32 patients who underwent a treatment plan for maxillofacial surgery, followed by orthognathic surgery. Pre- and post-operative 3D facial recordings acquired by the 3dMD facial scanner and cone-beam CT unit were analyzed and compared using the FIDENTIS Analyst Software and 3D Dolphin Imaging Software. The statistics were computed using Statistica 13 and PAST 3.0 packages. The results indicate that maxillofacial surgery is a factor that increases significantly the within-sample variation. The acquisition technologies were seen as a minor agent, which, however, should be of concern if both, the point-based as well as the Iterative Closest Point (ICP) registration is to be employed. The 3D methods of surgery planning and therapy provided specific information not only about surgical treatment options, but also for 3D comparison procedures applicable in forensic dentistry and facial identification.

Decentralized Adaptive Optimal Tracking Control for Massive Autonomous Vehicle Systems With Heterogeneous Dynamics: A Stackelberg Game.

In this article, a decentralized optimal tracking control problem has been studied for a large-scale autonomous vehicle system with heterogeneous system dynamics. Due to the ultralarge number of agents, the notorious "curse of dimension" problem as well as the unrealistic assumption of the existence of reliable very large-scale communication links in uncertain environments have challenged the traditional multiagent system (MAS) algorithms for decades. The emerging mean-field game (MFG) theory has recently been widely adopted to generate a decentralized control method that deals with those challenges by encoding the large scale MASs' information into a novel time-varying probability density functions (PDF) which can be obtained locally. However, the traditional MFG methods assume all agents are homogeneous, which is unrealistic in practical industrial applications, e.g., Internet of Things (IoTs), and so on. Therefore, a novel mean-field Stackelberg game (MFSG) is formulated based on the Stackelberg game, where all the agents have been classified as two different categories where one major leader's decision dominates the other minor agents. Moreover, a hierarchical structure that treats all minor agents as a mean-field group is developed to tackle the assumption of homogeneous agents. Then, the actor-actor-critic-critic-mass ( A2C2M ) algorithm with five neural networks is designed to learn the optimal policies by solving the MFSG. The Lyapunov theory is utilized to prove the convergence of A2C2M neural networks and the closed-loop system's stability. Finally, a series of numerical simulations are conducted to demonstrate the effectiveness of the developed method.

Hierarchical Optimal Synchronization for Linear Systems via Reinforcement Learning: A Stackelberg-Nash Game Perspective.

Considering the fact that in the real world, a certain agent may have some sort of advantage to act before others, a novel hierarchical optimal synchronization problem for linear systems, composed of one major agent and multiple minor agents, is formulated and studied in this article from a Stackelberg-Nash game perspective. The major agent herein makes its decision prior to others, and then, all the minor agents determine their actions simultaneously. To seek the optimal controllers, the Hamilton-Jacobi-Bellman (HJB) equations in coupled forms are established, whose solutions are further proven to be stable and constitute the Stackelberg-Nash equilibrium. Due to the introduction of the asymmetric roles for agents, the established HJB equations are more strongly coupled and more difficult to solve than that given in most existing works. Therefore, we propose a new reinforcement learning (RL) algorithm, i.e., a two-level value iteration (VI) algorithm, which does not rely on complete system matrices. Furthermore, the proposed algorithm is shown to be convergent, and the converged values are exactly the optimal ones. To implement this VI algorithm, neural networks (NNs) are employed to approximate the value functions, and the gradient descent method is used to update the weights of NNs. Finally, an illustrative example is provided to verify the effectiveness of the proposed algorithm.

Linear-Quadratic Mixed Stackelberg–Nash Stochastic Differential Game with Major–Minor Agents

In this paper, we study a controlled linear-quadratic-Gaussian large population system combining three types of interactive agents mixed, which are respectively, major leader, minor leaders, and minor followers. In reality, they may represent three typical types of participants involved in market price formation: major supplier, minor suppliers and minor producers. The Stackelberg–Nash–Cournot (SNC) approximate equilibrium is derived from the combination of a major–minor mean-field game (MFG) and a leader–follower Stackelberg game. Although all agents are of forward states in that only initial conditions are specified in their dynamics, our SNC analysis provides an MFG framework that is naturally in a forward–backward state in that both initial and terminal conditions are specified. This result differs from those reported in the literature on standard MFG frameworks, mainly as a result of the adoption of a Stackelberg structure. Through variational analysis, the consistency condition system can be represented by some fully-coupled forward–backward-stochastic-differential-equations with a high-dimensional block structure in an open-loop case. To sufficiently address the related solvability, we also derive the feedback form of the SNC approximate. equilibrium strategy via some coupled Riccati equations. Our study includes various mean-field game models as its special cases.

$\epsilon$-Nash Equilibria for Major–Minor LQG Mean Field Games With Partial Observations of All Agents

The partially observed major minor LQG and nonlinear mean field game (PO MM LQG MFG) systems where it is assumed the major agent's state is partially observed by each minor agent, and the major agent completely observes its own state have been analysed in the literature. In this paper, PO MM LQG MFG problems with general information patterns are studied where (i) the major agent has partial observations of its own state, and (ii) each minor agent has partial observations of its own state and the major agent's state. The assumption of partial observations by all agents leads to a new situation involving the recursive estimation by each minor agent of the major agent's estimate of its own state. For a general case of indefinite LQG MFG systems, the existence of $\epsilon$-Nash equilibria together with the individual agents' control laws yielding the equilibria are established via the Separation Principle.

Convex analysis for LQG systems with applications to major–minor LQG mean–field game systems

We develop a convex analysis approach for solving LQG optimal control problems and apply it to major–minor (MM) LQG mean–field game (MFG) systems. The approach retrieves the best response strategies for the major agent and all minor agents that attain an ϵ-Nash equilibrium. An important and distinctive advantage to this approach is that unlike the classical approach in the literature, we are able to avoid imposing assumptions on the evolution of the mean–field. In particular, this provides a tool for dealing with complex and non–standard systems.

Lightning is a major cause of large tree mortality in a lowland neotropical forest.

The mortality rates of large trees are critical to determining carbon stocks in tropical forests, but the mechanisms of tropical tree mortality remain poorly understood. Lightning strikes thousands of tropical trees every day, but is commonly assumed to be a minor agent of tree mortality in most tropical forests. We use the first systematic quantification of lightning-caused mortality to show that lightning is a major cause of death for the largest trees in an old-growth lowland forest in Panama. A novel lightning strike location system together with field surveys of strike sites revealed that, on average, each strike directly kills 3.5 trees (>10cm diameter) and damages 11.4 more. Given lightning frequency data from the Earth Networks Total Lightning Network and historical total tree mortality rates for this site, we conclude that lightning accounts for 40.5% of the mortality of large trees (>60cm diameter) in the short term and probably contributes to an additional 9.0% of large tree deaths over the long term. Any changes in cloud-to-ground lightning frequency due to climatic change will alter tree mortality rates; projected 25-50% increases in lightning frequency would increase large tree mortality rates in this forest by 9-18%. The results of this study indicate that lightning plays a critical and previously underestimated role in tropical forest dynamics and carbon cycling.

Mean-Field Game Strategies for Optimal Execution

ABSTRACTAlgorithmic trading strategies for execution often focus on the individual agent who is liquidating/acquiring shares. When generalized to multiple agents, the resulting stochastic game is notoriously difficult to solve in closed-form. Here, we circumvent the difficulties by investigating a mean-field game framework containing (i) a major agent who is liquidating a large number of shares, (ii) a number of minor agents (high-frequency traders (HFTs)) who detect and trade against the liquidator, and (iii) noise traders who buy and sell for exogenous reasons. Our setup accounts for permanent price impact stemming from all trader types inducing an interaction between major and minor agents. Both optimizing agents trade against noise traders as well as one another. This stochastic dynamic game contains couplings in the price and trade dynamics, and we use a mean-field game approach to solve the problem. We obtain a set of decentralized feedback trading strategies for the major and minor agents, and express the solution explicitly in terms of a deterministic fixed point problem. For a finite population of HFTs, the set of major-minor agent mean-field game strategies is shown to have a -Nash equilibrium property where as .

Linear-Quadratic-Gaussian Mixed Mean-Field Games with Heterogeneous Input Constraints

We consider a class of linear-quadratic-Gaussian mean-field games with a major agent and considerable heterogeneous minor agents with mean-field interactions. The individual admissible controls are constrained in closed convex subsets $Γ k$ of full space $R m$. The decentralized strategies for individual agents and consistency condition system are represented in an unified manner via a class of mean-field forward-backward stochastic differential equations involving projection operators on $Γ k$. The well-posedness of consistency system is established in the local and global cases both by the contraction mapping and discounting method respectively. The related $e$−Nash equilibrium property is also verified.

$\epsilon$-Nash Equilibria for Partially Observed LQG Mean Field Games With a Major Player

Huang (2010) and Nguyen and Huang (2012) solved the linear quadratic mean field systems and control problem in the case where there is a major agent (i.e. non-asymptotically vanishing as the population size goes to infinity) together with a population of minor agents (i.e. individually asymptotically negligible). The new feature in this case is that the mean field becomes stochastic and then, by minor agent state extension, the existence of $\epsilon$ -Nash equilibria together with the individual agents' control laws that yield the equilibria may be established. This paper presents results initially announced by Caines and Kizilkale (2013, 2014) where it is shown that if the major agent's state is partially observed by the minor agents, and if the major agent completely observes its own state, all agents can recursively generate estimates (in general individually distinct) of the major agent's state and the mean field, and thence generate feedback controls yielding $\epsilon$ -Nash equilibria.

An Optimal Execution Problem in Finance with Acquisition and Liquidation Objectives: an MFG Formulation

Partially observed major minor LQG mean field game theory is applied to an optimal execution problem in finance; following standard financial models, controlled linear system dynamics are postulated where an institutional investor (interpreted as a major agent) in the market aims to liquidate a specific amount of shares and has partial observations of its own state (which includes its inventory). Furthermore, the market is assumed to have two populations of high frequency traders (interpreted as minor agents) who wish to liquidate or acquire a certain number of shares within a specific time, and each one of them has partial observations of its own state and the major agent’s state (which include the corresponding inventories). The objective for each agent is to maximize its own wealth and to avoid the occurrence of large execution prices, large rates of trading and large trading accelerations which are appropriately weighted in the agent’s performance function. The existence of e-Nash equilibria together with the individual agents’ trading strategies yielding the equilibria, were established. A simulation example is provided.

Abstract 51: Characterization of the mechanism of action, pharmacodynamics and preclinical safety of ADCT-402, a pyrrolobenzodiazepine (PBD) dimer-containing antibody-drug conjugate (ADC) targeting CD19-expressing hematological malignancies

Abstract ADCT-402, currently in Phase I clinical trials for B-cell hematological malignancies, is an ADC composed of a recombinant humanized IgG1 against human CD19, stochastically conjugated via a cleavable linker to a PBD dimer cytotoxin (DAR of 2.3). PBD dimers, DNA minor groove interstrand cross-linking agents, are gaining increasing attention and are currently being tested as the ADC warheads in several clinical trials. ADCT-402 has potent and targeted cytotoxicity against a panel of human lymphoma and leukemia cell lines in vitro. In vivo, ADCT-402 demonstrates dose-dependent antitumor activity against Burkitt’s lymphoma xenograft models. Moreover, ADCT-402 is markedly superior to maytansinoid- and auristatin-based CD19-targeting ADCs in the Ramos xenograft model. In a rat toxicology study, a single dose of ADCT-402 at 2 mg/kg is well tolerated with a favorable PK profile and excellent stability in vivo. The current study aimed to further define the mechanism of action (MOA) of ADCT-402 and validate its pharmacology and preclinical safety for clinical development. CD19 is a clinically validated target with restricted normal tissue expression and a widespread expression in the majority of B-cell malignancies. Importantly, we show here the consistent expression of CD19 in matched samples (initial diagnosis and relapsed/refractory) from panels of lymphoma patients, indicating that relapsed/refractory patients are appropriate for treatment with ADCT-402. ADCT-402 was shown to be efficiently internalized by CD19+ cells in vitro. Moreover, in line with the PBD dimer MOA, following a 2 hour exposure to ADCT-402, DNA interstrand cross-links reached a peak between 8 - 12 hours and persisted for up to 36 hours post-treatment. In contrast, the peak of cross-link formation for the PBD dimer warhead alone was observed immediately after 2 hour incubation, while a non-targeted PBD-ADC did not yield any appreciable DNA cross-links. In SCID mice s.c. implanted with Ramos cells, a single dose of ADCT-402 was administered at 0.33 or 1 mg/kg. Twenty-four hours after treatment, excised tumors showed a dose proportional increase in intensity of staining by an anti-PBD payload antibody, as well as in DNA cross-linking and in γ-H2AX formation. In contrast, no DNA cross-linking was observed in matched lymphocyte samples. The toxicity of ADCT-402 was further evaluated in a repeat dose cynomolgus monkey study. ADCT-402 was clinically well tolerated with an acceptable off-target safety profile. The PK of the ADC was consistent with normal antibody clearance with a half-life of about 12 to 17 days. These data confirm the MOA of ADCT-402 and provide relevant pharmacodynamic assays and preclinical safety assessment to guide the clinical development of this promising ADC in B-cell malignancies. Citation Format: Francesca Zammarchi, Simon Corbett, Karin Havenith, Narinder Janghra, Konstantinos Kiakos, Teresa Marafioti, David G. Williams, Simon Chivers, Phil W. Howard, John A. Hartley, Patrick H. van Berkel. Characterization of the mechanism of action, pharmacodynamics and preclinical safety of ADCT-402, a pyrrolobenzodiazepine (PBD) dimer-containing antibody-drug conjugate (ADC) targeting CD19-expressing hematological malignancies [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2017; 2017 Apr 1-5; Washington, DC. Philadelphia (PA): AACR; Cancer Res 2017;77(13 Suppl):Abstract nr 51. doi:10.1158/1538-7445.AM2017-51

Three French Accounts of Nineteenth-century Singapore and Penang

Three French Accounts of Nineteenth-century Singapore and Penang Colin Dyer (bio) Keywords Penang, Singapore, Malacca, French in Southeast Asia, Auguste Vaillant, Penang, J.W. Salmond, Straits Settlements, Malaya, Henri de Ponchalon, Whampoa, Paul Doumer [End Page 97] Penang in 1837, as seen by the French explorer Captain Auguste Vaillant Colin Dyer (bio) Abstract The French Captain Auguste Vaillant arrived in the Straits of Malacca in February, 1837, in the corvette La Bonite. He called briefly in Malacca, where he was received by the Resident-Councillor, Samuel Garling, and arrived in Penang on 3 March. His account describes Penang's strategic position, its population and the effect of Singapore's development on its trade. During his visit, he visited the Resident-Councillor, J. W. Salmond, who famously performed his duties from a house on Penang Hill. He also met French missionaries and saw their establishments for the young and the elderly. On 6 March, after inviting Georgetown's authorities aboard his ship to bid them farewell, he set sail for India. Introduction This article presents an original and integral translation of a document that may be useful to others in their research. Auguste Vaillant was 44 years of age when he arrived in Penang. He had joined the French Navy at an early age, and had served during the Napoleonic Wars. In 1836, he was given command of a circumglobal expedition and, on 6 February of that year, set out from Toulon in the 800-ton corvette La Bonite. His formal instructions were set out in a letter from the Minister for the Navy, the Baron Duperré, dated 28 December 1836, and they specifically mentioned Penang. After leaving Toulon he was to visit the Canary Islands and then Rio de Janeiro before going around Cape Horn and arriving in Valparaiso. From there he was to go to Peru and from there cross the Pacific to the Sandwich Islands (Hawaii), then to the Philippines and Macao before arriving in Vietnam. From Vietnam he was to sail to Singapore at the southern end of the Malacca Strait. [End Page 98] "As you go along the Malacca Strait", Duperré had written, "you will stop, if the occasion presents itself, either at Malacca, or especially at the beautiful English settlement of Pulo Penang [sic]". Here, as elsewhere, he was to "show the flag of the King [Louis-Philippe]", which was to "be seen everywhere as a friendly flag", but Duperré added "It is important that our nation should appear everywhere as being strong and powerful".1 On the morning of 23 February 1837 (just over a year after leaving France), that La Bonite arrived "in sight of Malacca and its white houses", and "cast anchor in the harbour". "Of its former splendour", Vaillant would note later, "there remain nothing but ruins and pieces of wall from the fortifications where the inhabitants withstood so many sieges against the Malay sultans". Nonetheless, he added, "the population of Malacca is still ten thousand souls, made up of Malays, Chinese and mixed bloods, There is a garrison with four hundred sepoys, but the only whites living here are the officers of the troops and the administration employees, that is to say a Resident-Councillor, a deputy-Resident and a few minor agents". While in Malacca Vaillant was invited to dinner by the Resident-Councillor, Mr Garling, who was happy "to arrange everything" for his visitors before inviting them to a "charmante soirée dansante" the following day. Then, "on 26 February at daybreak", Vaillant set sail for Penang.2 On 3 March, the travellers ("after having admired, while sailing along the coast, the fresh countryside of Pulo-Pinang [sic] and the pretty sight of its numerous Malay villages") arrived in front of "Georges-Town [sic]", and dropped anchor a mile and a half from Fort Cornwallis".3 The extract from Vaillant's journal here begins at volume three, chapter 32, p. 326. On his return to France, Vaillant himself did not write up his journal, but left this to "A. De Lasalle" [sic],4 a clerk employed by the naval ministry, who wrote "from various documents put at his disposal".5 For this reason, Vaillant is referred to...