Multi-agent Dynamical Systems Research Articles

Cooperation Dynamics in Multiagent Systems: Modeling Vehicular Cooperation through Game Theory

<div>Cooperation lies at the core of multiagent systems (MAS) and multiagent reinforcement learning (MARL), where agents must navigate between individual interests and collective benefits. Advanced driver assistance systems (ADAS), like collision avoidance systems and adaptive cruise control, exemplify agents striving to optimize personal and collective outcomes in multiagent environments. The study focuses on strategies aimed at fostering cooperation with the aid of game-theoretic scenarios, particularly the iterated prisoner’s dilemma, where agents aim to optimize personal and group outcomes. Existing cooperative strategies, such as tit-for-tat and win-stay lose-shift, while effective in certain contexts, often struggle with scalability and adaptability in dynamic, large-scale environments. The research investigates these limitations and proposes modifications to align individual gains with collective rewards, addressing real-world dilemmas in distributed systems. By analyzing existing cooperative strategies, the research investigates their effectiveness in encouraging group-oriented behavior in repeated games. It suggests modifications to align individual gains with collective rewards, addressing real-world dilemmas in distributed systems. Furthermore, it extends to scenarios with exponentially growing agent populations (<i>N</i> → +∞), addressing computational challenges using mean-field game theory to establish equilibrium solutions and reward structures tailored for infinitely large agent sets. Practical insights are provided by adapting simulation algorithms to create scenarios conducive to cooperation for group rewards. Additionally, the research advocates for incorporating vehicular behavior as a metric to assess the induction of cooperation, bridging theoretical constructs with real-world applications.</div>

Distributed Coordination D-Stabilization in Cyclic Pursuit Formations of Dynamical Multi-Agent Systems

In this study, the cyclic pursuit formation stabilization problem in target-enclosing operations by multiple homogeneous dynamic agents is investigated. To this end, a Lyapunov D-stability problem is first formulated to cover the transient performance requirements for multi-agent systems. Then, a simple diagrammatic Lyapunov D-stability criterion for cyclic pursuit formation is derived. The formation control scheme combined with a cyclic-pursuit-based distributed online path generator satisfying this condition guarantees both the required transient performance and global convergence properties with theoretical rigor. It is shown that the maximization of the connectivity gain in a cyclic-pursuit-based online path generator can be reduced to an optimization problem subject to linear matrix inequality constraints derived using the generalized Kalman-Yakubovich–Popov lemma. This approach provides a permissible range of connectivity gain, which not only guarantees global formation stability/convergence properties but also satisfies the required performance specification. Several numerical examples are provided to confirm the effectiveness of the proposed method.

Optimized distributed formation control using identifier–critic–actor reinforcement learning for a class of stochastic nonlinear multi-agent systems

This article is to propose an adaptive reinforcement learning (RL)-based optimized distributed formation control for the unknown stochastic nonlinear single-integrator dynamic multi-agent system (MAS). For solving the issue of unknown dynamic, an adaptive identifier neural network (NN) is developed to determine the stochastic MAS under expectation sense. And then, for deriving the optimized formation control, the RL is putted into effect via constructing a pair of critic and actor NNs. With regard of the traditional RL optimal controls, their algorithm exists the inherent complexity, because their adaptive RL algorithm are derived from negative gradient of the square of Hamilton–Jacobi–Bellman (HJB) equation. As a result, these methods are difficultly extended to stochastic dynamical systems. However, since this adaptive RL laws are derived from a simple positive function rather than the square of HJB equation, it can make optimal control with simple algorithm. Therefore, this optimized formation scheme can be smoothly performed to the stochastic MAS. Finally, according to theorem proof and computer simulation, the optimized method can realize the required control objective.

Non-local interaction in discrete Ricci curvature-induced biological aggregation

We investigate the collective dynamics of multi-agent systems in two- and three-dimensional environments generated by minimizing discrete Ricci curvature with local and non-local interaction neighbourhoods. We find that even a single effective topological neighbour suffices for significant order in a system with non-local topological interactions. We also explore topological information flow patterns and clustering dynamics using Hodge spectral entropy and mean Forman–Ricci curvature.

Graph Neural Network Based Asynchronous Federated Learning for Digital Twin-Driven Distributed Multi-Agent Dynamical Systems

The rapid development of artificial intelligence (AI) and 5G paradigm brings infinite possibilities for data annotation for new applications in the industrial Internet of Things (IIoT). However, the problem of data annotation consistency under distributed architectures and growing concerns about issues such as data privacy and cybersecurity are major obstacles to improving the quality of distributed data annotation. In this paper, we propose a reputation-based asynchronous federated learning approach for digital twins. First, this paper integrates digital twins into an asynchronous federated learning framework, and utilizes a smart contract-based reputation mechanism to enhance the interconnection and internal interaction of asynchronous mobile terminals. In addition, in order to enhance security and privacy protection in the distributed smart annotation system, this paper introduces blockchain technology to optimize the data exchange, storage, and sharing process to improve system security and reliability. The data results show that the consistency of our proposed FedDTrep distributed intelligent labeling system reaches 99%.

Competitive Equilibrium for Dynamic Multiagent Systems: Social Shaping and Price Trajectories

Competitive Equilibrium for Dynamic Multiagent Systems: Social Shaping and Price Trajectories

Switching adaptive dynamic event-triggered consensus of nonlinear multi-agent systems under DoS attacks and communication delay

This article addresses the issue of achieving security consensus in multi-agent systems (MASs) that are susceptible to denial-of-service attacks (DoS-As) and time-varying communication delays (TCD). We have developed a novel unified model to describe DoS-As and TCD. Firstly, a new switching adaptive dynamic event-triggered mechanism (SADETM) is proposed to conserve communication resources during DoS-As and TCD. The mechanism transforms into three event-triggered strategies adapted to highly dynamic MASs. Then, a consensus control protocol is designed through consensus errors based on SADETM. Finally, the sufficient conditions for ensuring the stability of MASs are solved by combining the Lyapunov-Krasovskii functional (LKF) method and the linear matrix inequality (LMI) technique. The effectiveness of this method is verified through simulation examples.

Decentralized leader-following consensus control design for discrete-time multi-agent systems with switching topology

The problem of consensus control of linear discrete-time multi-agent systems (MASs) with switching topology is considered in the presence of a leader. The goal of consensus control is to bring the states of all agents to the leader state while providing stability for local agents, as well as the MAS as a whole. In contrast to the traditional approach, which uses the concept of an extended dynamic multi-agent system model and communication topology graph Laplacian, this paper proposes a decomposition approach, which provides a separate design of local controllers. The control law is chosen in the form of distributed feedback with discrete PID controllers. The problem of local controllers’ design is reduced to a set of semidefinite programming problems using the method of invariant ellipsoids. Sufficient conditions for agents’ stabilization and global consensus condition fulfillment are obtained using the linear matrix inequality technique. The availability of information about a finite set of possible configurations between agents allows us to design local controllers offline at the design stage. A numerical example demonstrates the effectiveness of the proposed approach.

Resilient coordination of multi-agent systems in the presence of unknown heterogeneous actuation efficiency and coupled dynamics: distributed approaches

To guarantee the stability and performance of an overall heterogeneous multi-agent system, feedback control structures must be capable of addressing the presence of heterogeneous agent-based nonlinear uncertainties, unknown actuation performance, and coupling effects. Additionally, some tasks require agents to track the leader's state and reach different states in a distributed way. Therefore, this paper proposes a distributed adaptive controller that drives a set of agents to user-assigned positions in the presence of the mentioned heterogeneous system anomalies. To address actuator dynamics, a hedging-based reference model is used to decouple adaptive updating from the actuator dynamics. To deal with the coupled dynamics, an observer-based estimation algorithm is proposed. Finally, the low-frequency learning method is used to deal with the high-frequency control response and, therefore, to obtain a better transient performance. The overall closed-loop system's stability is investigated using the Lyapunov Stability Theory, and the Linear Matrix Inequalities (LMI) technique is employed to identify stability limits regarding the actuator bandwidths of each agent. Illustrative numerical examples are presented to demonstrate the designed controller's performance in a heterogeneous uncertain multi-agent system's dynamics with unknown actuation capabilities and coupled dynamics.

Optimized leader–follower consensus control for high-order nonlinear multi-agent system modeled in canonical dynamic form

This paper addresses the optimized leader–follower consensus control problem of nonlinear canonical dynamic multi-agent system (MAS). Since every agent is modeled in high-order dynamic, they contain the various state variables having derivative relations. Consequently, to attain the optimized consensus control, backstepping technique and reinforcement learning (RL) are combined together. In the first backstepping step, the virtual control is formulated to have the consensus error term consisting of neighbor agents’ output states. In the last backstepping step, since the nonlinearity dynamic is arisen, the optimized actual control is derived by performing the critic-actor RL. It is worth mentioning that, in the traditional RL optimal control, the critic and actor updating laws are obtained by performing the gradient descent algorithm to square of the approximated Hamilton–Jacobi-Bellman (HJB) equation, which includes multiple nonlinearity items, as a result, the algorithm is very intricate. However, in this optimized consensus control, since the RL training laws are yielded in the light of negative gradient of a simple positive function that is relevant to the HJB equation, its algorithm is very simple. At the same time, it can also avoid the persistence excitation conditions. Finally, theory and simulation confirm the validity of this optimized consensus method.

Time-Varying Optimal Formation Control for Second-Order Multiagent Systems Based on Neural Network Observer and Reinforcement Learning.

This article addresses a distributed time-varying optimal formation protocol for a class of second-order uncertain nonlinear dynamic multiagent systems (MASs) based on an adaptive neural network (NN) state observer through the backstepping method and simplified reinforcement learning (RL). Each follower agent is subjected to only local information and measurable partial states due to actual sensor limitations. In view of the distributed optimized formation strategic needs, the uncertain nonlinear dynamics and undetectable states may jointly affect the stability of the time-varying cooperative formation control. Furthermore, focusing on Hamilton-Jacobi-Bellman optimization, it is almost incapable of directly dealing with unknown equations. Above uncertainty and immeasurability processed by adaptive state observer and NN simplified RL are further designed to achieve desired second-order formation configuration at the least cost. The optimization protocol can not only solve the undetectable states and realize the prescribed time-varying formation performance on the premise that all the errors are SGUUB, but also prove the stability and update the critics and actors easily. Through the above-mentioned approaches offer an optimal control scheme to address time-varying formation control. Finally, the validity of the theoretical method is proven by the Lyapunov stability theory and digital simulation.

Prosocial dynamics in multiagent systems

AbstractMeeting today's major scientific and societal challenges requires understanding dynamics of prosociality in complex adaptive systems. Artificial intelligence (AI) is intimately connected with these challenges, both as an application domain and as a source of new computational techniques: On the one hand, AI suggests new algorithmic recommendations and interaction paradigms, offering novel possibilities to engineer cooperation and alleviate conflict in multiagent (hybrid) systems; on the other hand, new learning algorithms provide improved techniques to simulate sophisticated agents and increasingly realistic environments. In various settings, prosocial actions are socially desirable yet individually costly, thereby introducing a social dilemma of cooperation. How can AI enable cooperation in such domains? How to understand long‐term dynamics in adaptive populations subject to such cooperation dilemmas? How to design cooperation incentives in multiagent learning systems? These are questions that I have been exploring and that I discussed during the New Faculty Highlights program at AAAI 2023. This paper summarizes and extends that talk.

Distributed Localization for Multi-Agent Systems With Random Noise Based on Iterative Learning.

This article is concerned with the real-time localization problem for the dynamic multi-agent systems with measurement and communication noises under directed graphs. The barycentric coordinates are introduced to describe the relative position between agents. A novel robust distributed localization estimation algorithm based on iterative learning is proposed. The relative-distance unbiased estimator constructed from the historical iterative information is used to suppress the measurement noise. The designed stochastic approximation method with two iterative-varying gains is used to inhibit the communication noise. Under the zero-mean and independent distributed conditions on the measurement and communication noises, the asymptotic convergence of the proposed methods is derived. The numerical simulation and the QBot-2e robot experiment are conducted to test and verify the effectiveness and the practicability of the proposed methods.

Opinion Dynamics in Multiagent Systems with Optimal Choice of Opinion Verification Moments

Opinion Dynamics in Multiagent Systems with Optimal Choice of Opinion Verification Moments

Flocking and swarming in a multi-agent dynamical system.

Over the past few decades, the research community has been interested in the study of multi-agent systems and their emerging collective dynamics. These systems are all around us in nature, such as bacterial colonies, fish schools, and bird flocks, as well as in technology, such as microswimmers and robotics, to name a few. Flocking and swarming are two key components of the collective behaviors of multi-agent systems. In flocking, the agents coordinate their direction of motion, but in swarming, they congregate in space to organize their spatial position. We propose a minimal mathematical model of a locally interacting multi-agent system where the agents simultaneously swarm in space and exhibit flocking behavior. Various cluster structures are found depending on the interaction range. When the coupling strength value exceeds a crucial threshold, flocking behavior is observed. We do in-depth simulations and report the findings by changing the other parameters and with the incorporation of noise.

Submodularity-based false data injection attack scheme in multi-agent dynamical systems

Consensus in multi-agent dynamical systems is prone to be sabotaged by the adversary, which has attracted much attention due to its key role in broad applications. In this paper, we study a new false data injection (FDI) attack design problem, where the adversary with limited capability aims to select a subset of agents and manipulate their local multi-dimensional states to maximize the consensus convergence error. We first formulate the FDI attack design problem as a combinatorial optimization problem and prove it is NP-hard. Then, based on the submodularity optimization theory, we provide the necessary and sufficient conditions to guarantee that the convergence error is a submodular function of the set of compromised agents, satisfying the property of diminishing marginal returns. In other words, the benefit of adding an extra agent to the compromised set decreases as that set becomes larger. With this property, we exploit the greedy scheme to find the optimal compromised agent set that can produce the maximum convergence error when adding one extra agent to that set each time. Thus, the FDI attack set selection algorithms are developed to obtain the near-optimal subset of the compromised agents. Furthermore, we derive the analytical suboptimality bounds and the worst-case running time under the proposed algorithm. Extensive simulation results are conducted to show the effectiveness of the proposed algorithm.

A kinetic approach to investigate the collective dynamics of multi-agent systems

A kinetic approach to investigate the collective dynamics of multi-agent systems

Leader–follower consensus control for a class of nonlinear multi-agent systems using dynamical neural networks

This paper addresses the formation consensus tracking control problem of uncertain dynamical multi-agent system with nonlinear dynamics based on a leader–follower configuration based on a undirected communication topology. The control design is based on a novel combination of consensus protocols and dynamical neural networks, known as differential neural networks, to approximate the modeling uncertainties and external disturbances of the follower agents. The proposed controller-identifier structure forces the state of each agent to maintain a formation with respect to the leader ensuring limited residual errors. The stability proof supports the control design and guarantees the multi-agent system states to be ultimately bounded. In order to show the effectiveness of the proposed tracking controller and the superior performance against the static RBFNNs approach, some simulation results are presented considering a multi-robotic system with five agents following a helical three-dimensional reference signal of a virtual leader.

Towards privacy-preserving cooperative control via encrypted distributed optimization

Abstract Cooperative control is crucial for the effective operation of dynamical multi-agent systems. Especially for distributed control schemes, it is essential to exchange data between the agents. This becomes a privacy threat if the data are sensitive. Encrypted control has shown the potential to address this risk and ensure confidentiality. However, existing approaches mainly focus on cloud-based control and distributed schemes are restrictive. In this paper, we present a novel privacy-preserving cooperative control scheme based on encrypted distributed optimization. More precisely, we focus on a secure distributed solution of a general consensus problem, which has manifold applications in cooperative control, by means of the alternating direction method of multipliers (ADMM). As a unique feature of our approach, we explicitly take into account the common situation that local decision variables contain copies of quantities associated with neighboring agents and ensure the neighbor’s privacy. We show the effectiveness of our method based on a numerical case study dealing with the formation of mobile robots.

Power Degrees in Dynamic Multi-Agent Systems

Power Degrees in Dynamic Multi-Agent Systems