Heterogeneous Multiagent Research Articles

Reinforcement Learning for Synchronization of Heterogeneous Multiagent Systems by Improved Q-Functions.

This article dedicates to investigating a methodology for enhancing adaptability to environmental changes of reinforcement learning (RL) techniques with data efficiency, by which a joint control protocol is learned using only data for multiagent systems (MASs). Thus, all followers are able to synchronize themselves with the leader and minimize their individual performance. To this end, an optimal synchronization problem of heterogeneous MASs is first formulated, and then an arbitration RL mechanism is developed for well addressing key challenges faced by the current RL techniques, that is, insufficient data and environmental changes. In the developed mechanism, an improved Q-function with an arbitration factor is designed for accommodating the fact that control protocols tend to be made by historic experiences and instinctive decision-making, such that the degree of control over agents' behaviors can be adaptively allocated by on-policy and off-policy RL techniques for the optimal multiagent synchronization problem. Finally, an arbitration RL algorithm with critic-only neural networks is proposed, and theoretical analysis and proofs of synchronization and performance optimality are provided. Simulation results verify the effectiveness of the proposed method.

Distributed Cooperative Control of Heterogeneous Multiagent Systems for Air‐to‐Ground Formation

ABSTRACTThis paper is concerned with a distributed cooperative formation problem for a heterogeneous multiagent system with a car‐like mobile robot, a quadrotor, and an omnidirectional wheeled mobile robot. In the heterogeneous multiagent system, a three‐layer distributed cooperative formation control strategy is constructed to achieve air‐to‐ground formation, the car‐like mobile robot tracks a virtual leader, the quadrotor, and the omnidirectional wheeled mobile robot track the car‐like mobile robot. Two nonlinear leader‐follower controllers based on exponential virtual control laws are designed to maintain formation for the quadrotor and the omnidirectional wheeled mobile robot. A kinematics controller with continuous functions, an improved adaptive sliding mode controller and a suitable back‐stepping controller are introduced in the heterogeneous multiagent system, respectively. Multiagent communication is one of the key steps to accomplish multiagent time synchronization. Experimental results are given to illustrate the applicability of the proposed distributed cooperative control strategy.

Persistent surveillance by heterogeneous multi-agents using mutual information based on observation capability

Persistent surveillance by heterogeneous multi-agents using mutual information based on observation capability

Robust Predefined Output Containment for Heterogeneous Nonlinear Multiagent Systems Under Unknown Nonidentical Leaders' Dynamics.

This article discusses the robust predefined output containment (RPOC) control problem for heterogeneous nonlinear multiagent systems having multiple uncertain nonidentical leaders. In order to solve this problem, a new kind of distributed observer-based RPOC control framework is presented. First, for obtaining the information of nonidentical leaders' dynamics, including uncertain parameters in leaders' system matrices, output matrices, states, and outputs, four kinds of adaptive observers are constructed in a fully distributed form without any knowledge of the dynamics of nonidentical leaders, exactly. Second, on the basis of adaptive learning technique, a new RPOC controller is then developed by using the presented observers, where the adaptive observers can make up for the uncertain parameter in followers' dynamics, and the solutions of output regulation equations can be obtained adaptively by the developed adaptive strategy. Furthermore, with the help of the output regulation method and Lyapunov stability theory, the RPOC criteria for the considered system under unknown nonidentical leaders' dynamics are derived from the constructed controller. Finally, a simulation example is provided to demonstrate the effectiveness of the proposed RPOC controller.

Guest Editorial Special Issue on Robust Cooperative Control for Heterogeneous Nonlinear Multiagent Systems

Guest Editorial Special Issue on Robust Cooperative Control for Heterogeneous Nonlinear Multiagent Systems

Collision-Free Formation Control for Heterogeneous Multiagent Systems Under DoS Attacks.

A safe time-varying formation (TVF) control framework is proposed in this article for heterogeneous multiagent systems under the constraints of denial of service (DoS) attacks, noncooperative dynamic obstacles, and input saturation. The framework integrates both the cyber-layer and physical-layer components to address the challenges posed by these adverse conditions. In the cyber-layer, a distributed resilient observer is provided based on a control Lyapunov function (CLF)-quadratic program (QP). This observer estimates a reference exosystem, effectively decoupling heterogeneous dynamics from unsafe networks and optimizing the system resilience against DoS attacks. At the physical-layer, for the first time, a collision-free TVF controller is presented based on the CLF-exponential control barrier function-QP. The controller guarantees high-order heterogeneous agents' operation safety under noncooperative obstacles and input saturation. The effectiveness and advantages of the proposed algorithms are verified through the comparative simulations and experiments conducted on a physical system comprising unmanned aerial vehicles and unmanned ground vehicles.

Robust Formation Tracking Control for Noncooperative Heterogeneous Multiagent Systems.

The robust formation tracking problems (FTPs) are investigated for noncooperative heterogeneous multiagent systems (MASs). The parameters and state dimensions of each agent in the MASs can be heterogeneous. The leader is noncooperative and has an unknown external input. The followers are affected by uncertainties and disturbances. A distributed extended state observer is designed to estimate the leader's state and the unknown external input. Then a robust formation tracking control method is proposed. It turns out that the output formation tracking error can reduce to any size neighborhood near the origin by selecting appropriate parameters. Finally, the effectiveness of the proposed observer and controller is demonstrated by a numerical simulation example and a hardware-in-the-loop (HITL) simulation example.

Consensus for Heterogeneous Multiagent Systems: Output Rate-Coded Secure Control.

This article investigates an output rate-coded secure control based on backstepping for the consensus of heterogeneous multiagent systems (MASs) with output-triggering condition. Due to the nondifferentiable virtual control inputs caused by discontinuous triggering signals, it presents a technical obstacle in implementing the recursive backstepping, rendering previous results inapplicable. To address this problem, an auxiliary high-order filter is elaborately constructed to guarantee the required-order derivatives of virtual control inputs. As this filter is driven by local triggering consensus error, it allows our MASs to have different system orders and relative degrees. Besides, to reduce communication bit and strengthen communication security, we propose a novel distributed rate-coded algorithm from an encoding-decoding viewpoint. When it is triggered, agent's output is encrypted into an L -length codeword, then transmitted to neighbors without exposing any sensitive system states or real control inputs. It is shown that all the closed-loop system signals are ultimately bounded, and the mean square consensus tracking error can be reduced by appropriately selecting control parameters. Simulations illustrate our theoretical finding.

Neural Network-Based Fixed-Time Tracking and Containment Control of Second-Order Heterogeneous Nonlinear Multiagent Systems.

This study concentrates on the fixed-time tracking consensus and containment control of second-order heterogeneous nonlinear multiagent systems (MASs) with and without measurable velocity under directed topology. By defining a time-varying scaling function and approximating the unknown nonlinear dynamics with radial basis function neural networks (RBFNNs), a novel distributed protocol for solving the fixed-time tracking consensus and containment control problems of second-order heterogeneous nonlinear MASs with full states available is proposed based on a nonsingular sliding-mode control method constructed by designing a prescribed-time convergent sliding surface. For the scenario of immeasurable velocity, a fixed-time convergent states' observer is designed to reveal the velocity information when the unknown linearity is bounded. Subsequently, a distributed fixed-time consensus protocol based on observed velocity information is proposed for the extended results. Ultimately, the acquired results are verified by three simulation examples.

Simplified ADP-Based Distributed Event-Triggered Fault-Tolerant Control of Heterogeneous Nonlinear Multiagent Systems With Full-State Constraints

Simplified ADP-Based Distributed Event-Triggered Fault-Tolerant Control of Heterogeneous Nonlinear Multiagent Systems With Full-State Constraints

Neuroadaptive Cooperative Fault-Tolerant Control of Heterogeneous Multiagent Systems Based on Fully Actuated System Approaches.

The leader-following cooperative problem in heterogeneous multiagent systems (HMASs) with unmodeled dynamics and actuator faults is investigated in this article. The HMASs, which include unmanned ground vehicles and unmanned aerial vehicles, are first described using a fully actuated system model (FASM). The FASM, as opposed to the first-order state-space model, preserves the physical significance of original systems and makes it feasible to apply the control rule entirely. In order to approximate unknown system dynamics, novel neuroadaptive laws with few learning parameters are then suggested. To counteract the negative effects of actuator faults, the Nussbaum function and adaptive approach are utilized. In addition, a cooperative fault-tolerant protocol is suggested, wherein consensus errors are uniformly ultimately bounded. The lack of virtual control variables in the proposed protocol reduces its complexity. The theoretical results are then validated by numerical simulations.

Leader-Following Secure Output Consensus of Heterogeneous Multiagent Systems Based on Two Sampling Mechanisms Under Hybrid Cyber-Attacks.

This article investigates the leader-following secure output consensus (LFSOC) problem of the heterogeneous multiagent systems (MASs) under the hybrid cyber-attacks. A novel hybrid cyber-attack model consisting of aperiodic additive deception (AAD) attacks and aperiodic denial of service (ADoS) attacks is proposed for characterizing cyber-attacks in a real network, where the aperiodicity is reflected in the fact that the duration of each cyber-attack can be different. First, a compensator is introduced for each agent to estimate the leader's state. Second, two sampling mechanisms consisting of a multirate sampling (MRS) mechanism and a periodic sampling mechanism are employed for heterogeneous MASs. The MRS mechanism is used to obtain real-time sampled data on the different physical variables of each agent. The periodic sampling mechanism is applied to sample the agents' compensators and the sampled data are broadcast to their neighbors immediately through a network. By selecting an appropriate sampling period (i.e., a communication period of the compensators), the robustness of heterogeneous MASs against hybrid cyber-attacks can be enhanced. Then, the appropriate communication period is selected for the compensators in different network environments by taking into consideration the cyber-attack parameters. Under these two sampling mechanisms, a sync controller is developed to achieve the LFSOC of heterogeneous MASs. Finally, an example is presented to verify the effectiveness of the proposed approach.

Event-Triggered Adaptive Containment Control for Heterogeneous Stochastic Nonlinear Multiagent Systems.

This article investigates the event-triggered adaptive containment control problem for a class of stochastic nonlinear multiagent systems with unmeasurable states. A stochastic system with unknown heterogeneous dynamics is established to describe the agents in a random vibration environment. Besides, the uncertain nonlinear dynamics are approximated by radial basis function neural networks (NNs), and the unmeasured states are estimated by constructing the NN-based observer. In addition, the switching-threshold-based event-triggered control method is adopted with the hope of reducing communication consumption and balancing system performance and network constraints. Moreover, we develop the novel distributed containment controller by utilizing the adaptive backstepping control strategy and the dynamic surface control (DSC) approach such that the output of each follower converges to the convex hull spanned by multiple leaders, and all signals of the closed-loop system are cooperatively semi-globally uniformly ultimately bounded in mean square. Finally, we verify the efficiency of the proposed controller by the simulation examples.

Output Consensus of Heterogeneous Multiagent Systems With Mismatched Uncertainties and Measurement Noises: An ADRC Approach

Output Consensus of Heterogeneous Multiagent Systems With Mismatched Uncertainties and Measurement Noises: An ADRC Approach

Distributed Fault-Tolerant Control for Nonlinear Heterogeneous Multiagent System With Fault Indication Under Safety Constraint

Distributed Fault-Tolerant Control for Nonlinear Heterogeneous Multiagent System With Fault Indication Under Safety Constraint

Adaptive Event-Triggered Time-Varying Output Group Formation Containment Control of Heterogeneous Multiagent Systems

Adaptive Event-Triggered Time-Varying Output Group Formation Containment Control of Heterogeneous Multiagent Systems

Neural-Network-Based Adaptive Fault-Tolerant Cooperative Control of Heterogeneous Multiagent Systems With Multiple Faults and DoS Attacks.

In this article, the issue of adaptive fault-tolerant cooperative control is addressed for heterogeneous multiple unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) with actuator faults and sensor faults under denial-of-service (DoS) attacks. First, a unified control model with actuator faults and sensor faults is developed based on the dynamic models of the UAVs and UGVs. To handle the difficulty introduced by the nonlinear term, a neural-network-based switching-type observer is established to obtain the unmeasured state variables when DoS attacks are active. Then, the fault-tolerant cooperative control scheme is presented by utilizing an adaptive backstepping control algorithm under DoS attacks. According to Lyapunov stability theory and improved average dwell time method by integrating the duration and frequency characteristics of DoS attacks, the stability of the closed-loop system is proved. In addition, all vehicles can track their individual references, while the synchronized tracking errors among vehicles are uniformly ultimately bounded. Finally, simulation studies are given to demonstrate the effectiveness of the proposed method.

Event-Based Distributed Average Tracking for Second-Order Heterogeneous Multiagent Systems With Uncertain Dynamics

Event-Based Distributed Average Tracking for Second-Order Heterogeneous Multiagent Systems With Uncertain Dynamics

Fully Data-Driven Robust Output Formation Tracking Control for Heterogeneous Multiagent System With Multiple Leaders and Actuator Faults.

This article investigates a fully data-driven method to solve the robust output formation tracking control problem for the multiagent system (MAS) under actuator faults. The outputs of the followers are controlled to track those of multiple leaders with respect to a convex point while achieving an expected time-varying formation. To obviate the requirement of various system prior knowledge in typical MAS control, a hierarchical frame is developed with three learning and control stages using the online measured data. First, a distributed adaptive observer is designed to coordinate the state convex of multiple leaders while estimating unknown dynamics. The adaptive mechanism relaxes the demand for global topology. Second, by collecting and reusing the online system data, an off-policy reinforcement learning (RL) method is proposed in a continuous form to acquire nominal feedback gains from partial observations of the followers. Essential system models are learned along with the RL process, while solutions to the output regulation equations are implicitly obtained. Third, a comprehensive robust controller is further presented based on the previous learning results. To address the actuator faults with efficiency loss and bias, the adaptive neural networks and robust compensations are utilized in a model-free manner. The output formation tracking is achieved under a derived feasibility condition while stabilities of the learning and control methods are analyzed. Finally, simulation results demonstrate the validity of this fully data-driven control frame.

Finite-Time Time-Varying Formation Tracking for Heterogeneous Nonlinear Multiagent Systems Using Adaptive Output Regulation.

The finite-time output time-varying formation tracking (TVFT) problem for heterogeneous nonlinear multiagent system (MAS) is investigated in this article, where the dynamics of the agents can be nonidentical, and leader's input is unknown. The target of this article is that the outputs of followers need to track leader's output and realize the desired formation in finite time. First, for removing the assumption that all agents are required to know the information of leader's system matrices and the upper boundary of its unknown control input in previous studies, a kind of finite-time observer is constructed by exploiting the neighboring information, which can estimate not only the leader's state and system matrices but also can compensate for the effects of unknown input. On the basis of the developed finite-time observers and adaptive output regulation method, a novel finite-time distributed output TVFT controller is proposed with the help of the technique of coordinate transformation by introducing an extra variable, which removes the assumption that the generalized inverse matrix of follows' input matrix needs to be found in the existing results. By means of the Lyapunov and finite-time stability theory, it is proven that the expected finite-time output TVFT can be realized by the considered heterogeneous nonlinear MASs within a finite time. Finally, simulation results demonstrate the efficacy of the proposed approach.