Multi-agent Formation Control Research Articles

Multiagent Formation Control for Obstacle Traversing Using Distance-Only Measurements in GPS-Denied Environments

Multiagent Formation Control for Obstacle Traversing Using Distance-Only Measurements in GPS-Denied Environments

Safe Region Multi-Agent Formation Control With Velocity Tracking

This paper provides a solution to the problem of safe region formation control with reference velocity tracking for a second-order multi-agent system without velocity measurements. Safe region formation control is a control problem where the agents are expected to attain the desired formation while reaching the target region and simultaneously ensuring collision and obstacle avoidance. To tackle this control problem, we break it down into two distinct objectives: safety and region formation control, to provide a completely distributed algorithm. Region formation control is modeled as a high-level abstract objective, whereas safety and actuator saturation are modeled as a low-level objective designed independently, without any knowledge of the former, and being minimally invasive. Our approach incorporates connectivity preservation, actuator saturation, safety considerations, and absence of velocity measurement from other agents with second-order system dynamics which are important constraints in practical applications. Both internal safety for collision avoidance among agents and external safety for avoiding unsafe regions are ensured using exponential control barrier functions. We provide theoretical results for asymptotic convergence and numerical simulation to show the approach’s effectiveness.

Bearing-Based Formation Tracking Control of Nonholonomic Mobile Agents With a Persistently Exciting Leader

This paper investigates the leader-following formation control problem of nonholonomic agents with a persistently exciting leader based on bearing measurements. A follower needs to maintain a desired relative position to the leader in its local coordinate frame using bearing measurements. An integrated method consisting of a bearing-based estimator and a tracking controller is proposed to achieve the leader-following control objective. The estimator is designed to estimate the relative position to the leader in the follower's local coordinate frame. Based on the estimated relative position, the controller is proposed to drive the follower to achieve the desired formation. Different from existing integrated methods, our method does not require any conditions on the relative bearing between the leader and the follower in the global frame, which are difficult to guarantee. We characterize the conditions locally on the leader only, namely, conditions on the leader's linear and angular velocities with persistently exciting steering speed, such that the overall system is asymptotically stable with our integrated method. Besides, the proposed method is extended to the multi-agent formation control problem. A reasonable solution is proposed to the multi-agent case with a minimal leader follower structure. Simulations prove the effectiveness of the proposed methods in both cases.

Collision avoidance and connectivity preservation using asymmetric barrier Lyapunov function with time-varying distance-constraints

Collision avoidance and connectivity preservation are two contrasting issues in multi-agent formation control problems. This paper proposes a distributed control design methodology to stabilize a desired formation shape in a multi-agent system while incorporating these two factors simultaneously. Contrary to a simplifying point mass assumption, we consider that agents in the group are characterized by a circular disk-type structure of different radii. Collision avoidance and connectivity preservation in the group are handled by applying time-varying constraints on the inter-center distances among neighboring agents. The concept of asymmetric time-varying barrier Lyapunov function is exploited to derive the stabilizing distributed control law, under a mild assumption on the initial conditions and a feasibility condition relating dimensions of the desired formation with the lower and upper limits of the barrier functions. We also obtain analytical bounds on the various post-design signals and illustrate the results through a simulation example. Finally, the effect of input saturation is discussed, and a comparison with the sensing-region-dependent potential function-based control design approach from the existing literature is provided.

Fourier-Based Multi-Agent Formation Control to Track Evolving Closed Boundaries

Fourier-Based Multi-Agent Formation Control to Track Evolving Closed Boundaries

Hierarchical Multiagent Formation Control Scheme via Actor-Critic Learning.

This article presents a nearly optimal solution to the cooperative formation control problem for large-scale multiagent system (MAS). First, multigroup technique is widely used for the decomposition of the large-scale problem, but there is no consensus between different subgroups. Inspired by the hierarchical structure applied in the MAS, a hierarchical leader-following formation control structure with multigroup technique is constructed, where two layers and three types of agents are designed. Second, adaptive dynamic programming technique is conformed to the optimal formation control problem by the establishment of performance index function. Based on the traditional generalized policy iteration (PI) algorithm, the multistep generalized policy iteration (MsGPI) is developed with the modification of policy evaluation. The novel algorithm not only inherits the advantages of high convergence speed and low computational complexity in the generalized PI algorithm but also further accelerates the convergence speed and reduces run time. Besides, the stability analysis, convergence analysis, and optimality analysis are given for the proposed multistep PI algorithm. Afterward, a neural network-based actor-critic structure is built for approximating the iterative control policies and value functions. Finally, a large-scale formation control problem is provided to demonstrate the performance of our developed hierarchical leader-following formation control structure and MsGPI algorithm.

A Deep Reinforcement Learning Approach Combined With Model-Based Paradigms for Multiagent Formation Control With Collision Avoidance

Generating collision-free formation control strategy for multiagent systems faces huge challenges in collaborative navigation tasks, especially in a highly dynamic and uncertain environment. Two typical methodologies for solving this problem are the conventional model-based paradigm and the data-driven paradigm, particularly the widely used deep reinforcement learning (DRL) method. However, both the model-based and data-driven paradigms encounter inherent drawbacks. In this paper, we present two novel general schemes that combine these two paradigms together in an online mode. Specifically, the two paradigms are combined in a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">parallel</i> and a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">serial</i> structure in these two schemes, respectively. In the parallel scheme, the outputs of the model-based and DRL-based controllers are lumped together. In the serial scheme, the output of the model-based controller is fed as an input of the DRL-based controller. The interpretation of the two combined schemes is suggested from a control-oriented perspective, where the parallel DRL controller is viewed as a complementary uncertainty compensator and the serial DRL controller is taken as an inverse dynamics estimator. Finally, comprehensive simulations are conducted to demonstrate the superiority of the proposed schemes, and the effectiveness is further verified by deploying our schemes to a physical experiment platform based on a set of three-wheeled omnidirectional robots.

Distributed sliding-mode cloud predictive formation control of networked multi-agent systems with application to air-bearing spacecraft simulators

This paper investigates the application of distributed sliding-mode cloud predictive control scheme in networked multi-agent control systems (NMACS). A sliding-mode cloud predictive control (SMCPC) scheme based on the cloud control system is designed. Combining the advantages of variable structure control (VSC) of sliding-mode control(SMC) and the advantages of cloud predictive control (CPC) in handling network delays, the network delay is actively compensated while ensuring the stability and output consistency of the NMACS. Also taking advantage of the small online computation of SMC, reduces the burden of CPC in processing big data and increases the computation rate. In addition, the scheme includes multi-step state prediction and output prediction as well as coordinated optimization among multiple agents. By optimizing the cost function design, the SMCPC scheme is guaranteed to achieve the desired control performance in multi-agent formation control. And the feasibility of the control scheme is verified by stability and consistency analysis. The applicability of the control scheme in practical applications is verified by air-bearing spacecraft simulator (ABS) formation control experiments.

A Survey of Multi-Agent Systems on Distributed Formation Control

Multi-agent formation control is an important part of distributed perception and cooperation, which is convenient to complete various complex tasks and would be a key research direction in the future. This paper reviews the corresponding problems of formation control and the existing centralized and distributed formation control strategies. In particular, we discuss four types of distributed formation control methods based on position and displacement in the global coordinate system and distance and bearing in the nonglobal coordinate system, respectively. Moreover, this paper analyzes affine formation which does not require the global coordinate system. Combined with the current practical applications of multi-agent systems, the latest research for the formation control of the unmanned aerial vehicle (UAV), unmanned ground vehicle (UGV), unmanned surface vehicle (USV) and autonomous underwater vehicle (AUV) is given. Finally, the challenges and opportunities in this burgeoning field are discussed.

Study on learning algorithm of transfer reinforcement for multi-agent formation control

Considering the obstacle avoidance and collision avoidance for multi-agent cooperative formation in multi-obstacle environment, a formation control algorithm based on transfer learning and reinforcement learning is proposed. Firstly, in the source task learning stage, the large storage space required by Q-table solution is avoided by using the value function approximation method, which effectively reduces the storage space requirement and improves the solving speed of the algorithm. Secondly, in the learning phase of the target task, Gaussian clustering algorithm was used to classify the source tasks. According to the distance between the clustering center and the target task, the optimal source task class was selected for target task learning, which effectively avoided the negative transfer phenomenon, and improved the generalization ability and convergence speed of reinforcement learning algorithm. Finally, the simulation results show that this method can effectively form and maintain formation configuration of multi-agent system in complex environment with obstacles, and realize obstacle avoidance and collision avoidance at the same time.

Multi-Agent Formation Control under Switching Network and Input Constraints

This work discusses formation control of heterogeneous Multi-Agent Systems (MASs) under discrete time setting where its formation size is scalable by a scaling factor. The case with and without input constraints are discussed. The communication network is assumed to be jointly connected and the leader-follower network is assumed for the unconstrained case. Compared to the previous work on distributed scalable formation control for heterogeneous agents, this work considers input constraints in the formulation. The proposed algorithm is based on the discrete time version of the internal model principle and the constraint is handled by the command governor. Numerical examples are shown to illustrate the proposed method.

Dual quaternion matrices in multi-agent formation control

Dual quaternion matrices in multi-agent formation control

A Multilayer Graph for Multiagent Formation and Trajectory Tracking Control Based on MPC Algorithm.

This article studies the formation and trajectory tracking control of multiagent systems. We present a novel multilayer graph for the multiagent system to enable extensibility of the interaction network. Based on the multilayer graph, a formation control law by using the potential function approach is developed for autonomous formation, formation maintenance, collision, and obstacle avoidance. When the desired formation is achieved, the barycentric of the formation shape is viewed as a virtual leader, and a model predictive control (MPC) scheme is applied to the virtual leader for tracking a reference trajectory; meanwhile, the agents will maintain the desired angles and distances via the formation control law. By applying the proposed schemes, the tasks of formation maintenance and trajectory tracking in a constrained space are fulfilled. Comprehensive simulation studies under different environmental constraints and trajectories confirm the effectiveness of the proposed approaches in addressing the formation and trajectory tracking problems.

Learning-Based Cooperative Multiagent Formation Control With Collision Avoidance

This article presents a learning-based controller to solve the cooperative formation control problem for multiagent system (MAS) with collision avoidance. First, the consensus problem of first-order MAS is mostly solved by linear matrix inequality (LMI) without consideration of energy loss. To overcome these difficulties, an adaptive dynamic programming (ADP) technique is fit to solve the consensus problem and similar formation control problem for second-order MAS by the establishment of a performance index function. Besides, we introduce the generalized policy iteration (GPI) algorithm as a kind of ADP technique without the problem of low convergence speed and high computational complexity. Combined with previous works, it can be found that our proposed structure can be extended to high-order cases based on the structure of local neighborhood formation error and algorithm. Afterward, the convergence analysis, optimality analysis, and stability analysis are given. Neural networks (NNs) are also implemented to approximate the iterative control policies and value functions, respectively. Moreover, we realize that many collisions may occur in the formation control problem. Inspired by the idea of the artificial potential field (APF) technique, the concept of the repulsive force field is introduced based on our proposed learning-based structure to avoid collisions simply and efficiently. Finally, a simulation is provided to demonstrate the effectiveness of our proposed method.

A decentralized learning strategy to restore connectivity during multi-agent formation control

In this paper, we propose a decentralized learning algorithm to restore communication connectivity during multi-agent formation control. The time-varying connectivity profile of a mobile multi-agent system represents the dynamic information exchange capabilities among agents. While connected to the neighbors, each mobile agent in the proposed scheme learns to raise the team connectivity. When the inter-agent communication is lost, the associated trained neural network generates appropriate control actions to restore connectivity. The proposed learning technique leverages an adaptive control formalism, wherein a neural network tries to mimic the negative gradient of a value that relies on the agent-to-neighbor distances. All agents use the conventional consensus protocol during the connected multi-agent dynamics, and under communication loss, only the lost agent executes the neural network predicted actions to come back to the fleet. Simulation results demonstrate the effectiveness of our proposed approach for single/multiple agent loss even in the presence of velocity disturbances.

Singular Values of Dual Quaternion Matrices and Their Low-Rank Approximations

The dual quaternion matrix is an important and powerful tool for the study of multi-agent formation control. However, many basic properties of dual quaternion matrices are still not studied, due to the difficulty incurred by the non-commutativity of multiplication and existence of zero divisors of dual quaternion numbers. In this paper, we study some basic properties of dual quaternion matrices, including the polar decomposition theorem, the minimax principle and Weyl’s type monotonicity inequality for singular values, spectral norm and the Pythagoras theorem. Based upon these, we further present the best low-rank approximations for dual quaternion matrices, and discuss the relationship between the approximation degree characterizations of the best low-rank approximations in the sense of Frobenius norm and spectral norm, in addition to proving a fundamental property of the best low-rank approximation of dual quaternion matrices in a given subspace. These results are of importance to the applications in rigid body motion and data reduction.

A Visual Predictive Control Framework for Robust and Constrained Multi-Agent Formation Control

A Visual Predictive Control Framework for Robust and Constrained Multi-Agent Formation Control

A distributed FDI cyber-attack detection in discrete-time nonlinear multi-agent systems using neural networks

This paper proposes a distributed, false data injection (FDI) cyber-attack detection method in communication channels for a class of discrete-time, nonlinear, heterogeneous, multi-agent systems controlled by our formation-based controller. A distributed neural network (NN)-based observer is proposed that generates the residual signal which is used in detection of FDI attacks on agents’ sensors, actuators, and neighboring communication channels in a multi-agent formation control setting. A radial basis function neural network (RBFNN) is used to approximate the unknown nonlinearity in the dynamics. A Lyapunov stability theory is used to prove that the attack detection residual and the multi-agent formation error are uniformly ultimately bounded (UUB), and to explicitly derive the NN weights tuning law and the attack detectability threshold. The proposed method’s attack detectability properties are analyzed, and simulation results are provided to demonstrate performance of the detection methodology.

Gradient-based bearing-only formation control: An elevation angle approach

This paper proposes an elevation angle rigidity theory in both 2D and 3D spaces, and applies it to solve multi-agent formation control with only inter-agent bearing/direction measurements in agents’ local coordinate frames. Motivated by the sensor technology in measuring elevation angle and angular diameter, we develop elevation angle rigidity by attaching each agent in a multi-agent framework with a rod in 2D and a ball in 3D, respectively. By defining the elevation angle rigidity matrix, conditions for infinitesimal elevation angle rigidity are derived. Compared to previously developed angle rigidity-based and bearing rigidity-based formation control laws, the proposed elevation angle rigidity-based control law can maintain the gradient-based control property. Compared to distance-based formation control laws, less sensor measurements are required. The formation maneuvering with desired translation and rotation is also realized by using only local bearing measurements. Simulation examples illustrate the advantages and effectiveness of the proposed approach.

A Distributed Multi-Agent Formation Control Method Based on Deep Q Learning.

Distributed control method plays an important role in the formation of a multi-agent system (MAS), which is the prerequisite for an MAS to complete its missions. However, the lack of considering the collision risk between agents makes many distributed formation control methods lose practicability. In this article, a distributed formation control method that takes collision avoidance into account is proposed. At first, the MAS formation control problem can be divided into pair-wise unit formation problems where each agent moves to the expected position and only needs to avoid one obstacle. Then, a deep Q network (DQN) is applied to model the agent's unit controller for this pair-wise unit formation. The DQN controller is trained by using reshaped reward function and prioritized experience replay. The agents in MAS formation share the same unit DQN controller but get different commands due to various observations. Finally, through the min-max fusion of value functions of the DQN controller, the agent can always respond to the most dangerous avoidance. In this way, we get an easy-to-train multi-agent collision avoidance formation control method. In the end, unit formation simulation and multi-agent formation simulation results are presented to verify our method.