Control Of Multi-agent Systems Research Articles

Cooperative control of heterogeneous multi-agent systems under spatiotemporal constraints

A current trend in research on multi-agent control systems is to consider high-level task specifications that go beyond traditional control objectives and take into account the heterogeneity of each agent in the system, i.e., the different capabilities of the agents in terms of actuation, sensing, communication and computation. This article provides an overview of our work on the problem of control of heterogeneous multi-agent systems under both spatial and temporal constraints as well as our perspective on the challenges and open problems associated with the consideration of such spatiotemporal constraints. Initially, we review a set of control strategies introduced by the authors addressing the satisfaction of cooperative tasks such as formation control as well as individual objectives such as reference tracking. The satisfaction of those objectives is ensured using prescribed performance control. Building upon these approaches we then review recent results on control under high-level spatiotemporal objectives expressed in Signal Temporal Logic, a formal language that allows to express complex spatial tasks that must be satisfied within pre-defined deadlines. Theoretical results considering multi-agent systems with various capabilities under spatiotemporal constraints are presented.

All-in-One Framework for Design, Simulation, and Practical Implementation of Distributed Multiagent Control Systems

All-in-One Framework for Design, Simulation, and Practical Implementation of Distributed Multiagent Control Systems

Coordinated predictive control of networked multiagent systems via distributed cloud computing with time-varying transmission delays

This article examines the complex issue of coordinating networked multi-agent control systems through distributed cloud computing, with a specific focus on compensating for time-varying transmission delays using a predictive control scheme. Unlike the general approach, which treats the time-varying delays as a constant at their maximum value, this paper aims to minimize transmission delays as much as possible. This objective leads to a consensus problem characterized by a time-varying upper-triangular topology. In this study, stability criteria are established to ensure the stability of the time-varying error systems, and a simultaneous similar method is proposed to facilitate feedback designation. The effectiveness of the devised scheme is demonstrated through illustrative examples.

Coordinated tracking control of multi agent systems with full-state constraints

This paper presents a solution of the distributed coordinated tracking for multiagents modeled by Euler-Lagrange systems (ELSs), where model uncertainties and full-state constraints (FSCs) are considered. We first introduce a distributed estimator to obtain accurate estimations of the target trajectory. To guarantee the FSCs of ELSs, we next employ the barrier Lyapunov function to design a distributed coordinated tracking law combined with Moore-Penrose inverse. In order to handle the model uncertainties, the adaptive neural networks are then integrated into the proposed control law. Through the application of the Lyapunov stability theory, state errors are guaranteed that semi-global uniform ultimate boundedness. Simulation results finally demonstrate the feasibility of the developed strategies.

A deep multi-agent reinforcement learning framework for autonomous aerial navigation to grasping points on loads

Deep reinforcement learning, by taking advantage of neural networks, has made great strides in the continuous control of robots. However, in scenarios where multiple robots are required to collaborate with each other to accomplish a task, it is still challenging to build an efficient and scalable multi-agent control system due to increasing complexity. In this paper, we regard each unmanned aerial vehicle (UAV) with its manipulator as one agent, and leverage the power of multi-agent deep deterministic policy gradient (MADDPG) for the cooperative navigation and manipulation of a load. We propose solutions for addressing navigation to grasping point problem in targeted and flexible scenarios, and mainly focus on how to develop model-free policies for the UAVs without relying on a trajectory planner. To overcome the challenges of learning in scenarios with an increasing number of grasping points, we incorporate the demonstrations from an Optimal Reciprocal Collision Avoidance (ORCA) algorithm into our framework to guide the policy training and adapt two novel techniques into the architecture of MADDPG. Furthermore, curriculum learning with the attention mechanism is utilized by reusing knowledge from fewer grasping points to facilitate the training of a load with more points. Our experiments were validated by a load with three, four and six grasping points respectively in Coppeliasim simulator and then transferred into the real world with Crazyflie quadrotors. Our results show that the average tracking deviations from the desirable grasping point to the final position of the UAV can be less than 10 cm in some real-world experiments. Compared with state-of-the-art model-free reinforcement learning and swarm optimization algorithms, results show that our proposed methods outperform other baselines with a reasonable success rate especially in the scenarios with more grasping points. Furthermore, the learned optimal policies enable UAVs to reach and hover over all the grasping points before manipulation without any collision. We conducted a comprehensive analysis of both targeted and flexible navigation, highlighting their respective advantages and disadvantages.

Distributed sliding-mode cloud predictive formation control of networked multi-agent systems with application to air-bearing spacecraft simulators

This paper investigates the application of distributed sliding-mode cloud predictive control scheme in networked multi-agent control systems (NMACS). A sliding-mode cloud predictive control (SMCPC) scheme based on the cloud control system is designed. Combining the advantages of variable structure control (VSC) of sliding-mode control(SMC) and the advantages of cloud predictive control (CPC) in handling network delays, the network delay is actively compensated while ensuring the stability and output consistency of the NMACS. Also taking advantage of the small online computation of SMC, reduces the burden of CPC in processing big data and increases the computation rate. In addition, the scheme includes multi-step state prediction and output prediction as well as coordinated optimization among multiple agents. By optimizing the cost function design, the SMCPC scheme is guaranteed to achieve the desired control performance in multi-agent formation control. And the feasibility of the control scheme is verified by stability and consistency analysis. The applicability of the control scheme in practical applications is verified by air-bearing spacecraft simulator (ABS) formation control experiments.

Hierarchical Integration of Model Predictive and Fuzzy Logic Control for Combined Coverage and Target-Oriented Search-and-Rescue via Robots with Imperfect Sensors

Search-and-rescue (SaR) in unknown environments is a crucial task with life-threatening risks. SaR requires precise, optimal, and fast decisions to be made. Robots are promising candidates expected to execute various SaR tasks autonomously. While humans use heuristics to effectively deal with uncertainties of SaR, optimisation of multiple objectives (e.g., the mission time, the area covered, the number of victims detected), in the presence of physical and control constraints, is a mathematical challenge that requires machine computations. Thus including both human-inspired and mathematical capabilities in decision making of SaR robots is highly desired. However, developing control approaches that exhibit both capabilities has been significantly ignored in literature. Moreover, coordinating the decisions of the robots in large-scale SaR missions with affordable computation costs is an open challenge. Finally, in real-life, due to defects (e.g., in the sensors of the robots) or environmental factors (e.g., smoke) data perceived by SaR robots may be prone to uncertainties. We introduce a hierarchical multi-agent control architecture that simultaneously provides the following advantages: exploiting non-homogeneous and imperfect perception capabilities of SaR robots; improving the global performance as it is provided by centralised controllers; computational efficiency and robustness to failure of the central controller as offered by decentralised control methods. The integrated structure of the proposed control framework allows to combine human-inspired and mathematical decision making methods, via respectively fuzzy logic and model predictive control, in a coordinated and computationally efficient way. Our results for various computer-based simulations show that while the area coverage with the proposed control approach is comparable to existing heuristic methods that are particularly developed for coverage-oriented SaR, our approach has a significantly better performance regarding locating the trapped victims. Furthermore, with comparable computation times, the proposed control approach successfully avoids conflicts that may appear in non-cooperative control methods. In summary, the proposed multi-agent control system is capable of combining coverage-oriented and target-oriented SaR in a balanced and coordinated way.

Hierarchical multi-agent control framework for energy efficiency and carbon emission reduction in multi-zone buildings

Centralized optimization approaches to trade-off between comfort, energy and carbon emissions are widely adopted in the control of building heating, ventilation, and air-conditioning (HVAC) systems. However, the high computational complexity in each control horizon, single point of failure risks, and the limited number of zones to control make the centralized approach unattractive. Unlike centralized controls, multi-agent control (MAC) systems are flexible and modular. This paper proposes a scalable multi-agent based distributed approach for optimized control of a multi-zone smart building based on a set of local agents which represent individual zones in the building, coordinated by a central agent. For each control horizon, the coordinator minimizes the overall carbon emissions and assigns an individual energy budget to each local agent. Each local agent minimizes the discomfort in its zone while respecting the energy budget assigned by the coordinator. We propose a heuristic search based on a genetic algorithm to find the optimized control sequences in each zone, and formulate an integer linear programming (ILP) model for the coordinator problem which can be solved using an ILP solver. For a representative winter test day, the proposed methodology gave an energy savings of 8.8% and reduced the carbon footprint by 23.4%.

Tracking Control of Multi-Agent Systems Using a Networked Predictive PID Tracking Scheme

With the rapid development of network technology and control technology, a networked multi-agent control system is a key direction of modern industrial control systems, such as industrial Internet systems. This paper studies the tracking control problem of networked multi-agent systems with communication constraints, where each agent has no information on the dynamics of other agents except their outputs. A networked predictive proportional integral derivative (PPID) tracking scheme is proposed to achieve the desired tracking performance, compensate actively for communication delays, and simplify implementation in a distributed manner. This scheme combines the past, present and predictive information of neighbour agents to form a tracking error signal for each agent, and applies the proportional, integral, and derivative of the agent tracking error signal to control each individual agent. The criteria of the stability and output tracking consensus of multi-agent systems with the networked PPID tracking scheme are derived through detailed analysis on the closed-loop systems. The effectiveness of the networked PPID tracking scheme is illustrated via an example.

Reduction in optimal control with broken symmetry for collision and obstacle avoidance of multi-agent system on Lie groups

<abstract><p>We study the reduction by symmetry for optimality conditions in optimal control problems of left-invariant affine multi-agent control systems, with partial symmetry breaking cost functions for continuous-time and discrete-time systems. We recast the optimal control problem as a constrained variational problem with a partial symmetry breaking Lagrangian and obtain the reduced optimality conditions from a reduced variational principle via symmetry reduction techniques in both settings continuous-time, and discrete-time. We apply the results to a collision and obstacle avoidance problem for multiple vehicles evolving on $ SE(2) $ in the presence of a static obstacle.</p></abstract>

APPLICATION OF CLUSTER ANALYSIS IN SMART GRID NETWORKS

The task of developing and improving intelligent multi-agent control systems, which provide opportunities for intelligent online analysis of the electrical power supply system provided by the smart grid system, is considered. The purpose of the article is the formation of clusters of sections of the electrical network, the initial parameters of which are the main indicators of the quality of electrical energy in accordance with the accepted standards. Total harmonic distortion coefficient of voltages, coefficient of unsymmetry of supply voltages, reactive power of loads are chosen as such parameters. The results of the hierarchical clustering of power supply network sections are summarized in the corresponding dendogram. The method of full connection is used to form a dendogram of sections of the power supply system. This method defines the distance between clusters as the largest distance between any objects in different clusters or the most distant neighbors. The measure of closeness was determined by the Euclidean distance. The obtained treeshaped diagram demonstrates the distribution of the sections of the power supply system into four natural clusters, which visually divides the sections of the power supply system into separate groups according to certain characteristics, namely the main parameters of the quality of the power system. It is shown that the mathematical apparatus of cluster analysis allows solving the problem of classification of sections of the power supply system when the main parameters of the quality of electric energy deviate from the normally permissible values. It is proved that the classification can be carried out not only by one parameter, but also by using a combination of several parameters. The results of the conducted analysis provide an opportunity to further form production rules for the selection of measures to improve the quality of electric energy, which are applied to one or another section of the power supply system.)

Active Power Distribution Networks with Decentralized Multi-Agent Control Mode. Part 2

The article considers the solution of system tasks concerned with controlling the overall mode of an electric network with distributed generation: maintaining active and reactive power balances in normal modes, emergency control when large disturbances of the normal mode occur, maintaining the power supply system operability in post-emergency modes, restoring the power system integrity and normal operation after it was disintegrated into isolated parts by an accident or emergency control system operation. Methods of solving these tasks in a decentralized manner are proposed, which have been tested on physical models of power systems and in a pilot project on Minigrid. The developed Minigrid is integrated with an external electric network using a control system that is an agent of a multiagent system. A conclusion is made about the possibility of developing a technology for decentralized control of the modes of electric networks with distributed small generation sources. In the first part [1], the terminology used is given, the concept of a subject of the multiagent control system of electric networks with distributed generation is given, and matters concerned with multiagent decentralized control of frequency and voltage in such networks are considered. The second part of the article presents the results of a study, on an active electric network physical model, of decentralized multiagent restoration of the integrity and normal operation of the network after its disintegration into isolated parts by an accident or emergency control system operation.

Artifi cial Intelligence Technologies in the Autonomous Underwater Vehicle Control Systems

The purpose of the paper is to present the experience of developing algorithms implemented in the AUV control system using artificial intelligence (AI) technologies. The author’s view on the content of the concept of AI in relation to the creation of complex technical systems is presented. Two of the most promising, according to the author, AI technologies are identified: 1) development of the basic algorithm and its improvement based on the results of comprehensive modeling in various operating conditions; 2) creation of a problem-oriented artificial neural network and its deep learning using a large amount of experimentally obtained training material. It is stated that both technologies are quite time-consuming and require a long time to implement. But if the completeness and adequacy of modeling of the conditions in which the system being created is supposed to function plays a key role in modeling technology, then in machine learning technology, the availability of a sufficient amount of training material comes to the fore (in the case of developing vision systems — images of recognizable objects, the number of which can number many thousands).The paper presents the structure of the multi-agent control system of AUV, focuses on the complexity of the tasks it solves and the need to use AI technologies in its creation. It is shown that of all the tasks solved by the AUV control system, AI methods are most in demand for solving two categories of tasks: 1) recognition of the current situation and making an adequate decision in this situation in the interests of completing the route task; 2) search for the designated bottom object among many other bottom objects of natural and artificial origin. The use of AI technologies is demonstrated by the example of the development using a specially created simulation stand of the AUV control algorithm when bypassing an extended. It is proposed to solve the problem of detecting and recognizing a designated bottom object using deep learning technology of a problem-oriented artificial neural network with the peculiarity that the training material is formed programmatically in the form of digital images of the desired bottom object at the output of hydroacoustic, optical and electromagnetic means of monitoring the bottom in various conditions of their observation.

Developing a Learning Algorithm for a Multiagent Control System of an Electrical-Engineering Facility for an Oil- and Gas-Production Enterprise with Distributed Generation

Developing a Learning Algorithm for a Multiagent Control System of an Electrical-Engineering Facility for an Oil- and Gas-Production Enterprise with Distributed Generation

Разработка мультиагентной системы управления электрическими режимами электротехнического комплекса нефтегазодобывающего предприятия с распределенной генерацией

Разработка мультиагентной системы управления электрическими режимами электротехнического комплекса нефтегазодобывающего предприятия с распределенной генерацией

A blockchain-enabled trust aware energy trading framework using games theory and multi-agent system in smat grid

Recently, multi-agent systems (MASs) have received attention due to the consideration of distributed optimization and control in blockchain (BC) based energy trading applications. However, due to the dynamic behavior of uncertain and random variables, the coordination and control of MAS are still challenging to attain resilience and dynamicity. Traditional trust systems, which rely on access control and cryptography, cannot deal with dynamic behavior of agents. Furthermore, they are inefficient in addressing the computational overhead of cryptographic primitives. To overcome these limitations, this work proposes a BC-based trusted suite (TS) for MAS to handle privacy and anonymity issues during energy trading in a smart grid (SG). In this work, three objectives are simultaneously achieved: trust, cooperation, and confidentiality. Firstly, the proposed trust system is employed to perform trust credibility of agents based on trust deformation, coherence, and stability. The credibility evaluation is used to determine the dynamic behavior of agents and to detect dishonest agents in the system. Secondly, a tri-tit-for-tat (TTFT) repeated game approach is used to improve the cooperation among agents. The proposed strategy is more forgiving than the existing Di-TFT (DTFT) and TFT techniques. It motivates scammers and deceptive agents to regain their trust by cooperating in three consecutive rounds of a game. Furthermore a proof-of-cooperation (PoC) consensus mechanism is introduced to facilitate agent cooperation in block creation and validation. Thirdly, the publicly verifiable secret sharing (PVSS) technique is introduced to preserve the privacy of the agents. Unlike VSS, PVSS provides the immunity against different types of security threats. Where, the dealer agent maintained the trust, while the verification of the dealer and combiner is maintained within agent-to-agent cooperation. Simulation results show that the devised BTS model is superior to the existing benchmark model such as fuzzy logic trust (FLT) in terms of detecting the cheating and deceptive behavior of agents in the system. Besides, the devised TTFT allows cheating agents to effectively regain the trust if they cooperate thrice in a row as compared to the existing DTFT and TFT strategies. Furthermore, This study analyzes two trust-related attacks: bad-mouthing and on-off. Analysis shows that the proposed system is protected from trust related attacks.

Decentralized Holonic Control System Model for Line-less Mobile Assembly Systems

The Line-less Mobile Assembly System paradigm (short LMAS) provides necessitated flexibility, especially for large-scale products as customer demands for individualized products persist and product life-cycles remain short. To make LMAS advantages operationally usable in an industrial context, it requires a suitable control system to connect multi-purpose assembly resources and to autonomously configure transient assembly stations. Therefore this paper reviews the relevant literature to identify the necessary components of such architectures. Depending on the control system’s intention and use cases, the properties of the organizational paradigm, an adequate ontology and basic design patterns regarding the distribution and order-relationships of the system entities are to be defined conceptually. For the implementation, a role model, an interaction model, and a data model are required. Due to the utilization of mobile multipurpose resources and the possibility of factory shopfloor reconfiguration by transient stations, existing approaches do not meet LMAS inherent properties. Consequently, we present a suitable decentralized multi-agent control system approach.

Architecture design of battery energy storage coordinated control system based on Multi-Agent mechanism

Abstract Energy storage can realize rapid voltage, frequency support and energy adjustment, and is an important guarantee for promoting the efficient consumption of renewable energy and the safe and stable operation of the power system. The flexible adjustment of energy storage is not only affected by its own characteristics, but also closely related to the operation control mode of energy storage. Based on the PQ constant power and virtual synchronization control strategy of the battery energy storage system, this paper constructs the operation architecture of the battery energy storage system based on the Multi-Agent cooperative mechanism, and further gives the energy storage system Multi-Agent cooperative control system’s application scenarios in active frequency modulation and reactive voltage regulation. Research shows that this architecture helps to fully tap the potential of aggregation and regulation of multi-point decentralized energy storage facilities and meet the flexible regulation needs of the power system under the access of high-permeability intermittent renewable energy.

An adaptive multiagent control system for autonomous economic operation and resilience assurance in a hybrid-energy islanded microgrid

Multi-agent control is widely recognized as an effective management system for modern microgrids. This paper proposes a hierarchical distributed multi-agent control system for an islanded microgrid with hybrid-energy resources and a battery-storage bank. The dispatchable dispersed generators are to be online-controlled to stabilize the islanded microgrid and minimize operation cost. A non-dispatchable agent is assigned to each bus with a non-dispatchable generator or a load. A dispatchable agent is assigned to each bus with a dispatchable generator. The control scheme is organized into three layers. The primary control layer comprises only the agent which runs the very fast local voltage/frequency droop-control of the battery-storage bank. It achieves a quasi-instantaneous power balance under any disturbance to maintain the frequency and voltage stability of the microgrid. The secondary control layer is formed by fast control algorithms on the dispatchable agents working at 2 ms time step. It is responsible for regulating the islanded microgrid frequency and voltage at minimum participation time of the battery-storage bank. The tertiary control layer is made by relatively slow control algorithms on the dispatchable agents working at a time step of 200 ms. It copes with the economic operation of the microgrid. Each agent behaves autonomously and shares data with others via communication links. The architecture and control algorithms of each layer are presented. Besides, the coordination protocol between different layers is described. The scheme is implemented by a parallel Matlab-Jade computing setup. Results of a case study islanded microgrid are analysed and compared to literature.

Development of a learning algorithm for a multi-agent control system for an electrotechnical complex at oil and gas producing enterprise with microgrid

Development of a learning algorithm for a multi-agent control system for an electrotechnical complex at oil and gas producing enterprise with microgrid