Application Of Multi-agent Systems Research Articles

Deep reinforcement learning algorithm based on multi-agent parallelism and its application in game environment

Deep reinforcement learning has become a prominent area of research in artificial intelligence in recent years. Its application in solving complex tasks and game environments has garnered significant attention. This study aims to develop a deep reinforcement learning algorithm based on multi-agent parallelism to enhance intelligent decision-making in game environments. The algorithm combines a deep Q-network with a multi-agent cooperation strategy. Through parallel training of multiple agents, the learning process is accelerated, and decision accuracy is improved. The experimental results indicated that the Actor-Critic algorithm, when combined with precision rate, recall rate, and average fitness of multi-agent parallel, achieves a relatively high accuracy rate index, which stabilizes above 0.95. The recall rate index was also above 0.8, and the average fitness was in a relatively high range. The research shows that the deep reinforcement learning algorithm based on multi-agent parallelism performs better and is more effective in game environments. It can learn the optimal strategy faster and obtain higher rewards. This not only provides a new technical means for game development but also offers a useful reference for the application of multi-agent systems in complex environments.

Application of geospatial multi-agent system for simulation of different aspects of tuberculosis transmission

BACKGROUND. The article presents the modeling of the epidemiological picture of tuberculosis (TB) using a multi-agent system. This is to provide an in-depth multi-agent modeling study with comprehensive strategies related to TB treatment and prevention.
 OBJECTIVE. To develop an algorithm that uses the capabilities of artificial intelligence and neural networks to build a geospatial model of TB transmission.
 MATERIALS AND METHODS. This study involves the development of an algorithm that uses the potential of artificial intelligence to create a geospatial model that highlights the different routes of TB transmission. The simulation process itself is characterized by a number of key stages, including initialization of the city, calibration of health parameters, simulation of the working day, spread of infection, evolution of disease trajectories, rigorous statistical calculations and transition to the next day. A comprehensive description of the course of active TB according to official data of the World Health Organization is given. Each agent is provided with an appropriate and consistent model that includes relevant health attributes and necessary rules for their dynamic evolution.
 RESULTS AND DISCUSSION. The process of modeling the transmission of TB in the urban landscape has been thoroughly investigated. Attributes and functional dependencies are divided into three different groups: static (or deterministic), stochastic, and empirical. Empirical parameters that determine the contagiousness of TB patients are subjected to careful analysis, during which the dynamic nature of their actual contagiousness is carefully compared. Therefore, the empirical parameters that determine infectivity are carefully established, which additionally allows for the quantitative determination of the level of TB infection. A complex simulation is presented that illustrates the spread of TB in a completely healthy environment devoid of any preventive or therapeutic measures. This simulation serves as a crucial study of the spread of the disease. To test the adequacy of the model and its sensitivity to the main parameters that determine the course of TB, a series of experiments using three different approximations, namely: a basic model, a model that includes mortality factors and a comprehensive model covering all relevant aspects, was conducted. Statistics used in this context include the number of infected and latent persons, recoveries and deaths. The article presents a comparative analysis between statistical information and a model that offers a comprehensive picture of the state of health of each person. The model allows careful monitoring of various factors, such as interpersonal contacts, place of residence, workplace, modes of transport used to move from place to place, and other vital aspects of a person’s life, together contributing to a more complete understanding of the dynamics of the disease.
 CONCLUSIONS. The calculations show the stability of the results and the absence of large fluctuations. The obtained statistical values of infected, latent and recovered people correlate with known medical data, and therefore, confirm the adequacy of the proposed model. This model allows tracking and analyzing the life and behavior of each agent, which allows for a detailed assessment and analysis of the spread of TB, as well as to develop a strategy to prevent the spread of this infection.

Multi-agent Continuous Control with Generative Flow Networks

Generative Flow Networks (GFlowNets) aim to generate diverse trajectories from a distribution in which the final states of the trajectories are proportional to the reward, serving as a powerful alternative to reinforcement learning for exploratory control tasks. However, the individual-flow matching constraint in GFlowNets limits their applications for multi-agent systems, especially continuous joint-control problems. In this paper, we propose a novel Multi-Agent generative Continuous Flow Networks (MACFN) method to enable multiple agents to perform cooperative exploration for various compositional continuous objects. Technically, MACFN trains decentralized individual-flow-based policies in a centralized global-flow-based matching fashion. During centralized training, MACFN introduces a continuous flow decomposition network to deduce the flow contributions of each agent in the presence of only global rewards. Then agents can deliver actions solely based on their assigned local flow in a decentralized way, forming a joint policy distribution proportional to the rewards. To guarantee the expressiveness of continuous flow decomposition, we theoretically derive a consistency condition on the decomposition network. Experimental results demonstrate that the proposed method yields results superior to the state-of-the-art counterparts and better exploration capability. Our code is available at https://github.com/isluoshuang/MACFN.

Tailoring Gradient Methods for Differentially Private Distributed Optimization

Decentralized optimization is gaining increased traction due to its widespread applications in large-scale machine learning and multi-agent systems. The same mechanism that enables its success, i.e., information sharing among participating agents, however, also leads to the disclosure of individual agents' private information, which is unacceptable when sensitive data are involved. As differential privacy is becoming a de facto standard for privacy preservation, recently results have emerged integrating differential privacy with distributed optimization. However, directly incorporating differential privacy design in existing distributed optimization approaches significantly compromises optimization accuracy. In this paper, we propose to redesign and tailor gradient methods for differentially-private distributed optimization, and propose two differential-privacy oriented gradient methods that can ensure both rigorous <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\epsilon$</tex-math></inline-formula> -differential privacy and optimality. The first algorithm is based on static-consensus based gradient methods, and the second algorithm is based on dynamic-consensus (gradient-tracking) based distributed optimization methods and, hence, is applicable to general directed interaction graph topologies. Both algorithms can simultaneously ensure almost sure convergence to an optimal solution and a finite privacy budget, even when the number of iterations goes to infinity. To our knowledge, this is the first time that both goals are achieved simultaneously. Numerical simulations using a distributed estimation problem and experimental results on a benchmark dataset confirm the effectiveness of the proposed approaches.

Adaptive Fault-Tolerant Tracking Control for Discrete-Time Nonstrict-Feedback Nonlinear Systems With Stochastic Noises

This paper discusses the problem of fault-tolerant tracking control for discrete-time nonstrict-feedback nonlinear systems in the presence of stochastic noises and actuator faults. The system is characterized by a discrete-time nonstrict-feedback structure and multiplicative stochastic noise related to the states, which poses a challenge for the design and analysis of the fault-tolerant controller. Moreover, the considered actuator faults include multiplicative and additive faults without prior information. By integrating the properties of the backstepping framework and neural networks, and by introducing an adaptive fault compensation term, a novel fault-tolerant tracking control strategy is proposed. The effects of faults are compensated and the difficulties caused by the system structure are overcome, avoiding the algebraic loop problem and overcoming the causal contradiction. Given specific parameters, the designed fault-tolerant controller can ensure that the tracking error converges to an adjustable region regarding the origin and that all system signals are uniformly bounded concerning the mean-square sense. The simulation examples illustrate the effectiveness of the designed fault-tolerant control method. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —As system complexity has been increasing, practical systems are often affected by faults when performing tracking control. Fault-tolerant control can provide acceptable robustness and improve system safety. On the one hand, many practical systems are nonlinear, difficult to model mathematically and accurately, and affected by stochastic noises. On the other hand, with digital control, the object is a discrete-time system. In practical applications, the upper bounds of the faults and the noises are often unknown. In this paper, a novel adaptive fault-tolerant control method is proposed for the more general uncertain discrete-time nonlinear systems, i.e., nonstrict-feedback nonlinear systems, with stochastic noises and actuator faults. Neural networks and an adaptive fault compensation term are integrated into the fault-tolerant controller. By adjusting the controller parameters properly, it is ensured that the tracking error converges to a small neighborhood of zero in the event of a fault occurring. In future research, sensor faults, input constraints, optimal control design, and applications in multi-agent systems will be considered.

Sampling-Based Path Planning Algorithm for a Plug &amp; Produce Environment

The purpose of this article is to investigate a suitable path planning algorithm for a multi-agent-based Plug &amp; Produce system that can run online during manufacturing. This is needed since in such systems, resources can move around frequently, making it hard to manually create robot paths. To find a suitable algorithm and verify that it can be used online in a Plug &amp; Produce system, a comparative study between various existing sampling-based path planning algorithms was conducted. Much research exists on path planning carried out offline; however, not so much is performed in online path planning. The specific requirements for Plug &amp; Produce are to generate a path fast enough to eliminate manufacturing delays, to make the path energy efficient, and that it run fast enough to complete the task. The paths are generated in a simulation environment and the generated paths are tested for robot configuration errors and errors due to the target being out of reach. The error-free generated paths are then tested on an industrial test bed environment, and the energy consumed by each path was measured and validated with an energy meter. The results show that all the implemented optimal sampling-based algorithms can be used for some scenarios, but that adaptive RRT and adaptive RRT* are more suitable for online applications in multi-agent systems (MAS) due to a faster generation of paths, even though the environment has more constraints. For each generated path the computational time of the algorithm, move-along time and energy consumed are measured, evaluated, compared, and presented in the article.

A Study of Algorithms, Systems, and Applications of Multi-Agent Systems for Distributed Data Mining

Data mining is a powerful technology that converts data into competitive intelligence which businesses can use to proactively predict future trends, uncover meanings to historical happenings and discover business imperatives which was hitherto unknown to business.&#x0D; Data mining is a “process of discovering and interpreting previously unknown patterns in data to aid business in better decision-making”. Data mining is a set of automated techniques used to extract buried or previously unknown pieces of information from large databases.&#x0D; Data mining by nature is an iterative process and gets refined for further probing into data in a continuous manner. From a data management point of view, the data mining process requires exploration of data, creating the analytic data sets for evaluation, generating patterns and forecasting models.

Advancements in the Practical Applications of Agents, Multi-Agent Systems and Simulating Complex Systems

Introduction [...]

Application of multi agent systems for advanced energy management in cyber physical hybrid microgrid systems

This paper develops a multi agent system in real time for hybrid microgrids as advanced energy management (ADEM) protocol using a Java agent development environment (JADE) frame work. The proposed system is configured with two micro-grids, each having 1kw solar photo voltaic power, 1.5Kw wind power, 24 V, 150AH battery along with a local load, thus making it a hybrid Microgrid, modelled in MATLAB/Simulink. Due stability constraints while using MATLAB/Simulink, in a multi-threading environment, a middle ware called Multi agent Control Using Simulink with Jade Extension (MACSimJX) is adopted. The microgrid environment variables are captured through sensors, delivered to the agents operations in JADE, for the implementation of dynamic operation in hybrid microgrids. This results in maximizing operational efficiency of hybrid microgrids system. The simulation suggests that MAS can successfully manage dynamic loads, generated power in real time microgrids.

An Experience-Correction Method of Fractional-Order Systems Subject to Intermittent Sampled-Data Transmission: A Forgetting Curve Perspective

In this article, we establish an experience-correction (E-C) framework to deal with sampled-data fractional-order systems in the manner of general integer-order discrete-time model at sampling instants by coping with the memory effect. The designed E-C controller can be partitioned into local input and history input, respectively. Generated by current sampling, the local input is in a sense analogous to the input of integer-order system. In particular, the history input can correct all previous experience via incorporating earlier local inputs. Emphasize that the weights of earlier local inputs, norm of E-C controller together with corresponding sampling error are proved to descend to origin no slower than the Pareto function, i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal{O}(t^{-\alpha})$</tex-math> </inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula> represents the derivative’s order. Similar to the well-known forgetting curve, we witness a rapid decline in the proportion of earlier past in E-C controller, which is with respect to one of the memory failures: transience, the deterioration of memory with the passage of time. The closer the order <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha$</tex-math> </inline-formula> is to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1$</tex-math> </inline-formula> , the faster the forgetting is. The above analysis accounts for a phenomenon not previously understood: well-performed simulation results could still be expected in existing research who ignored memory effect and only considered local input. This is owing to the fractional orders are usually hypothesized to be larger than <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$0.5$</tex-math> </inline-formula> and thus the weight of earlier past decays sharply over time. Furthermore, both absolute and proportional decline of earlier past’s weight will slow down when time goes by, which is consistent with memory consolidation and Jost’s law in psychology. Overall, the E-C method keeps an elegant formulation which theoretically describes the evolution of hereditary effect. Meanwhile, it is also suitable for first-order systems. Thereafter, we address related applications in fractional-order multiagent systems (MASs) subject to intermittent sampled-data communication. Numerical simulations are provided finally to substantiate the effectiveness of theoretical results.

A Survey of Multi-Agent Systems on Distributed Formation Control

Multi-agent formation control is an important part of distributed perception and cooperation, which is convenient to complete various complex tasks and would be a key research direction in the future. This paper reviews the corresponding problems of formation control and the existing centralized and distributed formation control strategies. In particular, we discuss four types of distributed formation control methods based on position and displacement in the global coordinate system and distance and bearing in the nonglobal coordinate system, respectively. Moreover, this paper analyzes affine formation which does not require the global coordinate system. Combined with the current practical applications of multi-agent systems, the latest research for the formation control of the unmanned aerial vehicle (UAV), unmanned ground vehicle (UGV), unmanned surface vehicle (USV) and autonomous underwater vehicle (AUV) is given. Finally, the challenges and opportunities in this burgeoning field are discussed.

Bipartite-tracking quasi-consensus of nonlinear uncertain multi-agent systems: neural network-based adaptive state-constraint impulsive control approach

For the application of multi-agent systems (MASs), the uncertain dynamics and the actuator saturation are inevitable. Considering these factors, the bipartite-tracking quasi-consensus of nonlinear uncertain MASs over signed graph is discussed in this paper. Two kinds of neural network-based adaptive state-constraint impulsive control protocols are proposed, where the communication among the leader and followers only takes place at some discrete instants. Based on the superior approximation ability of radial basis function neural networks (RBFNNs), we design an adaptive weight updating strategy to compensate the uncertain nonlinear dynamics of the system. By means of linear matrix inequality (LMI), convex analysis, impulsive system theory and Lyapunov stability theory, we derive some simple sufficient conditions for the bipartite-tracking quasi-consensus under the assumption that the communication topology is structurally balanced. Furthermore, we estimate the region of attraction of the error system. Three simulation examples are also provided to show the effectiveness of the proposed protocols.

Multi-Agent-Based Controller for Microgrids: An Overview and Case Study

A microgrid can be defined as a grid of interconnected distributed energy resources, loads and energy storage systems. In microgrid systems containing renewable energy resources, the coordinated operation of distributed generation units is important to ensure the stability of the microgrid. A microgrid needs a successful control scheme to achieve its design goals. Undesirable situations such as distorted voltage profile and frequency fluctuations are significantly reduced by installing the appropriate hardware such as energy storage systems, and control strategies. The multi-agent system is one of the approaches used to control microgrids. The application of multi-agent systems in electric power systems is becoming popular because of their inherent benefits such as autonomy, responsiveness, and social ability. This study provides an overview of the agent concept and multi-agent systems, as well as reviews of recent research studies on multi-agent systems’ application in microgrid control systems. In addition, a multi-agent-based controller and energy management system design is proposed for the DC microgrid in the study. The designed microgrid is composed of a photovoltaic system consisting of 30 series-connected PV modules, a wind turbine, a synchronous generator, a battery-based energy storage system, critical and non-critical DC loads, the grid and the control system. The microgrid is controlled by the designed multi-agent-based controller. The proposed multi-agent-based controller has a distributed generation agent, battery agent, load agent and grid agent. The roles of each agent and communication among the agents are designed properly and coordinated to achieve control goals, which basically are the DC bus voltage quality and system stability. The designed microgrid and proposed multi-agent-based controller are tested for two different scenarios, and the performance of the controller has been verified with MATLAB/Simulink simulations. The simulation results show that the proposed controller provides constant DC voltage for any operation condition. Additionally, the system stability is ensured with the proposed controller for variable renewable generation and variable load conditions.

Latin Matchings and Ordered Designs OD(n−1, n, 2n−1)

This paper revisits a combinatorial structure called the large set of ordered design (LOD). Among others, we introduce a novel structure called Latin matching and prove that a Latin matching of order n leads to an LOD(n−1, n, 2n−1); thus, we obtain constructions for LOD(1, 2, 3), LOD(2, 3, 5), and LOD(4, 5, 9). Moreover, we show that constructing a Latin matching of order n is at least as hard as constructing a Steiner system S(n−2, n−1, 2n−2); therefore, the order of a Latin matching must be prime. We also show some applications in multiagent systems.

Prototype multi-agent robotic debris removal system: principles of development and experimental studies

Objectives. The article substantiates the relevance of the creation and the prospects of application of multi-agent robotic systems for elimination of consequences of emergency situations. The purpose of this work was to test the practical feasibility of algorithms for controlling a group of autonomous robots when performing multi-stage missions.Methods. The theses of the finite automata theory in planning actions of a multi-agent system, methods of automatic control in organizing a goal-directed movement of robots, and methods of computer vision in searching and analyzing debris geometry were used.Results. The principles of development, architecture, and composition are described for the software and algorithms of a prototype of the multi-agent robotic system created at RTU MIREA as part of integrated research for the creation of tools and methods of group control of robots. The multi-stage task of searching and removing debris in the process of eliminating the consequences of emergency situations is analyzed. A proposed algorithm for planning the actions of robotic agents determines the time sequence of the mission stages. Tasks are allocated among the performing robots according to assessments of their suitability. The autonomous functioning of robotic agents is determined by commands coming from the group control level, as well as an a priori embedded knowledge base with scenario models of appropriate actions. Compensation of local environmental uncertainties in the process of robot movement is based on a comprehensive analysis of visual and navigation information. Along with the main elements of the multi-agent system, the developed infrastructure of hardware and software for visual navigation and wireless communication is described.Conclusions. The results of the experimental studies demonstrated the efficiency of the developed approaches to the creation of intelligent technologies for group control of autonomous robots on the example of debris search and removal tasks. The feasibility of the multi-agent robotic system is demonstrated by the development and integration of a number of information management and infrastructure subsystems.

Applications of Multi-Agent Systems in Unmanned Surface Vessels

The comprehensive and safe application of unmanned surface vessels is certainly one of the biggest challenges currently facing maritime science. Such vessels can be implemented within a wide range of autonomy levels that goes from remote-controlled vessels to fully autonomous vessels in which intelligent vessel systems completely perform all necessary operations. One of the ways to achieve autonomous vessel systems is to implement multi-agent systems that take over all functions performed by the crew in classical manned crew vessels. A vessel is a complex system that conceptually can be considered as a set of interconnected subsystems. Theoretically, the functions of these subsystems could be performed using appropriate multi-agent systems. In this paper we analyzed 24 relevant papers. A review of the current state of implementation of multi-agent systems for performing the functions of unmanned surface vessels is presented.

Distributed output tracking via event-triggered control for nonlinear multi-agent systems

This paper investigates the problem of distributed output tracking for nonlinear multi-agent systems via event-triggered control under a directed graph topology inspired by the advantage that event-triggered control can effectively save resources and the wide application of multi-agent systems. By backstepping and emulation method, the event-triggered control strategy is put forward, by which a series of event-triggered controllers and event-triggering mechanisms with absolute or relative threshold are designed. An inevitable problem on event-triggered control, called Zeno phenomenon, is avoided by guaranteeing a positive minimum inter-execution interval. We can see from the event-triggered control strategy that the closed-loop system has a unique solution and tracking error could be adjusted to arbitrarily small while ensuring the boundedness of all states. Finally, a numerical example is shown to verify the validity of the conclusion.

A review of multi-agent mobile robot systems applications

&lt;span&gt;A multi-agent robot system (MARS) is one of the most important topics nowadays. The basic task of this system is based on distributive and cooperative work among agents (robots). It combines two important systems; multi-agent system (MAS) and multi-robots system (MRS). MARS has been used in many applications such as navigation, path planning detection systems, negotiation protocol, and cooperative control. Despite the wide applicability, many challenges still need to be solved in this system such as the communication links among agents, obstacle detection, power consumption, and collision avoidance. In this paper, a survey of the motivations, contributions, and limitations for the researchers in the MARS field is presented and illustrated. Therefore, this paper aims at introducing new study directions in the field of MARS.&lt;/span&gt;

Encoding topological optimisation logical structure rules into multi-agent system for architectural design and robotic fabrication

Natural phenomena have been explored as a source of architectural and structural design inspiration with different approaches undertaken within architecture and engineering. The research proposes a connection between two dichotomous principles: architectural complexity and structural efficiency through a hybrid of natural phenomena, topology optimisation and generative design. Both Bi-directional Evolutionary Structural Optimisation (BESO) and multi-agent algorithms are emerging technologies developed into new approaches that transform architectural and structural design, respectively, from the logic of topology optimisation and swarm intelligence. This research aims to explore a structural behaviour feedback loop in designing intricate functional forms through encoding BESO logical structure rules into the multi-agent algorithm. This research intends to study and evaluate the application of topology optimisation and multi-agent system in form-finding and later robotic fabrication through a series of prototypes. It reveals a supposition that the structural behaviour-based design method matches the beauty and function of natural appearance and structure. Thus, a new exploration of architectural design and fabrication strategy is introduced, which benefits the collaboration among architects, engineers and manufacturers. There is the potential to seek the ornamental complexities in architectural forms and the most efficient use of material based on structural performance in the process of generating complex geometry of the building and its various elements.

Multi-Agent System Development and Application for Supply Chain Management Tasks

In today’s dynamically changing environment, we need to be able to respond in a timely manner to changes in supply chain processes. Software agents are successfully used in supply chain management tasks for a variety of purposes. The behaviour of agents is determined by the purpose of their development, and the effectiveness of the use of agents is considered in accordance with the purpose of their development. The paper presents research on the development of a multi-agent system for supply chain management, focusing on the steps of developing a multi-agent system. The choice of each algorithm for agents is analysed and argued. The application of the developed multi-agent system for supply chain management is also described in the paper. The efficiency of application of the developed multi-agent system is presented as well.