Multi-agent Approach Research Articles

Utility-Driven End-to-End Network Slicing for Diverse IoT Users in MEC: A Multi-Agent Deep Reinforcement Learning Approach.

Mobile Edge Computing (MEC) is crucial for reducing latency by bringing computational resources closer to the network edge, thereby enhancing the quality of services (QoS). However, the broad deployment of cloudlets poses challenges in efficient network slicing, particularly when traffic distribution is uneven. Therefore, these challenges include managing diverse resource requirements across widely distributed cloudlets, minimizing resource conflicts and delays, and maintaining service quality amid fluctuating request rates. Addressing this requires intelligent strategies to predict request types (common or urgent), assess resource needs, and allocate resources efficiently. Emerging technologies like edge computing and 5G with network slicing can handle delay-sensitive IoT requests rapidly, but a robust mechanism for real-time resource and utility optimization remains necessary. To address these challenges, we designed an end-to-end network slicing approach that predicts common and urgent user requests through T distribution. We formulated our problem as a multi-agent Markov decision process (MDP) and introduced a multi-agent soft actor-critic (MAgSAC) algorithm. This algorithm prevents the wastage of scarce resources by intelligently activating and deactivating virtual network function (VNF) instances, thereby balancing the allocation process. Our approach aims to optimize overall utility, balancing trade-offs between revenue, energy consumption costs, and latency. We evaluated our method, MAgSAC, through simulations, comparing it with the following six benchmark schemes: MAA3C, SACT, DDPG, S2Vec, Random, and Greedy. The results demonstrate that our approach, MAgSAC, optimizes utility by 30%, minimizes energy consumption costs by 12.4%, and reduces execution time by 21.7% compared to the closest related multi-agent approach named MAA3C.

Bio-inspired distributed load frequency control in Islanded Microgrids: A multi-agent deep reinforcement learning approach

To prevent errors in load frequency control (LFC) decision-making in an islanded microgrid and reduce the waste of frequency regulation resources, a fully distributed “starfish” load frequency control method is proposed. For this method, a novel deep reinforcement learning algorithm called the distributed decomposed multirole multiagent deep deterministic policy gradient (DDMR-MADDPG) algorithm is introduced. This algorithm treats each unit as an agent, enabling centralized training of all the agents. Through the coordination and control of multiple agents, a global optimal strategy can be obtained. During online application, the algorithm employs a distributed implementation strategy, enabling each unit to make decisions autonomously. This decision-making strategy is akin to the distributed neural network of a starfish, and it can fundamentally improve the efficiency and correctness of decision-making. In addition, it employs multiple techniques, including value function decomposition, multirole learning, imitation learning, and curriculum learning approaches. The performance of the proposed method compared with 24 existing methods is demonstrated on a simulated LFC model of the Zhuzhou Island microgrid in China.

A Two-Stage Hybrid Stochastic–Robust Coordination of Combined Energy Management and Self-Healing in Smart Distribution Networks Incorporating Multiple Microgrids

This paper presents a two-stage hybrid stochastic–robust coordination of energy management and self-healing in smart distribution networks with multiple microgrids. A multi-agent systems approach is first used for coupling energy management and self-healing strategies of microgrids, based on expert system rules. The second stage problem, a framework similar to that of the first stage, is then established for the smart distribution networks. Then, hybrid stochastic–robust optimization is used to model the uncertainties of demand, energy price, power generation of renewable energy sources, demand of electric vehicles, and accessibility of zone agents. Further, the grey wolf algorithm is used to solve the formulated optimization problem and achieve an optimal and reliable solution. The proposal is validated on a 69-bus distribution network consisting of three microgrids. The results validate that the proposal minimizes microgrids’ utilization indices, such as energy costs, energy losses, and network voltage drops, while simultaneously managing a flexible distribution network. It is also verified that the proposed multi-agent system design provides a high-speed and optimized self-healing solution for the network.

Augmented multi-agent algorithm utilizing intelligent search range detection heuristic to solve assembly line sequencing problem: A case study in the truck industry

Car sequencing is a widely adopted approach to implicitly enhance the short-term balancing of mixed-model assembly lines in practical applications. In this research, we investigate this problem in the context of a real-world case taken from a truck final assembly line. Our study considers cross-ratio constraints that address limitations related to dependent features of the products. Additionally, to manage the distribution of violation errors in cases where predefined congestion level thresholds (soft rules) exist, we propose a piecewise linear penalty function. To tackle these considerations, we formulate a mathematical programming model aimed at minimizing the total weighted penalty incurred from violating both independent and dependent sequencing rules. As the problem falls into the NP-hard category, we develop a multi-start parallel local search algorithm based on multi-agent approach to efficiently solve it. The algorithm is augmented by incorporating a self-adaptive diversification mechanism and a novel intelligent search range detection heuristic. The proposed heuristic enables the searching agents to intelligently explore the solution space rather than relying on a purely random search. The numerical experiments are conducted using 30 real-world instances from three truck assembly lines to validate the quality of the proposed algorithm. The comparison of the results indicates that our algorithm outperforms state-of-the-art algorithms. Furthermore, computational experiments demonstrate the effectiveness of each proposed mechanism in enhancing the algorithm's performance.

Authentication communication by using visualization cryptography for UAV networks

Utilizing unmanned aerial vehicles (UAVs) to support V2X requirements leverages their versatility and line-of-sight communication. Our prior work explored a multi-agent learning approach for resource optimization, maximizing task offloading while maintaining QoS. This paper focuses on securing UAV communication, particularly authentication. Traditional methods are often unsuitable due to UAV limitations. We propose a novel authentication mechanism for a single ground control station (GCS) interacting with multiple UAVs across flight sessions. The system utilizes a key generation method based on chaotic maps to create unique flight session keys for each pre-defined flight plan. These keys, along with flight plans, are registered in a secure database. During flight, the GCS verifies UAV identity by employing the flight session key and corresponding flight plan for message authentication. This approach reduces computational and communication overhead compared to traditional certificate exchanges and asymmetric cryptography, which are energy-intensive for UAVs. While not a comprehensive security solution, this method provides an initial layer of protection for the UAV network.

Collaborative optimization of multi-energy multi-microgrid system: A hierarchical trust-region multi-agent reinforcement learning approach

In the context of the expanding diversity of energy demands, an increasing number of heterogeneous Multi-energy Microgrids (MEMGs) are engaging in the collaborative framework of the Multi-energy Multi-microgrid System (MEMMG). However, following this trend, the existing centralized Integrated Energy Management System (IEMS) control strategy is unreliable for future energy systems, characterized by more complex optimization control and a flexible system structure. This paper introduces a hierarchical Multi-agent Deep Reinforcement Learning (HMADRL) approach for distributed IEMS in MEMMG. Firstly, by employing a hierarchical approach, this method simplifies control complexity by segmenting the overarching control challenge into tasks of collaborative planning and action control, which are distributed across varied multi-agent scenes. Considering both macro and microeconomic factors, alongside carbon emissions, the optimal operation of MEMMG is realized through a bottom-up edge multi-agent control approach, in contrast to traditional top-down centralized methods. Secondly, in the phase of the inter-MEMG collaborative strategy, the Centralized Training Decentralized Execution (CTDE) framework is adopted to overcome the problems of unstable training environments and large-scale agent training, and each heterogeneous microgrid can develop local strategies independently with the assurance that their internal data will not be overly exposed. Thirdly, within each MEMG, the Trust-Region (TR) model is introduced for multi-agent action control, adeptly addressing the effects of mutual exclusion in decision-making time series. Simultaneously, an initialized Hot Experience Pool (HEP) is implemented, efficiently reducing exploration in complex, high-dimensional spaces. Finally, the off-time agent model is integrated into the HMADRL environment and undergoes secondary training based on real interactions, thereby deriving the optimal energy management policy. The proposed method markedly reduces reliance on exact physical modeling systems. Comparative simulations validate the proposed control scheme’s efficacy.

Visual Reasoning and Multi-Agent Approach in Multimodal Large Language Models (MLLMs): Solving TSP and mTSP Combinatorial Challenges

Multimodal Large Language Models (MLLMs) harness comprehensive knowledge spanning text, images, and audio to adeptly tackle complex problems. This study explores the ability of MLLMs in visually solving the Traveling Salesman Problem (TSP) and Multiple Traveling Salesman Problem (mTSP) using images that portray point distributions on a two-dimensional plane. We introduce a novel approach employing multiple specialized agents within the MLLM framework, each dedicated to optimizing solutions for these combinatorial challenges. We benchmarked our multi-agent model solutions against the Google OR tools, which served as the baseline for comparison. The results demonstrated that both multi-agent models—Multi-Agent 1, which includes the initializer, critic, and scorer agents, and Multi-Agent 2, which comprises only the initializer and critic agents—significantly improved the solution quality for TSP and mTSP problems. Multi-Agent 1 excelled in environments requiring detailed route refinement and evaluation, providing a robust framework for sophisticated optimizations. In contrast, Multi-Agent 2, focusing on iterative refinements by the initializer and critic, proved effective for rapid decision-making scenarios. These experiments yield promising outcomes, showcasing the robust visual reasoning capabilities of MLLMs in addressing diverse combinatorial problems. The findings underscore the potential of MLLMs as powerful tools in computational optimization, offering insights that could inspire further advancements in this promising field.

MADDPG-Based Deployment Algorithm for 5G Network Slicing

One of the core features of 5G networks is the ability to support multiple services on the same infrastructure, with network slicing being a key technology. However, existing network slicing architectures have limitations in efficiently handling slice requests with different requirements, particularly when addressing high-reliability and high-demand services, where many issues remain unresolved. For example, predicting whether actual physical resources can meet network slice request demands and achieving flexible, on-demand resource allocation for different types of slice requests are significant challenges. To address the need for more flexible and efficient service demands, this paper proposes a 5G network slicing deployment algorithm based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG). Firstly, a new 5G network slicing deployment system framework is established, which measures resources for three typical 5G network slicing scenarios (eMBB, mMTC, uRLLC) and processes different types of slice requests by predicting slice request traffic. Secondly, by adopting the multi-agent approach of MADDPG, the algorithm enhances cooperation between multiple service requests, decentralizes action selection for requests, and schedules resources separately for the three types of slice requests, thereby optimizing resource allocation. Finally, simulation results demonstrate that the proposed algorithm significantly outperforms existing algorithms in terms of resource efficiency and slice request acceptance rate, showcasing the advantages of multi-agent approaches in slice request handling.

Computation Rate Maximization for SCMA-Aided Edge Computing in IoT Networks: A Multi-Agent Reinforcement Learning Approach

Computation Rate Maximization for SCMA-Aided Edge Computing in IoT Networks: A Multi-Agent Reinforcement Learning Approach

Cooperative Multi-UAV Positioning for Aerial Internet Service Management: A Multi-Agent Deep Reinforcement Learning Approach

Cooperative Multi-UAV Positioning for Aerial Internet Service Management: A Multi-Agent Deep Reinforcement Learning Approach

Path planning and collision avoidance approach for a multi-agent system in grid environments

Abstract In multi-robot systems, ensuring efficient, safe, and conflict-free navigation for each robot is crucial, especially in scenarios requiring the coordination of complex tasks and interactions with the environment. Currently, most related research focuses on simplified grid environments, making it difficult to apply these findings to real-world scenarios. This paper introduces actual physical constraints into traditional grid environment path planning and designs a set of motion rules suitable for grid environments and a collision avoidance algorithm. We also propose a novel multi-agent path planning approach that integrates deep reinforcement learning with collision avoidance algorithms. Through a series of experiments, we have demonstrated that this method effectively guides agents in navigating from their starting points to their targets within grid environments that possess physical constraints, and significantly reduces the probability of collisions when used in conjunction with our collision avoidance algorithm. These findings not only showcase the efficacy of our approach but also open new avenues for deploying multi-agent systems in more complex real-world scenarios.

HOLONIC MULTI-AGENT SYSTEMS ALGORITHM FOR AUGMENTED REALITY INTERACTIVE ASSISTANCE

In this paper is presented an algorithm based on the holonic multi-agent approach concerning augmented reality assistance to be applied in assembly or disassembly operations. The variation in production and fast start of production requires operators to learn assembly operations for large range of products. In order to avoid errors, operators might be trained and corrected using augmented reality application able to assist and provide important information to allow the operation according to technical required specification. The holonic multi-agent system allows solving complex calculus concerning the best choice. The proposed algorithm is easier to apply when the operator needs to be assisted in assembly operations while meeting technical specifications. By applying the proposed algorithm, the operators learn very fast the operations for assembly/disassembly.

Risk contagion in interbank lending networks: A multi-agent-based modeling and simulation perspective

The interbank lending network, while facilitating transactions, poses risk contagion in the banking system. To address challenges in existing research, this paper models the interbank lending network using a multi-agent approach and generates a dynamic network through a bidirectional matching mechanism(BiMM). Additionally, We divide banks into three categories based on realistic bank balance sheet data. Given the fact that banks of different categories exhibit different degrees of contagion, we propose the weighted DebtRank (W-DebtRank). Then, comprehensive simulation analysis is conducted on the risk contagion of heterogeneous banks in the dynamic network using W-DebtRank. The proposed method addresses four issues: the heterogeneity of banks, the dynamic nature of networks, the practicality of lending strategies, and the comprehensiveness of risk analysis. The model is validated using the SRISK metric and the Monte Carlo-based SYMBOL simulation model. Through multiple perspectives in simulation experiments, the risk contagion process among banks is deeply investigated in our paper.

Unveiling the impact of heterogeneous driving behaviors on traffic flow: A mesoscale multi-agent modeling approach

There are fewer simulation studies that comprehensively consider the impact of collision events due to heterogeneous driving behaviour on multi-lane traffic flow. This comprehensive study utilized multi-agent modeling to simulate the driver-vehicle-road system at the mesoscale level, integrating the characteristics of heterogeneous driving behavior. We developed a traffic flow simulation model for a three-lane urban arterial road segment and used rule-based algorithms to simulate vehicle following, lane changing, and collision behavior. A visualization simulation program was developed using NetLogo software to observe and analyze vehicle behavior. The findings revealed that collision events were the leading cause of traffic bottlenecks, impacting the continuity and coherence of traffic flow. Even after the collision event subsided and congestion eased, the average vehicle speed could not fully recover to its original level, reaching only about 70% of the maximum vehicle speed. Aggressive drivers exhibited distinct speed control strategies compared to conservative and ordinary drivers. This study demonstrates the effectiveness of multi-agent modeling in capturing the relationship between traffic bottlenecks and collision events, highlighting the influence of dynamic traffic events on driving behavior.

Transition from machine intelligence to knowledge intelligence: A multi-agent simulation approach to technology transfer

Abstract The traditional machine intelligence system lacks deep understanding and reasoning ability. This study took the automatic driving system in multi-agent as an example to bring higher-level intelligence and decision-making ability to automatic driving through knowledge intelligence. It obtained real-world geographic information data from OpenStreetMap, preprocessed the data, and built a virtual environment. The inception model was used to identify information in environmental images, and the knowledge information of traffic regulations, road signs, and traffic accidents was expressed to build a knowledge map. The knowledge related to automatic driving was integrated, and automatic driving training was carried out through the reward mechanism and the deep Q-network (DQN) model. About 13 kinds of traffic situations were set up in the virtual environment, and the traditional machine intelligence autonomous driving and knowledge fusion autonomous driving multi-agent were compared. The results show that the average number of accidents in 100,000 km of traditional machine intelligence autonomous driving and knowledge fusion autonomous driving multi-agents was 3 and 1.4, and the average number of violations in 100,000 km was 4.3 and 1.8, respectively. The average graphics processing unit (GPU) utilization rate of knowledge fusion autonomous driving in 13 virtual environments was 75.9%, and the average peak GPU utilization rate was 96.1%. Knowledge fusion of autonomous driving multi-agents can effectively improve the safety of autonomous driving and enable autonomous driving multi-agents to have a higher level of decision-making ability.

Enhancing Grid-Interactive Buildings Demand Response: Sequential Update-Based Multiagent Deep Reinforcement Learning Approach

Enhancing Grid-Interactive Buildings Demand Response: Sequential Update-Based Multiagent Deep Reinforcement Learning Approach

A Multi-Agent Approach for the Optimized Operation of Modular Electrolysis Plants

In response to the energy transition to renewable resources, green hydrogen production via electrolysis is gaining momentum. Modular electrolysis plants provide a flexible and scalable solution to meet rising hydrogen demand and adapt to renewable energy fluctuations. However, optimizing their operation poses challenges, especially when dealing with heterogeneous electrolyzer modules. In this work, a combination of decentralized Multi-Agent Systems and the Module Type Package concept is presented that enhances the cost-optimized operation of modular electrolysis plants. This approach synergizes the individual strengths of Multi-Agent Systems in handling complex operational dynamics with the efficiency of the Module Type Package for integration and control capabilities. By integrating these technologies, the approach addresses the heterogeneity of electrolyzer modules and increases the adaptability, scalability, and operational flexibility of electrolysis plants. The approach was validated through a case study, demonstrating its effectiveness in achieving cost-optimized load scheduling, dynamic response to demand–supply fluctuations, and resilience against electrolyzer module malfunctions. In summary, the presented approach offers a comprehensive solution for the effective coordination and optimization of modular electrolysis plants.

Multi-task scheduling in vehicular edge computing: a multi-agent reinforcement learning approach

Multi-task scheduling in vehicular edge computing: a multi-agent reinforcement learning approach

An innovative multi-agent approach for robust cyber–physical systems using vertical federated learning

Federated learning presents a compelling approach to training artificial intelligence systems in decentralized settings, prioritizing data safety over traditional centralized training methods. Understanding correlations among higher-level threats exhibiting abnormal behavior in the data stream becomes paramount to developing cyber–physical systems resilient to diverse attacks within a continuous data exchange framework. This work introduces a novel vertical federated multi-agent learning framework to address the challenges of modeling attacker and defender agents in stationary and non-stationary vertical federated learning environments. Our approach uniquely applies synchronous Deep Q-Network (DQN) based agents in stationary environments, facilitating convergence towards optimal strategies. Conversely, in non-stationary contexts, we employ synchronous Advantage Actor–Critic (A2C) based agents, adapting to the dynamic nature of multi-agent vertical federated reinforcement learning (VFRL) environments. This methodology enables us to simulate and analyze the adversarial interplay between attacker and defender agents, ensuring robust policy development. Our exhaustive analysis demonstrates the effectiveness of our approach, showcasing its capability to learn optimal policies in both static and dynamic setups, thus significantly advancing the field of cyber-security in federated learning contexts. To evaluate the effectiveness of our approach, we have done a comparative analysis with its baseline schemes. The findings of our study show significant enhancements compared to the standard methods, confirming the efficacy of our methodology. This progress dramatically enhances the area of cyber-security in the context of federated learning by facilitating the formulation of substantial policies. The proposed scheme attains 15.93%, 32.91%, 31.02%, and 47.26% higher results as compared to the A3C, DDQN, DQN, and Reinforce, respectively.

Long-term resilience in wastewater management: Optimizing treated wastewater allocation with a dynamic multi-agent approach

Water scarcity poses a significant challenge to sustainable development, necessitating innovative approaches to manage limited resources efficiently. Effective water resource management involves not just the conservation and distribution of freshwater supplies but also the strategic reuse of treated wastewater (TWW). This study proposes a novel approach for the optimal allocation of treated wastewater among three key sectors (user agents): agriculture, industry, and urban green space. Recognizing the intricate interplays among these sectors, System Dynamics (SD) and Agent-Based Modeling (ABM) were integrated in a Complex Adaptive System (CAS) to capture the interactions and feedback mechanisms inherent within treated wastewater allocation systems. The Non-dominated Sorting Genetic Algorithm II (NSGA-II) serves as the optimization tool, enabling the identification of optimal allocation strategies across various management scenarios over a 25-year simulation period. Our research navigates the complexities of long-term resource management, accounting for each sector's evolving its objectives and guidelines along the whole system objectives and strategies. The outcomes demonstrate how treated wastewater can be effectively distributed to support economic and social equity –as the system objectives-while supporting agricultural and industrial growth and enhancing efficiency and social well-being –reflecting individual agent objectives-within the CAS framework. The research explores four distinct management scenarios, each prioritizing different sectors to address water resource management challenges. Notably, all four scenarios align with the strategies required by the ruler (government), providing strategic guidance to water resource managers for decision-making. The simulation results reveal a scenario where all sectors' demands are met, with Scenario 4 emerging as the most effective. Scenario 4 aligned with the objectives and guidelines of each sector, demonstrating significant improvements in the CY (Agriculture agent index; increased from 0.2 to 0.68), IGI (Industry agent index; increased from 1 to 1.63), and GAI (Urban Green Space agent index; increased from 1 to 1.23) indices over the 25-year simulation period. By providing a strategic blueprint for policymakers and stakeholders, this study contributes significantly to the discourse on sustainable water resource management, presenting a replicable model for similar contexts globally, where judicious allocation of treated wastewater is paramount for achieving harmony between human activity and ecological preservation.