Multiple Intelligent Agents Research Articles

Multiagent trajectory prediction with global‐local scene‐enhanced social interaction graph network

AbstractTrajectory prediction is essential for intelligent autonomous systems like autonomous driving, behavior analysis, and service robotics. Deep learning has emerged as the predominant technique due to its superior modeling capability for trajectory data. However, deep learning‐based models face challenges in effectively utilizing scene information and accurately modeling agent interactions, largely due to the complexity and uncertainty of real‐world scenarios. To mitigate these challenges, this study presents a novel multiagent trajectory prediction model, termed the global‐local scene‐enhanced social interaction graph network (GLSESIGN), which incorporates two pivotal strategies: global‐local scene information utilization and a social adaptive attention graph network. The model hierarchically learns scene information relevant to multiple intelligent agents, thereby enhancing the understanding of complex scenes. Additionally, it adaptively captures social interactions, improving adaptability to diverse interaction patterns through sparse graph structures. This model not only improves the understanding of complex scenes but also accurately predicts future trajectories of multiple intelligent agents by flexibly modeling intricate interactions. Experimental validation on public datasets substantiates the efficacy of the proposed model. This research offers a novel model to address the complexity and uncertainty in multiagent trajectory prediction, providing more accurate predictive support in practical application scenarios.

Analysis of Multi-Intelligent Distributed Japanese Language Block Recognition Based on Knowledge Recognition Corpus

The Knowledge Recognition Corpus (KRC) serves as a comprehensive repository designed to capture and organize diverse forms of knowledge for computational analysis and understanding. This corpus encompasses a wide array of data sources, including text documents, images, videos, and structured datasets, covering various domains and topics. The concept of Multi-Intelligent Distributed Knowledge Recognition Corpus (MI-DKRC) represents an innovative approach to harnessing the collective intelligence of distributed systems for knowledge recognition tasks. This paper presents an analysis of Multi-Intelligent Distributed Japanese Language Block Recognition (MI-DJLBR) based on the Knowledge Recognition Corpus (KRC), employing multi-factor bi-gram Sentimental Classification (MFbi-SC). The MI-DJLBR system is designed to recognize and classify Japanese language blocks within a distributed framework, leveraging the collective intelligence of multiple intelligent agents. Through the utilization of the KRC, which encompasses a diverse array of Japanese text samples, images, and multimedia content, MI-DJLBR aims to enhance the efficiency and accuracy of Japanese language block recognition tasks. The incorporation of MFbi-SC further refines this process by considering multiple factors, including syntax, semantics, context, and sentiment, to classify Japanese language blocks with greater precision. Simulation demonstrated that a dataset of 1,000 Japanese language blocks, MI-DJLBR achieves an average recognition accuracy of 94.5%, demonstrating its effectiveness in accurately identifying and classifying Japanese text segments. Furthermore, the incorporation of MFbi-SC enhances the system's classification accuracy by an average of 7.2%, indicating the significance of multi-factor sentiment analysis in refining classification outcomes. In terms of computational efficiency, MI-DJLBR exhibits impressive processing times, with an average recognition speed of 200 blocks per second. This highlights the scalability and responsiveness of the distributed framework, enabling efficient processing of large-scale Japanese language datasets in real-time scenarios.

An Extensive-Form Game Paradigm for Visual Field Testing via Deep Reinforcement Learning.

Glaucoma is the leading cause of irreversible but preventable blindness worldwide, and visual field testing is an important tool for its diagnosis and monitoring. Testing using standard visual field thresholding procedures is time-consuming, and prolonged test duration leads to patient fatigue and decreased test reliability. Different visual field testing algorithms have been developed to shorten testing time while maintaining accuracy. However, the performance of these algorithms depends heavily on prior knowledge and manually crafted rules that determine the intensity of each light stimulus as well as the termination criteria, which is suboptimal. We leverage deep reinforcement learning to find improved decision strategies for visual field testing. In our proposed algorithms, multiple intelligent agents are employed to interact with the patient in an extensive-form game fashion, with each agent controlling the test on one of the testing locations in the patient's visual field. Through training, each agent learns an optimized policy that determines the intensities of light stimuli and the termination criteria, which minimizes the error in sensitivity estimation and test duration at the same time. In simulation experiments, we compare the performance of our algorithms against baseline visual field testing algorithms and show that our algorithms achieve a better trade-off between estimation accuracy and test duration. By retaining testing accuracy with reduced test duration, our algorithms improve test reliability, clinic efficiency, and patient satisfaction, and translationally affect clinical outcomes.

Collaborative operation optimization of distribution system and virtual power plants using multi‐agent deep reinforcement learning with parameter‐sharing mechanism

AbstractWith the increasing integration of distributed energy resources (DERs) into distribution systems, the optimization of system operation has become complex, facing challenges such as inadequate consideration of market participants’ benefits, poor computational efficiency, and data privacy concerns. This paper introduces the concept of a virtual power plant (VPP) as a solution for energy integration and management. To strike a balance between operational safety and the interests of market participants, a dual‐layer model is proposed. This model considers the benefits of both Distribution System Operators (DSO) and VPP, while also enhancing the consideration of distribution network constraints. The DSO considers AC optimal power flow and utilizes penalty functions to ensure network security in case of violations. To enhance computational efficiency and privacy, the paper presents the parameter‐sharing twin delayed deep deterministic policy gradient approach. This approach allows multiple intelligent agents to share a neural network model, effectively reducing the computational load. During the training process, only essential data is exchanged among the agents, ensuring the privacy of sensitive information. The effectiveness of the proposed model and the algorithm is validated through a case study on an IEEE 33‐node system.

Research on task offloading optimization strategies for vehicular networks based on game theory and deep reinforcement learning

With the continuous development of the 6G mobile network, computing-intensive and delay-sensitive onboard applications generate task data traffic more frequently. Particularly, when multiple intelligent agents are involved in tasks, limited computational resources cannot meet the new Quality of Service (QoS) requirements. To provide a satisfactory task offloading strategy, combining Multi-Access Edge Computing (MEC) with artificial intelligence has become a potential solution. In this context, we have proposed a task offloading decision mechanism (TODM) based on cooperative game and deep reinforcement learning (DRL). A joint optimization problem is presented to minimize both the overall task processing delay (OTPD) and overall task energy consumption (OTEC). The approach considers task vehicles (TaVs) and service vehicles (SeVs) as participants in a cooperative game, jointly devising offloading strategies to achieve resource optimization. Additionally, a proximate policy optimization (PPO) algorithm is designed to ensure robustness. Simulation experiments confirm the convergence of the proposed algorithm. Compared with benchmark algorithms, the presented scheme effectively reduces delay and energy consumption while ensuring task completion.

Coordinated control of multiple converters in model-free AC/DC distribution networks based on reinforcement learning

Taking into account the challenges of obtaining accurate physical parameters and uncertainties arising from the integration of a large number of sources and loads, this paper proposes a real-time voltage control method for AC/DC distribution networks. The method utilizes model-free generation and coordinated control of multiple converters, and employs a combination of agent modeling and multi-agent soft actor critic (MASAC) techniques for modeling and solving the problem. Firstly, a complex nonlinear mapping relationship between bus power and voltage is established by training an power-voltage model, to address the issue of obtaining physical parameters in AC/DC distribution networks. Next, a Markov decision process is established for the voltage control problem, with multiple intelligent agents distributed to control the active and reactive power at each converter, in response to the uncertainties of photovoltaic (PV) and load variations. Using the MASAC method, a centralized training strategy and decentralized execution policy are implemented to achieve distributed control of the converters, with each converter making optimal decisions based on its local observation state. Finally, the proposed method is verified by numerical simulations, demonstrating its sound effectiveness and generalization ability.

Multi-agent based scheduling method for tandem automated guided vehicle systems

We consider the scheduling method that allows to minimize the traveling time of a AGV (and thus energy consumption) at each of zones, while to minimize the waiting time of the automated guided vehicles (AGVs) and/or containers for pickup and delivery (PD) at the transfer points (TPs) installed between adjacent zones in tandem systems. Scheduling for these tandem AGV systems is often not possible from a single point by a single intelligent agent. Intead, scheduling has to be performed using multiple intelligent agents. We consider the cooperative multi-agent scheduling (CMAS) scheme in which each agent employs a multi-offspring genetic algorithm. Coordination between the agents is used to improve their schedulings. This coordination can be in the form of serial scheme. We propose a novel CMAS method based on Lagrange theory and compare this to a non-cooperative scheduling approach between agents. The proposed method is verified at AGV system for a book printing process in Pyongyang Textbook Print Shop as the real-world experiment and it is shown that the CMAS method for tandem AGV system is effective.

Multi-Agent Advisor Q-Learning

In the last decade, there have been significant advances in multi-agent reinforcement learning (MARL) but there are still numerous challenges, such as high sample complexity and slow convergence to stable policies, that need to be overcome before wide-spread deployment is possible. However, many real-world environments already, in practice, deploy sub-optimal or heuristic approaches for generating policies. An interesting question that arises is how to best use such approaches as advisors to help improve reinforcement learning in multi-agent domains. In this paper, we provide a principled framework for incorporating action recommendations from online suboptimal advisors in multi-agent settings. We describe the problem of ADvising Multiple Intelligent Reinforcement Agents (ADMIRAL) in nonrestrictive general-sum stochastic game environments and present two novel Q-learning based algorithms: ADMIRAL - Decision Making (ADMIRAL-DM) and ADMIRAL - Advisor Evaluation (ADMIRAL-AE), which allow us to improve learning by appropriately incorporating advice from an advisor (ADMIRAL-DM), and evaluate the effectiveness of an advisor (ADMIRAL-AE). We analyze the algorithms theoretically and provide fixed point guarantees regarding their learning in general-sum stochastic games. Furthermore, extensive experiments illustrate that these algorithms: can be used in a variety of environments, have performances that compare favourably to other related baselines, can scale to large state-action spaces, and are robust to poor advice from advisors.

Review of the progress of communication-based multi-agent reinforcement learning

Reinforcement learning (RL) technology has been successfully applied to various continuous decision environments in decades of development. Nowadays, RL is attracting more attention, even being touted as one of the closest approaches to general artificial intelligence. However, real-world problems often involve multiple intelligent agents interacting with each other. Thus, we focus on multi-agent reinforcement learning (MARL) to deal with such multi-agent systems in practice. In the past decade, the combination of multi-agent system and RL has become increasingly close, gradually forming and enriching the research field of MARL. Reviewing the studies on MARL, we found that researchers mainly solve MARL problems from three perspectives: learning framework, joint action learning, and communication-based MARL. In this paper, we focus from the studies on the communication perspective. We first state the reasons for choosing communication-based MARL and then list the president studies falling into the MARL category but different in nature. We hope that this article can provide a reference for developing MARL methods that can solve practical problems for the national welfare.

Robust flocking of multiple intelligent agents with multiple disturbances

The cooperation control of multiple intelligent agents (MIAs), which can solve complex engineering problems in practice, has received increasing attention. However, there are multiple disturbances in wireless sensor networks, which has a great effect on the collaboration of MIAs. In this paper, the problem of the flocking motion of second-order MIAs is addressed with collision avoidance and multiple disturbances. To estimate the matched/mismatched disturbances, the disturbance observers are designed. It is noted that the assumption that the differentials of disturbances converge to zeros in the literature is removed. A novel compound strategy is designed by using the robust auxiliary function and the potential function. The asymptotic properties of the flocking system are studied based on the Input-to-State Stability Theorem and the robust stability. Numerical simulation results verify the validity of the proposed protocol.

Neurocomputations of strategic behavior: From iterated to novel interactions.

Strategic interactions, where an individual's payoff depends on the decisions of multiple intelligent agents, are ubiquitous among social animals. They span a variety of important social behaviors such as competition, cooperation, coordination, and communication, and often involve complex, intertwining cognitive operations ranging from basic reward processing to higher‐order mentalization. Here, we review the progress and challenges in probing the neural and cognitive mechanisms of strategic behavior of interacting individuals, drawing an analogy to recent developments in studies of reward‐seeking behavior, in particular, how research focuses in the field of strategic behavior have been expanded from adaptive behavior based on trial‐and‐error to flexible decisions based on limited prior experience. We highlight two important research questions in the field of strategic behavior: (i) How does the brain exploit past experience for learning to behave strategically? and (ii) How does the brain decide what to do in novel strategic situations in the absence of direct experience? For the former, we discuss the utility of learning models that have effectively connected various types of neural data with strategic learning behavior and helped elucidate the interplay among multiple learning processes. For the latter, we review the recent evidence and propose a neural generative mechanism by which the brain makes novel strategic choices through simulating others' goal‐directed actions according to rational or bounded‐rational principles obtained through indirect social knowledge.This article is categorized under:Economics > Interactive Decision‐MakingPsychology > Reasoning and Decision MakingNeuroscience > Cognition

RAMARL: Robustness Analysis with Multi-Agent Reinforcement Learning - Robust Reasoning in Autonomous Cyber-Physical Systems

A key driver to ofering smart services is an infrastructure of Cyber-Physical systems (CPS)s. By definition, CPSs are intertwined physical and computational components that integrate physical behaviour with computation. The reason is to autonomously execute a task or a set of tasks providing a service or a list of end-users services. In real-life applications, CPSs operate in dynamically changing surroundings characterized by unexpected or unpredictable situations. Such operations involve complex interactions between multiple intelligent agents in a highly non-stationary environment. For safety reasons, a CPS should withstand a certain amount of disruption and exert the operations in a stable and robust manner when performing complex tasks. Recent advances in reinforcement learning have proven suitable for enabling multi-agents to robustly adapt to their environment, yet they often depend on a massive amount of training data and experiences. In these cases, robustness analysis outlines necessary components and specifications in a framework, ensuring reliable and stable behaviour while considering the dynamicity of the environment.This paper presents a combination of multi-agent reinforcement learning with robustness analysis shaping a cyber-physical system infrastructure that reasons robustly in a dynamically changing environment. The combination strengthens the reinforcement learning, increasing the reliability and flexibility of the system by applying robustness analysis. Robustness analysis identifies vulnerability issues when the system interacts within a dynamically changing environment. Based on this identification, when incorporated into the system, robustness analysis suggests robust solutions and actions rather than optimal ones provided by reinforcement learning alone. Results from the combination show that this infrastructure can enable reliable operations with the flexibility to adapt to the changing environment dynamics.

Dynamics of Multiagent Reinforcement Learning Compared to Synchronisation Dynamics of Kuramoto Oscillators

Multiagent reinforcement learning is a widespread approach to optimisation problem on systems with possibility of feedback links. Multiple intelligent agents interact with each other to enhance search of the optimal behavioural patterns. Structure of interactions was shown to affect system dynamics, in particular, on learning dynamics of intelligent agents, resulting in emergent behaviour, which make topicality of optimal topology exploration for different task formulations. To effectively solve certain problems, the ability of agents to cooperate in the frame of environments is extremely important. The main idea of coordinated reinforcement learning is that agents, when performing actions, take into account the actions of their neighbors, and this is taken into account when updating their parameters. However, due to the fact that multi-agent environments are very dynamic, a large amount of computation is required to effectively train MARL system. We believe the development of graph formation method for multiagent reinforcement learning systems is able to decrease resource consumption. In order to contribute to this problem, we explore effects of complete, star, path, and ring graphs on learning dynamics in a predator-prey environment, which is compared to synchronisation dynamics on graphs. In contrast to synchronisation case, showing dependence on average shortest paths, predator-prey system has two types of agents, which results in different patterns of influence.

Simultaneous Past and Current Social Interaction-aware Trajectory Prediction for Multiple Intelligent Agents in Dynamic Scenes

Trajectory prediction of multiple agents in a crowded scene is an essential component in many applications, including intelligent monitoring, autonomous robotics, and self-driving cars. Accurate agent trajectory prediction remains a significant challenge because of the complex dynamic interactions among the agents and between them and the surrounding scene. To address the challenge, we propose a decoupled attention-based spatial-temporal modeling strategy in the proposed trajectory prediction method. The past and current interactions among agents are dynamically and adaptively summarized by two separate attention-based networks and have proven powerful in improving the prediction accuracy. Moreover, it is optional in the proposed method to make use of the road map and the plan of the ego-agent for scene-compliant and accurate predictions. The road map feature is efficiently extracted by a convolutional neural network, and the features of the ego-agent’s plan is extracted by a gated recurrent network with an attention module based on the temporal characteristic. Experiments on benchmark trajectory prediction datasets demonstrate that the proposed method is effective when the ego-agent plan and the the surrounding scene information are provided and achieves state-of-the-art performance with only the observed trajectories.

Information-defined networks: A communication network approach for network studies

The research of complex networks facilitates the progress of various disciplines, including biology, chemistry, social science, computer, and communication engineering. Recently, it is popular to utilize complex networks to study the communication networks, such as designing efficient routing strategies and robust communication networks. However, exploiting the advantages of communication networks to investigate networks in various disciplines beyond telecommunications is still in infancy. Because of this situation, this paper proposes an information-defined network (IDN) framework by which a complex network can be abstracted as a communication network associated with multiple intelligent agents. Specifically, each component and dynamic process in this framework can be defined by information. We show that the IDN framework promotes the research of unsolved problems in the current complex network field, especially for detecting new interaction types in real-world networks.

Reducing Efficiency Degradation Due to Scheduled Agent Suspensions by Task Handover in Multi-Agent Cooperative Patrol Problems

This paper proposes a method to mitigate the significant performance degradation due to planned suspensions in the multi-agent cooperative patrol problem. In recent years, there has been an increased demand to utilize multiple intelligent agents that control robots. Furthermore, cooperation between multiple agents is required for performing tasks that are complex and/or cover large spaces. However, since robots are machines, they must be periodically inspected or replaced with new ones to prevent unintended breakdowns for continuous operation and to prolong the lifetime of agents as much as possible. However, such suspension of agents for inspection can cause a sudden deterioration in performance, which is not ignorable in some applications. Meanwhile, such suspensions are usually planned; thus, we can know in advance which agents will stop, and when, to anticipate a preparation period before the actual suspension time. Thus, we introduce a negotiation method in which the agents that are scheduled to be suspended hand over some responsible and important tasks to other agents to reduce the impact of a sudden performance degradation. The experimental results show that the proposed method considerably reduces the performance degradation, especially for security patrol applications.

A Kind of Reinforcement Learning to Improve Genetic Algorithm for Multiagent Task Scheduling

It is difficult to coordinate the various processes in the process industry. We built a multiagent distributed hierarchical intelligent control model for manufacturing systems integrating multiple production units based on multiagent system technology. The model organically combines multiple intelligent agent modules and physical entities to form an intelligent control system with certain functions. The model consists of system management agent, workshop control agent, and equipment agent. For the task assignment problem with this model, we combine reinforcement learning to improve the genetic algorithm for multiagent task scheduling and use the standard task scheduling dataset in OR-Library for simulation experiment analysis. Experimental results show that the algorithm is superior.

Relationship Identification Between Conversational Agents Using Emotion Analysis

Human relationships are influenced by the underlying emotions in their interactions. With the increasing use of social networks, relationships from textual data can also be inferred from online interactions. Such interactions result in massive amount of textual data which is available in the form of text messages, emails, and social media posts. Identification and analysis of human relationships are useful for numerous applications ranging from cybersecurity to public health. In this paper, we present a method called RIEA (Relationship Identification using Emotion Analysis), for identifying relationships between multiple intelligent agents by analyzing the conversation between them. The objective of our work is to combine concepts of cognitive psychology and natural language processing (NLP) to extract emotions and map them onto a set of relationships and analyze how relationships transform over time. We employ psychological models to label a large corpus of conversations and apply machine learning techniques to determine emotion-to-relationship mapping. We use four distinct association classes and four attachment styles using best-worst scaling method for classification. Combining the attachment and association styles given in research literature gives us the relationship combinations for our analysis. Additionally, this work studies the most common changes of behaviors and emotions and the corresponding transformations in human relationships. Our results show that RIEA can correctly detect interpersonal relationships with an accuracy of 85%. The evaluation shows that RIEA can accurately identify interpersonal relationships from conversations and can be extended for identifying more complex relationships. This study also highlights the effect of changes in emotional behavior in the development of relationships over time.

Dynamic Bayesian Collective Awareness Models for a Network of Ego-Things

A novel approach is proposed for multimodal collective awareness (CA) of multiple networked intelligent agents. Each agent is here considered as an Internet-of-Things (IoT) node equipped with machine learning capabilities; CA aims to provide the network with updated causal knowledge of the state of execution of actions of each node performing a joint task, with particular attention to anomalies that can arise. Data-driven dynamic Bayesian models learned from multisensory data recorded during the normal realization of a joint task (agent network experience) are used for distributed state estimation of agents and detection of abnormalities. A set of switching dynamic Bayesian network (DBN) models collectively learned in a training phase, each related to particular sensorial modality, is used to allow each agent in the network to perform synchronous estimation of possible abnormalities occurring when a new task of the same type is jointly performed. Collective DBN (CDBN) learning is performed by unsupervised clustering of generalized errors (GEs) obtained from a starting generalized model. A growing neural gas (GNG) algorithm is used as a basis to learn the discrete switching variables at the semantic level. Conditional probabilities linking nodes in the CDBN models are estimated using obtained clusters. CDBN models are associated with a Bayesian inference method, namely, distributed Markov jump particle filter (D-MJPF), employed for joint state estimation and abnormality detection. The effects of networking protocols and of communications in the estimation of state and abnormalities are analyzed. Performance is evaluated by using a small network of two autonomous vehicles performing joint navigation tasks in a controlled environment. In the proposed method, first the sharing of observations is considered in ideal condition, and then the effects of a wireless communication channel have been analyzed for the collective abnormality estimation of the agents. Rician wireless channel and the usage of two protocols (i.e., IEEE 802.11p and IEEE 802.15.4) along with different channel conditions are considered as well.

Survey on recent smart gateways for smart home: Systems, technologies, and challenges

AbstractA smart gateway, which plays a crucial role in a smart home, generally bridges the inside and outside home networks, as well as collaboratively manages the intelligent Internet of things (IoT) devices equipped in the smart home, due to it can convert different communication protocols, gather generated data from surrounding devices, and even conduct some local data processing tasks. In this survey, we first review the whole smart home research area with a focus on a framework based on edge computing and multiple intelligent agents. Then, relying on the time‐line and different metrics including quality of experience (QoE), artificial intelligence support, software‐defined network, and so on, we divide the evolution of smart gateways into three generations. Considering recent smart gateways mainly belong to the second generation, we further investigate and classify them into two categories from another perspective of user‐awareness and data‐awareness. Afterward, we discussed the key enabling technologies and components of smart gateways, such as operating systems, wireless communication protocols, and security. At last, we explore the challenges and trends of smart gateways and point out that maximizing QoE as well as data‐awareness is the key feature of future gateways. This survey has practical significance for building a smart home system, broadly speaking, even for constructing the IoT and smart space.