Multi-agent System Environment Research Articles

Distributed Formation Control of Multi-Agent Systems: A Novel Fast-Optimal Balanced Differential Game Approach

This paper proposes an efficient fast-optimal balanced differential game (DG) approach to address the formation control problem in dynamic environments for networked multi-agent systems (MASs). Compared to existing receding horizon distributed differential game (RH-DDG) approaches, the proposed approach employs a two-layer game structure to balance optimality and real-time performance, with a focus on formation control, collision avoidance and obstacle avoidance. In the offline layer, the problem is converted into a distributed differential game (DDG) where each agent computes strategies using distributed information from locally neighboring agents. The strategy of each agent self-enforces a unique global Nash equilibrium (G-NE) with a strongly connected communication topology, providing an optimal reference trajectory for the online game. In the online layer, a receding horizon differential game with an event-trigger mechanism (RH-DGET) is presented to track the G-NE trajectory. Ego players are triggered to update online Nash strategies only when the event-triggering condition is satisfied, ensuring the real-time safety certificate. Rigorous proofs demonstrate that the online Nash strategies converge to the offline G-NE until the trigger ends, and a certain dwell time condition is given to prevent the Zeno behavior. Simulation results validate the effectiveness of the proposed approach.

Fingerprint Networked Reinforcement Learning via Multiagent Modeling for Improving Decision Making in an Urban Food–Energy–Water Nexus

Food–energy–water (FEW) nexus analyses are critical to sustainable development. Nexus analyses form a unique multiagent decision-making arena that requires using a system engineering approach to simultaneously improve food and water security as well as energy efficiency. To tackle the complexity in decision making within a FEW nexus with respect to dynamic behaviors and interactive logics, we model the FEW nexus as a multiagent system (MAS) under a mixed competitive and cooperative environment from the perspective of a Markov game. Then, we propose a fingerprint networked reinforcement learning (FNRL) framework for the collective learning of a MAS by following the logic flows of human decision making. FNRL can alleviate the problems caused by stationary issues in a MAS environment by integrating a long short-term-memory-driven neural network model into the context of multiagent reinforcement learning (RL) to extract fingerprint information via historical data. Numerical simulations for an urban FEW nexus analysis in Florida (USA) demonstrate that applying the FNRL framework can drive agents in the MAS toward achieving optimality via RL in a dynamic environment.

A fuzzy scheduler for MAS applied to object tracking

Intelligent industry and Internet of Things (IoT) rely on sensors to monitor the surroundings and allow information exchange. More sensors deployed in an environment offer additional benefits, such as increased availability and more accurate observations, as they generate information from different perspectives. Nonetheless, dealing with large amounts of data and multiple input sources to deduct unified information often requires considerable processing, communication, and energy resources. To address such a problem, this work examines intelligent agents that cooperate to generate a unified output in a Multi-Agent System (MAS) environment. The proposed approach presents a fuzzy scheduler that evaluates the processing and accuracy level to determine the relevance of each agent by comparing its information with the unified output. The proposal orchestrates when an agent can have its processing suspended. Our evaluations demonstrate that this proposal improves service level metrics, such as the rate of video frames processed per second for an increasing number of agents. We consider three experimental setups using tracking agents. First, we evaluate the threshold and how long an agent should remain idle when its observations are irrelevant. The second experiment considers an increasing number of agents. Agent scheduling achieves a frame processing rate 12.5 times faster while suffering a 6% accuracy loss for 20 agents. The third experiment focuses on using different tracking algorithms simultaneously. In this case, compared to when no agent scheduling is applied, there is a 23.9 fold increase in the frame processing rate while losing some accuracy, limited to around 6%.

Improvement of Maneuverability Within a Multiagent Fuzzy Transportation System With the Use of Parallel Biobjective Real-Coded Genetic Algorithm

Over the past two decades, several simulation-based approaches have been developed to seek optimal solutions in complex multiagent systems (MASs). One example of these complex systems is the proposed multiagent fuzzy transportation system (FTS), in which agents, such as manned ground vehicles (MGVs) and unmanned ground vehicles (UGVs), use fuzzy logic for maneuvering while interacting with each other. The complexity of an FTS as an optimization problem is caused by two major factors, viz., inability to apply gradient-based optimization methods and uncertainty present in the FTS model. In these MASs, the objective functions are computed as an outcome of multiple agent-to-agent interactions. For the proposed FTS, two interconnected objectives are of prime importance while modeling the system: the number of potential traffic accidents, which should be minimized, and the output traffic that needs to be maximized. The resulting model is a simulation-based biobjective optimization problem. In an attempt to improve the maneuverability at various road network configurations (RNCs), in this work, a new parallel real-coded genetic algorithm based on fuzzy clustering (FCGA) is proposed that considers the specific requirements of the FTS model. Use of the FCGA aggregated with the FTS overcomes the difficulties associated with the optimization model and the limitations caused by simulation-based optimization (the large dimensionality of decision space, uncertainty present in the MAS environment, etc.). After showing superior performance in approximating the Pareto-optimal solutions for known test instances, the proposed FCGA is applied to the proposed multiagent FTS to approximate the optimal parameters. Because of the optimization of FTS parameters with the FCGA and the use of the proposed fuzzy clustering algorithm (FCA) for the traffic density estimate, the suggested approach substantially improves the maneuverability of UGVs and MGVs to change lanes while considering turning based on fuzzy rules and overtaking.

Dynamic Scheduling Framework of the Flexible Mixed-Model Assembly Line Based on the Internet of Manufacturing Things

In mass customization (MC), the assembly operations of modular products tend to be organized as the form of flexible mixed-model assembly line (FMMAL). The dynamic scheduling problem of FMMAL is quite complex with product sequencing, station allocation, material delivery, disruption events of station failure, product inserting and product reworking. To solve these problems, a comprehensive framework combining the architecture of Internet of Manufacturing Things (IoMT) with the dynamic scheduling algorithms is proposed. Firstly, the IoMT-based FMMAL are constructed via the multi-agent system (MAS) and ubiquitous environment. Secondly, a mathematical model of FMMAL is formulated with the decision variables, optimization objectives and constraint conditions. Thirdly, the IoMT-oriented algorithms for dynamic scheduling are proposed including the fuzzy analytic hierarchy process (FAHP) for normalization, weighted sum of properties-improved genetic algorithm (WSP-IGA) for prescheduling, priority weights search-simulated annealing (PWS-SA) for rescheduling. Lastly, a discrete event simulation of a numerical case is conducted to demonstrate the practicality and validity of proposed theories and algorithms. The results show that the proposed hyper-heuristics (WSP-IGA and PWS-SA) can respectively realize the prescheduling and rescheduling of FMMAL in four modes including the synthesized mode, time-efficient mode, just-in-time mode and energy-saving mode, which are superior to the four referenced meta-heuristics.

Spillover Algorithm: A decentralised coordination approach for multi-robot production planning in open shared factories

Open and shared manufacturing factories typically dispose of a limited number of robots that should be properly allocated to tasks in time and space for an effective and efficient system performance. In particular, we deal with the dynamic capacitated production planning problem with sequence independent setup costs where quantities of products to manufacture and location of robots need to be determined at consecutive periods within a given time horizon and products can be anticipated or backordered related to the demand period. We consider a decentralized multi-agent variant of this problem in an open factory setting with multiple owners of robots as well as different owners of the items to be produced, both considered self-interested and individually rational. Existing solution approaches to the classic constrained lot-sizing problem are centralized exact methods that require sharing of global knowledge of all the participants' private and sensitive information and are not applicable in the described multi-agent context. Therefore, we propose a computationally efficient decentralized approach based on the spillover effect that solves this NP-hard problem by distributing decisions in an intrinsically decentralized multi-agent system environment while protecting private and sensitive information. To the best of our knowledge, this is the first decentralized algorithm for the solution of the studied problem in intrinsically decentralized environments where production resources and/or products are owned by multiple stakeholders with possibly conflicting objectives. To show its efficiency, the performance of the Spillover Algorithm is benchmarked against state-of-the-art commercial solver CPLEX 12.8.

A Heuristic Lowest Unknown-Degree Target Search Strategy Under Non-Structured Environment for Multi-Agent Systems

Multi-target searching is a hotspot and foundation topic in multi-agent systems research. However, most of the research is based on simple environment or known environment, which greatly limits the application of target search. In the non-structured environment, the searching result can be greatly affected by the complex terrain constraints and detectability of targets especially when we have no prior knowledge about the environment. In the paper, a novel search strategy combining maximum visibility and particle swarm optimization is proposed for the target search problem in a completely unknown and non-structural environment. The strategy utilizes the concept of visibility to describe how well the agent detects the map, and guides the agent to perform online path planning to complete the search task. In addition, considering the limited communication distance and communication bandwidth, the strategy introduces a cooperative mechanism for each agent to improve the search efficiency. Finally, in the experimental part, the search strategy is compared with the commonly used search strategies. Compared with the methods combining advantages, the proposed strategy can still achieve similar results, which proves the feasibility and efficiency of the strategy.

MULTI-AGENT MANAGEMENT AND DECISION SUPPORT SYSTEMS

The questions of practical use of multi-agent technologies are considered. The description of the specific properties of multi-agent systems is presented, which determine the scope of their practical application, as well as examples of successful use of the multi-agent approach for solving important practical problems. The main attention is paid to the problem of agents interaction with an external environment. In this regard, the description of examples is given for the environment of several multi-agent systems. It also contains a description of the virtual reality model and virtual space, which reflect the full life cycle of multi-agent application systems. Taking into account the direction of the paper, the main attention is also paid the description of the environment to the description of the proposed solutions used to implement the mechanisms of interaction between agents and the environment.

Synchronizing production scheduling with resources allocation for precast components in a multi-agent system environment

The performance of precast construction is highly dependent on the effectiveness of production planning for the precast components (PCs). However, existing studies focused primarily on the separate mathematical optimization of production scheduling or resource allocations using heuristic algorithms without considering the interactions between each other and the flexible production environment. This paper proposes a multi-agent based precast production planning model to synchronize the production scheduling and resource allocation. In this model, the on-time delivery, minimum waiting and extension time are achieved using a two-hierarchy resource constraints based production scheduling optimization method. Further, the heuristic algorithm is presented to integrate the optimization techniques with the competing goals shown in a multi-agent system. Finally, a real case is conducted to verify the performance of the proposed model. The results demonstrate that 9.1% and 27.4% cost savings can be achieved by comparison to the actual scheduling method in practice and the traditional scheduling methods without considering the resource constraints, respectively. This proposed methodology will support flexible and optimized decision makings on the precast production planning.

DEVELOPMENT OF A MULTI-AGENT COLLISION RESOLUTION SYSTEM AT THE SUPPLY OF SPARE PARTS AND COMPONENTS TO THE PRODUCTION EQUIPMENT OF INDUSTRIAL ENTERPRISES

The approach to the creation of computer facilities for the automation of the technical maintenance of production equipment (TMPE) at industrial enterprises (IE) is outlined. Meaningful and formal statement of the problem of forming solutions for identifying and eliminating collisions that arise when delivering spare parts and components for TMPE are presented. The method of formation of coordinating decisions on maintenance with spare parts and accessories for carrying out TMPE at IE is described. The organization of intellectual support of formation of coordinating decisions by recognition of potential collision in the TMPE process is offered. This procedure involves checking the real existence of the collision and issuing a coordinating decision. In this case, the decision is formed in the event of a disagreement between the need for spare parts and components for the TMPE maintenance, with their availability in the PP warehouse. The ways of software implementation of this method in the environment of multi-agent system are considered. In particular, the description of the multi-agent system developed during the prototype research is given. The prototype is implemented using CORBA technology, in accordance with DSTU ISO/ EC 2382-15:2005. The calculation of the efficiency of the application of the developed computer tools in production is shown. To assess the quality of the system, a sliding control method based on leave-on-out cross-validation (LOOCV) is applied.

An Evolutionary Transfer Reinforcement Learning Framework for Multiagent Systems

In this paper, we present an evolutionary transfer reinforcement learning framework (eTL) for developing intelligent agents capable of adapting to the dynamic environment of multiagent systems (MASs). Specifically, we take inspiration from Darwin’s theory of natural selection and Universal Darwinism as the principal driving forces that govern the evolutionary knowledge transfer process. The essential backbone of our proposed eTL comprises several meme-inspired evolutionary mechanisms, namely meme representation , meme expression , meme assimilation , meme internal evolution , and meme external evolution . Our proposed approach constructs social selection mechanisms that are modeled after the principles of human learning to identify appropriate interacting partners. eTL also models the intrinsic parallelism of natural evolution and errors that are introduced due to the physiological limits of the agents’ ability to perceive differences, so as to generate “growth” and “variation” of knowledge that agents have of the world, thus exhibiting higher adaptivity capabilities on solving complex problems. To verify the efficacy of the proposed paradigm, comprehensive investigations of the proposed eTL against existing state-of-the-art TL methods in MAS, are conducted on the “minefield navigation tasks” platform and the “Unreal Tournament 2004” first person shooter computer game, in which homogeneous and heterogeneous learning machines are considered.

Directed Bee Colony Optimization Algorithm to Solve the Nurse Rostering Problem.

The Nurse Rostering Problem is an NP-hard combinatorial optimization, scheduling problem for assigning a set of nurses to shifts per day by considering both hard and soft constraints. A novel metaheuristic technique is required for solving Nurse Rostering Problem (NRP). This work proposes a metaheuristic technique called Directed Bee Colony Optimization Algorithm using the Modified Nelder-Mead Method for solving the NRP. To solve the NRP, the authors used a multiobjective mathematical programming model and proposed a methodology for the adaptation of a Multiobjective Directed Bee Colony Optimization (MODBCO). MODBCO is used successfully for solving the multiobjective problem of optimizing the scheduling problems. This MODBCO is an integration of deterministic local search, multiagent particle system environment, and honey bee decision-making process. The performance of the algorithm is assessed using the standard dataset INRC2010, and it reflects many real-world cases which vary in size and complexity. The experimental analysis uses statistical tools to show the uniqueness of the algorithm on assessment criteria.

Learning and relearning of target decision strategies in continuous coordinated cleaning tasks with shallow coordination1

We propose a method of autonomous learning of target decision strategies for coordination in the continuous cleaning domain. With ongoing advances in computer and sensor technologies, we can expect robot applications for covering large areas that often require coordinated/cooperative activities by multiple robots. We focus on the cleaning tasks by multiple robots or by agents which are programs to control the robots in this paper. We assumed situations where agents did not directly exchange deep and complicated internal information and reasoning results such as plans, strategies and long-term targets for their sophisticated coordinated activities, but rather exchanged superficial information such as the locations of other agents (using the equipment de- ployed) for their shallow coordination and individually learned appropriate strategies by observing how much dirt/dust had been vacuumed up in multi-agent system environments. We will first discuss the preliminary method of improving the coordinated activities by autonomously learning to select cleaning strategies to determine which targets to move to clear them. Although we could have improved the efficiency of cleaning, we observed a phenomenon where performance degraded if agents continued to learn strategies. This is because so many agents overly selected the same strategy (over-selection) by using autonomous learning. In addition, the preliminary method assumed information given about which regions in the environment easily became dirty. Thus, we propose a method that was extended by incorporating the preliminary method with (1) environmental learning to iden- tify which places were likely to be dirty and (2) autonomous relearning through self-monitoring the amount of vacuumed dirt to avoid strategies from being over-selected. We experimentally evaluated the proposed method by comparing its performance with those obtained by the regimes of agents with a single strategy and obtained with the preliminary method. The experimental results revealed that the proposed method enabled agents to select target decision strategies and, if necessary, to abandon the current strategies from their own perspectives, resulting in appropriate combinations of multiple strategies. We also found that environmental learning on dirt accumulation was effectively learned.

A systematic approach to evaluating domain-specific modeling language environments for multi-agent systems

Multi-agent systems (MASs) include multiple interacting agents within an environment to provide a solution for complex systems that cannot be easily solved with individual agents or monolithic systems. However, the development of MASs is not trivial due to the various agent properties such as autonomy, responsiveness, and proactiveness, and the need for realization of the many different agent interactions. To support the development of MASs various domain-specific modeling languages (DSMLs) have been introduced that provide a declarative approach for modeling and supporting the generation of agent-based systems. To be effective, the proposed DSMLs need to meet the various stakeholder concerns and the related quality criteria for the corresponding MASs. Unfortunately, very often the evaluation of the DSML is completely missing or has been carried out in idiosyncratic approach. If the DSMLs are not well defined, then implicitly this will have an impact on the quality of the MASs. In this paper, we present an evaluation framework and systematic approach for assessing existing or newly defined DSMLs for MASs. The evaluation is specific for MAS DSMLs and targets both the language and the corresponding tools. To illustrate the evaluation approach, we first present SEA_ML, which is a model-driven MAS DSML for supporting the modeling and generation of agent-based systems. The evaluation of SEA_ML is based on a multi-case study research approach and provides both qualitative evaluation and quantitative analysis. We report on the lessons learned considering the adoption of the evaluation approach as well as the SEA_ML for supporting the generation of agent-based systems.

Diffusion in Social Networks: A Multiagent Perspective

In recent years, significant attention has been paid to diffusion in social networks (SNs), which is, factually, the collective behavior of a set of autonomous social actors for interacting on something in SNs (such as opinions, viruses, or innovations). While this subject has been intensively reported, there have been relatively few systematic reviews concerning the typical diffusion elements and models that are relevant to this subject. Because multiagent computing has already been widely envisioned to be a powerful paradigm for modeling the collective interactions of autonomous multientity systems. In this survey, we review diffusion in SNs through a multiagent perspective. First, we review the following essential elements in diffusion: 1) diffusion actors (who will diffuse), which can be understood to be the interacting agents; 2) diffusion media (where to be diffused), which can be understood to be the interaction environments in multiagent systems (MASs); and 3) diffusion contents (what to be diffused), which can be understood to be the interaction objects in MASs. Next, based on varying situations of diffusion elements, we review the representative diffusion models (how to diffuse), which can be understood as the decision-making mechanisms and interaction protocols in MASs. For each class of diffusion elements and models, we summarize the existing studies and discuss the challenges for solving the complex diffusion problems by applying multiagent methodologies. Finally, we discuss the advantages and disadvantages of our multiagent perspective by comparing other typical perspectives (the empirical research perspective and the theoretical perspective in empirical research), and we conclude with suggestions for further research.

Behavioral Modeling of Multi Agent System

Modeling interactions between agents and the Multi-Agent System (MAS) behavior based on role based collaboration among the participating agents are the key factors to design of effective MAS dynamics. In this paper, a High level Multi Agent Petri Net called HMAP has been proposed which is capable of describing, analyzing and modeling dynamics of such MAS which are characterized as asynchronous, distributed, parallel and non-deterministic agent based systems. Proposed HMAP is also effective towards modeling roles, collaborations and interactions among the heterogeneous agents in MAS environment. Moreover the HMAP is useful in formal analysis of several behavioral properties of MAS like, Reachability, Home properties, Boundedness, Liveness and Fairness. The proposed mechanism has been illustrated using a suitable case study of Medical Emergency System. Moreover, to further validate the proposed concepts of HMAP, it has been simulated using Color Petri Net based tool called CPN Tool, with some restriction.

An Agent-Based Solution for the Berth Allocation Problem

This work presents the development of MABAP, a decision support system based on the agent technology that helps in solving the problem of berth allocation for ships within a port. The Berth Allocation Problem (BAP) regards the logistics involved in planning and controlling the berthing of vessels. A software architecture in terms of agents is presented; Berths and Ships representing the actors in the system, BerthRequest and BerthPlanner as representatives of ships and berths in the planning process, and finally the Dock and Central agents representing the dock or pier. The architecture modeling was done using PASSI methodology for the design of agent-oriented systems, and the implementation was done in JADE, a Javabased development environment for multiagent systems. To validate the resulting support system, tests were carried out in which the user can choose different portpolicy scenarios, ranging from maximizing vessels throughput to maximize berths use.

The role of the environment in agreement technologies

The notion of Multi-Agent System environment is currently considered as a mediating entity, functioning as enabler but possibly also as a manager and constrainer of agent actions, perceptions, and interactions. In this paper, we analyze how the environment could be a first class abstraction to support the building, the development and the management of Agreements in decentralized and open systems between autonomous agents. To this aim we analyze the synergies between the environment and the foundational dimensions of agreement technologies such as semantics, norms, organizations, argumentation & negotiation, trust.

A novel multi-objective directed bee colony optimization algorithm for multi-objective emission constrained economic power dispatch

In this paper, a multi-objective directed bee colony optimization algorithm (MODBC) is comprehensively developed and successfully applied for solving a multi-objective problem of optimizing the conflicting economic dispatch and emission cost with both equality and inequality constraints is showcased. Classical optimization techniques like direct search and gradient methods fail to give the global optimum solution. The proposed algorithm is an integration of the deterministic search, the multi-agent system (MAS) environment and the bee decision-making process. Thus making use of deterministic search, multi-agent environment and bee swarms, the MODBC realizes the purpose of optimization. The hybridization makes MODBC to obtain a unique and fast solution and hence generate a better pareto front for multi-objective problems. The above mentioned multi-objective evolutionary algorithms have been applied to the standard IEEE 30 bus six generator test system. Results of the proposed algorithm have been compared with traditional methods like linear programming (LP) and multi-objective stochastic search technique (MOSST). The performance of the introduced algorithm is also compared with other evolutionary algorithms like Non-dominated Sorting Genetic Algorithm (NSGA), Niched Pareto Genetic Algorithm (NPGA) and Strength Pareto Evolutionary Algorithm (SPEA) and Particle Swarm Optimization (PSO). The results show the robustness and accuracy of the proposed algorithm over the traditional methods and its other multi-objective evolutionary algorithm (MOEA) counterparts.

A user modeling approach for reasoning about interaction sensitive to bother and its application to hospital decision scenarios

In this paper, we present a framework for interacting with users that is sensitive to the cost of bother and then focus on its application to decision making in hospital emergency room scenarios. We begin with a model designed for reasoning about interaction in a single-agent single-user setting and then expand to the environment of multiagent systems. In this setting, agents consider both whether to ask other agents to perform decision making and at the same time whether to ask questions of these agents. With this fundamental research as a backdrop, we project the framework into the application of reasoning about which medical experts to interact with, sensitive to possible bother, during hospital decision scenarios, in order to deliver the best care for the patients that arrive. Due to the real-time nature of the application and the knowledge-intensive nature of the decisions, we propose new parameters to include in the reasoning about interaction and sketch their usefulness through a series of examples. We then include a set of experimental results confirming the value of our proposed approach for reasoning about interaction in hospital settings, through simulations of patient care in those environments. We conclude by pointing to future research to continue to extend the model for reasoning about interaction in multiagent environments for the setting of time-critical care in hospital settings.