Multi-agent Paradigm Research Articles

The convergence of computational thinking, computational intelligence, and multi-agency

The design of real-world industrial systems is subject to a natural tendency towards modularization in order to manage complexity. In addition, this article considers that patterns of self-similarity in many problem domains have made many such solutions naturally representable as holarchies. Likewise, the increasing need for autonomous local decision making as well as the demand to produce solutions at scale has increased the relevance of the multi-agent paradigm to the creation of modern software systems. A variety of software development patterns are explored for their compatibility with holonic multi-agency. The current skill sets required by software development workers and concomitant training activities focus on instilling computational thinking abilities, a set of related cognitive competencies useful in the development of such systems. Intelligent systems play an increasingly important role in modern development and often benefit from computational intelligence techniques for the purpose of parameter tuning. This position paper explores the intersections between holonic multi-agency, modern information systems development, the computational intelligence which train them and the computational thinking skills those developers should be trained in.

AutoDRIVE: A Comprehensive, Flexible and Integrated Digital Twin Ecosystem for Autonomous Driving Research & Education

Prototyping and validating hardware–software components, sub-systems and systems within the intelligent transportation system-of-systems framework requires a modular yet flexible and open-access ecosystem. This work presents our attempt to develop such a comprehensive research and education ecosystem, called AutoDRIVE, for synergistically prototyping, simulating and deploying cyber-physical solutions pertaining to autonomous driving as well as smart city management. AutoDRIVE features both software as well as hardware-in-the-loop testing interfaces with openly accessible scaled vehicle and infrastructure components. The ecosystem is compatible with a variety of development frameworks, and supports both single- and multi-agent paradigms through local as well as distributed computing. Most critically, AutoDRIVE is intended to be modularly expandable to explore emergent technologies, and this work highlights various complementary features and capabilities of the proposed ecosystem by demonstrating four such deployment use-cases: (i) autonomous parking using probabilistic robotics approach for mapping, localization, path-planning and control; (ii) behavioral cloning using computer vision and deep imitation learning; (iii) intersection traversal using vehicle-to-vehicle communication and deep reinforcement learning; and (iv) smart city management using vehicle-to-infrastructure communication and internet-of-things.

A New Cloud-Based Adaptive Mobile Learning System Using Multi-Agent Paradigm

Today, Mobile devices are receiving a wide recognition as a cutting-edge technology to assist teaching and learning strategies that harness individual learners’ context. As mobile technology is being resorted to in education systems, tools that help accessing information have changed. In similar fashion, concepts such as Mobile Learning have emerged. Mobile learning is widespread. Furthermore, students nowadays are able to learn regardless of time and place. This is accomplished by mobile technologies and wireless internet connections. Nevertheless, there are still critical challenges regarding this new learning paradigm. In fact, researchers are meeting a difficulty related to providing accessibility for all students. This paper introduces a New Cloud-based Adaptive Mobile Learning System using a Multi-agent paradigm. The system provides adaptive learning content and personalised to the learner’s style and preferences, to increase learner satisfaction and facilitate the learning process. The system was implemented and will be tested with students in a real educational environment.

CPN4M: Testing Multi-Agent Systems under Organizational Model Moise+ Using Colored Petri Nets

Multi-agent systems (MASs) are distributed and complex software that demand specific software engineering features. Testing is a critical phase when validating software, and it is also difficult to conceive and execute. Designing systems under a multi-agent paradigm is even more difficult because of properties such as autonomy, reactivity, pro-activity, and social skills. Any multi-agent system has at least three main dimensions: the individual and social levels and communication interfaces. Considering an approach for testing a dimension specifically, we deal with the social level as an organizational model in this paper. It imposes some restrictions on agents’ behavior through a set of behavioral constraints. In this sense, an error in the organization can occur when the allocated resources are not enough for executing plans and reaching goals. This work aims to present a whole framework for analyzing and testing MAS social level under organizational model Moise+. This framework uses a Moise+ specifications set as information artifact mapped in a colored Petri net (CPN) model, named CPN4M, as a test case generation mechanism. CPN4M uses two different test adequacy criteria: all-paths and state-transition path. In this paper, we have formalized the transition from Moise+ to CPN, the procedures for test case generation, and executed some tests in a case study. The results indicate that this methodology can increase the correction degree for a social level in a multi-agent system specified by a Moise+ model, indicating the system context that can lead the MAS for failures.

Agent-based Modelling of Manufacturing Systems in the Context of “Industry 4.0”

The requirements of the modern market are manifested mainly in the need to provide a diversified assortment and personalized products. The ability to diversify the product range and adjust the production volume to the market requirements is a desirable feature of modern production systems. The Fourth Industrial Revolution has introduced many changes in the process planning domain that increased the possibility of better allocation of resources by applying artificial intelligence methods in order to achieve the effect of self-organization of the production process. The Smart Factory concept, based on cyber-physical systems, has allowed the customers to have greater influence on the process of manufacturing the ordered goods and even to change the order details during the process. The implementation of Industry 4.0 assumptions, and in particular the emergence of autonomous units, requires a new approach to modelling of manufacturing systems. This paper considers the problem of applying the agent-based modelling in modelling of such systems, focusing especially on the problem of adjusting the current formalism of agents or holons to the principles of Industry 4.0. An important part is the analysis of the agent definition in the context of manufacturing systems, in accordance with the available literature. The next step involves the discussion on the possibility of representing the Industry 4.0-compliant systems in terms of multi-agent paradigm.

Decentralized Multi-Agent Control of a Manipulator in Continuous Task Learning

Many real-world tasks require multiple agents to work together. When talking about multiple agents in robotics, it is usually referenced to multiple manipulators in collaboration to solve a given task, where each one is controlled by a single agent. However, due to the increasing development of modular and re-configurable robots, it is also important to investigate the possibility of implementing multi-agent controllers that learn how to manage the manipulator’s degrees of freedom (DoF) in separated clusters for the execution of a given application (e.g., being able to face faults or, partially, new kinematics configurations). Within this context, this paper focuses on the decentralization of the robot control action learning and (re)execution considering a generic multi-DoF manipulator. Indeed, the proposed framework employs a multi-agent paradigm and investigates how such a framework impacts the control action learning process. Multiple variations of the multi-agent framework have been proposed and tested in this research, comparing the achieved performance w.r.t. a centralized (i.e., single-agent) control action learning framework, previously proposed by some of the authors. As a case study, a manipulation task (i.e., grasping and lifting) of an unknown object (to the robot controller) has been considered for validation, employing a Franka EMIKA panda robot. The MuJoCo environment has been employed to implement and test the proposed multi-agent framework. The achieved results show that the proposed decentralized approach is capable of accelerating the learning process at the beginning with respect to the single-agent framework while also reducing the computational effort. In fact, when decentralizing the controller, it is shown that the number of variables involved in the action space can be efficiently separated into several groups and several agents. This simplifies the original complex problem into multiple ones, efficiently improving the task learning process.

Activity-Aware Vital SignMonitoring Based on a Multi-Agent Architecture.

Vital sign monitoring outside the clinical environment based on wearable sensors ensures better support in assessing a patient’s health condition, and in case of health deterioration, automatic alerts can be sent to the care providers. In everyday life, the users can perform different physical activities, and considering that vital sign measurements depend on the intensity of the activity, we proposed an architecture based on the multi-agent paradigm to handle this issue dynamically. Different types of agents were proposed that processed different sensor signals and recognized simple activities of daily living. The system was validated using a real-life dataset where subjects wore accelerometer sensors on the chest, wrist, and ankle. The system relied on ontology-based models to address the data heterogeneity and combined different wearable sensor sources in order to achieve better performance. The results showed an accuracy of 95.25% on intersubject activity classification. Moreover, the proposed method, which automatically extracted vital sign threshold ranges for each physical activity recognized by the system, showed promising results for remote health status evaluation.

A Procedure To automate the assessment of generator protection

The evaluation of protection performance after a generator shutdown is mandatory to assess if all automatic actions resulted in system security and safety to its users. Nowadays, however, the number of digital fault recorders and protection devices with oscillographic functionalities has intensely increased, making traditional manual analysis excessively time consuming. In this context, the contribution of this work is to propose a procedure to automate the evaluation of generating units’ protection, making it possible for the engineers to focus on the most important data. The proposed procedure was evaluated through the development of a prototype designed using the multi-agent paradigm, taking advantage of its main computational benefits. The implemented prototype generated correct assessments for a large set of real disturbances, proving to be a very efficient and accurate tool for assessment and decision support to operators.

Social group membership does not modulate automatic imitation in a contrastive multi-agent paradigm

A key prediction of motivational theories of automatic imitation is that people imitate in-group over out-group members. However, research on this topic has provided mixed results. Here, we investigate the possibility that social group modulations emerge only when people can directly compare in- and out-group. To this end, we conducted three experiments in which we measured automatic imitation of two simultaneously shown hands: one in-group and one out-group hand. Our general hypothesis was that the in-group hand would be imitated more than the out-group hand. However, even though both explicit and implicit manipulation checks showed that we succeeded in manipulating participants’ feelings of group membership, we did not find support for the predicted influence of group membership on automatic imitation. In contrast to motivational theories, this suggests that group membership does not influence who we do or do not imitate, not even in a contrastive multi-agent paradigm.

DQR: An Efficient Deep Q-Based Routing Approach in Multi-Controller Software Defined WAN (SD-WAN)

Software Defined Networking (SDN) is a promising paradigm in the field of network technology. This paradigm suggests the separation between the control plane and the data plane which brings flexibility, efficiency and programmability to network resources. SDN deployment in large scale networks raises many issues which can be overcame using a collaborative multi-controller approaches. Such approaches can resolve problems of routing optimization and network scalability. In large scale networks, such as SD-WAN, routing optimization consists of achieving a trade-off between per-flow QoS, the load balancing in each domain as well as the resource utilization in inter-domain links. Multi-Agent Reinforcement Learning paradigm(MARL) is one of the most popular solutions that can be used to optimize routing strategies in SD-WAN. This paper proposes an efficient approach based on MARL which is able to ensure a load balancing among each network as well as optimized resource utilization of inter-domain links. This approach profits from our previous work, denoted SPFLR, and tries to balance the load of the whole network using Deep Q-Networks (DQN) algorithms. Simulation results show that the proposed solution performs better than parallel solutions such as BGP-based routing and random routing.

Exploring communication protocols and centralized critics in multi-agent deep learning

Tackling multi-agent environments where each agent has a local limited observation of the global state is a non-trivial task that often requires hand-tuned solutions. A team of agents coordinating in such scenarios must handle the complex underlying environment, while each agent only has partial knowledge about the environment. Deep reinforcement learning has been shown to achieve super-human performance in single-agent environments, and has since been adapted to the multi-agent paradigm. This paper proposes A3C3, a multi-agent deep learning algorithm, where agents are evaluated by a centralized referee during the learning phase, but remain independent from each other in actual execution. This referee’s neural network is augmented with a permutation invariance architecture to increase its scalability to large teams. A3C3 also allows agents to learn communication protocols with which agents share relevant information to their team members, allowing them to overcome their limited knowledge, and achieve coordination. A3C3 and its permutation invariant augmentation is evaluated in multiple multi-agent test-beds, which include partially-observable scenarios, swarm environments, and complex 3D soccer simulations.

Agent-based Modeling for Ontology-driven Analysis of Patient Trajectories

Patients are often required to follow a medical treatment after discharge, e.g., for a chronic condition, rehabilitation after surgery, or for cancer survivor therapies. The need to adapt to new lifestyles, medication, and treatment routines, can produce an individual burden to the patient, who is often at home without the full support of healthcare professionals. Although technological solutions –in the form of mobile apps and wearables– have been proposed to mitigate these issues, it is essential to consider individual characteristics, preferences, and the context of a patient in order to offer personalized and effective support. The specific events and circumstances linked to an individual profile can be abstracted as a patient trajectory, which can contribute to a better understanding of the patient, her needs, and the most appropriate personalized support. Although patient trajectories have been studied for different illnesses and conditions, it remains challenging to effectively use them as the basis for data analytics methodologies in decentralized eHealth systems. In this work, we present a novel approach based on the multi-agent paradigm, considering patient trajectories as the cornerstone of a methodology for modelling eHealth support systems. In this design, semantic representations of individual treatment pathways are used in order to exchange patient-relevant information, potentially fed to AI systems for prediction and classification tasks. This paper describes the major challenges in this scope, as well as the design principles of the proposed agent-based architecture, including an example of its use through a case scenario for cancer survivors support.

A PSO-inspired architecture to hybridise multi-objective metaheuristics

Hybridisation is a technique that exploits and unites the best features of individual algorithms. The literature includes several hybridisation methodologies, among which there are general procedures, termed architectures, that provide generic functionalities and features for solving optimisation problems. Successful hybridisation methodologies have applied concepts of the multi-agent paradigm, such as cooperation and agent intelligence. However, there is still a lack concerning architectures for the hybridisation of multi-objective metaheuristics that fully explore these concepts. This study proposes a new architecture, named MO-MAHM, based on concepts from Particle Swarm Optimisation, to hybridise multi-objective metaheuristics. We apply the MO-MAHM to the Bi-objective Spanning Tree Problem. Four algorithms were hybridised within the MO-MAHM: three evolutionary algorithms and a local search method. We report the results of experiments with 180 instances, analyse the behaviour of the MO-MAHM, and compare to the results produced by algorithms proposed for the Bi-objective Spanning Tree Problem.

A Preventive Multi-agent based Policy for Distributed Hardware Resource Balancing

The success of a business, especially a multi-site extended enterprise, depends on the good management of all its distributed resources. It is difficult for a company to be successful if it does not have a reliable and optimal management of resources by avoiding overstocking of certain resources on a site Sitem ∈ E, and at the same time, the sub-storing of the same resources on another site Sitep ∈ E. In both cases, there is a lack of profit. In this paper, we will try to resolve this situation, by the proposal of an architecture based on the cooperative multi-agent systems paradigm combined with the Contract-Net protocol. We bring in an intelligent agent whose role is to warn in advance and for each item itemi ∈ Sitem, the coming of breakdowns and stock excesses by balancing the level of inter-site availability by a flow of resources of the same itemi by calling on the other E sites whose levels are in over-storage or under-storage.

ПРОБЛЕМИ ІНФОРМАЦІЙНО-АНАЛІТИЧНОГО СУПРОВОДУ ФУНКЦІОНУВАННЯ І РОЗВИТКУ ПІДПРИЄМСТВ ПОЛІГРАФІЧНОЇ ГАЛУЗІ В УМОВАХ КОНКУРЕНЦІЇ

Small and medium-sized enterprises are a potential driver of Ukraine's modern economy. The large part of the market is occupied by the enterprises of printing industry. Their managers constantly come across the tasks of expanding production, re-profiling, upgrading, setting up branches or eliminating them. Given the dynamic nature of the operation of such enterprises and its duality of the process of multiagent systems functioning, it is proposed to solve such a problem on the basis of evolutionary and multiagent paradigms. As the enterprises of the industry evolve over time, it is shown that in their modeling the ideas of the concept of "artificial life" can be used, their peculiarities, advantages and disadvantages are established. The analytical review of models, methods and software-algorithmic tools, used in the processes of support of manufacturing enterprises by stages of their life cycle, is carried out. The analysis shows the benefits of using multiagent systems in decision support systems in homogeneous environments. Models have been built to modify multiple tasks or production structures or management strategies based on predefined rules. The features of building an intelligent decision support system and experimental verification of results are presented. The functional structure of modular interaction in the decision support system is proposed. Features of modules functioning are defined, their input and output data flows, features of functioning and critical modes are specified. The peculiarities of forming a knowledge base, including a database of transactions, a bank of mathematical models, a variety of mathematical methods and rules for obtaining new knowledge, are shown. The analysis of the recommendations contained in the knowledge base significantly increases the chances of the manager to make informed progressive decisions. The above results are, in aggregate, the solution to scientific and applied research problem.

Agent-based Model for Simulating Urban System

In this paper, we address the issue related to the modelling and simulation of urban systems. We propose a new approach for simulating urban system based on multi-agent paradigm. Our proposed model is based on the use of a coupling between cellular agents and vector agents. These agents made it possible to take into account the spatial dimension of urban systems as well as the modelling of all the rules that govern them. To allow reusability of our model, we apply the VOYELLE approach by defining an environment model, an organization model, an agent model and an interaction model. We test our proposed model with a case study on Casablanca city. We discuss the problem of urbanization of Casablanca by following an approach that reduces the problem into two sub-problems similar but are treated differently: first, predict the city’s need of housing (individual housing zone, multifamily housing zone, ...) and then anticipate the city’s need of public services and ensure better spatial distribution of these equipment to best serve the people needs. Then we did experimentation with two simulation scenarios by changing in each scenario the hypotheses concerning urban planning especially in terms of demographic growth rate and residential sprawl.

Towards a Multi-Agent based Network Intrusion Detection System for a Fleet of Drones

The objective of this research work is to propose a new model of intrusion detection system for a fleet of UAVs deployed with an ad hoc communication architecture. The security of a drone fleet is rarely addressed by the scientific community, and most research has focused on routing protocols and battery autonomy, while ignoring the security aspect. The multi-agent paradigm is considered the most adequate and appropriate solution to model an effective intrusion detection system capable of detecting intrusions targeting a drone fleet. Multi-agent systems can perfectly address the security problem of a drone fleet, given the mobility, autonomy, cooperation and distribution characteristics present in the network linking the different nodes of the fleet. The proposed model consists of a set of cooperative, autonomous, communicating, learning and intelligent agents that collaborate with each other to carry out intrusion and suspicious activity detection missions that can target the network of a fleet of drones. Our system is autonomous and can detect known and unknown cyber attacks in real time without the need for human experts, who generally design the signatures of known attacks for conventional intrusion detection systems.

Towards automated SCADA forensic investigation: challenges, opportunities and promising paradigms

Modern supervisory control and data acquisition (SCADA) networks represent a challenging domain for forensic investigators who have the responsibility to determine the main causes of the catastrophic incidents that could happen in SCADA systems and to provide precise and logical evidences supported with comprehensive technical reports to the legal organisations. They are characterised to be complex, large-scale, and highly distributed systems comprising diversities of proprietary components such as field devices, embedded control systems, computers, communication networks, etc. Providing forensic investigators with fully or partially automated forensic investigation can be an effective solution against the challenging nature of modern SCADA networks. This review paper discusses the challenges and opportunities towards achieving that goal and highlights the emerging technological paradigms that can be considered as promising in the realisation of such a framework. Finally, this paper proposes a conceptual framework for automated forensic investigation in modern SCADA networks accompanied with a possible realisation architecture based on the multi-agent systems and wireless sensor networks promising technological paradigms.

Is deep learning the new agent for map generalization?

ABSTRACTThe automation of map generalization has been keeping researchers in cartography busy for years. Particularly great progress was made in the late 90s with the use of the multi-agent paradigm. Although the current use of automatic processes in some national mapping agencies is a great achievement, there are still many unsolved issues and research seems to stagnate in the recent years. With the success of deep learning in many fields of science, including geographic information science, this paper poses the controversial question of the title: is deep learning the new agent, i.e. the technique that will make generalization research bridge the gap to fully automated generalization processes? The paper neither responds a clear yes nor a clear no but discusses what issues could be tackled with deep learning and what the promising perspectives. Some preliminary experiments with building generalization or data enrichments are presented to support the discussion.

Long-Term Adaptivity in Distributed Intelligent Systems: Study of ViaBots in a Simulated Environment

This paper proposes a long-term adaptive distributed intelligent systems model which combines an organization theory and multi-agent paradigm—ViaBots. Currently, the need for adaptivity in autonomous intelligent systems becomes crucial due to the increase in the complexity and diversity of the tasks that autonomous robots are employed for. To deal with the design complexity of such systems within the ViaBots model, each part of the modeled system is designed as an autonomous agent and the entire model, as a multi-agent system. Based on the viable system model, which is widely used to ensure viability, (i.e., long-term autonomy of organizations), the ViaBots model defines the necessary roles a system must fulfill to be capable to adapt both to changes in its environment (like changes in the task) and changes within the system itself (like availability of a particular robot). Along with static role assignments, ViaBots propose a mechanism for role transition from one agent to another as one of the key elements of long term adaptivity. The model has been validated in a simulated environment using an example of a conveyor system. The simulated model enabled the multi-robot system to adapt to the quantity and characteristics of the available robots, as well as to the changes in the parts to be processed by the system.