Distributed Artificial Intelligence Systems Research Articles

Constructing Executing and Overcoming Challenges in Distributed AI Systems: A Study of Federated Learning Framework

Federated learning stands out as a promising approach within the realm of distributed artificial intelligence (AI) systems, facilitating collaborative model training across decentralized devices while safeguarding data privacy. This study presents a thorough investigation into federated learning architecture, covering its foundational design principles, implementation methodologies, and the significant challenges encountered in distributed AI systems. We delve into the fundamental mechanisms underpinning federated learning, elucidating its merits in diverse environments and its prospective applications across various domains. Additionally, we scrutinize the technical complexities associated with deploying federated learning systems, including considerations such as communication efficiency, model aggregation techniques, and security protocols. By amalgamating insights gleaned from recent research endeavors and practical deployments, this study furnishes valuable guidance for both researchers and practitioners aiming to harness federated learning for the development of scalable and privacy-preserving AI solutions.

Constructing Executing and Overcoming Challenges in Distributed AI Systems: A Study of Federated Learning Framework

Federated learning stands out as a promising approach within the realm of distributed artificial intelligence (AI) systems, facilitating collaborative model training across decentralized devices while safeguarding data privacy. This study presents a thorough investigation into federated learning architecture, covering its foundational design principles, implementation methodologies, and the significant challenges encountered in distributed AI systems. We delve into the fundamental mechanisms underpinning federated learning, elucidating its merits in diverse environments and its prospective applications across various domains. Additionally, we scrutinize the technical complexities associated with deploying federated learning systems, including considerations such as communication efficiency, model aggregation techniques, and security protocols. By amalgamating insights gleaned from recent research endeavors and practical deployments, this study furnishes valuable guidance for both researchers and practitioners aiming to harness federated learning for the development of scalable and privacy-preserving AI solutions.

Artificial Intelligence in Medicine

Artificial Intelligence (AI) is the technology related to simulating human behaviour in machines. Machine intelligence is a subfield of AI in which available raw data is processed to learn inherent patterns and build a model to adapt to new data. Deep learning models utilize very large amount of data and extract important features and classify the data. Multiagent systems or distributed AI systems are autonomous, proactive, reactive and have ability to interact with humans and other agents. Medicine includes all the processes involved in preventing, diagnosing and curing diseases. It includes medical staff and supporting staff records, drug information, decision support information for medical professionals, clinical lab tests, X-rays, magnetic resonance images, surgeries, and so on. AI has a number of applications in medicine including expert systems, medical robots, medial image analysis and distributed medical agents etc. Expert systems can function as medical experts and helpful for patients who are unable to reach a medical specialist due to cost, or being in a remote area. The role of AI is significant in radiology as abnormal data is labelled in medical images obtained from computed tomography, X-rays, and magnetic resonance imaging etc. more accurately. Medical robots assist in patient care, clinical settings, surgeries and in many other ways. Distributed medical agents enable the availability of a number of medical experts online to examine critical cases. In this paper the role of artificial intelligence in the above mentioned medical applications is elaborated with relevant examples. It is concluded that AI is indispensable in medicine for effective and efficient healthcare.

Self-Organizing Democratized Learning: Toward Large-Scale Distributed Learning Systems.

Emerging cross-device artificial intelligence (AI) applications require a transition from conventional centralized learning systems toward large-scale distributed AI systems that can collaboratively perform complex learning tasks. In this regard, democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. The outlined principles are meant to study a generalization in distributed learning systems that go beyond existing mechanisms such as federated learning (FL). Moreover, such learning systems rely on hierarchical self-organization of well-connected distributed learning agents who have limited and highly personalized data and can evolve and regulate themselves based on the underlying duality of specialized and generalized processes. Inspired by Dem-AI philosophy, a novel distributed learning approach is proposed in this article. The approach consists of a self-organizing hierarchical structuring mechanism based on agglomerative clustering, hierarchical generalization, and corresponding learning mechanism. Subsequently, hierarchical generalized learning problems in recursive forms are formulated and shown to be approximately solved using the solutions of distributed personalized learning problems and hierarchical update mechanisms. To that end, a distributed learning algorithm, namely DemLearn, is proposed. Extensive experiments on benchmark MNIST, Fashion-MNIST, FE-MNIST, and CIFAR-10 datasets show that the proposed algorithm demonstrates better results in the generalization performance of learning models in agents compared to the conventional FL algorithms. The detailed analysis provides useful observations to further handle both the generalization and specialization performance of the learning models in Dem-AI systems.

Federated Learning Architecture: Design, Implementation, and Challenges in Distributed AI Systems

Federated learning has emerged as a promising paradigm in the domain of distributed artificial intelligence (AI) systems, enabling collaborative model training across decentralized devices while preserving data privacy. This paper presents a comprehensive exploration of federated learning architecture, encompassing its design principles, implementation strategies, and the key challenges encountered in distributed AI systems. We delve into the underlying mechanisms of federated learning, discussing its advantages in heterogeneous environments and its potential applications across various domains. Furthermore, we analyse the technical intricacies involved in deploying federated learning systems, including communication efficiency, model aggregation techniques, and security considerations. By synthesizing insights from recent research and practical implementations, this paper offers valuable guidance for researchers and practitioners seeking to leverage federated learning in the development of scalable and privacy-preserving AI solutions.

A compositional approach to creating architecture frameworks with an application to distributed AI systems

Artificial intelligence (AI) in its various forms finds more and more its way into complex distributed systems. For instance, it is used locally, as part of a sensor system, on the edge for low-latency high-performance inference, or in the cloud, e.g. for data mining. Modern complex systems, such as connected vehicles, are often part of an Internet of Things (IoT). This poses additional architectural challenges. To manage complexity, architectures are described with architecture frameworks, which are composed of a number of architectural views connected through correspondence rules. Despite some attempts, the definition of a mathematical foundation for architecture frameworks that are suitable for the development of distributed AI systems still requires investigation and study.In this paper, we propose to extend the state of the art on architecture framework by providing a mathematical model for system architectures, which is scalable and supports co-evolution of different aspects for example of an AI system. Based on Design Science Research, this study starts by identifying the challenges with architectural frameworks in a use case of distributed AI systems. Then, we derive from the identified challenges four rules, and we formulate them by exploiting concepts from category theory. We show how compositional thinking can provide rules for the creation and management of architectural frameworks for complex systems, for example distributed systems with AI. The aim of the paper is not to provide viewpoints or architecture models specific to AI systems, but instead to provide guidelines based on a mathematical formulation on how a consistent framework can be built up with existing, or newly created, viewpoints. To put in practice and test the approach, the identified and formulated rules are applied to derive an architectural framework for the EU Horizon 2020 project “Very efficient deep learning in the IoT” (VEDLIoT) in the form of a case study.

The Use of Blockchain to Support Distributed AI Implementation in IoT Systems

This article presents a distributed and decentralized architecture for the implementation of distributed artificial intelligence (DAI) using hardware platforms provided by the Internet of Things (IoT). A trained DAI system has been implemented over IoT, where each IoT device acts as one or more of the neurons within the DAI layers. This is accomplished by the utilization of decentralized, self-managed blockchain technologies that allow trusted interactions and information to be exchanged between distributed neurons. The platform was built and customized to be used within the IoT system, and it is capable of handling DAI-related tasks. A new consensus mechanism based on Proof of Authority (PoA) and Proof of Work (PoW) has been designed and implemented, along with bespoke block and transaction formats. The proposed architecture was analyzed, implemented, and tested using a dedicated testbed with low-cost IoT devices. A quantitative measurement and performance evaluation of the system based on a real-world IoT application was conducted. The implemented DAI is found to have an accuracy of 92%–98%, with an energy cost of 0.12 joules (J) when utilizing a Raspberry Pi to run one neuron. The measured hash per joule (h/J) when using a Raspberry Pi for mining is 13.8 Kh/J compared to 54 Kh/J using an ESP32. The results showed that it is feasible to implement a DAI system utilizing the IoT hardware platform while maintaining the system’s accuracy. The integration of the blockchain has added an element of security and trust to the data and the interaction between system components.

Distributed and Democratized Learning: Philosophy and Research Challenges

Due to the availability of huge amounts of data and processing abilities, current artificial intelligence (AI) systems are effective in solving complex tasks. However, despite the success of AI in different areas, the problem of designing AI systems that can truly mimic human cognitive capabilities such as artificial general intelligence, remains largely open. Consequently, many emerging cross-device AI applications will require a transition from traditional centralized learning systems towards large-scale distributed AI systems that can collaboratively perform multiple complex learning tasks. In this paper, we propose a novel design philosophy called democratized learning (Dem-AI) whose goal is to build large-scale distributed learning systems that rely on the self-organization of distributed learning agents that are wellconnected, but limited in learning capabilities. Correspondingly, inspired by the societal groups of humans, the specialized groups of learning agents in the proposed Dem-AI system are selforganized in a hierarchical structure to collectively perform learning tasks more efficiently. As such, the Dem-AI learning system can evolve and regulate itself based on the underlying duality of two processes which we call specialized and generalized processes. In this regard, we present a reference design as a guideline to realize future Dem-AI systems, inspired by various interdisciplinary fields. Accordingly, we introduce four underlying mechanisms in the design such as plasticity-stability transition mechanism, self-organizing hierarchical structuring, specialized learning, and generalization. Finally, we establish possible extensions and new challenges for the existing learning approaches to provide better scalable, flexible, and more powerful learning systems with the new setting of Dem-AI.

Bridging the Gap: On the Collaboration between Symbolic Interactionism and Distributed Artificial Intelligence in the Field of Multi‐Agent Systems Research

This article discusses the relationship in the U.S. between current symbolic interactionism and computer sciences—specifically, distributed artificial intelligence (DAI). The general thesis is twofold. First, current interactionist approaches to organization, science, and technology show a special affinity to goals and problems of DAI research, and in research style, methods, and theoretical concepts, symbolic interactionism can provide useful suggestions in the design of DAI systems. Second, a good way to analyze the relationship between computer sciences and symbolic interactionism is reflexive of theoretical concepts provided by interactionist approaches. In this sense, DAI is a “going concern” which extends across various fields and intersecting social worlds connected through a set of conceptual “boundary objects.” It is concluded that the interaction between technology and sociological thought must go beyond a mere exchange of ideas. What is required is continual, hands‐on, trans‐disciplinary collaboration.

A comparison of three information gathering strategies in DAI systems under noisy conditions

This paper investigates a problem of task allocation in Distributed Artificial Intelligence (DAI) systems, where a coordinator allocates tasks among multiple agents in an optimal manner. In making the task allocation, the coordinator needs to understand the preference orders of the agents for the different task bundles. The coordinator does this by adopting an information acquisition strategy that leads to an optimal system welfare. Three different information acquisition strategies are investigated here. The strategies are compared in a noisy environment for the quality of information they provide in terms of the deviation from optimal system welfare. Copyright 1996 Elsevier Science Ltd

Designing a reusable co-ordination module for co-operative industrial control applications

Distributed artificial intelligence (DAI) systems, in which multiple agents communicate and co-operate with one another to achieve their individual and collective goals, are a promising enabling technology for constructing large, real-world industrial control applications. To facilitate the development of such systems a number of generic DAI frameworks have been devised. These frameworks typically aid the development process by providing a language, a set of structures, and/or some tools with which the necessary infrastructure and support mechanisms for interacting agents can be instantiated. The paper reports on one such framework, called ARCHON™, which has been used to build DAI systems in the following industrial control domains: electricity distribution management, electricity transportation management, cement factory control, particle accelerator control and flexible assembly robotic cells. A distinguishing and novel feature of the ARCHON framework is that it extends the level of support offered to the system builder — it provides generic and reusable knowledge about the process of co-operation, in addition to the more standard development facilities. This generic knowledge is embedded in a domain-independent co-ordination module and it is the rationale, design, implementation and evaluation of this module which forms the major contribution of the paper.

Distributed Artificial Intelligence in manufacturing systems control

This research uses a Distributed Artificial Intelligence (DAI) framework to efficiently utilize the infrastructure available for process planning in a batch processing PWB assembly facility. The DAI approach decomposes the entire production control task into several sub-tasks. Then, the sub-tasks are implemented by the basic elements of the DAI system called ‘intelligent agents’. By working collectively, the intelligent agents of the DAI system can arrive at a solution for the problem. The DAI system initially proposes all possible solutions generated by the intelligent agents. Then, a fuzzy coordination technique is utilized to evaluate the solutions and to find the most appropriate one for shopfloor implementation. Using inputs such as the short-term production plan, design data, shopfloor observation data, and CAD information, the DAI system provides applicable production plans with ranks for the feasibility of current assembly activities.

A formal semantics for DAI language NUML

Traditional AI systems are brittle in the sense that they fail miserably when presented with problems even slightly outside of their limited range of expertise. A powerful, extensible strategy of Distributed Artificial Intelligence (DAI) for overcoming such bounds is to put the system in a society of systems. So the ability to coordinate group activities of individuals and to communicate between each other is necessary for a language describing DAI systems. Agent-oriented language NUML is such a language. It is a specific kind of object-oriented language. To give formal semantics to NUML, there is the problem to formalise object-oriented programming paradigm which is still open. The theory of higher-order π-calculus is a concurrent computation model with sufficient capability, which provides us a mathematical tool to do the formalization. This paper tries to use higher-order π-calculus to formalise NUML.

Toward a theory and method for cooperative systems, with application to auditing

Cooperative distributed knowledge-base systems—sometime termed distributed artificial Intelligence (DAI)—offer computer methods for coordinating group expertise in the solving of problems. Much of the research to the present has been empirical testing of informal methods. There is a need for more formal methods to guide the design and development of DAI. At the same time, object-oriented programming methods provide a natural means or representing real-world objects in a way that facilitates communication among those objects. In this paper we propose a methodology for integrating formal methods of knowledge theory with object-oriented methods in a way that may facilitate the development of more robust DAI systems. Our exposition takes the reader from conceptual development to fundamentals of an application in auditing.

Agent language NUML and its reduction implementation model based on HOπ

Higher--order π--calculus(HOπ)[3.4.6]] is a computation model with sufficient computation capability. It is suitable to describe the concurrency in computation. In this paper, the agent language NUML which is designed to describe DAI (distributed artificial intelligence) systems is presented; also its semantics in HOπ is given and finalliy a reduction implementation model of NUML based on HOπ is submitted.

An architecture for real-time distributed artificial intelligent systems

This paper introduces a new architecture for a real-time distributed artificial intelligence system: DENIS—a Dynamic Embedded Noticeboard Information System. The fundamental idea underlying the architecture draws heavily upon a distributed human system analogy, as seen, for example, in the workplace. The aim of DENIS is to provide a simple, meaningful means by which autonomous intelligent agents can cooperate and coordinate their actions in order to enhance the reliability and effectiveness of a real-time distributed control system. Based on a human paradigm, the architecture inherently allows for the control of an intelligent agent to be taken over by a human operator, yet still to maintain consistency in the distributed system. The key to the thinking in this new approach is to try to model how humans work together, and to implement this in a distributed architecture. One of the main issues raised is that humans owe much of their flexibility to their ability to reason, not only logically, but also in terms of time.

Reasoning about knowledge to understand distributed AI systems

The use of reasoning about agent knowledge to understand computation in distributed artificial intelligence (DAI) systems is explored. In particular, the negotiation model and the blackboard model are examined, using a temporal, epistemic logic to characterize knowledge and its evolution among interacting agents in these systems. For negotiation, knowledge that agents require to commit consistently to the outcome is determined; also derived are communication requirements for attaining that knowledge in several computational settings. For the blackboard model, the kinds of knowledge statements that can be built up via communication during a distributed computation, independent of the application domain, is discussed. It is suggested that reasoning about knowledge can help one to understand the role of communication in achieving coherence and coordination. The utility of reasoning about knowledge in DAI systems is considered. >

MACE: High-level distributed objects in a flexible testbed for distributed AI research

Distributed Artificial Intelligence (DAI) is a subfield of AI concerned with the problems of describing and constructing multiple intelligent systems which interact. MACE (for Multi-Agent Computing Environment) is a language, programming environment, and testbed for DAI systems. Parallelism in AI problem-solving applications can be exploited at many different levels: in hardware, in the implementation language (e.g., a production system language), in a problem-solving paradigm, or directly in the application. MACE is an instrumented testbed for building a wide range of experimental Distributed Artificial Intelligence systems. The goal of MACE is to support experimentation with different styles of distributed AI systems, at different levels of complexity. The dominant metaphor of MACE is a collection of intelligent, semi-autonomous agents interacting in organized ways. MACE is a distributed, object-oriented system. MACE computational units (called agents) run in parallel, and communicate via messages. They provide optional facilities for knowledge representation (world knowledge, models of other agents, their goals and plans, their roles and capabilities, etc.) and reasoning capabilities. The MACE environment maps agents onto processors, handles inter-agent communication, and provides several features: A collection of agents. Agents are the basic computational units of the MACE system. Rather than being simple generic object structures, MACE agents are inherently “social” in nature: they know about some other agents in their environment, and expect to draw upon and coordinate with the expertise of others they know about. MACE agents represent this information in the form of models of other agents in the world in a special database called acquaintances , which is a property of each MACE agent. Current agent models are predicate-calculus based. MACE agents may be composed into sub-units or coalitions; they respond as organized groups or composites. The MACE Agent Description Language includes facilities for describing organized clusters of agents. A community of system agents. A number of pre-defined system agents provide MACE command interpretation, a standard user interface, the agent-builder cluster, error handling, tracing, execution monitoring, etc. Specific MACE agents serve as interactive tools for building and executing MACE programs. A collection of facilities which all agents may use, including a pattern matcher, a simulator, several standard agent engines, handlers for standard errors, and standard messages understood by all agents. MACE incorporates extensive pattern-matching facilities for interpreting messages, for pattern-directed invocation of asynchronous events (e.g., demons and other event-monitors), and for associative database access within agents. Instrumentation. The MACE testbed is instrumented to allow for measurements of the characteristics of problem-solvers during experimental runs. Message traffic, queue and database sizes, work done by an agent (in terms of elapsed real time or number of invocations), and load on a processor node are common measurements. A description database. Agent descriptions are maintained in a description database by a system-agent cluster which constructs new descriptions, verifies descriptions, and constructs executable agents from descriptions. MACE enforces a separation of description and execution to handle some problems of distribution of objects. A collection of kernels. MACE kernels collectively handle communication and message routing, perform I/O to terminals, files, or other devices, map agents onto processors, and schedule agents for execution. Reflection capabilities. A MACE agent's acquaintances database can include, of course, models of the behaviors, plans, roles, or implementation of the agent itself, providing high-level and behavioral reflection capabilities. An evolutionary philosophy. MACE agents are intended to be a high-level approach to distributed object-oriented programming. Since we are using MACE as a testbed to explore coordination strategies for intelligent multi-agent systems, the nature of generic MACE agents changes and specializes over time. All MACE agents now have acquaintance databases, react to certain specialized messages, and include particular attributes (e.g. the attribute “ME” which includes a partial self-description). Over time, as more coordination and modeling features are found useful, they will be standardized as features of MACE agents. In this way, MACE agents are explicitly evolving toward higher-level objects. MACE has been used to build numerous experimental DAI systems including models of lower-level parallelism (sets of distributed production rule systems without global database or inference engine, where each rule is an agent) and higher-level distributed problem-solving architectures (e.g., domain-independent distributed blackboard and contract-net schemes). MACE is implemented on a 16-node INTEL iPSC/1 large-memory hypercube and in a LISP machine environment, and will soon be available on the iPSC/2 and a Sun network.