Machine Agents Research Articles

Knowledge graph-enhanced multi-agent reinforcement learning for adaptive scheduling in smart manufacturing

AbstractSelf-organizing manufacturing network has emerged as a viable solution for adaptive manufacturing control within the mass personalization paradigm. This approach involves three critical elements: system modeling and control architecture, interoperable communication, and adaptive manufacturing control. However, current research often separates interoperable communication from adaptive manufacturing control as isolated areas of study. To address this gap, this paper introduces Knowledge Graph-enhanced Multi-Agent Reinforcement Learning (MARL) method that integrates interoperable communication via Knowledge Graphs with adaptive manufacturing control through Reinforcement Learning. We hypothesize that implicit domain knowledge obtained from historical production job allocation records can guide each agent to learn more effective scheduling policies with accelerated learning rates. This is based on the premise that machine assignment preferences effectively could reduce the Reinforcement Learning search space. Specifically, we redesign machine agents with new observation, action, reward, and cooperation mechanisms considering the preference of machines, building upon our previous MARL base model. The scheduling policies are trained under extensive simulation experiments that consider manufacturing requirements. During the training process, our approach demonstrates improved training speed compared with individual Reinforcement Learning methods under the same training hyperparameters. The obtained scheduling policies generated by our Knowledge Graph-enhanced MARL also outperform both individual Reinforcement Learning methods and heuristic rules under dynamic manufacturing settings.

Block Chain-Powered Cloud Data Integrity Verification System with Excellent Effectiveness

The "next-generation financial technology," or blockchain, is praised for its peer authentication security features, which include virtual money, data encryption, and hash value creation. Cloud computing is widely used due to its efficiency, and the global financial sector sees a vast market for security. Blockchain technology is examined in this article along with its developments and application to cloud computing in healthcare and electronic vehicle security. Verification, oversight, and dependable data storage are made possible by the virtual machine agent mechanism, which is crucial for maintaining blockchain integrity. The Merkel hash tree's unique hash values are used by blockchain smart contracts to keep track of data modifications. Users are notified of data manipulation through a "block-and-response" procedure. Data security and trustworthy computing are still issues in spite of the growth of cloud computing. Different approaches, such as data integrity tests and secure multi-party computations, have been proposed by researchers; however, these frequently encounter problems related to complexity and scalability. This paper investigates the ways in which blockchain technology can help with these issues by providing a decentralized method for improving cloud computing security and storage. Keywords— Blockchain, money, security, cloud service, technology, big-data, data, healthcare ;

In human–machine trust, humans rely on a simple averaging strategy

With the growing role of artificial intelligence (AI) in our lives, attention is increasingly turning to the way that humans and AI work together. A key aspect of human–AI collaboration is how people integrate judgements or recommendations from machine agents, when they differ from their own judgements. We investigated trust in human–machine teaming using a perceptual judgement task based on the judge–advisor system. Participants (n=89\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$n=89$$\\end{document}) estimated a perceptual quantity, then received a recommendation from a machine agent. The participants then made a second response which combined their first estimate and the machine’s recommendation. The degree to which participants shifted their second response in the direction of the recommendations provided a measure of their trust in the machine agent. We analysed the role of advice distance in people’s willingness to change their judgements. When a recommendation falls a long way from their initial judgement, do people come to doubt their own judgement, trusting the recommendation more, or do they doubt the machine agent, trusting the recommendation less? We found that although some participants exhibited these behaviours, the most common response was neither of these tendencies, and a simple model based on averaging accounted best for participants’ trust behaviour. We discuss implications for theories of trust, and human–machine teaming.

Negotiation-based scheduling considering agent emotion

Scheduling in decentralized manufacturing systems is being expanded and transformed to a smart level where manufacturing tasks are executed autonomously. However, achieving self-organized manufacturing resources that can make collaborative decisions remains challenging. This paper presents a novel negotiation model for decentralized scheduling in an autonomous manufacturing system. A variety of emotions represent the willingness of autonomous intelligent agents (IAs) to negotiate for a consensus despite their diverse goals. The proposed communication protocol considers the emotional aspects of the autonomous machine agents in decentralized decision-making. The negotiation model focuses on analyzing the effects of two key emotions: selfishness and cooperation. This study constitutes the basis for experiments on multi-goal machine tasks and artificial emotion mechanism in a flow shop production system. The complexities of negotiation dynamics between IAs, where the interplay of selfish and cooperative strategies influences the outcomes and efficiency of reaching agreements, are explored. The experimental results show that selfish IAs eventually reach agreements through gradual compromise, but cooperative IAs, with their flexibility, can quickly achieve resolutions, especially when bargaining power is balanced. Aligning with the dual concern theory, this study reveals that while selfish IAs can dominate cooperative ones, the cooperative approach often leads to faster and more effective outcomes, regardless of the level of selfishness involved.

Repensando a inteligência humana e da máquina sob o determinismo

This paper proposes a metaphysical framework for distinguishing between human and machine intelligence. It posits two identical deterministic worlds -- one comprising a human agent and the other a machine agent. These agents exhibit different information processing mechanisms despite their apparent sameness in a causal sense. Providing a conceptual modeling of their difference, this paper resolves what it calls “the vantage point problem” – namely, how to justify an omniscient perspective through which a determinist asserts determinism from within the supposedly deterministic universe.

Shannon Holes, Black Holes, and Knowledge: The Essential Tension for Autonomous Human–Machine Teams Facing Uncertainty

We develop a new theory of knowledge with mathematics and a broad-based series of case studies to seek a better understanding of what constitutes knowledge in the field and its value for autonomous human–machine teams facing uncertainty in the open. Like humans, as teammates, artificial intelligence (AI) machines must be able to determine what constitutes the usable knowledge that contributes to a team’s success when facing uncertainty in the field (e.g., testing “knowledge” in the field with debate; identifying new knowledge; using knowledge to innovate), its failure (e.g., troubleshooting; identifying weaknesses; discovering vulnerabilities; exploitation using deception), and feeding the results back to users and society. It matters not whether a debate is public, private, or unexpressed by an individual human or machine agent acting alone; regardless, in this exploration, we speculate that only a transparent process advances the science of autonomous human–machine teams, assists in interpretable machine learning, and allows a free people and their machines to co-evolve. The complexity of the team is taken into consideration in our search for knowledge, which can also be used as an information metric. We conclude that the structure of “knowledge”, once found, is resistant to alternatives (i.e., it is ordered); that its functional utility is generalizable; and that its useful applications are multifaceted (akin to maximum entropy production). Our novel finding is the existence of Shannon holes that are gaps in knowledge, a surprising “discovery” to only find Shannon there first.

A Novel Framework for Cloud Data Security with Blockchain Technology and Distributed Virtual Machine Agents

A Novel Framework for Cloud Data Security with Blockchain Technology and Distributed Virtual Machine Agents

A Novel Framework for Cloud Data Security with Blockchain Technology and Distributed Virtual Machine Agents

Even though cloud computing has advanced over the years, the two major challenges currently faced by cloud computing are data security and trusted computing. In cloud computing applications, blockchain technology can enhance data security and trusted computing when paired with virtual machine agents and mobile agent technologies. A decentralized new framework for distributed computing is blockchain. A potential area of study is the integration of blockchain technology with cloud computing, leveraging the former's security mechanisms to enhance the latter's secure computing and storage capabilities. The usage of the proposed framework builds on the inherent security features of blockchain technology in an effort to secure data and information integrity, confidentiality and authentication in cloud environments. Through the introduction of distributed virtual machine agents, the framework increases the scalability and effectiveness of data protection mechanisms for virtual data environments that can be used to store information protected by cloud security measures. Thus, the proposed study substantiates the applicability and efficiency of the existing and emerging problems of cloud data security and this approach develops satisfactory data protection solutions.

Writing at a Distance: Notes on Authorship and Artificial Intelligence

ABSTRACT: This essay provides an overview of notions of authorship in AI and natural language processing systems and discusses past and current debates about computers as literary authors. It suggests the concept of causal authorship to measure the types of distance between human and machine agents, acknowledging the anthropocentric bias of this idea. To balance this, the essay reflects on the notion of distributed authorship , which considers the network of actors involved in the creation of a text and has its own limitations. Both concepts are elements of a future theory of authorship in the age of machine learning.

A Survey of Algorithmic Methods for Competency Self-Assessments in Human-Autonomy Teaming

Humans working with autonomous artificially intelligent systems may not be experts in the inner workings of their machine teammates, but need to understand when to employ, trust, and rely on the system. A critical challenge is to develop machine agents with the capacity to understand their own capabilities and limitations, and the ability to communicate this information to human partners. Self-assessment is an emerging field that tackles this challenge through the development of algorithms that enable autonomous agents to understand and communicate their competency. These methods can engender appropriate trust and align human expectations with autonomous assistant abilities. However, current research in self-assessment is dispersed across many fields, including artificial intelligence, robotics, and human factors. This survey connects work from these disparate areas and reviews state-of-the-art methods for algorithmic self-assessments that enable autonomous agents to estimate, understand, and communicate valuable information pertaining to their competency, with focus on methods that can improve interactions within human-machine teams. To better understand the landscape of self-assessment approaches, we present a framework for categorizing work in self-assessment based on underlying algorithm type: test-based , learning-based , or knowledge-based . We synthesize common features across these approaches and discuss relevant future directions for research in this emerging space.

Exploring the constraints on artificial general intelligence: A game-theoretic model of human vs machine interaction

The potential emergence of artificial general intelligence (AGI) systems has sparked intense debate among researchers, policymakers, and the public due to their potential to surpass human intelligence in all domains. This note argues that for an AI to be considered “general”, it should achieve superhuman performance not only in zero-sum games but also in general-sum games, where winning or losing is not clearly defined. In this note, I propose a game-theoretic framework that captures the strategic interactions between a representative human agent and a potential superhuman machine agent. Four assumptions underpin this framework: Superhuman Machine, Machine Strategy, Rationality, and Strategic Unpredictability. The main result is an impossibility theorem, establishing that these assumptions are inconsistent when taken together, but relaxing any one of them results in a consistent set of assumptions. This note contributes to a better understanding of the theoretical context that can shape the development of superhuman AI.

Cognitive work in future manufacturing systems: Human-centered AI for joint work with models

Manufacturers perpetually adapt their systems to meet unforeseen events, new objectives, competition, and improved understanding of processes. In that human-directed work, models mediate an enduring relationship between production resources and engineers. Accommodating new understanding in the models controlling production can lead to more effective manufacturing. That work has previously been the province of quality programs such as Six Sigma, but is now fertile ground to study human-computer interaction about that enduring relationship mediated by models. Can AI augment human capability in the arcane work of formulating and refining models? This question is relevant to complex system engineering generally, not just manufacturing. In answering this question, this paper adapts Klein’s flexecution for use in adaptable manufacturing systems. Theory flexecution, the methodical refinement of models, points to human-computer interactions that emphasize the roles of models, explanation, and machine agents that recognize the engineer’s goals. This perspective article illustrates these ideas with an example of formulating models for production scheduling.

Operational safety hazard identification methodology for automated driving systems fleets

The safety of Automated Driving Systems (ADS) operating as Mobility as a Service (MaaS) depends on multiple factors in addition to the vehicle’s functionality, reliability, and performance. Currently, no comprehensive approach has been formally developed to identify operational safety hazards and define the operational safety responsibilities of the key agents involved in Level 4 (L4) ADS MaaS operations. This work develops and applies a structured hazard identification methodology for this operation. The methodology leverages and complements the strengths of various hazard identification and modeling methods, including Event Sequence Diagram (ESD), Concurrent Task Analysis (CoTA), System-Theoretic Process Analysis (STPA), and Fault Tree Analysis (FTA). The methodology is applied to analyze the operation of a fleet of L4 ADS vehicle fleets without a safety driver, monitored and supervised by remote operators. The results highlight the fleet operator’s role in ensuring the correct vehicle operation and preventing and mitigating incidents. The analysis demonstrates the developed methodology’s strengths and suitability for operational safety analysis of complex systems’ operations, considering the inherent complexity of the interactions between multiple human and machine agents.

Probing an Easy-to-Deploy Multi-Agent Manufacturing System Based on Agent Computing Node: Architecture, Implementation, and Case Study

Abstract Due to the widespread adoption of personalized customization services, the application contexts within discrete manufacturing workshops have become increasingly intricate, necessitating the modern industry to evolve toward a more adaptable production trajectory. The pre-established production rules in a traditional centralized control manufacturing system present difficulties in accommodating dynamic situations. Although a multi-agent manufacturing system (MAMS) yields natural advantages in handling dynamic emergencies, the current research is limited to the computer simulation level and lacks integration with the underlying physical devices. In order to mitigate said challenges, the standardization modeling approach for the agent computing node (ACN) to facilitate the implementation of a readily deployable MAMS was proposed in the present study. Initially, adapters encompassing communication, decision, and control modules were developed within the industrial personnel computer-based computing node to accommodate the heterogeneous interface protocols of diverse machines. These adapters enable communication and interaction among machines while laying the computational foundation for the ACN. Accordingly, the models of the machine agent, the part agent, and the monitoring agent were constructed based on ACNs and could perceive the dynamic production information and support the enabling application. Subsequently, to guide ACNs in making scheduling decisions beneficial to global performance, an improved negotiation mechanism in MAMS was achieved in real-time. Finally, the proposed MAMS based on the ACN was deployed in an actual flexible machining workshop. Comparative experiments were implemented and, as exhibited from the experimental results, the proposed ACNs possessed the capabilities of achieving optimal global decision-making and facilitating straightforward deployment.

Framing, more than speech, affects how machine agents are perceived

ABSTRACT We tested how speech phenomena (discourse markers like oh and you know and fillers like uh and um) and the conceptualisation of voices (as human or machine) affected speaker quality ratings (trustworthiness, friendliness, intelligence, and nervousness) across dialogues and monologues. Listeners preferred human voices over synthesised voices in dialogue containing discourse markers and fillers (Study 1). Shifting the location of discourse markers to new locations did not negatively impact ratings of what participants heard (Study 2), but hearing shifted markers did lead participants to rate imagined machine speech containing markers more negatively on a post-experiment questionnaire. Backstories also impacted perceptions of recorded speech: People rated the same synthesised speech more positively if they believed it was created by a person than if they believed it was created by machine (Study 3). Similarly, people's opinions about robots varied depending on context (Study 4). How people rate machine agents in the future will likely reflect how machine agents are portrayed to the public, capabilities aside.

Introduction of Reinforcement Learning and Its Application Across Different Domain

In the modern era of rapid development in Deep Neural Networks, Reinforcement Learning (RL) has evolved into a pivotal and transformative technology. RL, a learning process where these machine agent interacts with several unknown environment through trial and error. The agent, responsive to the learning machine, go through these interaction, and start receiving feedback in the form of positive rewards or negative rewards like penalties from the environment, and constantly refines its behavior. This research paper offers an in-depth introduction to the foundational concepts of RL, focusing on Markov Decision Processes and various RL algorithms.</p> <p>Machine Learning (ML) is a subset of Artificial Intelligence, which deals with ‘‘the question of how to develop software agents (Machine) that improve automatically with experience’’. The basic three categories of Machine Learning are.</p> <ol> <li>Supervised Learning</li> <li>Unsupervised Learning</li> <li>Reinforcement Learning</li> </ol> <p>RL method is that in any situation the agent has to choose between using its acquired knowledge of the environment i.e. using an action already tried or performed previously or exploring actions never tried before in that situation.</p> <p>In this review paper, we will discuss the most used learning algorithms in games robotics and healthcare, autonomous control as well as communication and networking, natural language processing.[1]</p>

Joint Dynamic Dispatching and Preventive Maintenance for Unrelated Parallel Machines With Equipment Health Considerations

A decomposed dynamic dispatching and preventive maintenance (D-DDPM) framework is presented to overcome the computational challenges of solving large DDPM problems. Stochastic deterioration of machine health is explicitly considered. By efficient allocation of individual machines capacity under different machine health, the D-DDPM framework decomposes the global DDPM problems into single-machine DDPM problems while ensuring throughput optimality. Intelligent machine agents are then designed to solve the single-machine DDPM problems efficiently and to perform real-time decisions for individual machines. After the decomposition, the overall computational complexity grows only linearly with the number of machines and the proposed framework could be adopted in large systems. In the numerical study, the D-DDPM framework is near-optimal in small systems that have optimal DDPM solutions. In which, the computation time is reduced by 90.72% and the average cycle time is within 6% of optimal. For large systems without optimal DDPM solutions, numerical results show that the D-DDPM maintains throughput optimality and reduces average cycle time by up to 90% against other control policies.

A multi-agent double Deep-Q-network based on state machine and event stream for flexible job shop scheduling problem

With the industry's rapid shift to large-scale personalized development, the need for a ramose multi-agent system capable of flexible job shop scheduling problem (FJSP) is apparent. As such, this paper proposes a novel framework that transforms the combinatorial optimization problem into a multi-stage sequence decision problem and introduces a multi-agent double Deep-Q-network algorithm (MADDQN) for FJSP. The event-driven workshop environment model based on state machine and event stream mechanism is constructed to transform FJSP into a Markov Decision Process (MDP) with more robust general performance and decouple the workshop environment model from the decision analysis model. The developed multi-agent reinforcement learning consisting of job agents and machine agents applies dispatching rules to maximize the cumulative reward of all agents by Boltzmann exploitation to avoid the local optimum. Results of numerical experiments reflect that the proposed MADDQN performs better than the traditional approaches in large-scale instances, and the real-time performance of the scheduling decision is guaranteed.

Actions, not Actor Identities, Affect Beliefs About Cooperation in Human-Machine Teaming

The efficacy of future human-machine teams will be determined by the ability of machine agents to work cooperatively with their human teammates. The ways teammates engage in cooperative teamwork behaviors might affect human beliefs about expressed cooperativity, and these changes in beliefs might depend on the identity of a teammate as a human or machine. In the current experiment we investigated how violations of cooperation expectations, presented to participants in a narrative vignette, influenced ratings on a measure of cooperation derived from social interdependence theory. As expected, participants rated vignette actors as less cooperative if the story included violated cooperation expectations, but consistent with previous research, the identities of the actors as humans or robots did not appear to influence ratings. Overall, this is a useful expansion on previous work describing the factors that influence perceptions of cooperation in human-machine teaming.

On the Integration of Machine Agents into Live Coding

Co-creation strategies for human–machine collaboration have recently been explored in various creative disciplines and more opportunities for human–machine collaborations are materialising. In this article, we outline how to augment musical live coding by considering how human live coders can effectively collaborate with a machine agent imbued with the ability to produce its own patterns of executable code. Using machine agents allows live coders to explore not-yet conceptualised patterns of code and supports them in asking new questions. We argue that to move away from scenarios where machine agents are used in a merely generative way, or only as creative impetus for the human, and towards a more collaborative relationship with the machine agent, consideration is needed for system designers around the aspects of reflection, aesthetics and evaluation. Furthermore, owing to live coding’s close relationship with exposing processes, using agents in such a way can be a useful manner to explore how to make artificial intelligence processes more open and explainable to an audience. Finally, some speculative futures of co-creative and artificially intelligent systems and what opportunities they might afford the live coder are discussed.