Autonomous Agents Research Articles

A Social Group Chatbot System by Multiple Topics Tracking and Atkinson‐Shiffrin Memory Model Using AI Agents Collaboration

ABSTRACTThe widespread use of Internet has accelerated the explosive growth of data, which in turn leads to information overload and information confusion. This makes it difficult for us to communicate effectively in social groups, thereby intensifying the demands for emotional companionship. Therefore, we propose a novel social group chatting framework based on Large Language Model (LLM) powered multiple autonomous agents collaboration in this article. Specifically, BERTopic is used to extract topics from history chatting content for each social group everyday, and then multiple topics tracking is realised through multi‐level association by adaptive time sliding‐window mechanism and optimal matching. Furthermore, we use topic tracking architecture and prompts to design and implement an AI Chatbot system with different characters that can conduct natural language conversations with users in online social group. LLM, as the controller and coordinator of the whole AI Chatbot for sub‐tasks, allows different AI Agents to autonomously decide whether to participate in current topic, how to generate response, and whether to propose a new topic. Each AI Agent has their own multi‐store memory system based on the Atkinson‐Shiffrin model. Finally, we construct a verification environment based on online game that is consistent with real society. Subjective and objective evaluation methods were deployed to perform qualitative and quantitative analyses to demonstrate the performance of our AI Chatbot system.

NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila.

Discovering principles underlying the control of animal behavior requires a tight dialogue between experiments and neuromechanical models. Such models have primarily been used to investigate motor control with less emphasis on how the brain and motor systems work together during hierarchical sensorimotor control. NeuroMechFly v2 expands Drosophila neuromechanical modeling by enabling vision, olfaction, ascending motor feedback and complex terrains that can be navigated using leg adhesion. We illustrate its capabilities by constructing biologically inspired controllers that use ascending feedback to perform path integration and head stabilization. After adding vision and olfaction, we train a controller using reinforcement learning to perform a multimodal navigation task. Finally, we illustrate more bio-realistic modeling involving complex odor plume navigation, and fly-fly following using a connectome-constrained visual network. NeuroMechFly can be used to accelerate the discovery of explanatory models of the nervous system and to develop machine learning-based controllers for autonomous artificial agents and robots.

Robotic Warehousing Operations: A Learn-Then-Optimize Approach to Large-Scale Neighborhood Search

The rapid deployment of robotics technologies requires dedicated optimization algorithms to manage large fleets of autonomous agents. This paper supports robotic parts-to-picker operations in warehousing by optimizing order–workstation assignments, item–pod assignments, and the schedule of order fulfillment at workstations. The model maximizes throughput, managing human workload at the workstations and congestion in the facility. We solve it via large-scale neighborhood search with a novel learn-then-optimize approach to subproblem generation. The algorithm relies on an off-line machine learning procedure to predict objective improvements based on subproblem features and an online optimization model to generate a new subproblem at each iteration. In collaboration with Amazon Robotics, we show that our model and algorithm generate much stronger solutions for practical problems than state-of-the-art approaches. In particular, our solution enhances the utilization of robotic fleets by coordinating robotic tasks for human operators to pick multiple items at once and by coordinating robotic routes to avoid congestion in the facility. Funding: This research was partially funded by Amazon.com Services LLC under award number 2D-12552417. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoo.2024.0033 .

The Use of Generative Artificial Intelligence for Business Decision-Making

The purpose of this study was to develop autonomous artificial intelligence agents capable of working as a cohesive group to solve intricate decision-making processes, ultimately pushing the boundaries of artificial intelligence’s applicability in high-stakes and real-world scenarios. An autonomous multi-agent model was designed and deployed using the Analytic Hierarchy Process (AHP) decision-making framework, with autonomous AI agents, such as CrewAI and AutoGen, orchestrating multiple stages of activities performed by four distinct agents leveraging generative AI (GenAI). These agents were applied to a use case for proof of concept. The agents used a qualitative dataset to generate management recommendations based on cost and market feasibility considerations. The model was tested with publicly available customer feedback data on oatmeal cookies. It synthesized 913 customer reviews, identified common complaints, provided a summary of potential solutions, and generated a summary of market opportunities, along with relevant challenges. Although a subject matter expert's review is necessary to evaluate the practicality and relevance of the recommendations, the results demonstrated the high potential of multi-agent models in synthesizing and distilling large datasets into actionable insights, thereby augmenting decision-making processes in business contexts. The model's modular design allows for enhancements in quality, accuracy, and practicality by incorporating additional datasets and new agents. This study underscores that the integration of GenAI within a multi-agent model empowers businesses to swiftly transform vast amounts of business intelligence data into practical recommendations.

Multi-agent reinforcement learning with synchronized and decomposed reward automaton synthesized from reactive temporal logic

Multi-agent systems (MAS) consist of multiple autonomous agents interacting to achieve collective objectives. Multi-agent reinforcement learning (MARL) enhances these systems by enabling agents to learn optimal behaviors through interaction, thus improving their coordination in dynamic environments. However, MARL faces significant challenges in adapting to complex dependencies on past states and actions, which are not adequately represented by the current state alone in reactive systems. This paper addresses these challenges by considering MAS operating under task specifications formulated as Generalized Reactivity of rank 1 (GR(1)). These synthesized strategies are used as a priori knowledge to guide the learning. To tackle the difficulties of handling non-Markovian tasks in reactive systems, we propose a novel synchronized decentralized training paradigm that guides agents to learn within the MARL framework using a reward structure constructed from decomposed synthesized strategies of GR(1). We initially formalize the synthesis of GR(1) strategies as a reachability problem of winning states of the system. Subsequently, we develop a decomposition mechanism that constructs individual reward structures for decentralized MARL, incorporating potential values calculated through value iteration. Theoretical proofs are provided to verify that the safety and liveness properties are preserved. We evaluate our approach against other state-of-the-art methods under various GR(1) specifications and scenario maps, demonstrating superior learning efficacy and optimal rewards per episode. Additionally, we show that the decentralized training paradigm outperforms the centralized training paradigm. The value iteration strategy used to calculate potential values for the reward structure is compared against two other strategies, showcasing its advantages.

Exploring action-oriented models via active inference for autonomous vehicles

Being able to robustly interact with and navigate a dynamic environment has been a long-standing challenge in intelligent transportation systems. Autonomous agents can use models that mimic the human brain to learn how to respond to other participants’ actions in the environment and proactively coordinate with the dynamics. Modeling brain learning procedures is challenging for multiple reasons, such as stochasticity, multimodality, and unobservant intents. Active inference may be defined as the Bayesian modeling of a brain with a biologically plausible model of the agent. Its primary idea relies on the free energy principle and the prior preference of the agent. It enables the agent to choose an action that leads to its preferred future observations. An exploring action-oriented model is introduced to address the inference complexity and solve the exploration–exploitation dilemma in unobserved environments. It is conducted by adapting active inference to an imitation learning approach and finding a theoretical connection between them. We present a multimodal self-awareness architecture for autonomous driving systems where the proposed techniques are evaluated on their ability to model proper driving behavior. Experimental results provide the basis for the intelligent driving system to make more human-like decisions and improve agent performance to avoid a collision.

Formal contracts mitigate social dilemmas in multi-agent reinforcement learning

Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group incentives diverge. Humans are remarkably capable at solving these social dilemmas. It is an open problem in MARL to replicate such cooperative behaviors in selfish agents. In this work, we draw upon the idea of formal contracting from economics to overcome diverging incentives between agents in MARL. We propose an augmentation to a Markov game where agents voluntarily agree to binding transfers of reward, under pre-specified conditions. Our contributions are theoretical and empirical. First, we show that this augmentation makes all subgame-perfect equilibria of all Fully Observable Markov Games exhibit socially optimal behavior, given a sufficiently rich space of contracts. Next, we show that for general contract spaces, and even under partial observability, richer contract spaces lead to higher welfare. Hence, contract space design solves an exploration-exploitation tradeoff, sidestepping incentive issues. We complement our theoretical analysis with experiments. Issues of exploration in the contracting augmentation are mitigated using a training methodology inspired by multi-objective reinforcement learning: Multi-Objective Contract Augmentation Learning. We test our methodology in static, single-move games, as well as dynamic domains that simulate traffic, pollution management, and common pool resource management.

Automation of time series analysis of Google trends data in R studio using autonomous AI agents

Automation of time series analysis of Google trends data in R studio using autonomous AI agents

Merging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Powered Autonomous Agents

Merging Paradigms: The Synergy of Symbolic and Connectionist AI in LLM-Powered Autonomous Agents

Adaptive Multi-Agent Reinforcement Learning for Optimizing Dynamic Electric Vehicle Charging Networks in Thailand

The rapid growth of electric vehicles (EVs) necessitates efficient management of dynamic EV charging networks to optimize resource utilization and enhance service reliability. This paper explores the application of adaptive multi-agent reinforcement learning (MARL) to address the complexities of EV charging infrastructure in Thailand. By employing MARL, multiple autonomous agents learn to optimize charging strategies based on real-time data by adapting to fluctuating demand and varying electricity prices. Building upon previous research that applied MARL to static network configurations, this study extends the application to dynamic and real-world scenarios, integrating real-time data to refine agent learning processes and also evaluating the effectiveness of adaptive MARL in maximizing rewards and improving operational efficiency compared to traditional methods. Experimental results indicate that MARL-based strategies increased efficiency by 20% and reduced energy costs by 15% relative to conventional algorithms. Key findings demonstrate the potential of extending MARL in transforming EV charging network management, highlighting its benefits for stakeholders, including EV owners, operators, and utility providers. This research contributes insights into advancing electric mobility and energy management in Thailand through innovative AI-driven approaches. The implications of this study include significant improvements in the reliability and cost-effectiveness of EV charging networks, fostering greater adoption of electric vehicles and supporting sustainable energy initiatives. Future research directions include enhancing MARL adaptability and scalability as well as integrating predictive analytics for proactive network optimization and sustainability. These advancements promise to further refine the efficacy of EV charging networks, ensuring that they meet the growing demands of Thailand’s evolving electric mobility landscape.

PenGym: Realistic training environment for reinforcement learning pentesting agents

Penetration testing, or pentesting, refers to assessing network system security by trying to identify and exploit any existing vulnerabilities. Reinforcement Learning (RL) has recently become an effective method for creating autonomous pentesting agents. However, RL agents are typically trained in a simulated network environment. This can be challenging when deploying them in a real network infrastructure due to the lack of realism of the simulation-trained agents.In this paper, we present PenGym, a framework for training pentesting RL agents in realistic network environments. The most significant features of PenGym are its support for real pentesting actions, full automation of the network environment creation, and good execution performance. The results of our experiments demonstrated the advantages and effectiveness of using PenGym as a realistic training environment in comparison with a simulation approach (NASim). For the largest scenario, agents trained in the original NASim environment behaved poorly when tested in a real environment, having a high failure rate. In contrast, agents trained in PenGym successfully reached the pentesting goal in all our trials. Even after fixing logical modeling issues in simulation to create the revised version NASim(rev.), experiment results with the largest scenario indicated that agents trained in PenGym slightly outperformed, and were more stable, than those trained in NASim(rev.). Thus, the average number of steps required to reach the pentesting goal was 1.4 to 8 steps better for PenGym. Consequently, PenGym provides a reliable and realistic training environment for pentesting RL agents, eliminating the need to model agent actions via simulation.

No Laughing Matter: How Autonomous Agents Can Improve Well-Being and Teamwork

Autonomous agents are a critical factor for maintaining the well-being and health of astronaut teams in long duration spaceflight. As autonomous agents become more independent and interdependent the need to understand how they can interact effectively with humans not only becomes more necessary but will become the paradigm for future team research. In our paper, we describe a conceptual model that explains how autonomous agents and human team members can learn to implicitly coordinate through humorous social interaction and effective task interdependence. We argue that to build a shared mental model between autonomous agents and human team members, HATs need to develop trust with one another, albeit through different mechanisms, to work together effectively.

Specification, validation and verification of social, legal, ethical, empathetic and cultural requirements for autonomous agents

Autonomous agents are increasingly being proposed for use in healthcare, assistive care, education, and other applications governed by complex human-centric norms. To ensure compliance with these norms, the rules they induce need to be unambiguously defined, checked for consistency, and used to verify the agent. In this paper, we introduce a framework for formal specification, validation and verification of social, legal, ethical, empathetic and cultural (SLEEC) rules for autonomous agents. Our framework comprises: (i) a language for specifying SLEEC rules and rule defeaters (that is, circumstances in which a rule does not apply or an alternative form of the rule is required); (ii) a formal semantics (defined in the process algebra tock-CSP) for the language; and (iii) methods for detecting conflicts and redundancy within a set of rules, and for verifying the compliance of an autonomous agent with such rules. We show the applicability of our framework for two autonomous agents from different domains: a firefighter UAV, and an assistive-dressing robot.

Joint antenna selection and resource allocation for mm‐wave directional D2D communications using distributed deep reinforcement learning

AbstractIn this paper, with the promising assumption of using adaptive directional microstrip antenna on user equipment, the problem of joint antenna selection, spectrum assignment, and transmit power allocation for mm‐wave device‐to‐device communications underlying cellular networks is tackled, with the goal of enhancing system throughput and energy efficiency. To address the complexity of this problem, a method based on multi‐agent distributed deep reinforcement learning in which an autonomous intelligent agent is deployed for each user equipment is proposed. The performance evaluation demonstrates its superiority over existing strategies, resulting in improved system performance, reduced outage probability, and enhanced energy efficiency.

ChatEDA: A Large Language Model Powered Autonomous Agent for EDA

ChatEDA: A Large Language Model Powered Autonomous Agent for EDA

Enhancing Autonomous Agents Through Narrative Memory: A Comprehensive Framework for Improved Performance and Interaction

The process of organizing information and experiences into a narrative style is known as narrative memory, and it presents a cutting-edge method for improving the performance of autonomous agents. This essay describes the idea of narrative memory, as well as its elements, construction of narrative structures, and uses. The four main elements of narrative memory are timelines, characters, events, and context. Events are important things that happen, such interactions or acts, and characters are people, other agents, and the autonomous agent. The context gives the underlying knowledge required to comprehend these events and individuals, while the chronology sets the order of occurrences, creating a cause-and-effect link. The process of creating narrative frameworks includes gathering data, identifying events, crafting stories, and refreshing memories. Three methods are used to collect data: external sources, user input, and sensors. This data is used to identify significant events, which are subsequently arranged into logical tales. New situations and experiences are constantly added to the narrative memory. Improved predictive analysis, easier learning and adaptability, and improved human-robot interaction are just a few of the uses of narrative memory in autonomous agents. Robots can interact with people more successfully if they can comprehend human narratives and reply with responses that are contextually relevant. Agents may learn and adapt by identifying patterns in their prior experiences, and they can make better decisions by forecasting future occurrences with the help of historical narratives. Managing the complexity of real-world events, addressing ambiguity and uncertainty, and guaranteeing the ethical use of narrative data are challenges in the implementation of narrative memory. In order to improve narrative memory frameworks, future research options include creating sophisticated story generating tools, integrating narrative memory with other memory systems, and working across disciplines. The goal of this roadmap is to enhance autonomous systems' comprehension, communication, and decision-making skills by utilizing the power of tales.

A Fault-Tolerant Multi-Agent Reinforcement Learning Framework for Unmanned Aerial Vehicles–Unmanned Ground Vehicle Coverage Path Planning

In this paper, we introduce a fault-tolerant multi-agent reinforcement learning framework called SERT-DQN to optimize the operations of UAVs with UGV central control in coverage path planning missions. Our approach leverages dual learning systems that combine individual agent autonomy with centralized strategic planning, thus enhancing the efficiency of cooperative path planning missions. This framework is designed for high performance in environments with fault uncertainty detected and operational challenges such as interruptions in connectivity and compromised sensor reliability. With the integration of an innovative communication system between agents, our system appropriately handles both static and dynamic environments. Also, we introduce similarity-based shared experience replay to attain faster convergence and sample efficiency in the multi-agent system. The architecture is specially designed to respond adaptively to such irregularities by effectively showing enhanced resilience in scenarios where data integrity is impaired due to faults or the UAV faces disruptions. Simulation results indicate that our fault tolerance algorithms are very resilient and do indeed improve mission outcomes, especially under dynamic and highly uncertain operating conditions. This approach becomes critical for the most recent sensor-based research in autonomous systems.

Utilizing ChatGPT for On-the-Fly Decision-Making in Autonomous Systems

In the realm of autonomous systems, rapid and efficient decision-making is critical to ensure optimal performance and adaptability in dynamic environments. This paper explores the application of ChatGPT, a large-scale language model, in on-the-fly decision-making within autonomous systems. By leveraging ChatGPT's capabilities in natural language processing, contextual understanding, and real-time response generation, the study demonstrates how it can support autonomous agents in interpreting situations, generating actionable insights, and making informed decisions. The integration of ChatGPT into autonomous systems provides a flexible and scalable approach to improving decision accuracy and response times in unpredictable scenarios. This research investigates the benefits, challenges, and potential use cases, highlighting the model's contributions to enhancing autonomy in various fields such as robotics, self-driving vehicles, and intelligent drones.

How optimal is the minimum-time manoeuvre of an artificial race driver?

Minimum-lap-time optimal control problems (MLT-OCPs) are a popular tool to assess the best lap time of a vehicle on a racetrack. However, MLT-OCPs with high-fidelity dynamic vehicle models are computationally expensive, which limits them to offline use. When using autonomous agents in place of an MLT-OCP for online trajectory planning and control, the question arises of how far the resulting manoeuvre is from the maximum performance. In this paper, we improve a recently proposed artificial race driver (ARD) for online trajectory planning and control, and we compare it with a benchmark MLT-OCP. The novel challenge of our study is that ARD controls the same high-fidelity vehicle model used by the benchmark MLT-OCP, which enables a direct comparison of ARD and MLT-OCP. Leveraging its physics-driven structure and a new formulation of the g-g-v performance constraint, ARD achieves lap times comparable to the offline MLT-OCP (few milliseconds difference). We analyse the different trajectories resulting from the ARD and MLT-OCP solutions, to understand how ARD minimises the effect of local execution errors in search of the minimum-lap-time. Finally, we show that ARD consistently maintains its performance when tested on unseen circuits, even with unmodelled changes in the vehicle's mass.

Scaling Learning based Policy Optimization for Temporal Logic Tasks by Controller Network Dropout

This paper introduces a model-based approach for training feedback controllers for an autonomous agent operating in a highly nonlinear (albeit deterministic) environment. We desire the trained policy to ensure that the agent satisfies specific task objectives and safety constraints, both expressed in Discrete-Time Signal Temporal Logic (DT-STL). One advantage for reformulation of a task via formal frameworks, like DT-STL, is that it permits quantitative satisfaction semantics. In other words, given a trajectory and a DT-STL formula, we can compute the robustness , which can be interpreted as an approximate signed distance between the trajectory and the set of trajectories satisfying the formula. We utilize feedback control, and we assume a feed forward neural network for learning the feedback controller. We show how this learning problem is similar to training recurrent neural networks (RNNs), where the number of recurrent units is proportional to the temporal horizon of the agent's task objectives. This poses a challenge: RNNs are susceptible to vanishing and exploding gradients, and naïve gradient descent-based strategies to solve long-horizon task objectives thus suffer from the same problems. To tackle this challenge, we introduce a novel gradient approximation algorithm based on the idea of dropout or gradient sampling. One of the main contributions is the notion of controller network dropout , where we approximate the NN controller in several time-steps in the task horizon by the control input obtained using the controller in a previous training step. We show that our control synthesis methodology, can be quite helpful for stochastic gradient descent to converge with less numerical issues, enabling scalable backpropagation over long time horizons and trajectories over high dimensional state spaces. We demonstrate the efficacy of our approach on various motion planning applications requiring complex spatio-temporal and sequential tasks ranging over thousands of time-steps.