AI Agents Research Articles

Unveiling the Power of Data: A Journey Through Machine Learning Techniques

Machine Learning Techniques: Unveiling the Power of Data-Driven Solutions Machine learning (ML) has become a transformative force across disciplines, revolutionizing how computers extract knowledge from data and tackle complex tasks without explicit programming. This research paper delves into the core principles of machine learning, exploring its fundamental techniques and their far-reaching applications. We embark on a journey through the two main paradigms that govern learning: supervised and unsupervised. Supervised learning equips algorithms to make informed predictions based on labeled data, encompassing tasks like spam filtering, disease diagnosis, and stock market forecasting. The algorithm ingests this data, which includes both the input features and the corresponding desired output, and learns the intricate relationships between them. This empowers it to generalize this knowledge to unseen data, enabling accurate predictions. Unsupervised learning, in contrast, thrives on unlabeled data, where the data points lack predefined categories or target variables. Here, the objective is to uncover the inherent structures or patterns within the data itself. Unsupervised learning techniques are invaluable for tasks like data clustering, where similar data points are grouped together to reveal underlying relationships. Additionally, they play a crucial role in dimensionality reduction, a process of compressing high-dimensional data into a lower-dimensional space for efficient processing and analysis. This is particularly beneficial when dealing with massive datasets that may overwhelm traditional computational methods. Beyond these fundamental paradigms, the landscape of machine learning extends further. Deep learning, a subfield inspired by the structure and function of the human brain, utilizes artificial neural networks with multiple layers. These complex networks are adept at learning intricate patterns from vast amounts of data, achieving remarkable success in areas like computer vision, natural language processing, and speech recognition. Deep learning models, for instance, power facial recognition technology, enabling our devices to identify individuals with exceptional accuracy. In the realm of natural language processing, deep learning algorithms fuel machine translation systems, allowing seamless communication across languages. Reinforcement learning presents another powerful approach. Here, an agent interacts with an environment, learning through trial and error. The agent receives rewards for desired actions and penalties for undesired ones, constantly refining its decision-making process. This approach has applications in robotics, where robots can learn to navigate complex environments and perform tasks with increasing autonomy. Reinforcement learning is also used in game playing, where AI agents can learn optimal strategies through self-play and achieve superhuman levels of performance. As machine learning continues its exponential growth, ethical considerations become paramount. Potential biases within the training data can lead to discriminatory outcomes in areas like loan approvals or facial recognition software. Additionally, the explainability and transparency of complex models, particularly deep learning models, raise concerns. To ensure responsible development and deployment of ML, addressing these issues is crucial. The field requires robust methodologies to mitigate bias in training data and develop techniques that make complex models interpretable, allowing us to understand the rationale behind their predictions. Looking towards the horizon, the future of machine learning is brimming with potential. From personalized healthcare with tailored treatment plans to self-driving cars that navigate roads with exceptional precision, ML promises to revolutionize various aspects of our lives. Continuous research and development will enhance the capabilities and robustness of machine learning models, further expanding their reach and impact. By harnessing the power of data and fostering responsible development practices, machine learning will undoubtedly play a pivotal role in shaping a more intelligent and data-driven future

A multiplayer VR showdown game for people with visual impairment

ABSTRACT Researchers have developed virtual reality (VR) systems for people with visual impairment using various audio feedback techniques. However, few studies have investigated collaborative VR systems where people with and without visual impairment can participate together. We developed a VR Showdown, similar to a real showdown game, where players with and without visual impairment can play together in a remote virtual environment. We incorporated auditory distance perception using a head-related transfer function based on the spatial position of the Showdown ball in VR. We developed two modes of VR Showdown. One is the player vs. AI agent (PvA) mode, where visually impaired people can play alone. The other is the player vs. player (PvP) mode, in which visually impaired people can play with another player in a network. The user study results showed that people with and without visual impairment enjoyed playing VR Showdown games that were well-matched.

Preferences for AI Explanations Based on Cognitive Style and Socio-Cultural Factors

Designing AI systems with the capacity to explain their behaviour is paramount to enable human oversight, facilitate trust, promote acceptance of technology and, ultimately, empower users and improve their experience. There are, however, several challenges to explainable AI, one of which is the generation and selection of explanations from the causal history of a given event. Causal attribution, among other cognitive processes, has been found to be influenced by socio-cultural factors, which suggests that there could be systematic differences in preferences for AI explanations between communities of users according to their cognitive style and socio-cultural characteristics. In this paper, we investigate the relationship between preferences in the explanations provided by belief-desire-intention AI agents, cognitive style (holistic vs analytical), and socio-cultural factors, such as gender, education, social class, and political and religious beliefs. We found a relationship between explanation preference, cognitive style and various socio-cultural characteristics. Holistic cognitive style is associated with preference for goal explanations while analytic cognitive style is associated with preference for belief explanations. Socio-cultural variables that affect explanation preference are gender, religious beliefs, educational attainment, some fields of education, and political party affiliation.

Immersive multi-modal pedagogical conversational artificial intelligence for early childhood education: An exploratory case study in the wild

Educational technology research has found that parents of young children widely share concerns about extended screen time, lack of physical activity, and lack of social interaction. Kid Space was developed to address these concerns by enabling multi-modal and immersive collaborative play-based learning. Kid Space utilizes multiple sensing technologies with an immersive physical space through a human-scale wall projection and incorporates a conversational AI agent to interact with children, understand individual progress, and personalize learning experiences in a blended physical and digital environment. To evaluate Kid Space in the wild, we conducted a multi-method user study involving a quasi-experimental design and exploratory case study with 14 students and three educators in an elementary school. Mixed methods for data collection and analysis were used to understand the students' and educators' perceptions of Kid Space and its impact on the students’ educational outcomes (learning engagement, experience, and performance). The findings showed (1) positive perceptions toward Kid Space, (2) high levels of engagement - with decreased screen time (41% of the time), increased physical activity (99.3% of the time), and increased social interactions with conversational AI agent and the other collaborating student (52% of the time), and (3) significant learning gains after experiencing Kid Space (24% gain, paired t-test: p < 0.01). These positive results are accompanied by critical user insights for improving future iterations of Kid Space to validate long-term educational outcomes.

Survival games for humans and machines

Survival games can be described as video games where the player searches for food and treasures, while avoiding obstacles and hostile attacks. Ms.Pac-Man and Minecraft are two well-known examples. Currently there are AI models that outperform human players at Ms.Pac-Man, while AI models playing Minecraft above the human level has been a long-standing challenge. This paper concerns what we call pure survival games, which take place in previously unseen worlds containing only food, water, and obstacles. The challenge of the player is to navigate and survive in those worlds by continuously finding resources and avoiding obstacles. Arguably, animals need to master physical analogues of pure survival games in order to survive and reproduce. Here we begin to explore human and machine performance on pure survival games. We define two games called the Grid game and the Terrain game and two corresponding AI agents based on deep reinforcement learning: the Grid agent and the Terrain agent. We explore to what extent these agents can match human performance and how their performance is affected by variations in their perception, memory, and reward models. We find that (1) the Terrain agent performs above human level, while the Grid agent performs below human level; (2) the smell, touch, and interoception models contribute significantly to the performance of the Grid agent; (3) the memory model contributes significantly to the performance of the Grid agent; and (4) the performance of the Grid agent is relatively stable under three quite different reward signals, including one that rewards survival and nothing else.

FGeo-DRL: Deductive Reasoning for Geometric Problems through Deep Reinforcement Learning

Human-like automatic deductive reasoning has always been one of the most challenging open problems in the interdisciplinary field of mathematics and artificial intelligence. This paper is the third in a series of our works. We built a neural-symbolic system, named FGeo-DRL, to automatically perform human-like geometric deductive reasoning. The neural part is an AI agent based on deep reinforcement learning, capable of autonomously learning problem-solving methods from the feedback of a formalized environment, without the need for human supervision. It leverages a pre-trained natural language model to establish a policy network for theorem selection and employ Monte Carlo Tree Search for heuristic exploration. The symbolic part is a reinforcement learning environment based on geometry formalization theory and FormalGeo, which models geometric problem solving (GPS) as a Markov Decision Process (MDP). In the formal symbolic system, the symmetry of plane geometric transformations ensures the uniqueness of geometric problems when converted into states. Finally, the known conditions and objectives of the problem form the state space, while the set of theorems forms the action space. Leveraging FGeo-DRL, we have achieved readable and verifiable automated solutions to geometric problems. Experiments conducted on the formalgeo7k dataset have achieved a problem-solving success rate of 86.40%.

ProAgent: Building Proactive Cooperative Agents with Large Language Models

Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems. Current approaches to developing cooperative agents rely primarily on learning-based methods, whose policy generalization depends heavily on the diversity of teammates they interact with during the training phase. Such reliance, however, constrains the agents' capacity for strategic adaptation when cooperating with unfamiliar teammates, which becomes a significant challenge in zero-shot coordination scenarios. To address this challenge, we propose ProAgent, a novel framework that harnesses large language models (LLMs) to create proactive agents capable of dynamically adapting their behavior to enhance cooperation with teammates. ProAgent can analyze the present state, and infer the intentions of teammates from observations. It then updates its beliefs in alignment with the teammates' subsequent actual behaviors. Moreover, ProAgent exhibits a high degree of modularity and interpretability, making it easily integrated into various of coordination scenarios. Experimental evaluations conducted within the Overcooked-AI environment unveil the remarkable performance superiority of ProAgent, outperforming five methods based on self-play and population-based training when cooperating with AI agents. Furthermore, in partnered with human proxy models, its performance exhibits an average improvement exceeding 10% compared to the current state-of-the-art method. For more information about our project, please visit https://pku-proagent.github.io.

NarrativePlay: An Automated System for Crafting Visual Worlds in Novels for Role-Playing

In this demo, we present NarrativePlay -- an innovative system enabling users to role-play a fictional character and interact with dynamically generated narrative environments. Unlike existing predefined sandbox approaches, NarrativePlay centres around the main storyline events extracted from the narrative, allowing users to experience the story from the perspective of a character they chose. To design versatile AI agents for diverse scenarios, we employ a framework built on a Large Language Models (LLMs) to extract detailed character traits from text. We also incorporate automatically generated visual displays of narrative settings, character portraits, and character speech, greatly enhancing the overall user experience.

WebVLN: Vision-and-Language Navigation on Websites

Vision-and-Language Navigation (VLN) task aims to enable AI agents to accurately understand and follow natural language instructions to navigate through real-world environments, ultimately reaching specific target locations. We recognise a promising opportunity to extend VLN to a comparable navigation task that holds substantial significance in our daily lives, albeit within the virtual realm: navigating websites on the Internet. This paper proposes a new task named Vision-and-Language Navigation on Websites (WebVLN), where we use question-based instructions to train an agent, emulating how users naturally browse websites. Unlike the existing VLN task that only pays attention to vision and instruction (language), the WebVLN agent further considers underlying web-specific content like HTML, which could not be seen on the rendered web pages yet contain rich visual and textual information. Toward this goal, we contribute a dataset, WebVLN-v1, and introduce a novel approach called Website-aware VLN Network (WebVLN-Net), which is built upon the foundation of state-of-the-art VLN techniques. Experimental results show that WebVLN-Net outperforms current VLN and web-related navigation methods. We believe that the introduction of the newWebVLN task and its dataset will establish a new dimension within the VLN domain and contribute to the broader vision-and-language research community. Code is available at: https://github.com/WebVLN/WebVLN.

Designing an age-friendly conversational AI agent for mobile banking: the effects of voice modality and lip movement

Age factor is a significant barrier in adopting mobile banking mainly due to low competence in using the system and security concerns. Thus, this study presents a conversational artificial intelligence agent-based mobile banking app that is specially designed for seniors considering their needs and physical characteristics. Voice modality and lip-movement were identified as the important factors to improve seniors’ mobile banking service experience and the effect of them were empirically tested by conducting a controlled experiment with between-subject design. In total, 91 participants who were 55 years or older used the app to complete the given tasks of online banking. Results showed that those in the two voice modality conditions, compared to text agent condition, reported higher levels of trust, perceived ease of use, and social presence. The effect of lip movement of the agent also showed significant results in those three dependant variables. The result of mediation analysis revealed that agent's lip movement increased social presence and perceived ease of use, which further increased trust toward the agent. The results are meaningful especially because trust and perceived ease of use were listed as the main reason for not using online banking services among non-users. The implications of the study were discussed.

A neurosymbolic cognitive architecture framework for handling novelties in open worlds

“Open world” environments are those in which novel objects, agents, events, and more can appear and contradict previous understandings of the environment. This runs counter to the “closed world” assumption used in most AI research, where the environment is assumed to be fully understood and unchanging. The types of environments AI agents can be deployed in are limited by the inability to handle the novelties that occur in open world environments. This paper presents a novel cognitive architecture framework to handle open-world novelties. This framework combines symbolic planning, counterfactual reasoning, reinforcement learning, and deep computer vision to detect and accommodate novelties. We introduce general algorithms for exploring open worlds using inference and machine learning methodologies to facilitate novelty accommodation. The ability to detect and accommodate novelties allows agents built on this framework to successfully complete tasks despite a variety of novel changes to the world. Both the framework components and the entire system are evaluated in Minecraft-like simulated environments. Our results indicate that agents are able to efficiently complete tasks while accommodating “concealed novelties” not shared with the architecture development team.

Walking and Survival AI Using Reinforcement Learning - Simulation

This research paper presents a novel approach to training an AI agent for walking and survival tasks using reinforcement learning (RL) techniques. The primary research question addressed in this study is how to develop an AI system capable of autonomously navigating diverse terrains and environments while ensuring survival through adaptive decision-making. To investigate this question, we employ RL algorithms, specifically deep Q-networks (DQN) and proximal policy optimization (PPO), to train an AI agent in simulated environments that mimic real-world challenges. Our methodology involves designing a virtual environment where the AI agent learns to walk and make survival-related decisions through trial and error. The agent receives rewards or penalties based on its actions, encouraging the development of strategies that optimize both locomotion and survival skills. We evaluate the performance of our approach through extensive experimentation, testing the AI agent's adaptability to various terrains, obstacles, and survival scenarios.

On the Motivations to Seek Information From Artificial Intelligence Agents Versus Humans: A Risk Information Seeking and Processing Perspective

This study investigates how anticipating an artificial intelligence agent versus human information source moderates the risk information seeking and processing model. It focuses on a behavioral proxy of seeking intention—how long a participant waited for an online consultant whose identity was manipulated. In two samples ( N1 = 182 students and N2 = 800 mturkers), the source identity consistently moderated the model in two ways: First, informational subjective norms encouraged seeking from humans but discouraged seeking from AI agents. Second, information insufficiency drove favoritism toward humans–when perceived information-gathering capacity was high. When the capacity was low, AI agents were favored.

Team formation through an assessor: choosing MARL agents in pursuit–evasion games

Team formation in multi-agent systems usually assumes the capabilities of each team member are known, and the best formation can be derived from that information. As AI agents become more sophisticated, this characterisation is becoming more elusive and less predictive about the performance of a team in cooperative or competitive situations. In this paper, we introduce a general and flexible way of anticipating the outcome of a game for any lineups (the agents, sociality regimes and any other hyperparameters for the team). To this purpose, we simply train an assessor using an appropriate team representation and standard machine learning techniques. We illustrate how we can interrogate the assessor to find the best formations in a pursuit–evasion game for several scenarios: offline team formation, where teams have to be decided before the game and not changed afterwards, and online team formation, where teams can see the lineups of the other teams and can be changed at any time.

A research of artificial intelligence game agent application

Currently, large language models are on the rise with breakthrough progress in artificial intelligence. Existing reviews of AI game agents have not covered these latest developments, requiring a combing and analysis of the newest research advancements in game AI agents. This paper summarizes the application scenarios of game AI agents in four aspects: combat AI, Non-Player Character (NPC) interaction, automated testing, and Artificial General Intelligence (AGI) testing. In combat AI, there is a progressive developmental trend, with the introduction of Monte Carlo tree search and reinforcement learning enabling AI game agents to fully surpass humans in traditional board games. In NPC interaction, full AI is unnecessary. Game developers only need to incorporate AI for abilities related to player experience to increase appeal, with controllable generation results. In automated testing, game AI agents lack generalizability for testing so far. In AGI testing, academia has helpfully explored general game AI, but capabilities remain limited to certain games. Introducing large language models to game AI agents shows unprecedented capabilities. Finally, this paper provides an outlook on the hot topics and future directions of this research subject.

MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledge

Solving mechanics problems using numerical methods requires comprehensive intelligent capability of retrieving relevant knowledge and theory, constructing and executing codes, analyzing the results, a task that has thus far mainly been reserved for humans. While emerging AI methods can provide effective approaches to solve end-to-end problems, for instance via the use of deep surrogate models or various data analytics strategies, they often lack physical intuition since knowledge is baked into the parametric complement through training, offering less flexibility when it comes to incorporating mathematical or physical insights. By leveraging diverse capabilities of multiple dynamically interacting large language models (LLMs), we can overcome the limitations of conventional approaches and develop a new class of physics-inspired generative machine learning platform, here referred to as MechAgents. A set of AI agents can solve mechanics tasks, here demonstrated for elasticity problems, via autonomous collaborations. A two-agent team can effectively write, execute and self-correct code, in order to apply finite element methods to solve classical elasticity problems in various flavors (different boundary conditions, domain geometries, meshes, small/finite deformation and linear/hyper-elastic constitutive laws, and others). For more complex tasks, we construct a larger group of agents with enhanced division of labor among planning, formulating, coding, executing and criticizing the process and results. The agents mutually correct each other to improve the overall team-work performance in understanding, formulating and validating the solution. Our framework shows the potential of synergizing the intelligence of language models, the reliability of physics-based modeling, and the dynamic collaborations among diverse agents, opening novel avenues for automation of solving engineering problems.

User Preferences on AI Psychotherapy Based on Moderating Effects of Individual Personality Traits: Employing a Clustering Analysis

The recent appearance of AI psychological counseling services is expected to lower physical, economic, and psychological burdens for individuals seeking psychological counseling and increase participation in the service. AI psychological counseling research is still in its infancy, and therefore it is important to consider client preference to achieve positive effects. However, no research has been conducted to investigate client preferences regarding AI therapists. Therefore, this study explores the characteristics of AI counselors preferred by clients (e.g., rapport, trust, expertise, and attraction), as well as which AI agent therapist characteristics have positive effects on the attitudes of clients (users). This study confirms the characteristics that affect client intention to use AI counseling. Groups were divided using the K-means clustering technique based on the individual’s degree of introversion/extroversion and depression because characteristics desired in AI counselors may differ depending on individual client inclinations and examined differences between groups. The results of this study will help enhance and supplement the capabilities of developers and actual counselors through client-tailored counseling.

ADVISE: Accelerating the Creation of Evidence Syntheses for Global Development Using Natural Language Processing-Supported Human-Artificial Intelligence Collaboration

Abstract When designing evidence-based policies and programs, decision-makers must distill key information from a vast and rapidly growing literature base. Identifying relevant literature from raw search results is time and resource intensive, and is often done by manual screening. In this study, we develop an artificial intelligence (AI) agent based on a bidirectional encoder representations from transformers (BERT) model and incorporate it into a human team designing an evidence synthesis product for global development. We explore the effectiveness of the human–AI hybrid team in accelerating the evidence synthesis process. To further improve team efficiency, we enhance the human–AI hybrid team through active learning (AL). Specifically, we explore different sampling strategies, including random sampling, least confidence (LC) sampling, and highest priority (HP) sampling, to study their influence on the collaborative screening process. Results show that incorporating the BERT-based AI agent into the human team can reduce the human screening effort, i.e., the number of documents that humans need to screen, by 68.5% compared to the case of no AI assistance and by 16.8% compared to the industry-standard case of using a frequency-based language model and support vector machine-based classifier for identifying 80% of all relevant documents. When we apply the HP sampling strategy, the human screening effort can be reduced even more: by 78.3% for identifying 80% of all relevant documents compared to no AI assistance. We apply the AL-enhanced human–AI hybrid teaming workflow in the design process of three evidence gap maps for U.S. Agency for International Development and find it to be highly effective. These findings demonstrate how AI can accelerate the development of evidence synthesis products and promote timely evidence-based decision-making in global development.

Towards a Natural Language Interface for Flexible Multi-Agent Task Assignment

Task assignment and scheduling algorithms are powerful tools for autonomously coordinating large teams of robotic or AI agents. However, the decisions these system make often rely on components designed by domain experts, which can be difficult for non-technical end-users to understand or modify to their own ends. In this paper we propose a preliminary design for a flexible natural language interface for a task assignment system. The goal of our approach is both to grant users more control over a task assignment system's decision process, as well as render these decisions more transparent. Users can direct the task assignment system via natural language commands, which are applied as constraints to a mixed-integer linear program (MILP) using a large language model (LLM). Additionally, our proposed system can alert users to potential issues with their commands, and engage them in a corrective dialogue in order to find a viable solution. We conclude with a description of our planned user-evaluation in the simulated environment Overcooked and describe next steps towards developing a flexible and transparent task allocation system.

Generative Retrieval-Augmented Ontologic Graph and Multiagent Strategies for Interpretive Large Language Model-Based Materials Design.

Transformer neural networks show promising capabilities, in particular for uses in materials analysis, design, and manufacturing, including their capacity to work effectively with human language, symbols, code, and numerical data. Here, we explore the use of large language models (LLMs) as a tool that can support engineering analysis of materials, applied to retrieving key information about subject areas, developing research hypotheses, discovery of mechanistic relationships across disparate areas of knowledge, and writing and executing simulation codes for active knowledge generation based on physical ground truths. Moreover, when used as sets of AI agents with specific features, capabilities, and instructions, LLMs can provide powerful problem-solution strategies for applications in analysis and design problems. Our experiments focus on using a fine-tuned model, MechGPT, developed based on training data in the mechanics of materials domain. We first affirm how fine-tuning endows LLMs with a reasonable understanding of subject area knowledge. However, when queried outside the context of learned matter, LLMs can have difficulty recalling correct information and may hallucinate. We show how this can be addressed using retrieval-augmented Ontological Knowledge Graph strategies. The graph-based strategy helps us not only to discern how the model understands what concepts are important but also how they are related, which significantly improves generative performance and also naturally allows for injection of new and augmented data sources into generative AI algorithms. We find that the additional feature of relatedness provides advantages over regular retrieval augmentation approaches and not only improves LLM performance but also provides mechanistic insights for exploration of a material design process. Illustrated for a use case of relating distinct areas of knowledge, here, music and proteins, such strategies can also provide an interpretable graph structure with rich information at the node, edge, and subgraph level that provides specific insights into mechanisms and relationships. We discuss other approaches to improve generative qualities, including nonlinear sampling strategies and agent-based modeling that offer enhancements over single-shot generations, whereby LLMs are used to both generate content and assess content against an objective target. Examples provided include complex question answering, code generation, and execution in the context of automated force-field development from actively learned density functional theory (DFT) modeling and data analysis.