Agent Behavior Model Research Articles

Intelligent Agent and NPC Behavior Modeling: From Traditional Methods to AI Driven Interactive Game Design

In recent years, artificial intelligence (AI) related technologies have been widely applied in the gaming industry, bringing enormous commercial value. Research has shown that non-player characters (NPCs) embedded with AI technology are more intelligent, popular among players, and can make them feel challenged and amused. Major game companies have established their own AI studios to invest energy in designing more intelligent NPCs, making the NPCs deployed in games increasingly superior. However, there is still a lack of comprehensive induction and summary of the behavior modeling design methods for NPCs. Therefore, this article focuses on providing a more comprehensive summary of this field, by searching and comparing relevant examples in existing game and research results of NPC design, summarizing and generalizing different types of methods, then analyzing and comparing the advantages and disadvantages of traditional algorithms and AI algorithms, as well as their usage scenarios, and proposing potential methods for the future in the end.

Shared Protentions in Multi-Agent Active Inference.

In this paper, we unite concepts from Husserlian phenomenology, the active inference framework in theoretical biology, and category theory in mathematics to develop a comprehensive framework for understanding social action premised on shared goals. We begin with an overview of Husserlian phenomenology, focusing on aspects of inner time-consciousness, namely, retention, primal impression, and protention. We then review active inference as a formal approach to modeling agent behavior based on variational (approximate Bayesian) inference. Expanding upon Husserl's model of time consciousness, we consider collective goal-directed behavior, emphasizing shared protentions among agents and their connection to the shared generative models of active inference. This integrated framework aims to formalize shared goals in terms of shared protentions, and thereby shed light on the emergence of group intentionality. Building on this foundation, we incorporate mathematical tools from category theory, in particular, sheaf and topos theory, to furnish a mathematical image of individual and group interactions within a stochastic environment. Specifically, we employ morphisms between polynomial representations of individual agent models, allowing predictions not only of their own behaviors but also those of other agents and environmental responses. Sheaf and topos theory facilitates the construction of coherent agent worldviews and provides a way of representing consensus or shared understanding. We explore the emergence of shared protentions, bridging the phenomenology of temporal structure, multi-agent active inference systems, and category theory. Shared protentions are highlighted as pivotal for coordination and achieving common objectives. We conclude by acknowledging the intricacies stemming from stochastic systems and uncertainties in realizing shared goals.

Agent behavior modeling method based on reinforcement learning and human in the loop

Computer generated force (CGF) is one of the increasingly important research topics in the field of simulation. However, low modeling efficiency and lack of adaptability are acute problems of traditional CGF modeling. In this study, a method for modeling the agent behavior based on reinforcement learning and human in the loop is proposed to improve the ability and efficiency of agent behavior modeling. First, an overall framework for modeling the behavior of intelligent agents is constructed based on the deep reinforcement learning algorithm Soft Actor Critic (SAC) framework. Second, in order to overcome the slow convergence speed of the SAC framework, a method for human interaction and value evaluation in the loop is introduced, and the specific algorithm flow is designed. Third, in order to verify the performance of the proposed method, experiments are conducted and compared with algorithms using a pure SAC framework based on an example of agent completing specific tasks. Result shows that after 100 episodes of training, the task completion rate of the agent can approach 100% while a pure SAC framework require at least 500 episodes of training to gradually improve the completion rate. Finally, the results demonstrate that the proposed method can significantly improve the efficiency of agent behavior modeling and the task completion rate increases with the number of human interventions in the loop.

Modeling Voters in Multi-Winner Approval Voting

In many real world situations, collective decisions are made using voting and, in scenarios such as committee or board elections, employing voting rules that return multiple winners. In multi-winner approval voting (AV), an agent submits a ballot consisting of approvals for as many candidates as they wish, and winners are chosen by tallying up the votes and choosing the top-k candidates receiving the most approvals. In many scenarios, an agent may manipulate the ballot they submit in order to achieve a better outcome by voting in a way that does not reflect their true preferences. In complex and uncertain situations, agents may use heuristics instead of incurring the additional effort required to compute the manipulation which most favors them. In this paper, we examine voting behavior in single-winner and multi-winner approval voting scenarios with varying degrees of uncertainty using behavioral data obtained from Mechanical Turk. We find that people generally manipulate their vote to obtain a better outcome, but often do not identify the optimal manipulation. There are a number of predictive models of agent behavior in the social choice and psychology literature that are based on cognitively plausible heuristic strategies. We show that the existing approaches do not adequately model our real-world data. We propose a novel model that takes into account the size of the winning set and human cognitive constraints; and demonstrate that this model is more effective at capturing real-world behaviors in multi-winner approval voting scenarios.

A Comparative Study on the Impact of One-Way and Two-Way Matching Strategies on the Evolution of Cloud Manufacturing Ecosystems

The supply-demand matching (SDM) strategy is an important part of the transaction mechanism design of cloud manufacturing (CMfg) platforms, which has a significant impact on the evolution trend of cloud manufacturing ecosystems (CMEs). To help CMfg platform operators choose the appropriate SDM strategy, first, the evolution process of the CME was qualitatively analyzed, and the evolution process was divided into three stages: the germination period, the growth period and the stable period. Then, three types of market agent behavior models, service demanders (SDs), service providers (SPs) and platform operators (POs), were established, and a multiagent behavior simulation experiment was conducted. Finally, the evolution of CMEs with one-way and two-way SDM strategies for POs was compared and analyzed from three aspects: the overall utilization rate of SPs, the diversity of CMEs and the total output of CMEs. Simulation experiments show that, compared with the CME that adopts the one-way SDM strategy, the CME that adopts the two-way SDM strategy is approximately 33% faster to reach ecological balance, the overall utilization rate is approximately 98.7% higher, the diversity is approximately 6% higher, and the total output is approximately 91% higher. The two-way SDM strategy that comprehensively considers the respective preferences of SDs and SPs is more conducive to the healthy development of CMEs.

Mathematical Definitions of Scene and Scenario for Analysis of Automated Driving Systems in Mixed-Traffic Simulations

This paper introduces a unified mathematical definition for describing commonly used terms encountered in systematical analysis of automated driving systems in mixed-traffic simulations. The most significant contribution of this work is in translating the terms that are clarified previously in literature into a mathematical set and function based format. Our work can be seen as an incremental step towards further formalisation of Domain-Specific-Language (DSL) for scenario representation. We also extended the previous work in the literature to allow more complex scenarios by expanding the model-incompliant information using set-theory to represent the perception capacity of the road-user agents. With this dynamic perception definition, we also support interactive scenarios and are not limited to reactive and pre-defined agent behavior. Our main focus is to give a framework to represent realistic road-user behavior to be used in simulation or computational tool to examine interaction patterns in mixed-traffic conditions. We believe that, by formalising the verbose definitions and extending theprevious work in DSL, we can support automatic scenario generation and dynamic/evolving agent behavior models for simulating mixed traffic situations and scenarios. In addition, we can obtain scenarios that are realistic but also can represent rare-conditions that are difficult to extract from field-tests and real driving data repositories.

Numerical simulation of the fire emergency evacuation for a metro platform accident

A coupled analysis of agent behavior and Computational Fluid Dynamics (CFD) model is applied to investigate the fire evacuation effectiveness in a popular metro station in Guangzhou, China. Due to the high density and complexity of traffic, the concept of Required Safe Escape Time and Available Safe Escape Time (RSET/ASET), which is more flexible and adaptable than the “6 minutes” principle, is applied in the safety assessment of fire evacuation. To pursue a stable simulation of the coupled model, the standard Critical Radiant Flux is used to deter the tenability criteria for exposure to fire and heat. Various related factors, including the fire location, the Heat Release Rate (HRR) of fire, the crowd density, and the operation mode of escalators, are analyzed through a series of simulations. Results indicate that the interaction between fire and humans should not be neglected in the evacuation simulation. Both the fire location and the crowd density have a significant effect on the evacuation, while the HRR of fire has a minor impact. When the accident happens at the entrance of an escalator, RSET is 58.3% longer than that when the accident occurs in the middle of the platform. RSET grows with the increase of the crowd density linearly. Besides, the evacuation efficiency could be partly improved by changing escalators that usually operate in the descending mode into ascending mode.

Contextual Recurrent Predictive Model for Long-Term Intent Prediction of Vulnerable Road Users

Recently, the problem of intent and trajectory prediction of vulnerable road users (VRUs) in urban traffic environments has got some attention from the intelligent transportation research community. One of the main challenges that make this problem even harder is the uncertainty exists in the actions of pedestrians in urban traffic environments, as well as the difficulty in inferring their end goals. In this paper, we are proposing a data-driven framework based on inverse reinforcement learning (IRL) and the bidirectional recurrent neural network architecture (B-LSTM) for long-term prediction of VRUs' intention. We evaluated our framework on real-life datasets for agent behavior modeling in traffic environments, and it has achieved an overall average displacement error of only 2.93 and 4.12 pixels over 2.0 and 3.0 s ahead prediction horizons, respectively. In addition, we compared our framework against other baseline models based on sequence prediction models and planning-based approaches. We have outperformed these approaches with the lowest margin of average displacement error of more than 5 pixels. Furthermore, the performance of the proposed framework was evaluated on an additional vehicle-based video sequence dataset for path prediction of pedestrians and it continued to achieve robust results with higher generalization capabilities.

МЕТОД АДАПТАЦІЇ ПОВЕДІНКИ АГЕНТІВ В ІНТЕЛЕКТУАЛЬНІЙ НАВЧАЛЬНІЙ СИСТЕМІ ПІДГОТОВКИ ДИСПЕТЧЕРІВ УПРАВЛІННЯ ПОВІТРЯНИМ РУХОМ

В рамках агентно-орієнтованого підходу з метою підвищення варіативності середовища підготовки диспетчерів управління повітряним рухом у статті удосконалено метод адаптації поведінки агентів навчального середовища інтелектуальної навчальної системи, що ґрунтується на знаннях про результати взаємодії агентів. На етапі внесення змін до параметрів підсистеми планування при настоюванні систем моделювання поведінки агентів запропоновано використовувати механізм самонастроювання. Для управління процесом самонастроювання поведінки інтелектуальних агентів в статті розроблено інструментарій адаптації системи планування поведінки агента. Застосування моделей поведінки агентів із методами планування дозволить збільшити показник успішності реалізації очікуваної поведінки агентів при побудові інтелектуальних навчальних систем. Запропонований у статті підхід адаптації поведінки агентів навчального середовища інтелектуальної навчальної системи дозволить підвищити його варіативність. Як наслідок, така інтелектуальна навчальна система дозволить вивести на якісно новий рівень підготовку диспетчерів управління повітряним

Balancing Agent Retention and Waiting Time in Service Platforms

In many service industries the speed of service and support by experienced employees are two major drivers of service quality. When demand for a service is variable and the staffing requirements cannot be adjusted quickly, choosing capacity levels requires making a trade-off between service speed and operating costs both of which depend on worker utilization. However, recent business models have enabled service systems to access a large pool of employees with flexible working hours that are compensated through piece-rates. While this business model can operate at low levels of utilization without increasing operating costs, a different trade-off emerges: the service platform must control employee turnover, which may increase when employees are working at low levels of utilization. Hence, to make staffing decisions and manage workload, it is necessary to understand both customer conversion and employee retention, measuring their sensitivity to service time and workload, respectively. In our application, we study an outbound call-center that operates with a pool of flexible agents working remotely to sell auto insurance. We develop an econometric approach to model customer behavior that captures two key features of outbound calls: time-sensitivity and employee heterogeneity. We find a strong impact of waiting time on customer behavior: conversion rates drop by 33% when the time to make the first outbound call increases from 5 to 30 minutes. In addition, we use a survival model to measure how agent retention is affected by the assigned workload and find that a 10% increase in workload translates into a 25% percentage decrease in weekly agent attrition. These empirical models of customer and agent behavior are combined to illustrate how to balance customer conversion and employee retention, showing that both are relevant to plan staffing and allocate workload in the context of an on-demand service platform.

Choice sets for spatial discrete choice models in data rich environments

Failure to properly specify an agent's choice set in discrete choice models will generate biased parameter estimates resulting in inaccurate behavioral predictions as well as biased estimates of policy relevant metrics. We propose a method of constructing choice sets by sampling from specific points in space to model agent behavior when choice alternatives are unknown to the researcher, potentially infinite, and differ according to spatial and temporal factors. Using Monte Carlo analysis we compare the performance of this point-based sampling method to the commonly used approach of spatially aggregating choice alternatives. We then apply these alternative approaches to modelling location choice in the Pacific groundfish trawl fishery which has a complex spatial choice structure. Both the Monte Carlo and application results provide considerable support for the efficacy of the point-based approaches.

Developing an advanced classifier of threat for security agent behavior models

Developing an advanced classifier of threat for security agent behavior models

Computational Behavioral Models for Public Goods Games on Social Networks

Cooperation is a fundamental aspect of well-organized societies and public good games are a useful metaphor for modeling cooperative behavior in the presence of strong incentives to free ride. Usually, social agents interact to play a public good game through network structures. Here, we use social network structures and computational agent rules inspired by recent experimental work in order to develop models of agent behavior playing public goods games. The results of our numerical simulations based on a couple of simple models show that agents behave in a manner qualitatively similar to what has been observed experimentally. Computational models such as those presented here are very useful to interpret observed behavior and to enhance computationally the limited variation that is possible in the experimental domain. By assuming a priori reasonable individual behaviors, the easiness of running simulations could also facilitate exploration prior to any experimental work in order to vary and estimate a number of key parameters that would be very difficult, if not impossible, to change during the actual experiment.

Large Scale Learning of Agent Rationality in Two-Player Zero-Sum Games

With the recent advances in solving large, zero-sum extensive form games, there is a growing interest in the inverse problem of inferring underlying game parameters given only access to agent actions. Although a recent work provides a powerful differentiable end-to-end learning frameworks which embed a game solver within a deep-learning framework, allowing unknown game parameters to be learned via backpropagation, this framework faces significant limitations when applied to boundedly rational human agents and large scale problems, leading to poor practicality. In this paper, we address these limitations and propose a framework that is applicable for more practical settings. First, seeking to learn the rationality of human agents in complex two-player zero-sum games, we draw upon well-known ideas in decision theory to obtain a concise and interpretable agent behavior model, and derive solvers and gradients for end-to-end learning. Second, to scale up to large, real-world scenarios, we propose an efficient first-order primal-dual method which exploits the structure of extensive-form games, yielding significantly faster computation for both game solving and gradient computation. When tested on randomly generated games, we report speedups of orders of magnitude over previous approaches. We also demonstrate the effectiveness of our model on both real-world one-player settings and synthetic data.

Modeling the behavior of a hierarchy of command agents with context-based reasoning

Context-based reasoning is a paradigm for modeling agent behavior that is based on the idea that humans only use a small portion of their knowledge at any given time. It was specially designed to represent human tactical behavior and has been successfully implemented in systems with single agents or two agents working together. In this paper, we apply this idea in a hierarchical multi-agent system of command agents, where the agents’ actions are to command and coordinate subordinates, send reports to their superiors, and communicate with other agents at the same level. We focus on how contexts and actions can be defined for these higher level command agents and how the contexts and actions for the different command agents are related. The proposed methodology is implemented and tested for a hierarchy of command agents that are interpreting and planning an operational order at a battalion level and carrying it out in a computer generated forces environment.

建設組合方式における業務委託に関する制度について

Previous study has proved that a targeted removal approach for fire spreading buildings is effective in preventing wide-spread fire generated by an intense earthquake in areas densely crowded with wooden buildings, and that a building cooperative system can effectively function for the purpose of consolidating small and irregular plots of land without road access and meeting the financial requirement of landowners. A separate issue is how to design a mechanism that allows agents for cooperatives to work autonomously. The principal-agent theory provides a substantial framework for the issue, such as asymmetric information, risk, conflicts of interests, moral hazards, observability, and incentives. According to the theory, we make a realistic model of agent behavior that reduces project risks for building cooperatives by acquired information. The model analyses quantitative effects on agent effort levels by factors, i.e. allowance of dual employment, observability, and terms of payment. Analysis indicates unexpected synergistic effects. Although one factor alone has little influence on the agent effort level, multiple factors significantly lower the effort level. This suggests that regulations on agencies should require that operating contracts prohibit subsidiary work, deliverables, and payments by results other than by banning conflicts of interests. other than avoiding conflict of interest if agents are not well disciplined as professionals. This quantitative analysis on a realistic contract theory model could be an example to consider for an effective regulation structure that contains synergistic effects.

Effects of the Perspectives that Influenced on the Human Mental Stance in the Multiple-to-Multiple Human-Agent Interaction

Virtual agents are useful for simulations of real-world events or processes designed for solving a problem, such as serious games and gamifications. In several types of the virtual world simulations, multiple-to-multiple interaction is needed because our social interaction is multiple-to-multiple in many cases. In this study, we extended the agent behavior model that induces intentional stance in one-to-one human-agent interaction, which was proposed in our previous works, to the multiple-to-multiple human-agent interaction. By using the agent group, we investigated effects of the perspectives that influenced the human mental stance in the multiple-to-multiple human-agent interaction. To investigate the effects of the perspectives, we changed the viewpoints (subjective or objective) in different perspectives (physical or mental) when people interacted with a team of agents in the virtual futsal game. As a result, we can find that there is a different effect between the physical perspective and the mental perspective. The subjective viewpoint of the physical perspective enhanced the feeling of the agent team’s intelligence, but the objective viewpoint of the mental perspective enhanced the feeling of the agent team’s intelligence.

Data-driven agent-based modeling, with application to rooftop solar adoption

Agent-based modeling is commonly used for studying complex system properties emergent from interactions among many agents. We present a novel data-driven agent-based modeling framework applied to forecasting individual and aggregate residential rooftop solar adoption in San Diego county. Our first step is to learn a model of individual agent behavior from combined data of individual adoption characteristics and property assessment. We then construct an agent-based simulation with the learned model embedded in artificial agents, and proceed to validate it using a holdout sequence of collective adoption decisions. We demonstrate that the resulting agent-based model successfully forecasts solar adoption trends and provides a meaningful quantification of uncertainty about its predictions. We utilize our model to optimize two classes of policies aimed at spurring solar adoption: one that subsidizes the cost of adoption, and another that gives away free systems to low-income households. We find that the optimal policies derived for the latter class are significantly more efficacious, whereas the policies similar to the current California Solar Initiative incentive scheme appear to have a limited impact on overall adoption trends.

Modeling of Searching Agent Behavior by Means of Neural Gas

The current work describes the results of modeling of the behavior of the autonomous agent. The control system of the agent is based on a growing neuron gas. Computer simulations characterize the exploration behavior of the agent in one-dimensional and two-dimensional mazes.

Behavioral mechanism design

Incentives are more likely to elicit desired outcomes when they are based on accurate models of agent behavior. A growing literature---from behavioral economics, as well as online user studies---suggests, however, that people do not quite behave like standard economic agents in a variety of environments, both online and offline. What consequences might such differences have for the optimal design of these environments? In this note, we summarize our results from Easley and Ghosh [2015] which explores this question of behavioral design---how departures from standard economic models of agent behavior affect mechanism design---via the problem of the optimal choice of contract structure in crowdsourcing markets where workers make decisions according to prospect theory preferences [Kahneman and Tversky 1979] rather than classical expected utility theory: we show that a principal might indeed choose a fundamentally different kind of mechanism---an output-contingent contest versus a 'safe', output-independent, fixed-payment scheme---and do better as a result, if he accounts for deviations from the standard economic models of decision-making that are typically used in theoretical design.