Behavior Of Agents Research Articles

Distributed time-varying Nash equilibrium in resilient multi-objective formation control for cyber–physical systems

This paper investigates the problem of multi-objective time-varying formation control in cyber–physical systems (CPSs) with saturated constraints and malicious agents. By incorporating the concept of multi-agent systems and employing objective weight allocation method, the problem is reformulated as the seeking for the Nash equilibrium solution in a time-varying non-cooperative game. To evaluate the trustworthiness of agent behavior, we propose an adaptive weight learning algorithm based on reinforcement learning. Furthermore, we develop a novel smooth saturation control strategy and a predefined time controller to design a distributed resilient algorithm for CPSs with both known and unknown parameters. Additionally, we apply the designed algorithm to the resilient distributed multi-objective time-varying formation control problem of unmanned aerial vehicles (UAVs) to verify its effectiveness.

代理人行为能力规则的检视与适用 — 基于代理制度与行为能力制度视角

代理人行为能力规则的检视与适用 — 基于代理制度与行为能力制度视角

Improving algorithmic trading consistency via human alignment and imitation learning

Research on algorithmic trading using reinforcement learning has become increasingly popular in recent years. Although most of the current reinforcement learning methods are employed to train the agent for some kind of modeling or data problem, it is worthwhile to explore in aligning agents with human behavior in applications as crucial as financial trading. Achieving such consistency by incorporating human expert experience into agent behavior is a key for potential improvements in this field. Imitation learning learns directly from examples of humans or other agents performing tasks. However, using imitation learning alone suffers from the problem of transitionally fitting expert example data. By combining the advantages of imitation learning and the Advantage Actor–Critic method, the Human Alignment Advantage Actor–Critic (HA3C) algorithm is proposed, to enhance single-asset trading strategy. First, by adding daily and weekly frequency trading data as input features to TimesNet, which is specifically designed to extract correlated temporal patterns from time-series data, it can capture both short-term and long-term features, thus capturing time-series features more comprehensively. Second, an expert action labeling method is proposed to train a strategy prediction network through supervised learning of behavior imitation. Third, a pre-trained strategy network is transferred to balance the exploration and exploitation of the agent’s behavior. Imitation learning techniques leverage finance-specific knowledge to enhance algorithmic trading consistency. This approach enables algorithms to mimic and adapt human decision-making patterns in finance, ultimately improving overall performance. This paper introduces a novel return-based function that efficiently balances short-term and long-term returns over flexible time horizons. It considers the maximum return from different positions and uses flexible time windows to capture trends while maximizing returns. Finally, evaluation on six commonly used datasets, such as DJI and SP500, demonstrates the advantages of the proposed HA3C algorithm compared with other classical and reinforcement learning-based strategies. Notably, on the HSI dataset, the HA3C strategy significantly outperforms other methods, achieving an impressive cumulative return of 681.55% and a Sharpe ratio of 5.07. These results show the superior performance of the HA3C algorithm in enhancing stock trading strategies and its potential to impact algorithmic trading consistency through aligning agent behavior with human expertise.

Motivating People to Work: The Value Behind Diverse Assumptions

AbstractAgency theory and self‐determination theory have contrasting assumptions about what motivates human beings and, accordingly, suggest differing methods of motivating others – from extrinsic rewards aimed at controlling agency to facilitating human agentic behaviour. These different assumptions are consequential – organizations and societies would look wildly different with the adoption of the one or other perspectives, with important implications for human welfare and wellbeing. The introduction to this Point and Counterpoint (PCP) calls on prominent scholars in both perspectives to clarify as well as question their assumptions about human motivation. An invitation to take a step back and clarify one's beliefs with precision, elucidating both the ideas and the data they are based on. At the same time, this PCP constitutes an invitation to explore how one's own personal preferences or values might be guiding their own (selection of) research and argumentation. We hope such internal reflection and external dialogue moves the conversation from us‐versus‐them to shared passion, from contradictory to paradoxical, and from stalemates to practical solutions that are sufficiently integrative to address today's complex societal challenges.

K-PERM: Personalized Response Generation Using Dynamic Knowledge Retrieval and Persona-Adaptive Queries

Personalizing conversational agents can enhance the quality of conversations and increase user engagement. However, they often lack external knowledge to appropriately tend to a user’s persona. This is crucial for practical applications like mental health support, nutrition planning, culturally sensitive conversations, or reducing toxic behavior in conversational agents. To enhance the relevance and comprehensiveness of personalized responses, we propose using a two-step approach that involves (1) selectively integrating user personas and (2) contextualizing the response by supplementing information from a background knowledge source. We develop K-PERM (Knowledge-guided PErsonalization with Reward Modulation), a dynamic conversational agent that combines these elements. K-PERM achieves state-of-the- art performance on the popular FoCus dataset, containing real-world personalized conversations concerning global landmarks.We show that using responses from K-PERM can improve performance in state-of-the-art LLMs (GPT 3.5) by 10.5%, highlighting the impact of K-PERM for personalizing chatbots.

AI Health Agents: Pathway2vec, ReflectE, Category Theory, and Longevity

Health Agents are introduced as the concept of a personalized AI health advisor overlay for continuous health monitoring (e.g. 1000x/minute) medical-grade smartwatches and wearables for “healthcare by app” instead of “sickcare by appointment.” Individuals can customize the level of detail in the information they view. Health Agents “speak” natural language to humans and formal language to the computational infrastructure, possibly outputting the mathematics of personalized homeostatic health as part of their reinforcement learning agent behavior. As an AI health interface, the agent facilitates the management of precision medicine as a service. Healthy longevity is a high-profile area characterized by the increasing acceptance of medical intervention, longevity biotech venture capital investment, and global priority as 2 billion people will be over 65 in 2050. Aging hallmarks, biomarkers, and clocks provide a quantitative measure for intervention. Some of the leading interventions include metformin, rapamycin, spermidine, NAD+/sirtuins, alpha-ketoglutarate, and taurine. AI-driven digital biology, longevity medicine, and Web3 personalized healthcare come together in the idea of Health Agents. This Web3 genAI tool for automated health management, specifically via digital-biological twins and pathway2vec approaches, demonstrates human-AI intelligence amplification and works towards healthy longevity for global well-being.

Terrain-Aware Military Planning Agents

Terrain is important, often decisive, in military battles. This concept recurs across numerous historical examples, military theorists, and doctrinal manuals used by armies around the world. This poster discusses how Mission Command Agents, automated agents used to represent the behaviors of military forces in combat simulations, achieve this understanding of the importance of terrain. First, a review of tactical manuals from different nations consistently identified the importance of terrain to observation, fires, and mobility. Based on the unit’s mission, the geography afforded, or prevented, these activities with respect to the enemy force. The next step was to build an automated tool that could quickly calculate these effects based on the anticipated positions. The third step was to formulate these quantifiable terrain effects as objectives in a multi-objective search heuristic so that different places could be compared with each other. After excellent results with these techniques in a realistic military planning scenario, the team is further enhancing human-machine collaboration in this area by adding geospatial and military characteristics of terrain to an existing standard for Command and Control – Simulation Interoperation (C2SIM). With this enhancement, the reasoning employed by the agents is more explainable to military observers. It also allows military experts to adjust agent behaviors by adjusting their goals, without the need to change source code, while employing a user interface that they will be familiar with.

Learning Crowd Motion Dynamics with Crowds

Reinforcement Learning (RL) has become a popular framework for learning desired behaviors for computational agents in graphics and games. In a multi-agent crowd, one major goal is for agents to avoid collisions while navigating in a dynamic environment. Another goal is to simulate natural-looking crowds, which is difficult to define due to the ambiguity as to what is a natural crowd motion. We introduce a novel methodology for simulating crowds, which learns most-preferred crowd simulation behaviors from crowd-sourced votes via Bayesian optimization. Our method uses deep reinforcement learning for simulating crowds, where crowdsourcing is used to select policy hyper-parameters. Training agents with such parameters results in a crowd simulation that is preferred to users. We demonstrate our method's robustness in multiple scenarios and metrics, where we show it is superior compared to alternate policies and prior work.

Emergence of integrated behaviors through direct optimization for homeostasis

Homeostasis is a self-regulatory process, wherein an organism maintains a specific internal physiological state. Homeostatic reinforcement learning (RL) is a framework recently proposed in computational neuroscience to explain animal behavior. Homeostatic RL organizes the behaviors of autonomous embodied agents according to the demands of the internal dynamics of their bodies, coupled with the external environment. Thus, it provides a basis for real-world autonomous agents, such as robots, to continually acquire and learn integrated behaviors for survival. However, prior studies have generally explored problems pertaining to limited size, as the agent must handle observations of such coupled dynamics. To overcome this restriction, we developed an advanced method to realize scaled-up homeostatic RL using deep RL. Furthermore, several rewards for homeostasis have been proposed in the literature. We identified that the reward definition that uses the difference in drive function yields the best results. We created two benchmark environments for homeostasis and performed a behavioral analysis. The analysis showed that the trained agents in each environment changed their behavior based on their internal physiological states. Finally, we extended our method to address vision using deep convolutional neural networks. The analysis of a trained agent revealed that it has visual saliency rooted in the survival environment and internal representations resulting from multimodal input.

Stochastic pairwise preference convergence in Bayesian agents.

Beliefs inform the behavior of forward-thinking agents in complex environments. Recently, sequential Bayesian inference has emerged as a mechanism to study belief formation among agents adapting to dynamical conditions. However, we lack critical theory to explain how preferences evolve in cases of simple agent interactions. In this paper, we derive a Gaussian, pairwise agent interaction model to study how preferences converge when driven by observation of each other's behaviors. We show that the dynamics of convergence resemble an Ornstein-Uhlenbeck process, a common model in nonequilibrium stochastic dynamics. Using standard analytical and computational techniques, we find that the hyperprior magnitudes, representing the learning time, determine the convergence value and the asymptotic entropy of the preferences across pairs of agents. We also show that the dynamical variance in preferences is characterized by a relaxation time t^{★} and compute its asymptotic upper bound. This formulation enhances the existing toolkit for modeling stochastic, interactive agents by formalizing leading theories in learning theory, and builds towards more comprehensive models of open problems in principal-agent and market theory.

Evaluation of surfactant-aided polycyclic aromatic hydrocarbon biodegradation by molecular docking and molecular dynamic simulation in the marine environment

Marine oil spills directly cause polycyclic aromatic hydrocarbons (PAHs) pollution and affect marine organisms due to their toxic property. Chemical and bio-based dispersants composed of surfactants and solvents are considered effective oil spill-treating agents. Dispersants enhance oil biodegradation in the marine environment by rapidly increasing their solubility in the water column. However, the effect of dispersants, especially surfactants, on PAHs degradation by enzymes produced by microorganisms has not been studied at the molecular level. The role of the cytochrome P450 (CYP) enzyme in converting contaminants into reactive metabolites during the biodegradation process has been evidenced, but the activity in the presence of surfactants is still ambiguous. Thus, this study focused on the evaluation of the impact of chemical and bio-surfactants (i.e., Tween 80 (TWE) and Surfactin (SUC)) on the biodegradation of naphthalene (NAP), chrysene (CHR), and pyrene (PYR), the representative components of PAHs, with CYP enzyme from microalgae Parachlorella kessleri using molecular docking and molecular dynamics (MD) simulation. The molecular docking analysis revealed that PAHs bound to residues at the CYP active site through hydrophobic interactions for biodegradation. The MD simulation showed that the surfactant addition changed the enzyme conformation in the CYP-PAH complexes to provide more interactions between the enzyme and PAHs. This led to an increase in the enzyme's capability to degrade PAHs. Binding free energy (ΔG‌‌Bind) calculations confirmed that surfactant treatment could enhance PAHs degradation by the enzyme. The SUC gave a better result on NAP and PYR biodegradation based on ΔG‌‌Bind, while TWE facilitated the biodegradation of CHR. The research outputs could greatly facilitate evaluating the behaviors of oil spill-treating agents and oil spill response operations in the marine environment.

Proactive Language Learning Theory

AbstractSecond language acquisition theory has traditionally focused on the cognitive and psycholinguistic processes involved in additional language (L2) learning. In addition, research on learner psychology has primarily centered on learners’ cognitive abilities (e.g., aptitude and working memory) and internal traits or states (e.g., dispositions, motivations, and affect). Language learning behavior, however, has remained largely neglected and under‐theorized. To address this gap, this paper proposes the proactive language learning theory, which delineates the agentic and strategic behaviors that learners employ to learn an additional language. These behaviors include input‐seeking behavior, interaction‐seeking behavior, information‐seeking behavior, and feedback‐seeking behavior. This paper presents theoretical arguments supporting the proposal, describes the four behavioral dimensions of the theory, and outlines general hypotheses concerning the contextual and learner‐related antecedents of these behaviors and their effects on L2 outcomes. Finally, the potential implications of this theory for advancing our understanding of L2 learning and instruction are discussed.

Proactive Agent Behaviour in Dynamic Distributed Constraint Optimisation Problems

In multi-agent systems, the Dynamic Distributed Constraint Optimisation Problem (D-DCOP) framework is pivotal, allowing for the decomposition of global objectives into agent constraints. Proactive agent behaviour is crucial in such systems, enabling agents to anticipate future changes and adapt accordingly. Existing approaches, like Proactive Dynamic DCOP (PD-DCOP) algorithms, often necessitate a predefined environment model. We address the problem of enabling proactive agent behaviour in D-DCOPs where the dynamics model of the environment is unknown. Specifically, we propose an approach where agents learn local autoregressive models from observations, predicting future states to inform decision-making. To achieve this, we present a temporal experience-sharing message-passing algorithm that leverages dynamic agent connections and a distance metric to collate training data. Our approach outperformed baseline methods in a search-and-extinguish task using the RoboCup Rescue Simulator, achieving better total building damage. The experimental results align with prior work on the significance of decision-switching costs and demonstrate improved performance when the switching cost is combined with a learned model.

О МОДЕЛИРОВАНИИ ЗАКУПКИ ОБРАЗОВАТЕЛЬНОГО ОБОРУДОВАНИЯ И БЮДЖЕТНОЙ КОНКУРЕНЦИИ ШКОЛ

Educational establishments regularly acquire administrative and pedagogical equipment to ensure a quality educational process. In the context of limited budgets and competition among institutions, optimal resource allocation and efficient procurement management become crucial. This study builds upon approaches by applying game theory to model procurement in the educational sector. The first part extends single-stage auction models of procurement to account for heterogeneous participant information. Acquired knowledge enables participants to reduce uncertainty in procurement outcomes and outmaneuver stronger market participants with more favorable offers. Specifically, an optimal strategy is identified for the scenario where only one player possesses information, along with a quantitative order of neutralizing the undesired advantage through additional player inclusion. The second part of the study examines mechanisms of long-term budgetary competition among schools. Well-known Rubinstein, Baron, and Ferejohn game theory models are employed to investigate this setup. A substantive analogy between competition and multi-stage negotiations is drawn, and key parameters determining competitive outcomes are delineated. The purpose of the work is to evaluate the applicability of game theory models. Materials and methods. For modeling, the mathematical apparatus of dynamic games is used, including methods of probabilistic modeling, game theory and optimization. The determining criterion for the decision of the players is the greatest utility, calculated solely based on available information. Results of the modeling are performed analytically through optimization, considering the discounting of the utility function over time. Results. Novelty of work is coupled with domain interpretation of game theory models. Modeling allows predicting agents behavior under the regulation of the procurement market and the definition of their budgeting. Conclusion. The analysis of models of game theory in relation to the field of education is accomplished. It is shown that the degree of awareness of the players about the value functions can significantly influence their decisions and the results of procurement.

Comprehensible Extraction of Knowledge Bases for Learning Agents in Games

AbstractThis dissertation abstract summarizes results of the thesis “Comprehensible Knowledge Base Extraction for Learning Agents - Practical Challenges and Applications in Games” (accepted as dissertation at the Department of Computer Science of TU Dortmund University, Germany). The thesis presents approaches that allow for the automated creation of knowledge bases from agent behavior learned in the context of games. The aims are twofold: (1) The creation of human-readable knowledge that can provide insights into what an agent learned, and (2) the investigation of how learning agents themselves can benefit from incorporating these approaches into their learning processes. Applications are presented, e.g., in the context of general video game playing. Moreover, an outlook on the InteKRator toolbox is provided which implements the most essential approaches in a more general context for the potential use in other domains (e.g. in medical informatics).

AGENT-BASED MODELING OF UP DILIMAN INTRACAMPUS PEDESTRIAN MOBILITY USING NETLOGO AND GIS

Abstract. In the Philippines, campus mobility studies remain underexplored. Since small-scale models help in policy making that may also be applied in a large spatiotemporal context, intracampus mobility modeling and simulation offer support for future mobility research. In this study, we designed a NetLogo model to apply ABM and GIS in understanding intracampus student pedestrian mobility. We created a deterministic model using discrete student schedule and a probabilistic model using the multinomial logistic (MNL) regression model derived from the student demographics data to analyze student movement. To provide a realistic representation of student pedestrians and campus environment, GIS spatiotemporal data was proven crucial to the pedestrian mobility models. The generated pedestrian count heatmaps show a significant increase in student count from online learning switching to face–to–face instruction. The overall campus building occupancy served as the key factor influencing the changes in agent behaviors such as walking speed and student tardiness. Moreover, the probabilistic model showed erratic movements of students. The model evaluation depends on the heavily imbalanced datasets and its multiclass nature with relatively higher model performance (F1 score > 0.8) is observed during morning, noon, and end schedule times, whereas suboptimal performance (F1 score < 0.6) where the dataset has a higher number of classes. The simulations demonstrate that ABM is a useful tool for examining pedestrian and transport patterns and can be enhanced further to allow multimodal and multiagent systems.

Meso-institutions shaping arenas for policymaking: an exploratory study on front-of-package food labelling in Brazil, Chile, and Mexico

Front-of-package food labelling (FOPL) is a mechanism to inform consumers on food nutrients content and risks for health. Each country, within its own institutional framework, defines methods and protocols to formulate rules capable of influencing organizations and the behavior of agents. The paper aims at evaluating the role of some meso-institutions in shaping arenas for policymaking on FOPL based on an exploratory study in Brazil, Chile, and Mexico. A documental analysis and literature review were employed, followed by in-depth interviews. The relevant aspects of FOPL assessed were thresholds for sugar, fats, and sodium; marketing restrictions, and timeframe given for compliance with the new rules. The contribution of this article is to illuminate the way in which meso-institutions operate in a specific context, namely the development of food labelling standards. The focus is to find the rules that allowed the participation of stakeholders in the process of defining normative acts intended to regulate food labelling. The discussion of mandatory FOPL can be interpreted as a struggle to define proper rights for the food industry and the consumer. Meso-institutions, as translators of general normative principles in specific contexts, can encourage or restrict stakeholder participation in policymaking. Besides political arena, the participation can be done in other arenas, such as legislature, regulatory agencies, scientific community, media and public opinion. Institutions that encourage stakeholder participation increase the probability of obtaining widely accepted specific rules.

Improving Farm Yield through Agent Based Modeling

This paper explores the potential of Agent-Based Modeling (ABM) using NetLogo to enhance farm yield. Traditional approaches to agricultural management often overlook the complexity and interdependencies within farming systems. ABM offers a dynamic and flexible framework to simulate the behaviours of individual agents within a farming ecosystem, enabling a more nuanced understanding of the factors influencing yield. By modeling the interactions between agents such as farmers, crops, pests, weather conditions, and market dynamics, this study aims to identify optimal strategies for improving farm yield while minimizing input costs and environmental impacts. Through experimentation and scenario analysis, various farming practices and policies can be simulated and evaluated in silico, providing valuable insights for real-world decision-making. This interdisciplinary approach integrates concepts from computer science, economics, ecology, and agronomy to develop a holistic understanding of agricultural systems. The findings of this research contribute to the development of sustainable farming practices and policy interventions to address food security challenges in a rapidly changing world.

Learning agile soccer skills for a bipedal robot with deep reinforcement learning.

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent's tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

Mining threats in high-level biodiversity conservation policies.

Amid a global infrastructure boom, there is increasing recognition of the ecological impacts of the extraction and consumption of construction minerals, mainly processed as concrete, including significant and expanding threats to global biodiversity. We investigated how high-level national and international biodiversity conservation policies address mining threats, with a special focus on construction minerals. We conducted a review and quantified the degree to which threats from mining these minerals are addressed in biodiversity goals and targets under the 2011-2020 and post-2020 biodiversity strategies, national biodiversity strategies and action plans, and the assessments of the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services. Mining appeared rarely in national targets but more frequently in national strategies. Yet, in most countries, it was superficially addressed. Coverage of aggregates mining was greater than coverage of limestone mining. We outline 8 key components, tailored for a wide range of actors, to effectively mainstream biodiversity conservation into the extractive, infrastructure, and construction sectors. Actions include improving reporting and monitoring systems, enhancing the evidence base around mining impacts on biodiversity, and modifying the behavior of financial agents and businesses. Implementing these measures could pave the way for a more sustainable approach to construction mineral use and safeguard biodiversity.