Behavior Of Agents Research Articles

Developing teamwork: transitioning between stages in human-agent collaboration

IntroductionHuman-centric artificial intelligence (HCAI) focuses on systems that support and collaborate with humans to achieve their goals. To better understand how collaboration develops in human-AI teaming, further exploration grounded in a theoretical model is needed. Tuckman's model describes how team development among humans evolves by transitioning through the stages of forming, storming, norming, performing, and adjourning. The purpose of this pilot study was to explore transitions between the first three stages in a collaborative task involving a human and a human-centric agent.MethodThe collaborative task was selected based on commonly performed tasks in a therapeutic healthcare context. It involved planning activities for the upcoming week to achieve health-related goals. A calendar application served as a tool for this task. This application embedded a collaborative agent designed to interact with humans following Tuckman's stages of team development. Eight participants completed the collaborative calendar planning task, followed by a semi-structured interview. Interviews were transcribed and analyzed using inductive content analysis.ResultsThe results revealed that the participants initiated the storming stage in most cases (n = 7/8) and that the agent initiated the norming stage in most cases (n = 5/8). Additionally, three main categories emerged from the content analyses of the interviews related to participants' transition through team development stages: (i) participants' experiences of Tuckman's first three stages of team development; (ii) their reactions to the agent's behavior in the three stages; and (iii) factors important to the participants to team up with a collaborative agent.ConclusionResults suggest ways to further personalize the agent to contribute to human-agent teamwork. In addition, this study revealed the need to further examine the integration of explicit conflict management into human-agent collaboration for human-agent teamwork.

Artefact design and societal worldview.

Technological artefacts are created in accordance with the values and worldviews of their designers. In operation, they act as a medium, facilitating and constraining human interaction with, and perception of, the world. When used on a large scale, they may lastingly affect societal ethos. If institutional structures of domination allocate the resources necessary for artefact design and development to some population groups over others, the direction and extent of such an effect may lead to increased disparity and inequity. While the direct influence of technology on opinion is well-studied, the evaluation of non-epistemic values, assumptions and presuppositions is a hurdle in the way of a deeper understanding of the large-scale effects of asymmetries in worldviews embodied by artefacts. Here, we show that artefacts have a strong potential to bias societal worldviews when they are distributed unevenly across the value spectrum. They can affect the clustering behaviour of agents with regard to worldview, both aiding and hindering intra- and inter-cluster diversity, depending on their distribution and frequency. Our findings underline the distributional sensitivity of worldview dynamics to institutional structures of domination. We highlight the importance of procedural interventions such as participatory design, which explicitly acknowledges existing asymmetries and redistributes power accordingly.This article is part of the theme issue 'Co-creating the future: participatory cities and digital governance'.

Making a human's trust repair for an agent in a series of tasks through the agent's empathic behavior

As AI technology develops, trust in agents (including robot, AI and anthropomorphic agent) is becoming more important for more AI applications in human society. Possible ways to improve the trust relationship include empathy, success-failure series, and capability (performance). Appropriate trust is less likely to cause deviations between actual and ideal performance. In this study, we focus on the agent's empathic behavior and success-failure series to increase trust in agents. We experimentally examine the effect of empathic behavior from agent to person on changes in trust over time. The experiment was conducted with a two-factor mixed design: empathic behavior (available, not available) and success-failure series (phase 1 to phase 5). An analysis of variance (ANOVA) was conducted using data from 200 participants. The results showed an interaction between the empathic behavior factor and the success-failure series factor, with trust in the agent repairing when empathic behavior was present. This result supports our hypothesis. This study shows that designing agents to be empathic behavior is an important factor for trust and helps humans build appropriate trust relationships with agents.

Skill matters: Dynamic skill learning for multi-agent cooperative reinforcement learning

With the popularization of intelligence, the necessity of cooperation between intelligent machines makes the research of collaborative multi-agent reinforcement learning (MARL) more extensive. Existing approaches typically address this challenge through task decomposition of the environment or role classification of agents. However, these studies may rely on the sharing of parameters between agents, resulting in the homogeneity of agent behavior, which is not effective for complex tasks. Or training that relies on external rewards is difficult to adapt to scenarios with sparse rewards. Based on the above challenges, in this paper we propose a novel dynamic skill learning (DSL) framework for agents to learn more diverse abilities motivated by internal rewards. Specifically, the DSL has two components: (i) Dynamic skill discovery, which encourages the production of meaningful skills by exploring the environment in an unsupervised manner, using the inner product between a skill vector and a trajectory representation to generate intrinsic rewards. Meanwhile, the Lipschitz constraint of the state representation function is used to ensure the proper trajectory of the learned skills. (ii) Dynamic skill assignment, which utilizes a policy controller to assign skills to each agent based on its different trajectory latent variables. In addition, in order to avoid training instability caused by frequent changes in skill selection, we introduce a regularization term to limit skill switching between adjacent time steps. We thoroughly tested the DSL approach on two challenging benchmarks, StarCraft II and Google Research Football. Experimental results show that compared with strong benchmarks such as QMIX and RODE, DSL effectively improves performance and is more adaptable to difficult collaborative scenarios.

DOMINANTS FOR THE NATIONAL ECONOMIES’ COMPETITIVENESS

This article is devoted to innovation as a prerequisite for the national economies' sustainable development and a domain for their global competitiveness. The paper aims to highlight the internal essence of mutual relations between innovation, competition, and economic growth. It was assumed that innovation positively affected the competitive behaviour of economic agents and eventually contributed to an increase in public welfare. Regarding the period from 2007 to 2022, we considered the sample of Central European countries and the Baltic states. Theoretical generalization has been applied to systematize the factors determining competition and innovation. We used statistical methods and comparative analysis to investigate the indicators' dynamics. The interdependencies between macroeconomic dynamics and a set of competition and innovation indicators have been assessed. It was proved that the proportion of medium and high-tech industry value added in the total value added of manufacturing was quite important, yet not the ultimate criterion for ensuring high growth rates and sustainable development. The dynamics of commercial spending on research and development had been ambiguous and unsustainable. To ensure strategic competitiveness, national governments should encourage innovation in their respective business communities. The sampled economies have been examined considering a set of innovation and competition indicators (e.g., gross capital formation, research and development expenditures, logistics performance, export technological structure, carbon dioxide emission, and renewable energy consumption). Considering the implications of globalization, tax burden as an integral characteristic of the national economy's competitiveness has been evaluated as well. It was pointed out that: a) a stable, understandable and transparent tax policy should be implemented to protect the investors' property rights; b) a system of total anti-corruption measures aimed at preventing the practices of receiving undue benefits should be set; and c) the business sector should improve own environmental and social responsibility complexly, introduce resource-saving, green and circular economy.

Penetration of Non-Adhesive Gel-like Embolic Materials During Dural Vessels Embolization According to Characteristics of Tantalum Powder

Tantalum powder is included in the composition of Non-Adhesive Gel-like Embolic Materials (NAGLEMs) for X-ray opacity. The duration of X-ray opacity during embolization is primarily associated with the particle size, which differs in the most used NAGLEMs—ONYX (Medtronic) and SQUID (Balt). NAGLEMs are widely used for the embolization of branches of the middle meningeal artery (MMA) in patients with chronic subdural hematomas (CSDHs). Considering the size (5–15 microns) of the target dural vessels, we assumed that not only the viscosity of NAGLEMs, but also the size and shape of tantalum granules may be important for the penetration of these gel-like embolic agents and determine their behavior. A notable discrepancy in size was observed. The medium-sized granules in the SQUID 18 sample (0.443 ± 0.086 microns, M ± SD) were found to be approximately ten times smaller than the tantalum granules in the ONYX 18 sample (5.2 ± 0.33 microns, M ± SD).Tantalum granules in SQUID 18 have a regular spherical shape; in ONYX 18 they have an irregular angular shape. When comparing the behavior of gel-like embolic agents of the same viscosity during MMA embolization in patients with CSDHs (an average age of 62.2 ± 14.3 years) in the group where SQUID 18 (n = 8) was used, the gel-like embolic agent in dural vessels demonstrated significantly greater penetration ability compared with the group where ONYX 18 (n = 8) was used. Accordingly, not only the viscosity of NAGLEMs, but also the size and shape of tantalum granules can have a significant effect on the penetration ability of gel compositions.

Study on the Performances of Toughening UV-LED-Cured Epoxy Electronic Encapsulants

This study aims to investigate the effects of three toughening agents—core–shell rubber particles (CSR), nano-silica particles (NSPs), and epoxidized polybutadiene (EPB)—on the performance of UV-LED-cured epoxy electronic encapsulants. By systematically comparing the curing behavior, thermomechanical properties, and impact resistance of different toughening agents in alicyclic epoxy resins, their potential applications in more environmentally friendly UV-cured electronic encapsulation are evaluated. The results show that NSP and CSR toughened samples have fast cured speed under 365 nm UV-LED light, but it affects the depth of curing under low energy conditions. They maintain high Tg, high modulus, and low thermal expansion coefficient (CTE), especially in the NSP-toughened sample. The EPB-toughened sample has good transparency for LED, but it has negative effects on Tg and CTE. This research provides essential theoretical and experimental data to support the development of high-performance UV-LED-cured epoxy encapsulation materials.

Don't touch me! Mating not always preceded by courtship behavior in predatory coccinellids.

Knowledge of the ecology and behavior of biological control agents is essential for their effective use in biocontrol and Integrated Pest Management (IPM)programs. There is a lack of information regarding the courtship and mating traits that are crucial for successful mating in biocontrol population of coccinellids. To expand our current understanding in this area, 2 coccinellid species commonly used for the biocontrol of soft-bodied insects, Cryptolaemus montrouzieri (Mulsant) and Propylea quatuordecimpunctata (Linnaeus) (Coleoptera: Coccinellidae), were studied. A quantitative analysis was performed to compare their courtship and mating displays. Key behavioral traits of the precopulatory, copulatory, and postcopulatory phases were investigated. The courtship and mating sequences of C. montrouzieri and P. quatuordecimpunctata were similar to each other. During the mating phase, the key displays were the opening of the elytrae and wings followed by body shaking in C. montrouzieri and leg tapping followed by body shaking in P. quatuordecimpunctata. The mating success of both species was not correlated with any courtship display, such as antennal tapping. Only in C. montrouzieri mounting attempt showed a higher frequency from the backside without affecting male mating success. This research adds baseline knowledge about the courtship and mating behavior of the biocontrol population of coccinellids, contributing to the identification of potentiallyuseful benchmarks (e.g., body shaking, leg tapping) for monitoring prolonged mass-rearing processes, thusreducing mating failures.

Formal contracts mitigate social dilemmas in multi-agent reinforcement learning

Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group incentives diverge. Humans are remarkably capable at solving these social dilemmas. It is an open problem in MARL to replicate such cooperative behaviors in selfish agents. In this work, we draw upon the idea of formal contracting from economics to overcome diverging incentives between agents in MARL. We propose an augmentation to a Markov game where agents voluntarily agree to binding transfers of reward, under pre-specified conditions. Our contributions are theoretical and empirical. First, we show that this augmentation makes all subgame-perfect equilibria of all Fully Observable Markov Games exhibit socially optimal behavior, given a sufficiently rich space of contracts. Next, we show that for general contract spaces, and even under partial observability, richer contract spaces lead to higher welfare. Hence, contract space design solves an exploration-exploitation tradeoff, sidestepping incentive issues. We complement our theoretical analysis with experiments. Issues of exploration in the contracting augmentation are mitigated using a training methodology inspired by multi-objective reinforcement learning: Multi-Objective Contract Augmentation Learning. We test our methodology in static, single-move games, as well as dynamic domains that simulate traffic, pollution management, and common pool resource management.

Large language models for whole-learner support: opportunities and challenges.

In recent years, large language models (LLMs) have seen rapid advancement and adoption, and are increasingly being used in educational contexts. In this perspective article, we explore the open challenge of leveraging LLMs to create personalized learning environments that support the "whole learner" by modeling and adapting to both cognitive and non-cognitive characteristics. We identify three key challenges toward this vision: (1) improving the interpretability of LLMs' representations of whole learners, (2) implementing adaptive technologies that can leverage such representations to provide tailored pedagogical support, and (3) authoring and evaluating LLM-based educational agents. For interpretability, we discuss approaches for explaining LLM behaviors in terms of their internal representations of learners; for adaptation, we examine how LLMs can be used to provide context-aware feedback and scaffold non-cognitive skills through natural language interactions; and for authoring, we highlight the opportunities and challenges involved in using natural language instructions to specify behaviors of educational agents. Addressing these challenges will enable personalized AI tutors that can enhance learning by accounting for each student's unique background, abilities, motivations, and socioemotional needs.

A multi-theoretical approach to supply chain management

The article examines the supply chain as a modeling object. The multi-criteria and contradictory goals in supply chain management are: on the one hand, the maximum level of service, on the other hand, minimizing logistics costs. Solving strategic optimization problems for this class of objects using classical optimization methods is difficult. The theoretical and methodological basis of the supply chain was considered from the perspective of a set-theoretic approach. The purpose of the study is to analyze the behavior of many different agents (suppliers, clients, etc.) with different needs and goals when making decisions. An agent is an active element of the system, possessing a certain autonomy and the ability to make independent decisions, relying on the information it has about the state of the environment and the actions of other agents. In turn, supply chains are a complex organizational (active) system, i.e. associated with human participation. The study of behavioral aspects and interorganizational interaction, the formation of new organizational forms are an extremely important aspect in the management and modeling of supply chains, a factor in the dynamic behavior of the logistics system. The conducted research represents an important contribution to the development of knowledge about the application of the set-theoretic approach in supply chain management. It demonstrates the applicability of this approach in practice through an analysis of a specific company, LLP «Knauf», on the Kazakhstan market.

CONFLICT COMMUNICATIVE BEHAVIOR IN THE DIGITAL ENVIRONMENT

The article describes the features of the conflictual communicative behavior of agents of the modern digital environment. Such an environment guarantees a person new opportunities in the field of communication, which, on the one hand, opens up new freedoms for him, and on the other – new forms of conflicts. It was found that conflict communication occurs in various types of aggression and violence, the most famous of which are analyzed in detail: cyberbullying, hating, trolling, grifting, etc.

Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization.

A key challenge of continual reinforcement learning (CRL) in dynamic environments is to promptly adapt the reinforcement learning (RL) agent's behavior as the environment changes over its lifetime while minimizing the catastrophic forgetting of the learned information. To address this challenge, in this article, we propose DaCoRL, that is, dynamics-adaptive continual RL. DaCoRL learns a context-conditioned policy using progressive contextualization, which incrementally clusters a stream of stationary tasks in the dynamic environment into a series of contexts and opts for an expandable multihead neural network to approximate the policy. Specifically, we define a set of tasks with similar dynamics as an environmental context and formalize context inference as a procedure of online Bayesian infinite Gaussian mixture clustering on environment features, resorting to online Bayesian inference to infer the posterior distribution over contexts. Under the assumption of a Chinese restaurant process (CRP) prior, this technique can accurately classify the current task as a previously seen context or instantiate a new context as needed without relying on any external indicator to signal environmental changes in advance. Furthermore, we employ an expandable multihead neural network whose output layer is synchronously expanded with the newly instantiated context and a knowledge distillation regularization term for retaining the performance on learned tasks. As a general framework that can be coupled with various deep RL algorithms, DaCoRL features consistent superiority over existing methods in terms of stability, overall performance, and generalization ability, as verified by extensive experiments on several robot navigation and MuJoCo locomotion tasks.

Digital transition, data-and-tasks crowd-based economy, and the shared social progress: unveiling a new political economy from a European perspective

The current unequal distribution of the benefits of digital transition and data-and-tasks crowd-based economy is directly linked to an incomplete interpretation of the behaviour of economic agents and their effects. This article proposes a new economic vision of digitalisation based on overcoming the postulates of neoclassical (automation-oriented and implemented mainly in the US) and autocratic (control-oriented and implemented mainly in China) economics. It suggests recovering the evolutionary, social, institutional, ethical and humanistic precepts of political economy. Following this more plural and European conceptual orientation, the analysis shows that current inequality arises from a digital R&D&I process plagued by network, platform, bias and polarisation effects, from business models that foster automation, and from the emergence of large superstar firms or control organisations with excessive market and political power. The redirection of technological progress towards more democratic individual values and shared social welfare requires digital governance based on new incentives and taxes, a greater strategic and political orientation towards social sustainability, and a new transformative role for the State.

Nudging law enforcement: evidence from low priority initiatives

The effect of non-binding guidelines (nudges) on citizens’ compliance with law has been studied theoretically and empirically in behavioral law and economics. Less is known about the impact of non-binding guidelines on the behavior of law enforcement agents and prosecution. Our work fills this gap. We study whether non-binding guidelines affect law enforcement and prosecution practices when the guidelines are not necessarily aligned with legal actors’ preferences. Our empirical analysis focuses on the impact of the adoption of a low priority initiative (LPI) on police and prosecutor behavior in Los Angeles County. Our results suggest that following the introduction of an LPI there is a rise in the number of misdemeanor arrests, but not in the rate that misdemeanor marijuana offenses are dismissed. We conclude that law enforcement preferences have a counterbalancing effect given that police officers may have strong punitive preferences. Prosecutors do not appear to respond to non-binding guidelines as evidenced by no change in the rate of dismissing cases.

Femininity Is Favorable: Sexually Dimorphic Facial Features Affect Assessments of White Women's Leadership Abilities

Women remain underrepresented in leadership positions traditionally held by men. Research on role congruity and backlash has shown that aspiring women versus men leaders are more negatively evaluated when they enact agentic behaviors. We examined whether sexually dimorphic facial features, which are associated with agentic and communal trait impressions, constitute a nonverbal barrier to White women's leadership in a college setting. Manipulated masculinized versus feminized facial features elicited, respectively, higher dominance and lower warmth impressions. Aspiring women leaders with masculinized versus feminized facial features received less favorable evaluations for several leadership roles, whereas men's evaluations were unaffected by varying features. Contrasting past work, aspiring women leaders were overall more favorably evaluated. This difference related to beliefs that college-aged White women versus men are more competent, responsible, and warm. These findings provide novel evidence that feminized facial features offer a unique advantage to aspiring college-aged White women leaders.

Assessment of an Agent’s Wayfinding of the Urban Environment Through Reinforcement Learning

This simulation study explores wayfinding motivated behavioral patterns in the city through agent-based modelling. Agents were trained using Unity’s ML-Agents toolkit with reinforcement learning. The study uses the Sultan Ahmet Mosque and its surrounding boundary as a model environment for the training of an agent’s wayfinding. Agents are trained to locate the Sultan Ahmet Mosque target. The behaviors of agents trained with two different methods, “Complex” and “Simple” learning, comparing their navigation quests at various difficulty levels featuring respawn points. After the training of the agents, the alternative routes produced while attaining the target during the wayfinding process were analyzed. As an outcome of the analysis, it was observed that the agents were prone to go off-route, navigate to different locations they perceived in the urban space, and then would reach the target. This occurrence is justified as an agent’s curiosity trained through reinforcement learning. This study differs from the literature in a way that it attempts to understand the navigational behavior of agents that were trained with reinforcement learning. Moreover, this research discusses the perception of wayfinding through curiosity and aims to make a comprehension of the perception of the city, which is one of the key ideas in neurourbanism. The study contributes to the literature by showing that wayfinding behaviors acquired from agents’ curiosity-driven explorations and past experiences can be an input for neurourbanism, supporting urban design. It informs urban enhancements that are user-centric and rich in urban perception using the reinforcement learning method.

Learning in Inverse Optimization: Incenter Cost, Augmented Suboptimality Loss, and Algorithms

Enhancing the Efficiency and Accuracy of Inverse Optimization Inverse optimization (IO) is used to model the behavior of decision-making agents who solve optimization problems in response to external signals. Inspired by the geometry of IO problems, in “Learning in Inverse Optimization: Incenter Cost, Augmented Suboptimality Loss, and Algorithms,” Zattoni Scroccaro, Atasoy, and Mohajerin Esfahani propose the “incenter” concept to solve IO problems, which unlike previously proposed approaches, can be used to derive computationally tractable solutions to this modeling problem. Moreover, they also propose a novel loss function for IO problems and a tailored optimization algorithm to optimize it. Extensive numerical experiments showcase the improved efficiency and accuracy of the proposed IO formulations and algorithm.

Автономное следование безэкипажного судна по траектории и предотвращение столкновений

In this paper we propose a method to solve the problem of autonomous vessel navigation in highly uncertain conditions. The primary goal was to make the Reinforcement Learning agent to learn the policy allowing the AV to follow a certain path while avoid collisions with any other objects. Maneuvering of AV in the proposed conditions is the principal subject of this paper. Several scenarios were explored during the research with static and dynamic objects. The agent was trained with model free off-policy algorithms. The training process was divided in several parts where we were experimenting with meta-learning approach elements to achieve the robustness of the agent behavior.

A cooperative perception based adaptive signal control under early deployment of connected and automated vehicles

Connected vehicle-based adaptive traffic signal control requires certain market penetration rates (MPRs) to be effective, usually exceeding 10%. Cooperative perception based on connected and automated vehicles (CAVs) can effectively improve overall data collection efficiency and reduce required MPR. However, the distribution of observed vehicles under cooperative perception is highly skewed and imbalanced, especially under very low CAV MPRs (e.g., 1%). To address this challenge, this paper proposes a novel deep reinforcement learning-based adaptive traffic signal control (RL-TSC) method that integrates a traffic flow model, known as the cell transmission model (CTM), denoted as CAVLight. Traffic states estimated from the CTM are integrated with the data collected from the cooperative perception environment to update the states in the CAVLight model. The design of reward function aims for reducing total vehicle delays and stabilizing agent behaviors. Extensive numerical experiments under a real-world intersection with varying traffic demand levels and CAV MPRs are conducted to compare the performance of CAVLight and other benchmark algorithms, including a fixed-time controller, an actuated controller, the max pressure model, and an optimization-based adaptive TSC. Results demonstrate the superiority of CAVLight in performance and generalizability over benchmarks, especially under 1% CAV MPR scenario with high traffic demands. The influence of CTM integration on CAVLight is further explored through RL agent policy visualization and sensitivity analysis in CTM parameters and CAV perception capabilities (i.e., detection range and detection accuracy).