Behavior Of Agents Research Articles

Penetration of Non-Adhesive Gel-like Embolic Materials During Dural Vessels Embolization According to Characteristics of Tantalum Powder

Tantalum powder is included in the composition of Non-Adhesive Gel-like Embolic Materials (NAGLEMs) for X-ray opacity. The duration of X-ray opacity during embolization is primarily associated with the particle size, which differs in the most used NAGLEMs—ONYX (Medtronic) and SQUID (Balt). NAGLEMs are widely used for the embolization of branches of the middle meningeal artery (MMA) in patients with chronic subdural hematomas (CSDHs). Considering the size (5–15 microns) of the target dural vessels, we assumed that not only the viscosity of NAGLEMs, but also the size and shape of tantalum granules may be important for the penetration of these gel-like embolic agents and determine their behavior. A notable discrepancy in size was observed. The medium-sized granules in the SQUID 18 sample (0.443 ± 0.086 microns, M ± SD) were found to be approximately ten times smaller than the tantalum granules in the ONYX 18 sample (5.2 ± 0.33 microns, M ± SD).Tantalum granules in SQUID 18 have a regular spherical shape; in ONYX 18 they have an irregular angular shape. When comparing the behavior of gel-like embolic agents of the same viscosity during MMA embolization in patients with CSDHs (an average age of 62.2 ± 14.3 years) in the group where SQUID 18 (n = 8) was used, the gel-like embolic agent in dural vessels demonstrated significantly greater penetration ability compared with the group where ONYX 18 (n = 8) was used. Accordingly, not only the viscosity of NAGLEMs, but also the size and shape of tantalum granules can have a significant effect on the penetration ability of gel compositions.

Study on the Performances of Toughening UV-LED-Cured Epoxy Electronic Encapsulants

This study aims to investigate the effects of three toughening agents—core–shell rubber particles (CSR), nano-silica particles (NSPs), and epoxidized polybutadiene (EPB)—on the performance of UV-LED-cured epoxy electronic encapsulants. By systematically comparing the curing behavior, thermomechanical properties, and impact resistance of different toughening agents in alicyclic epoxy resins, their potential applications in more environmentally friendly UV-cured electronic encapsulation are evaluated. The results show that NSP and CSR toughened samples have fast cured speed under 365 nm UV-LED light, but it affects the depth of curing under low energy conditions. They maintain high Tg, high modulus, and low thermal expansion coefficient (CTE), especially in the NSP-toughened sample. The EPB-toughened sample has good transparency for LED, but it has negative effects on Tg and CTE. This research provides essential theoretical and experimental data to support the development of high-performance UV-LED-cured epoxy encapsulation materials.

Nudging law enforcement: evidence from low priority initiatives

AbstractThe effect of non-binding guidelines (nudges) on citizens’ compliance with law has been studied theoretically and empirically in behavioral law and economics. Less is known about the impact of non-binding guidelines on the behavior of law enforcement agents and prosecution. Our work fills this gap. We study whether non-binding guidelines affect law enforcement and prosecution practices when the guidelines are not necessarily aligned with legal actors’ preferences. Our empirical analysis focuses on the impact of the adoption of a low priority initiative (LPI) on police and prosecutor behavior in Los Angeles County. Our results suggest that following the introduction of an LPI there is a rise in the number of misdemeanor arrests, but not in the rate that misdemeanor marijuana offenses are dismissed. We conclude that law enforcement preferences have a counterbalancing effect given that police officers may have strong punitive preferences. Prosecutors do not appear to respond to non-binding guidelines as evidenced by no change in the rate of dismissing cases.

Don't touch me! Mating not always preceded by courtship behavior in predatory coccinellids.

Knowledge of the ecology and behavior of biological control agents is essential for their effective use in biocontrol and Integrated Pest Management (IPM)programs. There is a lack of information regarding the courtship and mating traits that are crucial for successful mating in biocontrol population of coccinellids. To expand our current understanding in this area, 2 coccinellid species commonly used for the biocontrol of soft-bodied insects, Cryptolaemus montrouzieri (Mulsant) and Propylea quatuordecimpunctata (Linnaeus) (Coleoptera: Coccinellidae), were studied. A quantitative analysis was performed to compare their courtship and mating displays. Key behavioral traits of the precopulatory, copulatory, and postcopulatory phases were investigated. The courtship and mating sequences of C. montrouzieri and P. quatuordecimpunctata were similar to each other. During the mating phase, the key displays were the opening of the elytrae and wings followed by body shaking in C. montrouzieri and leg tapping followed by body shaking in P. quatuordecimpunctata. The mating success of both species was not correlated with any courtship display, such as antennal tapping. Only in C. montrouzieri mounting attempt showed a higher frequency from the backside without affecting male mating success. This research adds baseline knowledge about the courtship and mating behavior of the biocontrol population of coccinellids, contributing to the identification of potentiallyuseful benchmarks (e.g., body shaking, leg tapping) for monitoring prolonged mass-rearing processes, thusreducing mating failures.

Formal contracts mitigate social dilemmas in multi-agent reinforcement learning

Multi-agent Reinforcement Learning (MARL) is a powerful tool for training autonomous agents acting independently in a common environment. However, it can lead to sub-optimal behavior when individual incentives and group incentives diverge. Humans are remarkably capable at solving these social dilemmas. It is an open problem in MARL to replicate such cooperative behaviors in selfish agents. In this work, we draw upon the idea of formal contracting from economics to overcome diverging incentives between agents in MARL. We propose an augmentation to a Markov game where agents voluntarily agree to binding transfers of reward, under pre-specified conditions. Our contributions are theoretical and empirical. First, we show that this augmentation makes all subgame-perfect equilibria of all Fully Observable Markov Games exhibit socially optimal behavior, given a sufficiently rich space of contracts. Next, we show that for general contract spaces, and even under partial observability, richer contract spaces lead to higher welfare. Hence, contract space design solves an exploration-exploitation tradeoff, sidestepping incentive issues. We complement our theoretical analysis with experiments. Issues of exploration in the contracting augmentation are mitigated using a training methodology inspired by multi-objective reinforcement learning: Multi-Objective Contract Augmentation Learning. We test our methodology in static, single-move games, as well as dynamic domains that simulate traffic, pollution management, and common pool resource management.

A multi-theoretical approach to supply chain management

The article examines the supply chain as a modeling object. The multi-criteria and contradictory goals in supply chain management are: on the one hand, the maximum level of service, on the other hand, minimizing logistics costs. Solving strategic optimization problems for this class of objects using classical optimization methods is difficult. The theoretical and methodological basis of the supply chain was considered from the perspective of a set-theoretic approach. The purpose of the study is to analyze the behavior of many different agents (suppliers, clients, etc.) with different needs and goals when making decisions. An agent is an active element of the system, possessing a certain autonomy and the ability to make independent decisions, relying on the information it has about the state of the environment and the actions of other agents. In turn, supply chains are a complex organizational (active) system, i.e. associated with human participation. The study of behavioral aspects and interorganizational interaction, the formation of new organizational forms are an extremely important aspect in the management and modeling of supply chains, a factor in the dynamic behavior of the logistics system. The conducted research represents an important contribution to the development of knowledge about the application of the set-theoretic approach in supply chain management. It demonstrates the applicability of this approach in practice through an analysis of a specific company, LLP «Knauf», on the Kazakhstan market.

Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualization.

A key challenge of continual reinforcement learning (CRL) in dynamic environments is to promptly adapt the reinforcement learning (RL) agent's behavior as the environment changes over its lifetime while minimizing the catastrophic forgetting of the learned information. To address this challenge, in this article, we propose DaCoRL, that is, dynamics-adaptive continual RL. DaCoRL learns a context-conditioned policy using progressive contextualization, which incrementally clusters a stream of stationary tasks in the dynamic environment into a series of contexts and opts for an expandable multihead neural network to approximate the policy. Specifically, we define a set of tasks with similar dynamics as an environmental context and formalize context inference as a procedure of online Bayesian infinite Gaussian mixture clustering on environment features, resorting to online Bayesian inference to infer the posterior distribution over contexts. Under the assumption of a Chinese restaurant process (CRP) prior, this technique can accurately classify the current task as a previously seen context or instantiate a new context as needed without relying on any external indicator to signal environmental changes in advance. Furthermore, we employ an expandable multihead neural network whose output layer is synchronously expanded with the newly instantiated context and a knowledge distillation regularization term for retaining the performance on learned tasks. As a general framework that can be coupled with various deep RL algorithms, DaCoRL features consistent superiority over existing methods in terms of stability, overall performance, and generalization ability, as verified by extensive experiments on several robot navigation and MuJoCo locomotion tasks.

Digital transition, data-and-tasks crowd-based economy, and the shared social progress: unveiling a new political economy from a European perspective

The current unequal distribution of the benefits of digital transition and data-and-tasks crowd-based economy is directly linked to an incomplete interpretation of the behaviour of economic agents and their effects. This article proposes a new economic vision of digitalisation based on overcoming the postulates of neoclassical (automation-oriented and implemented mainly in the US) and autocratic (control-oriented and implemented mainly in China) economics. It suggests recovering the evolutionary, social, institutional, ethical and humanistic precepts of political economy. Following this more plural and European conceptual orientation, the analysis shows that current inequality arises from a digital R&D&I process plagued by network, platform, bias and polarisation effects, from business models that foster automation, and from the emergence of large superstar firms or control organisations with excessive market and political power. The redirection of technological progress towards more democratic individual values and shared social welfare requires digital governance based on new incentives and taxes, a greater strategic and political orientation towards social sustainability, and a new transformative role for the State.

Femininity Is Favorable: Sexually Dimorphic Facial Features Affect Assessments of White Women's Leadership Abilities

Women remain underrepresented in leadership positions traditionally held by men. Research on role congruity and backlash has shown that aspiring women versus men leaders are more negatively evaluated when they enact agentic behaviors. We examined whether sexually dimorphic facial features, which are associated with agentic and communal trait impressions, constitute a nonverbal barrier to White women's leadership in a college setting. Manipulated masculinized versus feminized facial features elicited, respectively, higher dominance and lower warmth impressions. Aspiring women leaders with masculinized versus feminized facial features received less favorable evaluations for several leadership roles, whereas men's evaluations were unaffected by varying features. Contrasting past work, aspiring women leaders were overall more favorably evaluated. This difference related to beliefs that college-aged White women versus men are more competent, responsible, and warm. These findings provide novel evidence that feminized facial features offer a unique advantage to aspiring college-aged White women leaders.

Assessment of an Agent’s Wayfinding of the Urban Environment Through Reinforcement Learning

This simulation study explores wayfinding motivated behavioral patterns in the city through agent-based modelling. Agents were trained using Unity’s ML-Agents toolkit with reinforcement learning. The study uses the Sultan Ahmet Mosque and its surrounding boundary as a model environment for the training of an agent’s wayfinding. Agents are trained to locate the Sultan Ahmet Mosque target. The behaviors of agents trained with two different methods, “Complex” and “Simple” learning, comparing their navigation quests at various difficulty levels featuring respawn points. After the training of the agents, the alternative routes produced while attaining the target during the wayfinding process were analyzed. As an outcome of the analysis, it was observed that the agents were prone to go off-route, navigate to different locations they perceived in the urban space, and then would reach the target. This occurrence is justified as an agent’s curiosity trained through reinforcement learning. This study differs from the literature in a way that it attempts to understand the navigational behavior of agents that were trained with reinforcement learning. Moreover, this research discusses the perception of wayfinding through curiosity and aims to make a comprehension of the perception of the city, which is one of the key ideas in neurourbanism. The study contributes to the literature by showing that wayfinding behaviors acquired from agents’ curiosity-driven explorations and past experiences can be an input for neurourbanism, supporting urban design. It informs urban enhancements that are user-centric and rich in urban perception using the reinforcement learning method.

Learning in Inverse Optimization: Incenter Cost, Augmented Suboptimality Loss, and Algorithms

Enhancing the Efficiency and Accuracy of Inverse Optimization Inverse optimization (IO) is used to model the behavior of decision-making agents who solve optimization problems in response to external signals. Inspired by the geometry of IO problems, in “Learning in Inverse Optimization: Incenter Cost, Augmented Suboptimality Loss, and Algorithms,” Zattoni Scroccaro, Atasoy, and Mohajerin Esfahani propose the “incenter” concept to solve IO problems, which unlike previously proposed approaches, can be used to derive computationally tractable solutions to this modeling problem. Moreover, they also propose a novel loss function for IO problems and a tailored optimization algorithm to optimize it. Extensive numerical experiments showcase the improved efficiency and accuracy of the proposed IO formulations and algorithm.

Автономное следование безэкипажного судна по траектории и предотвращение столкновений

In this paper we propose a method to solve the problem of autonomous vessel navigation in highly uncertain conditions. The primary goal was to make the Reinforcement Learning agent to learn the policy allowing the AV to follow a certain path while avoid collisions with any other objects. Maneuvering of AV in the proposed conditions is the principal subject of this paper. Several scenarios were explored during the research with static and dynamic objects. The agent was trained with model free off-policy algorithms. The training process was divided in several parts where we were experimenting with meta-learning approach elements to achieve the robustness of the agent behavior.

A cooperative perception based adaptive signal control under early deployment of connected and automated vehicles

Connected vehicle-based adaptive traffic signal control requires certain market penetration rates (MPRs) to be effective, usually exceeding 10%. Cooperative perception based on connected and automated vehicles (CAVs) can effectively improve overall data collection efficiency and reduce required MPR. However, the distribution of observed vehicles under cooperative perception is highly skewed and imbalanced, especially under very low CAV MPRs (e.g., 1%). To address this challenge, this paper proposes a novel deep reinforcement learning-based adaptive traffic signal control (RL-TSC) method that integrates a traffic flow model, known as the cell transmission model (CTM), denoted as CAVLight. Traffic states estimated from the CTM are integrated with the data collected from the cooperative perception environment to update the states in the CAVLight model. The design of reward function aims for reducing total vehicle delays and stabilizing agent behaviors. Extensive numerical experiments under a real-world intersection with varying traffic demand levels and CAV MPRs are conducted to compare the performance of CAVLight and other benchmark algorithms, including a fixed-time controller, an actuated controller, the max pressure model, and an optimization-based adaptive TSC. Results demonstrate the superiority of CAVLight in performance and generalizability over benchmarks, especially under 1% CAV MPR scenario with high traffic demands. The influence of CTM integration on CAVLight is further explored through RL agent policy visualization and sensitivity analysis in CTM parameters and CAV perception capabilities (i.e., detection range and detection accuracy).

Multi-agent statistically discriminative sub-trajectory mining and an application to NBA basketball

Abstract Improvements in tracking technology through optical and computer vision systems have enabled a greater understanding of the movement-based behaviour of multiple agents, including in team sports. In this study, a multi-agent statistically discriminative sub-trajectory mining (MA-Stat-DSM) method is proposed that takes a set of binary-labelled agent trajectory matrices as input and incorporates Hausdorff distance to identify sub-matrices that statistically significantly discriminate between the two groups of labelled trajectory matrices. Utilizing 2015/16 SportVU NBA tracking data, agent trajectory matrices representing attacks consisting of the trajectories of five agents (the ball, shooter, last passer, shooter defender, and last passer defender), were truncated to correspond to the time interval following the receipt of the ball by the last passer, and labelled as effective or ineffective based on a definition of attack effectiveness that we devise in the current study. After identifying appropriate parameters for MA-Stat-DSM by iteratively applying it to all matches involving the two top- and two bottom-placed teams from the 2015/16 NBA season, the method was then applied to selected matches and could identify and visualize the portions of plays, e.g., involving passing, on-, and/or off-the-ball movements, which were most relevant in rendering attacks effective or ineffective.

Stochastic control of geological carbon storage operations using geophysical monitoring and deep reinforcement learning

Geological carbon storage (GCS) is the process of injecting and storing carbon dioxide (CO2) in the subsurface to reduce greenhouse gas emissions. Safe and profitable GCS operations require effective decision-making in the presence of uncertain geological models, a process which can often be facilitated with geophysical monitoring. In this study, we examine how sequential decision-making algorithms can be combined with geophysical measurements for the optimal control of GCS operations. Specifically, we develop an artificial intelligence model using deep reinforcement learning (DRL) that takes geophysical time-lapse gravity and well pressure monitoring data as input and delivers an optimal CO2 injection policy. The objective of the problem at hand is to maximize the profit of a hypothetical GCS operation while mitigating the potential for induced seismicity, by training DRL agents using combined geostatistical, reservoir and geophysical simulation. Comparisons against two benchmarks – a constant injection strategy and an injection schedule optimized using a commercial reservoir simulator toolbox – show that the stochastic control of such operations from subsurface monitoring data using deep reinforcement learning is feasible. Evaluation results show that DRL agent behavior generates profits which are on average 1 to 8 percent higher than what is possible through a constant injection approach. Furthermore, we show that DRL can generate optimal injection policies applicable to the true (yet previously unseen) subsurface given carefully managed levels of uncertainty.

A mean-field game model of price formation with price-dependent agent behavior

A mean-field game model of price formation with price-dependent agent behavior

Towards emotion-aware intelligent agents by utilizing knowledge graphs of experiences

Because of the increasing presence of intelligent agents in various aspects of human social life, social skills play a vital role in ensuring these systems exhibit acceptable and realistic behavior in social communication. The importance of emotional intelligence in social capabilities is noteworthy, so incorporating emotions into the behaviors of intelligent agents is essential. Therefore, some computational models of emotions have been presented to develop intelligent agents that exhibit emotional human-like behaviors. However, most current computational models of emotions neglect the dynamic learning of the affective meaning of events based on agents’ experiences. Such models evaluate the events in the environment according to emotional aspects without considering the context of the situations. Also, these models capture the emotional states of agents by using predefined rules determined according to psychological theories. Therefore, they disregard the data-driven methods that can obtain the relationships between appraisal variables and emotions based on natural human data with fewer assumptions on the nature of such relationships. To address these issues, we proposed a novel and unified affective-cognitive framework (EIAEC) to facilitate the development of emotion-aware intelligent agents. EIAEC uses appraisal theories to acquire the emotional states of the agent in various situations. This paper contains four main contributions: 1- We have designed an efficient episodic memory that uses events and their conditional contexts to store and retrieve knowledge and experiences. This memory facilitates emotional expressions and decision-making adapted to the situations of the agent. 2- A novel method has been proposed that learns context-dependent affective values associated with events by using the agent’s experiences in various contexts. Subsequently, we acquired appraisal variables using the elements and related meta-data in episodic memory. 3- We have proposed a new data-driven method that maps appraisal variables to emotional states. 4- Moreover, a method has been developed to update the activation values regarding actions by using the emotional states of the agent. This method models the influence of emotions on the agent’s decision-making. Finally, we simulate a driving scenarios in our proposed framework to manifest the generated emotions in different situations and conditions. Moreover, we show how the proposed method learns the affective meaning of events and actions used in appraisal computing.

Limitations in creating artificial populations in agent-based epidemic modeling: a systematic review

Introduction. The key step in agent-based modeling of epidemics, which allows researchers to take into account individual characteristics of people, is the creation of an artificial population. The main difficulty of this procedure is finding a balance between the detail of the population description and the computational efficiency of the calculations. The aim and objectives of the review: Critically analyze and summarize the current evidence on how to create artificial populations; evaluate the limitations and advantages of available approaches in solving various problems in epidemiology. Materials and methods. An analysis of literature sources devoted to agent-based modeling has been performed. The analysis is focused on algorithms for creating an artificial population with a given level of detail for modeling human respiratory infections. Results. The approaches to the creation of artificial populations are generalized. The main principles of realization of interaction between agents are revealed: by means of networks of contacts between agents and on the basis of taking into account the movement of agents between locations. The first approach is the most computationally efficient and simple; the second approach allows to better take into account the change in the behavior of agents during the development of the epidemic process. Conclusion. Agent-based modeling is an optimal tool for selecting the best scenario for epidemic control and investigating the role of individual characteristics of people in the development of epidemics. When creating an artificial population, it is important to include in the model factors that can be targeted for control. A significant limitation is the lack of factual data on population structure, but this can be overcome by using indirect data.

Özgür ÖZDAMAR and Sercan CANBOLAT, Leaders in the Middle East and North Africa: How Ideology Shapes Foreign Policy

It is almost always the case that any Introduction to International Relations course teaches the contributions of Kenneth Waltz to the field, firstly through his grandiose formulation of a theory: it should be simple, parsimonious, abstract, and accordingly, one needs to move away from reality as much as possible to increase the theory’s explanatory and predictive capacity. Moving the individual and state level of analyses aside, Waltz simplifies his theory of international politics at the system/structural level and, with his neorealist theory becoming the dominant approach for a large part of the twentieth century, it put the state in a sort of black box, purposefully ignored individual actors, and targeted the maximum degree of abstraction as possible. In Leaders in the Middle East and North Africa: How Ideology Shapes Foreign Policy, which consists of seven chapters (one introduction chapter, four empirical chapters, one theoretical conclusions chapter and one policy implications chapter), Özdamar and Canbolat aim to shed light on foreign policy belief patterns of leaders in the Middle East and North Africa (MENA), highlighting divergences across them and comparing them to the average world leadership. In the process, and in total opposition to Waltz’s claim, the authors argue that advancing “the actor-specific empirical studies zeroing in on agent behaviors and decisions in the future” is “the only way for IR to establish itself as a scientific discipline” (p. 148). In other words, as opposed to the structural approaches, the authors propound that less abstraction and a more actor-specific, nuanced, and tailored approach would provide significant opportunities for the IR discipline to be scientific.

Playful exercise focused on microeconomics, applying gamification: “Rompeconomía”

Microeconomics is a branch of economics that focuses on the behavior of individual economic agents, such as consumers, businesses, and workers. Coupled with this, it analyzes how they interact in the market to determine supply and demand, prices and the allocation of resources. It is a fundamental tool to understand the economy and how it works in daily life. Based on this, the development of a recreational activity was carried out in order to strengthen theoretical knowledge, as well as the different market structures, in Industrial Engineering students from the Industrial University of Santander who are taking the subject "Economic environment”. To develop the activity, we worked in small groups through phases, which consist of identifying necessary aspects to prepare them based on a case study, focused on a market structure, which they must analyze in detail. This research was carried out using the research - participative action, (IAP) methodology; which allowed the identification of recreational activities to develop skills in students as economic agents, while they are instructed in aspects such as market studies, microeconomics and group work. Gamification as a tool for learning constitutes a teaching alternative to the challenges faced by higher education in contexts where the way of obtaining, processing and transmitting knowledge is transformed.