Behavior Of Agents Research Articles

Streptomyces and their specialised metabolites for phytopathogen control - comparative in vitro and in planta metabolic approaches.

In the search for new crop protection microbial biocontrol agents, isolates from the genus Streptomyces are commonly found with promising attributes. Streptomyces are natural soil dwellers and have evolved as plant symbionts producing specialised metabolites with antibiotic and antifungal activities. Streptomyces biocontrol strains can effectively suppress plant pathogens via direct antimicrobial activity, but also induce plant resistance through indirect biosynthetic pathways. The investigation of factors stimulating the production and release of Streptomyces bioactive compounds is commonly conducted in vitro, between Streptomyces sp. and a plant pathogen. However, recent research is starting to shed light on the behaviour of these biocontrol agents in planta, where the biotic and abiotic conditions share little similarity to those of controlled laboratory conditions. With a focus on specialised metabolites, this review details (i) the various methods by which Streptomyces biocontrol agents employ specialised metabolites as an additional line of defence against plant pathogens, (ii) the signals shared in the tripartite system of plant, pathogen and biocontrol agent, and (iii) an outlook on new approaches to expedite the identification and ecological understanding of these metabolites under a crop protection lens.

A Unified and Efficient Coordinating Framework for Autonomous DBMS Tuning

Recently using machine learning (ML) based techniques to optimize the performance of modern database management systems (DBMSs) has attracted intensive interest from both industry and academia. With an objective to tune a specific component of a DBMS (e.g., index selection, knobs tuning), the ML-based tuning agents have shown to be able to find better configurations than experienced database administrators (DBAs). However, one critical yet challenging question remains unexplored -- how to make those ML-based tuning agents work collaboratively. Existing methods do not consider the dependencies among the multiple agents, and the model used by each agent only studies the effect of changing the configurations in a single component. To tune different components for DBMS, a coordinating mechanism is needed to make the multiple agents be cognizant of each other. Also, we need to decide how to allocate the limited tuning budget (e.g., time and resources) among the agents to maximize the performance. Such a decision is difficult to make since the distribution of the reward (i.e., performance improvement) corresponding to each agent is unknown and non-stationary. In this paper, we study the above question and present a unified coordinating framework to efficiently utilize existing ML-based agents. First, we propose a message propagation protocol that specifies the collaboration behaviors for agents and encapsulates the global tuning messages in each agent's model. Second, we combine Thompson Sampling, a well-studied reinforcement learning algorithm with a memory buffer so that our framework can allocate the tuning budget judiciously in a non-stationary environment. Our framework defines the interfaces adapted to a broad class of ML-based tuning agents, yet simple enough for integration with existing implementations and future extensions. Based on extensive evaluations, we show that this framework can effectively utilize different ML-based agents and find better configurations with 1.4~14.1x speedups on the workload execution time compared with baselines.

GENESIS OF THEORIES OF INSTITUTIONAL SYSTEM DEVELOPMENT

The institutional system has become an integral part of society's development. The birth and creation of new institutes and institutions, the demise of obsolete ones, the fluidity of the institutional environment, structural changes in the institutional system give rise to the constant need to study the existing institutional system, make adjustments to it, raise the question of compliance with the ever-transforming economic system. These processes observed in institutes and institutions, their interaction, in the institutional environment, society require systematic streamlining, continuous improvement of theoretical principles, their implementation in practice, clarifying the results of the recommendations. Goal. The aim of the article was to study theories of institutional system development and their evolution in the historical period, to identify current problems and trends in the functioning of the institutional system. Method. General scientific and special methods of scientific cognition are used for the purpose. During the work, logical, system-structural methods, the method of generalization was used for comprehensive consideration of the object of research and the completeness of the results. The methodological basis for study of genesis of theories of development of the institutional system are systems analysis, synergetics. Results. The study of the genesis of theories of institutional system development considers their evolution in the historical aspect, identifies current problems of institutional system development and analyzes trends in the functioning of the institutional system. We consider clarifying the conditions for the emergence of the system to be a modern problem in the development of the institutional system; definition of basic institutes and institutions; study of the influence of institutions and institutions on the behavior of economic agents; determination of the necessary list and number of institutions, which are empowered to control the implementation of the existing system of rules, forms and norms of the existing institutional system in society; creating an institutional environment appropriate to the existing institutional system. Scientific novelty. On the basis of theoretical analysis and practical experience, current problems and trends in the functioning of the institutional system have been identified. Practical significance. The genesis of theories of institutional system development is an interest for the correct perception of the institutional system and the adoption of appropriate decisions regarding alternatives to its development.

Investigating the role of the right inferior frontal gyrus in control perception: A double-blind cross-over study using ultrasonic neuromodulation

Being able to control inner and environmental states is a basic need of living creatures. The perception of such control is based on the perceived ratio of outcome probabilities given the presence and the absence of agentic behavior. If an organism believes that options exist to change the probability of a given outcome, control perception (CP) may emerge. Nonetheless, regarding this model, not much is known about how the brain processes CP from this information. This study uses low-intensity transcranial focused ultrasound neuromodulation in a randomized-controlled double blind cross-over design to investigate the impact of the right inferior frontal gyrus of the lateral prefrontal cortex (lPFC) on this process. 39 healthy participants visited the laboratory twice (once in a sham, once in a neuromodulation condition) and rated their control perception regarding a classical control illusion task. EEG alpha and theta power density were analyzed in a hierarchical single trial-based mixed modeling approach. Results indicate that the litFUS neuromodulation changed the processing of stimulus probability without changing CP. Furthermore, neuromodulation of the right lPFC was found to modulate mid-frontal theta by altering its relationship with self-reported effort and worrying. While these data indicate lateral prefrontal sensitivity to stimulus probability, no evidence emerged for the dependency of CP on this processing.

The Robustness and Vulnerability of a Complex Adaptive System With Co-Evolving Agent Behavior and Local Structure

Agents' irrational behavior would lead to local configuration of complex adaptive system percolation. The corresponding critical point is key to making decisions for improving the system or keeping the system from collapsing. The authors construct a complex adaptive system model where agent behavior and its local configuration co-evolve. This model shows, when an arbitrary agent and its neighbors change their strategy and local interactive configuration, how the properties of percolation critical point of this system would emerge under random attack and intentional attack. It is shown that the system is robust if it is attacked randomly, and there are always at least two large components keeping the system connected. However, if the system is attacked intentionally, the result is more interesting. The system is robust if the deleting probability is smaller than a certain critical probability c0, but the system is vulnerable if the probability is larger than c0. Furthermore, the critical probability c0 is determined by the agent payoff, system structure, and noise.

Imitation Learning for Robotics: Progress, Challenges, and Applications in Manipulation and Teleoperation

The evolution of robotics has shifted its applications from industrial environments to more intelligent service scenarios, requiring adaptability in complex and uncertain conditions. Traditional manual coding methods struggle in these dynamic environments, making imitation learning (IL) a valuable approach for robotic manipulation by leveraging expert demonstrations. In this paper, I provide a review of the state-of-the-art in IL for robotic manipulation, focusing on three key aspects: demonstration, representation, and learning algorithms. I outline IL’s development history, taxonomies, and key milestones while discussing the challenges associated with learning strategies, such as dependency on demonstration quality and task-specific limitations. I also highlight potential areas for future research, including learning from suboptimal demonstrations, incorporating voice instructions, and optimizing learning policies. Additionally, I review approaches like "Mimic," which enhances teleoperation by allowing users to record and reuse robot trajectories, and Interactive Imitation Learning techniques that use human feedback in state-space to improve agent behavior. This survey provides insights into the current challenges of IL in robotic manipulation and explores promising directions for further research.

Dynamic trajectory planning for ships in dense environment using collision grid with deep reinforcement learning

In areas with high maritime traffic, ship safety is of utmost importance when validating autonomous navigation models. While exact methods exist for specific situations, they are inadequate in a global context. This study employs an approximate deep reinforcement learning method to solve the navigation problem in a dense environment with numerous static and moving obstacles. Our model prioritizes ship integrity by enabling the agent to dynamically adapt its kinematics to its surroundings to reach a designated goal without colliding with obstacles. To achieve this, we incorporate collision grids in the form of danger zones as input to our model and train it using the proximal policy optimization algorithm. Additionally, we propose implementing the International Regulations for Preventing Collisions at Sea (COLREGs) in the collision grid as these navigation rules are necessary to obtain realistic behavior of an autonomous agent. The agent’s performance is evaluated on a set of randomly generated scenarios operating in an environment complexity similar to the one used during training. These tests demonstrate that this type of data structure allows a trained agent to navigate in a dense environment while adhering to the COLREGs with a success rate of 94.69%.

A Multi-agent Model for Cross-border Trading in the Continuous Intraday Electricity Market

The increasing importance of cross-border trade has brought the topic of allocating cross-border transmission capacity under the limelight. In this paper, we focus on two different transmission capacity calculation methods, namely, Available Transfer Capacity and Flow-based Capacity allocation, in the continuous intraday electricity market. The effect of these two methods is studied on the continuous intraday electricity market through an agent-based model comprising various types of trading agents. It also models market operator agent and transmission system operator agent inspired by the Single Intraday Coupling project in Europe. The price–volume decisions for the orders posted by the market participants are determined by two different strategies where one of them adapts to changing market conditions and new information while the other is naive. The model accounts for ramping constraints which enables the analysis of the trading behavior and interaction of agents across multiple delivery products simultaneously. A switch parameter is availed in the trading timeline for the thermal and storage agents to change from a less conservative approach of ignoring the ramping and charging/discharging rate constraints respectively to considering them. An interplay between different switch parameters and cross-border transmission capacity calculation methods is studied.

Free Agency and Determinism: Is There a Sensible Definition of Computational Sourcehood?

Can free agency be compatible with determinism? Compatibilists argue that the answer is yes, and it has been suggested that the computer science principle of "computational irreducibility" sheds light on this compatibility. It implies that there cannot, in general, be shortcuts to predict the behavior of agents, explaining why deterministic agents often appear to act freely. In this paper, we introduce a variant of computational irreducibility that intends to capture more accurately aspects of actual (as opposed to apparent) free agency, including computational sourcehood, i.e., the phenomenon that the successful prediction of a process' behavior must typically involve an almost-exact representation of the relevant features of that process, regardless of the time it takes to arrive at the prediction. We argue that this can be understood as saying that the process itself is the source of its actions, and we conjecture that many computational processes have this property. The main contribution of this paper is technical, in that we analyze whether and how a sensible formal definition of computational sourcehood is possible. While we do not answer the question completely, we show how it is related to finding a particular simulation preorder on Turing machines, we uncover concrete stumbling blocks towards constructing such a definition, and demonstrate that structure-preserving (as opposed to merely simple or efficient) functions between levels of simulation play a crucial role.

Using Machine Learning to drive social learning in a Covid-19 Agent-Based Model

Abstract. Disease transmission and governmental interventions influence the spread of Covid-19. Models can be essential tools to optimise these governmental interventions. This requires the exploration of various ways to implement government agent behaviour. In Agent-Based Models (ABMs), government agent behaviour can be rule-based or data-driven, and the agent can be an isolated learner (using only its own data) or a social learner. We explore the creation of a data-driven social approach in which behaviour is based on a Machine Learning (ML) algorithm, and the government considers data from other European countries as input for their decision-making. Governmental actions start with risk perception, based on several parameters, e.g. the number of disease cases, deaths, and hospitalisation rate. The interventions are measured via the stringency index, measuring the simultaneous number of interventions (working from home, wearing a facemask, closing schools, etc.) taken. We test four machine learning algorithms (Bayesian Network (BN), c4.5, Naïve Bayes (NB) and Random Forest (RF)), using a 5-class and a 3-class classification of the stringency level. The algorithms are trained on disease data from many European countries. The best-performing algorithms were c4.5 and RF. The next step is to implement these algorithms into the ABM and evaluate the outcomes compared to the original model.

Towards trustworthy multi‐modal motion prediction: Holistic evaluation and interpretability of outputs

AbstractPredicting the motion of other road agents enables autonomous vehicles to perform safe and efficient path planning. This task is very complex, as the behaviour of road agents depends on many factors and the number of possible future trajectories can be considerable (multi‐modal). Most prior approaches proposed to address multi‐modal motion prediction are based on complex machine learning systems that have limited interpretability. Moreover, the metrics used in current benchmarks do not evaluate all aspects of the problem, such as the diversity and admissibility of the output. The authors aim to advance towards the design of trustworthy motion prediction systems, based on some of the requirements for the design of Trustworthy Artificial Intelligence. The focus is on evaluation criteria, robustness, and interpretability of outputs. First, the evaluation metrics are comprehensively analysed, the main gaps of current benchmarks are identified, and a new holistic evaluation framework is proposed. Then, a method for the assessment of spatial and temporal robustness is introduced by simulating noise in the perception system. To enhance the interpretability of the outputs and generate more balanced results in the proposed evaluation framework, an intent prediction layer that can be attached to multi‐modal motion prediction models is proposed. The effectiveness of this approach is assessed through a survey that explores different elements in the visualisation of the multi‐modal trajectories and intentions. The proposed approach and findings make a significant contribution to the development of trustworthy motion prediction systems for autonomous vehicles, advancing the field towards greater safety and reliability.

Natural Language Specification of Reinforcement Learning Policies Through Differentiable Decision Trees

Human-AI policy specification is a novel procedure we define in which humans can collaboratively warm-start a robot's reinforcement learning policy. This procedure is comprised of two steps; (1) Policy Specification, i.e. humans specifying the behavior they would like their companion robot to accomplish, and (2) Policy Optimization, i.e. the robot applying reinforcement learning to improve the initial policy. Existing approaches to enabling collaborative policy specification are often unintelligible black-box methods, and are not catered towards making the autonomous system accessible to a novice end-user. In this paper, we develop a novel collaborative framework to allow humans to initialize and interpret an autonomous agent's behavior. Through our framework, we enable humans to specify an initial behavior model via unstructured, natural language (NL), which we convert to lexical decision trees. Next, we leverage these translated specifications, to warm-start reinforcement learning and allow the agent to further optimize these potentially suboptimal policies. Our approach warm-starts an RL agent by utilizing non-expert natural language specifications without incurring the additional domain exploration costs. We validate our approach by showing that our model is able to produce <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$&gt;$</tex-math></inline-formula> 80% translation accuracy, and that policies initialized by a human can match the performance of relevant RL baselines in two domains.

Generating and Adapting to Diverse Ad Hoc Partners in Hanabi

Hanabi is a cooperative game that brings the problem of modeling other players to the forefront. In this game, coordinated groups of players can leverage pre-established conventions to great effect, but playing in an ad-hoc setting requires agents to adapt to its partner's strategies with no previous coordination. Evaluating an agent in this setting requires a diverse population of potential partners, but so far, the behavioral diversity of agents has not been considered in a systematic way. This paper proposes Quality Diversity algorithms as a promising class of algorithms to generate diverse populations for this purpose, and generates a population of diverse Hanabi agents using MAP-Elites. We also postulate that agents can benefit from a diverse population during training and implement a simple "meta-strategy" for adapting to an agent's perceived behavioral niche. We show this meta-strategy can work better than generalist strategies even outside the population it was trained with if its partner's behavioral niche can be correctly inferred, but in practice a partner's behavior depends and interferes with the meta-agent's own behavior, suggesting an avenue for future research in characterizing another agent's behavior during gameplay.

Agent behavior modeling method based on reinforcement learning and human in the loop

Computer generated force (CGF) is one of the increasingly important research topics in the field of simulation. However, low modeling efficiency and lack of adaptability are acute problems of traditional CGF modeling. In this study, a method for modeling the agent behavior based on reinforcement learning and human in the loop is proposed to improve the ability and efficiency of agent behavior modeling. First, an overall framework for modeling the behavior of intelligent agents is constructed based on the deep reinforcement learning algorithm Soft Actor Critic (SAC) framework. Second, in order to overcome the slow convergence speed of the SAC framework, a method for human interaction and value evaluation in the loop is introduced, and the specific algorithm flow is designed. Third, in order to verify the performance of the proposed method, experiments are conducted and compared with algorithms using a pure SAC framework based on an example of agent completing specific tasks. Result shows that after 100 episodes of training, the task completion rate of the agent can approach 100% while a pure SAC framework require at least 500 episodes of training to gradually improve the completion rate. Finally, the results demonstrate that the proposed method can significantly improve the efficiency of agent behavior modeling and the task completion rate increases with the number of human interventions in the loop.

Intermittent Learning Through Operant Conditioning for Cyber-Physical Systems.

This article presents a novel scheme, namely, an intermittent learning scheme based on Skinner's operant conditioning techniques that approximates the optimal policy while decreasing the usage of the communication buses transferring information. While traditional reinforcement learning schemes continuously evaluate and subsequently improve, every action taken by a specific learning agent based on received reinforcement signals, this form of continuous transmission of reinforcement signals and policy improvement signals can cause overutilization of the system's inherently limited resources. Moreover, the highly complex nature of the operating environment for cyber-physical systems (CPSs) creates a gap for malicious individuals to corrupt the signal transmissions between various components. The proposed schemes will increase uncertainty in the learning rate and the extinction rate of the acquired behavior of the learning agents. In this article, we investigate the use of fixed/variable interval and fixed/variable ratio schedules in CPSs along with their rate of success and loss in their optimal behavior incurred during intermittent learning. Simulation results show the efficacy of the proposed approach.

Market Power Mitigation in Transmission Expansion Planning Problems

The exercise of market power through network constraints in electricity markets can lead to high energy prices far from competitive prices. Traditional transmission expansion planning problem formulations do not consider strategic behavior of market agents. Therefore, they cannot capture the potential exercise of market power. In this paper, a predictor-corrector iterative algorithm is proposed to deal with market power mitigation in market-oriented transmission expansion planning problems. The predictor step consists of the solution of an equilibrium market model based on the conjectured supply function. The corrector step is a conventional transmission expansion planning posed as a mixed integer linear programming problem, where the feasible region is dynamically updated taking into account the results from the predictor step. Lerner index and other indices are used to quantify the potential market power. The algorithm finds the minimum cost expansion plan that avoids the exercise of market power through network congestion. The cost of this expansion plan is only slightly greater than the cost of a conventional expansion plan. The approach is illustrated using the 6-Bus Garver and the IEEE-24 RTS test systems.

Higher Order Barrier Certificates for Leader–Follower Multiagent Systems

This paper presents control strategies based on time-varying convergent higher order control barrier functions for a class of leader-follower multi-agent systems under signal temporal logic (STL) tasks. Each agent is assigned a local STL task which may be dependent on the behavior of agents involved in other tasks. We consider one or more than one leader in the multi-agent system. The leader has knowledge on the associated tasks and controls the performance of the subgroup involved agents. The followers are not aware of the tasks, and do not have any control authority to reach them. They follow the leader commands indirectly, according to their dynamics interconnections, for the task satisfaction. We further assume that the input-to-state stability (ISS) property for the multi-agent system is fulfilled. First, robust solutions for the task satisfaction, based on the leader's accessibility to the follower agents' states are suggested. In addition, using the notion of higher order barrier functions, individual barrier certificates for each agent evolving in a formation dynamic structure are proposed. For the case of presence of more leaders in the subgroups, we provide decentralized barrier certificates. Our approach finds solutions to guarantee the satisfaction of STL tasks independent of the agents' initial conditions.

Sequential Bayesian inference for agent-based models with application to the Chinese business cycle

Agent-based models (ABMs), which model the evolving system through the interactive behavior of agents, are often challenged by inaccurate estimates. However, estimates provided by traditional approaches are sensitive to initial values due to irregular and multimodal likelihood functions. In this paper, we extend the Sequential Monte Carlo (SMC) algorithm for Bayesian posterior inference to ABMs, and used the kernel density estimation (KDE) to approximate the likelihood function. Our simulation results demonstrate that this method delivers advantages for parameters estimation in addressing multimodal likelihood, especially for large samples. Applying this approach to China’s macroeconomic data, we find that exogenous shocks in the agent-based model play a small role in explaining China’s business cycle. The SMC combined with KDE is straightforward to implement in empirical studies and yields estimates that align with the intuitions of ABMs, by mitigating estimation errors and reducing efficiency losses.

Modern behavioral paradigm and managerial decision making

The study of the motives and patterns of behavior of economic agents is one of the important areas of modern economic science. Understanding the patterns of evolution of the behavioral paradigm is important from both cognitive and prognostic points of view. A modern manager must be able to analyze both rational and irrational aspects of human behavior in order to effectively manage organizations. The author explores the evolution of the behavioral paradigm, identifies the main stages of its development and reveals the influence of the philosophy of rationalism of the Enlightenment, as well as sociological and philosophical knowledge of the 19th-20th centuries, on this process. The author substantiates that the behavior of an «economic person» and «a managerial person» is influenced not only by rational motives, but also by cultural values, values of economic activity, behavioral patterns, memes, sociocultural codes, as well as cognitive limitations and inherent deviations from rational behavior. The study of the relationship between the behavioral paradigm that a manager adheres to and his managerial decisions is of scientific importance, since it allows to deepen knowledge about the motives of managerial decisions and the impact on them of a wide range of economic and non–economic factors, rational and irrational motives and incentives.

Benefit allocation for combined heat and power dispatch considering mutual trust

Combined heat and power dispatch (CHPD) has attracted wide attention recently, which exploits the potential flexibility in the district heating system (DHS) and provides an economic-effective solution for better wind accommodation. However, DHS operators bear extra operation cost in CHPD, which is proved theoretically in this paper, so that they have insufficient incentive to coordinate with electrical power system. To cope with this issue, a cost allocation strategy based on the Aumann-Shapley value method is proposed, which can rationally allocate the benefit among multiple agents. And a new solution scheme with the Gauss-Legendre quadrature formula is developed to enhance computation efficiency. Also, considering the potential malicious behavior of dishonest agents in the process of allocation, the blockchain framework with Proof of Solution consensus mechanism is introduced to ensure mutual trust among multiple agents. Numerical simulations based on different cases demonstrate the effectiveness of the proposed method of benefit allocation among multiple agents. In addition, the validity of the proposed blockchain framework in preventing malicious behavior when CHPD is verified.