Decisions Of Agents Research Articles

Imperfect-Information Game AI Agent Based on Reinforcement Learning Using Tree Search and a Deep Neural Network

In the field of computer intelligence, it has always been a challenge to construct an agent model that can be adapted to various complex tasks. In recent years, based on the planning algorithm of Monte Carlo tree search (MCTS), a new idea has been proposed to solve the AI problems of two-player zero-sum games such as chess and Go. However, most of the games in the real environment rely on imperfect information, so it is impossible to directly use the normal tree search planning algorithm to construct a decision-making model. Mahjong, which is a popular multiplayer game with a long history in China, attracts great attention from AI researchers because it contains a large game state space and a lot of hidden information. In this paper, we utilize an agent learning approach that leverages deep learning, reinforcement learning, and dropout learning techniques to implement a Mahjong AI game agent. First, we improve the state transition of the tree search based on the learned MDP model, the player position variable and transition information are introduced into the tree search algorithm to construct a multiplayer search tree. Then, the model training based on a deep reinforcement learning method ensures the stable and sustainable training process of the learned MDP model. Finally, we utilize the strategy data generated by the tree search and use the dropout learning method to train the normal decision-making agent. The experimental results demonstrate the efficiency and stability performance of the agent trained by our proposed method compared with existing agents in terms of test data accuracy, tournament ranking performance, and online match performance. The agent plays against human players and acts like real humans.

Approaches to Economic Institutionalism: A Theoretical Review of its Characteristics and Elements of Study

Institutionalism emerges as a scientific construct of economics and sociology, becoming a fundamental element for achieving the development objectives of countries (Portes, 2009). The theoretical review of its approaches, characteristics and elements of study, based on the prominent works of Ronald Coase (1988), Douglas North (1990), Oliver Williamson (1988) and Elinor Ostrom (2009), are carried out with the objective of exposing from another perspective the theories of the Traditional Institutionalist approach (TI), the Economic Neo- Institutionalism (NE) and the New Institutional Economics (NEI). Through the description and exploration, qualitative and transversal, of primary and secondary sources, the convergent and differentiating elements of each approach are described, as well as the importance of their characteristics in the economic system. The study of Institutionalism allows the recognition of market failures, irrationality in the decisions of economic agents, agency problems in the implementation of public policies, the existence of transaction costs, taking as a reference the theories of Economic Institutionalism and its approaches.
 
 Received: 22 February 2023 / Accepted: 16 April 2023 / Published: 5 May 2023

Did the unemployment benefits extension between 2009–2013 deter entrepreneurship in the US? Some evidence and a model of unemployment benefits and entrepreneurship

The 2007–2008 financial crisis severely dampened the American entrepreneurial sector, and the rate of business formation remained low until 2014. This study empirically assesses whether the extensions of unemployment benefits (EUB) contributed to the delay in recovery in entrepreneurship. We then introduce a general equilibrium occupational choice model of unemployment benefits and entrepreneurship and use sharp discontinuity and variations across states in the maximum duration of benefits at the end of 2013 to identify the negative impact of EUB on startups. Eliminating the EUB increased entrepreneurship by 1.4–1.78 percentage points in 2014. Regarding the model, moderate increases in unemployment benefits primarily affect entrepreneurship through the extensive margin by deterring low-ability and necessity entrepreneurs while having little impact on decisions for high-ability agents; however, overly generous unemployment benefits can negatively impact entrepreneurship, the labor market and the overall economy.

An Improved NSGA-III Algorithm Based on Deep Q-Networks for Cloud Storage Optimization of Blockchain

As the underlying technology of cryptocurrencies, blockchain has gained a lot of attention in recent years. However, the storage problem needs to be solved with the increasing number of blocks in the blockchain network. Cloud storage optimization is an effective way to solve the storage issue, which selects and stores parts of blocks to the cloud. Precisely, block selection can be described as a multiobjective optimization problem (MOP) and solved by evolutionary algorithms (EAs). To obtain well results of block selection, an improved NSGA-III algorithm based on deep Q-networks (DQN), termed NSGA-DQN, is proposed in this paper, which aims to maintain well convergence and diversity of the population. This way, a set of suitable solutions is obtained to determine the number of blocks stored to the cloud, and the storage problem can be solved effectively. To be specific, DQN creates a decision-making agent to maximize the expected reward by learning a policy that evaluates Q values of each action in each state. In the proposed selection mechanism, the reward values are set according to the convergence and diversity of the population, and the actions correspond to the individuals. This way, our method can determine a set of individuals that maximizes the convergence and diversity of the population. In addition, an adaptive maximum reward enhancement module (AMREM) is developed to further enhance the maximum expected reward by updating the new better reward and modifying the replay memory. We conduct the experimental study on block selection, and the results demonstrate that the proposed algorithm is superior to five state-of-the-art algorithms.

Formation of trade networks by economies of scale and product differentiation

Understanding the structure and formation of networks is a central topic in complexity science. Economic networks are formed by decisions of individual agents and thus not properly described by established random graph models. In this article, we establish a model for the emergence of trade networks that is based on rational decisions of individual agents. The model incorporates key drivers for the emergence of trade, comparative advantage and economic scale effects, but also the heterogeneity of agents and the transportation or transaction costs. Numerical simulations show three macroscopically different regimes of the emerging trade networks. Depending on the specific transportation costs and the heterogeneity of individual preferences, we find centralized production with a star-like trade network, distributed production with all-to-all trading or local production and no trade. Using methods from statistical mechanics, we provide an analytic theory of the transitions between these regimes and estimates for critical parameters values.

Reinforcement Learning-Based Approach for Minimizing Energy Loss of Driving Platoon Decisions.

Reinforcement learning (RL) methods for energy saving and greening have recently appeared in the field of autonomous driving. In inter-vehicle communication (IVC), a feasible and increasingly popular research direction of RL is to obtain the optimal action decision of agents in a special environment. This paper presents the application of reinforcement learning in the vehicle communication simulation framework (Veins). In this research, we explore the application of reinforcement learning algorithms in a green cooperative adaptive cruise control (CACC) platoon. Our aim is to train member vehicles to react appropriately in the event of a severe collision involving the leading vehicle. We seek to reduce collision damage and optimize energy consumption by encouraging behavior that conforms to the platoon's environmentally friendly aim. Our study provides insight into the potential benefits of using reinforcement learning algorithms to improve the safety and efficiency of CACC platoons while promoting sustainable transportation. The policy gradient algorithm used in this paper has good convergence in the calculation of the minimum energy consumption problem and the optimal solution of vehicle behavior. In terms of energy consumption metrics, the policy gradient algorithm is used first in the IVC field for training the proposed platoon problem. It is a feasible training decision-planning algorithm for solving the minimization of energy consumption caused by decision making in platoon avoidance behavior.

Employees' Appraisals and Trust of Artificial Intelligences' Transparency and Opacity.

Artificial intelligence (AI) is being increasingly used as a decision agent in enterprises. Employees' appraisals and AI affect the smooth progress of AI-employee cooperation. This paper studies (1) whether employees' challenge appraisals, threat appraisals and trust in AI are different for AI transparency and opacity. (2) This study investigates how AI transparency affects employees' trust in AI through employee appraisals (challenge and threat appraisals), and (3) whether and how employees' domain knowledge about AI moderates the relationship between AI transparency and appraisals. A total of 375 participants with work experience were recruited for an online hypothetical scenario experiment. The results showed that AI transparency (vs. opacity) led to higher challenge appraisals and trust and lower threat appraisals. However, in both AI transparency and opacity, employees believed that AI decisions brought more challenges than threats. In addition, we found the parallel mediating effect of challenge appraisals and threat appraisals. AI transparency promotes employees' trust in AI by increasing employees' challenge appraisals and reducing employees' threat appraisals. Finally, employees' domain knowledge about AI moderated the relationship between AI transparency and appraisals. Specifically, domain knowledge negatively moderated the positive effect of AI transparency on challenge appraisals, and domain knowledge positively moderated the negative effect of AI transparency on threat appraisals.

Dynamic distributed decision-making for resilient resource reallocation in disrupted manufacturing systems

The COVID-19 pandemic brings many unexpected disruptions, such as frequently shifting markets and limited human workforce, to manufacturers. To stay competitive, flexible and real-time manufacturing decision-making strategies are needed to deal with such highly dynamic manufacturing environments. One essential problem is dynamic resource allocation to complete production tasks, especially when a resource disruption (e.g. machine breakdown) occurs. Though multi-agent methods have been proposed to solve the problem in a flexible and agile manner, the agent internal decision-making process and resource uncertainties have rarely been studied. This work introduces a model-based resource agent (RA) architecture that enables effective agent coordination and dynamic agent decision-making. Based on the RA architecture, a rescheduling strategy that incorporates risk assessment via a clustering agent coordination strategy is also proposed. A simulation-based case study is implemented to demonstrate dynamic rescheduling using the proposed multi-agent framework. The results show that the proposed method reduces the computational efforts while losing some throughput optimality compared to the centralised method. Furthermore, the case study illustrates that incorporating risk assessment into rescheduling decision-making improves the throughput.

Multi-agents in transport process control

Abstract. Problem The problem is computer science, car transfer, road maintenance, synergy, principles and paradigms of using car computer systems - ACS. The use of multi-agent systems in the virtual management of the transport process allows improving the quality of transport management. A multi-agent system is a collection of interacting agents that can work together to achieve a common goal. Multi-agent systems can be used for better management of logistics processes, in particular in the field of transport. The role of agents in decision-making, including in real time, is considered, which allows more accurate forecasting and control of the transport process. Goal. The goal is integration of the intelligent multi-agent system (IMAS) and agents in the task of virtual management of transport processes. Results. A program installed on a server that supports Internet functions can potentially be accessed from any workstation. This opens the way to the use of intellectual and production tools by almost all employees of the enterprise at minimal costs. The same application becomes available from any workstation connected to the Internet, which is critical for e-commerce applications. Such an opportunity opens access to applications to corporate partners for creating virtual enterprises and virtual management.

Scheme of Operation for Multi-Robot Systems with Decision-Making Based on Markov Chains for Manipulation by Caged Objects

This paper presents the design of a new control scheme for a group of omnidirectional robots in a multi-robot system operating in an environment with obstacles. The control scheme uses a decision agent based on discrete-time Markov chains and takes into account the state of the system, obstacle positions, and geometries to manipulate targets, providing robustness against measurement uncertainties. The decision process is dynamic, with state information updating at each time step and tasks being executed based on the hierarchy determined by quadratic hierarchical programming. The system’s stability in the mean-square sense is analyzed through the study of a closed-loop stochastic system, and the effectiveness of the proposed control scheme is demonstrated through numerical simulations, including a comparative analysis with a finite-state machine decision agent.

Optimized Reinforcement Learning Approach on Sustainable Rural Tourism Development for Economic Growth

A country’s economic development relies on different features such as export/import, industrial processes, and tourism. Rural tourism is a discussion-centric research field for analyzing its contribution to a country’s economic growth. This field generates voluptuous data for tourists, expenditure, location, etc. analysis; the information increases over the years and the density of visiting tourists. Therefore, this article introduces an optimized reinforcement data analysis approach (ORDAA) for generating precise guidance information. This information is two-faced, namely, summarized data for tourist guidance and summarized data for the country’s economic development. Data augmentation’s steep rise and downfall are analyzed using reinforcement learning, wherein decision agents are precise for a relevant summary. The relevance is identified using associated development targets over varying years. Besides, the guidance information that identifies low tourist summary or nonachievable development targets is separately identified. The identified targets are analyzed using reinforcement agents for economic growth improvements compared to the previous tourist densities. This improves the focus on rural tourism sights and economic contributions to an optimal level.

The ABC of algorithmic aversion: not agent, but benefits and control determine the acceptance of automated decision-making

While algorithmic decision-making (ADM) is projected to increase exponentially in the coming decades, the academic debate on whether people are ready to accept, trust, and use ADM as opposed to human decision-making is ongoing. The current research aims at reconciling conflicting findings on ‘algorithmic aversion’ in the literature. It does so by investigating algorithmic aversion while controlling for two important characteristics that are often associated with ADM: increased benefits (monetary and accuracy) and decreased user control. Across three high-powered (Ntotal = 1192), preregistered 2 (agent: algorithm/human) × 2 (benefits: high/low) × 2 (control: user control/no control) between-subjects experiments, and two domains (finance and dating), the results were quite consistent: there is little evidence for a default aversion against algorithms and in favor of human decision makers. Instead, users accept or reject decisions and decisional agents based on their predicted benefits and the ability to exercise control over the decision.

Enhancing Local Decisions in Agent-Based Cartesian Genetic Programming by CMA-ES

Cartesian genetic programming is a popular version of classical genetic programming, and it has now demonstrated a very good performance in solving various use cases. Originally, programs evolved by using a centralized optimization approach. Recently, an algorithmic level decomposition of program evolution has been introduced that can be solved by a multi-agent system in a fully distributed manner. A heuristic for distributed combinatorial problem-solving was adapted to evolve these programs. The applicability of the approach and the effectiveness of the used multi-agent protocol as well as of the evolved genetic programs for the case of full enumeration in local agent decisions has already been successfully demonstrated. Symbolic regression, n-parity, and classification problems were used for this purpose. As is typical of decentralized systems, agents have to solve local sub-problems for decision-making and for determining the best local contribution to solving program evolution. So far, only a full enumeration of the solution candidates has been used, which is not sufficient for larger problem sizes. We extend this approach by using CMA-ES as an algorithm for local decisions. The superior performance of CMA-ES is demonstrated using Koza’s computational effort statistic when compared with the original approach. In addition, the distributed modality of the local optimization is scrutinized by a fitness landscape analysis.

A coupled agent-based model for France for simulating adaptation and migration decisions under future coastal flood risk

In this study, we couple an integrated flood damage and agent-based model (ABM) with a gravity model of internal migration and a flood risk module (DYNAMO-M) to project household adaptation and migration decisions under increasing coastal flood risk in France. We ground the agent decision rules in a framework of subjective expected utility theory. This method addresses agent’s bounded rationality related to risk perception and risk aversion and simulates the impact of push, pull, and mooring factors on migration and adaptation decisions. The agents are parameterized using subnational statistics, and the model is calibrated using a household survey on adaptation uptake. Subsequently, the model simulates household adaptation and migration based on increasing coastal flood damage from 2015 until 2080. A medium population growth scenario is used to simulate future population development, and sea level rise (SLR) is assessed for different climate scenarios. The results indicate that SLR can drive migration exceeding 8000 and 10,000 coastal inhabitants for 2080 under the Representative Concentration Pathways 4.5 and 8.5, respectively. Although household adaptation to flood risk strongly impacts projected annual flood damage, its impact on migration decisions is small and falls within the 90% confidence interval of model runs. Projections of coastal migration under SLR are most sensitive to migration costs and coastal flood protection standards, highlighting the need for better characterization of both in modeling exercises. The modeling framework demonstrated in this study can be upscaled to the global scale and function as a platform for a more integrated assessment of SLR-induced migration.

Modeling interactions between the embodied and the narrative self: Dynamics of the self-pattern within LIDA

Despite lacking a generally accepted definition, Artificial General Intelligence (AGI) is commonly understood to refer to artificial agents possessing the capacity to build up a context-independent understanding of itself and the world and to generalize this knowledge across a multitude of contexts. In human agents, this capacity is, to a large degree, facilitated by processes of self-directed learning, during which agents voluntarily control the conditions under which episodes of learning and problem solving occur. Since self-directed learning depends on the degree of knowledge the agent has about various aspects of themselves (their bodily skills, their learning goal, etc.), an AGI implementation of this type of learning must build on a theory of how this self-knowledge is actualized and modified during the learning process. In this paper, we employ the pattern theory of self in order to characterize different aspects of an agent’s self that are relevant for self-directed learning. Such aspects include agent-internal cognitive states such as thoughts, emotions, and intentions, but also relational states such as action possibilities in the environment. Combinations of these aspects form a characteristic pattern, which is unique to each individual agent, with no one aspect being necessary or sufficient for the individuation of that agent’s self. Here, we focus on the interdependence of narrative and embodied aspects of the self-pattern, since they involve particularly salient challenges consisting in conceptualizing the interaction between propositional and motor representations.In our paper, we model the reciprocal interaction of these aspects of the self-pattern within an individual cognitive agent. We do so by extending an approach by Ryan, Agrawal, & Franklin (2020), who laid the groundwork for the implementation of the pattern theory of self in the LIDA (Learning Intelligent Decision Agent) model. We describe how embodied and narrative aspects of an agent’s self-pattern are realized by patterns of interaction between different LIDA modules over time, and how interactions at multiple temporal scales allow the agent’s self-pattern to be both dynamically variable and relatively stable. Finally, we investigate the implications this view has for the creation of artificial agents that can benefit from self-directed learning, both in the context of deliberate planning and adaptive motor execution.

On the Specialization of FDRL Agents for Scalable and Distributed 6G RAN Slicing Orchestration

Network slicing enables multiple virtual networks to be instantiated and customized to meet heterogeneous use case requirements over 5G and beyond network deployments. However, most of the solutions available today face scalability issues when considering many slices, due to centralized controllers requiring a holistic view of the resource availability and consumption over different networking domains. In order to tackle this challenge, we design a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">hierarchical architecture to manage network slices resources in a federated manner</i> . Driven by the rapid evolution of deep reinforcement learning (DRL) schemes and the Open RAN (O-RAN) paradigm, we propose a set of traffic-aware local decision agents (DAs) dynamically placed in the radio access network (RAN). These federated decision entities tailor their resource allocation policy according to the long-term dynamics of the underlying traffic, defining <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">specialized</i> clusters that enable faster training and communication overhead reduction. Indeed, aided by a traffic-aware agent selection algorithm, our proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Federated DRL</i> approach provides higher resource efficiency than benchmark solutions by quickly reacting to end-user mobility patterns and reducing costly interactions with centralized controllers.

Modeling Collaborative Behaviors in Energy Ecosystems

The notions of a collaborative virtual power plant ecosystem (CVPP-E) and a cognitive household digital twin (CHDT) have been proposed as contributions to the efficient organization and management of households within renewable energy communities (RECs). CHDTs can be modeled as software agents that are designed to possess some cognitive capabilities, enabling them to make autonomous decisions on behalf of their human owners based on the value system of their physical twin. Due to their cognitive and decision-making capabilities, these agents can exhibit some behavioral attributes, such as engaging in diverse collaborative actions aimed at achieving some common goals. These behavioral attributes can be directed to the promotion of sustainable energy consumption in the ecosystem. Along this line, this work demonstrates various collaborative practices that include: (1) collaborative roles played by the CVPP manager such as (a) opportunity seeking and goal formulation, (b) goal proposition/invitation to form a coalition or virtual organization, and (c) formation and dissolution of coalitions; and (2) collaborative roles played by CHDTs which include (a) acceptance or decline of an invitation based on (i) delegation/non-delegation and (ii) value system compatibility/non-compatibility, and (b) the sharing of common resources. This study adopts a simulation technique that involves the integration of multiple simulation methods such as system dynamics, agent-based, and discrete event simulation techniques in a single simulation environment. The outcome of this study confirms the potential of adding cognitive capabilities to CHDTs and further shows that these agents could exhibit certain collaborative attributes, enabling them to become suitable as rational decision-making agents in households.

Evolutionary Game of Digital-Driven Photovoltaic–Storage–Use Value Chain Collaboration: A Value Intelligence Creation Perspective

In the context of “carbon neutral”, distributed energy, including photovoltaic power generation and energy storage systems, is developing rapidly. Meanwhile, the new generation of information technology, such as “Cloud computing, Big data, the Internet of things, Mobile Internet, AI, Blockchain”, is driving the digital transformation of the energy industry. Under digital drive, how the agents in the photovoltaic–storage–use value chain collaborate and create value intelligently is a question worthy of deep consideration. Firstly, the value creation mechanism and collaborative process of the digital-driven photovoltaic–storage–use value chain are analyzed from a value intelligence creation perspective. Secondly, the tripartite evolutionary game model of photovoltaic power generator, energy storage provider and user is established. Finally, the influencing factors of digital- driven photovoltaic–storage–use value chain collaboration are explored through a numerical simulation, and management suggestions are put forward. The study finds the following: (1) The behavior choice of each agent in the value chain will affect the decision of other agents. In particular, the photovoltaic power generator has a great influence on the cooperative willingness of other agents. To promote value chain collaboration, the guiding role of the photovoltaic power generator should be fully realized. (2) Agents on the value chain can use a variety of digital technologies to improve enabling benefits, which is conducive to promoting value chain collaboration. (3) The driving costs and potential risks are obstacles for value chain collaboration. Cost reduction and risk prevention are effective ways to improve the willingness of collaboration. (4) Reasonable incentive compensation mechanisms and information asymmetry punishment measures are the keys to enhancing collective willingness. This research provides theoretical support for photovoltaic–storage–use value chain collaboration from a value intelligence creation perspective.

Reinforcement Learning for Clinical Applications.

Reinforcement Learning for Clinical Applications.

“I feel like marijuana is the only drug that wouldn’t kill me”: perceptions of cannabis use in previously incarcerated Black men who have sex with other men

BackgroundFragmented state laws have impacted cannabis uptake and perceptions in the USA. Little research has explored the attitudes, beliefs, and social network influences of young Black men who have sex with men (BMSM) who have experienced incarceration and use cannabis. While problematic cannabis use is not well defined and understudied, scholars have found that a person’s social network can mediate problematic substance use and reduce recidivism rates by providing both tangible and emotional support. This analysis examines how social networks contribute to cannabis perceptions and use among BMSM with criminal legal system involvement in Chicago, IL, and Houston TX.MethodsResearchers conducted interviews with 25 cis gender Black men informed by life course theory, with a focus on the role of social networks, incarceration, and other life experiences in substance use. All interviews were audio-recorded, de-identified, and transcribed; participants were compensated $50. A deductive-inductive thematic analysis was used to analyze all qualitative data collected.ResultsTwelve BMSM in Chicago and 13 BMSM in Houston (M = 26.6 years old, SD = 3.7) were interviewed. A majority identified as gay (56%), with 12 participants (48%) reporting having a high school diploma or equivalent; their average age of first substance use was 15.2 (SD = 2.9). Participants perceived cannabis usage to be categorically distinct from other intoxicating substance usage, with many describing it as not harmful and potentially beneficial. Three themes shaped their choices and attitudes regarding cannabis and “hard” drugs—social networks, need fulfillment, and knowledge of risk.ConclusionParticipant descriptions of cannabis use emphasize their drug-use behavior as being produced by agent decision-making and risk assessment. Future work should expand on how these decisions are made, and how social networks can be leveraged to encourage non-harmful drug consumption behaviors.