Experience-weighted Attraction Research Articles

Promoting owners’ shared parking willingness considering management mechanisms in community network

Abstract Shared parking effectively optimizes urban parking resources while making full use of private parking spaces and satisfying the growing demand for parking in large cities. However, some owners are unwilling to share private parking spaces and oppose the community in conducting shared parking projects. To promote the sharing of private parking spaces, we use a complex network evolutionary game method to depict the impact of owners’ unfavorable relationships on individuals’ decision processes in real time and explore the impact of management mechanisms on owners’ willingness to share. The results demonstrate that experience-weighted attraction learning strategies that focus on experiential learning and adaptability are more conducive to promoting owner cooperation, whereas neighbor-avoidance conflict costs resulting from interactions between owners restrict cooperative behavior, and a higher number of network owners is detrimental to cooperation. A platform that improves the rejection rate of parking requests and overtime inconvenience cost is conducive to cooperation, but overtime probability and time window conflict cost reduce owners’ willingness to share. The government can lessen these adverse effects by adding compensation to all owners and increasing the public opinion adjustment coefficient to promote cooperative behavior and increase the number of shared parking spaces.

Collaborative resource allocation in computing power networks: A game-theoretic double auction perspective

The growth of global data is increasing exponentially, leading to a greater demand for computing power. To address this requirement, expanding computing power from the cloud to the edge is essential. However, this transformation presents two significant challenges: how to share computing resources more efficiently and how to optimize resource allocation. To tackle these challenges, we propose a three-layer Computing Power Network (CPN) framework that focuses on implementing the collaborative allocation of computing nodes and user tasks. We formulate the resource allocation problem in CPN as a double auction game and use an experience-weighted attraction algorithm that enables participants to adjust bidding strategies based on environmental interactions. We implemented a prototype of our proposed CPN framework and conducted extensive experiments to verify our algorithm’s convergence and evaluate the benefits obtained by buyers (users) and sellers (computing nodes) from the perspective of transaction prices, rewards, and average pricing. The comprehensive experimental results demonstrate the effectiveness of our proposed method. Compared with state-of-the-art pricing strategies, our approach achieves a 20% increase in convergence speed and an 88% increase in overall returns. Furthermore, it also exhibits a 2.5% increase in deal prices and a substantial 83% rise in the income of individual users. These outcomes convincingly prove the superiority of our method in achieving better convergence, improving overall returns, and benefiting both buyers and sellers in the CPN resource auction market.

Multi-Tier Supply Chain Learning Networks: A Simulation Study Based on the Experience-Weighted Attraction (EWA) Model

Multi-Tier Supply Chain Learning Networks: A Simulation Study Based on the Experience-Weighted Attraction (EWA) Model

Strategic Control of Experience-Weighted Attraction Model in Human-AI Interactions

Strategic Control of Experience-Weighted Attraction Model in Human-AI Interactions

Memory loss can prevent chaos in games dynamics.

Recent studies have raised concerns on the inevitability of chaos in congestion games with large learning rates. We further investigate this phenomenon by exploring the learning dynamics in simple two-resource congestion games, where a continuum of agents learns according to a simplified experience-weighted attraction algorithm. The model is characterized by three key parameters: a population intensity of choice (learning rate), a discount factor (recency bias or exploration parameter), and the cost function asymmetry. The intensity of choice captures agents' economic rationality in their tendency to approximately best respond to the other agent's behavior. The discount factor captures a type of memory loss of agents, where past outcomes matter exponentially less than the recent ones. Our main findings reveal that while increasing the intensity of choice destabilizes the system for any discount factor, whether the resulting dynamics remains predictable or becomes unpredictable and chaotic depends on both the memory loss and the cost asymmetry. As memory loss increases, the chaotic regime gives place to a periodic orbit of period 2 that is globally attracting except for a countable set of points that lead to the equilibrium. Therefore, memory loss can suppress chaotic behaviors. The results highlight the crucial role of memory loss in mitigating chaos and promoting predictable outcomes in congestion games, providing insights into designing control strategies in resource allocation systems susceptible to chaotic behaviors.

Sharing hydrogen storage capacity planning for multi-microgrid investors with limited rationality: A differential evolution game approach

Bottom on the ripple of the multiplication of sharing economy, hydrogen energy storage (HES) shared calls for novel solutions to ameliorate the cleanness and economy of micro-grids under the dual carbon target in China. The traditional investment method may become impracticable due to conflicting interests, insufficient utilization, and poor interoperability. The practical and prospective design of HES relies on innovation at the joint investment mechanism under incomplete synchronized and the interested equilibrium under limited rationality. This paper studies the capacity planning method for HES by multiple micro-grids from the sharing perspective. The evolution game model is utilized for conflict of interest, which exactly accounts for limited rationality. The dynamic competitive as well as time-varying demand for HES among multiple micro-grids are established by a small-word complex network theory. The dual dynamic evolutionary game of investment decision in multi-micro-grid and capacity planning for sharing HES is realized by a proposed method of differential evolutionary game algorithm based on experience-weighted attraction learning. The numerical simulation is carried out to verify the effectiveness of the proposed method and interpret the development path for sharing HES. The result shows that, the sizing method with limited rationality reduces the cost by 4.75% compared with the traditional game under complete rationality and 9.15% compared with the multi-objective optimization. The willingness of the park to participate in joint investment in shared HES is influenced by the long-term regulatory demand for energy storage and the decisions of other parks in the neighbors.

Cultural diffusion dynamics depend on behavioural production rules.

Culture is an outcome of both the acquisition of knowledge about behaviour through social transmission, and its subsequent production by individuals. Acquisition and production are often discussed or modelled interchangeably, yet to date no study has explored the consequences of their interaction for cultural diffusions. We present a generative model that integrates the two, and ask how variation in production rules might influence diffusion dynamics. Agents make behavioural choices that change as they learn from their productions. Their repertoires may also change, and the acquisition of behaviour is conditioned on its frequency. We analyse the diffusion of a novel behaviour through social networks, yielding generalizable predictions of how individual-level behavioural production rules influence population-level diffusion dynamics. We then investigate how linking acquisition and production might affect the performance of two commonly used inferential models for social learning; network-based diffusion analysis, and experience-weighted attraction models. We find that the influence that production rules have on diffusion dynamics has consequences for how inferential methods are applied to empirical data. Our model illuminates the differences between social learning and social influence, demonstrates the overlooked role of reinforcement learning in cultural diffusions, and allows for clearer discussions about social learning strategies.

Reinforcement learning with foregone payoff information in normal form games

This paper studies the reinforcement learning of Erev and Roth with foregone payoff information in normal form games: players observe not only the realised payoffs but also foregone payoffs, the ones which they could have obtained if they had chosen the other actions. We provide conditions under which the reinforcement learning process almost surely converges to a regular quantal response equilibrium (Goeree et al. 2005). We also show that the reinforcement learning model and an adaptive learning model which nests experience-weighted attraction learning, payoff assessment learning and stochastic fictitious play learning models share the same asymptotic behaviour under the linear choice rule of the reinforcement learning model. In addition, we provide conditions under which the reinforcement learning process under the logit choice rule almost surely converges to a Nash equilibrium.

Towards a taxonomy of learning dynamics in 2 × 2 games

Do boundedly rational players learn to choose equilibrium strategies as they play a game repeatedly? A large literature in behavioral game theory has proposed and experimentally tested various learning algorithms, but a comparative analysis of their equilibrium convergence properties is lacking. In this paper we analyze Experience-Weighted Attraction (EWA), which generalizes fictitious play, best-response dynamics, reinforcement learning and also replicator dynamics. Studying 2×2 games for tractability, we recover some well-known results in the limiting cases in which EWA reduces to the learning rules that it generalizes, but also obtain new results for other parameterizations. For example, we show that in coordination games EWA may only converge to the Pareto-efficient equilibrium, never reaching the Pareto-inefficient one; that in Prisoner Dilemma games it may converge to fixed points of mutual cooperation; and that limit cycles or chaotic dynamics may be more likely with longer or shorter memory of previous play.

Empirical properties and identification of adaptive learning models in behavioral game theory

This paper studies the empirical properties and identification for a broad class of adaptive learning models in games (e.g., experience-weighted attraction (EWA), payoff assessment, and impulse-matching models). I first show that adding a constant c≠0 to the utility function could affect players’ choices during the learning periods. This property is different from standard theories, such as expected utility theory and discrete choice models, where adding c≠0 preserves the same choice. This result has two further implications. First, the constant c added to the utility function could have a structural interpretation, and this paper discusses how to interpret it. Second, the estimation of learning models usually normalizes the utility of one action profile to be zero (i.e., c=0); for instance, the utility of zero dollars is zero, as commonly imposed in experimental economics. This normalization is innocuous in most applications of discrete choice models but could be misspecified in the context of learning. To deal with the above issue, this paper proposes to specify a player’s utility as an unknown function of the action profile, without imposing the above normalization. Equivalently, c is specified as an unknown parameter to be estimated. Under weak conditions, I obtain the joint identification of both utility and learning parameters. Finally, I illustrate the importance of c in the estimation of learning models by conducting a Monte Carlo experiment and revisiting the experimental study by Feri et al. (2010).

Regional Cooperation in Marine Plastic Waste Cleanup in the South China Sea Region

This paper uses countries in the South China Sea Region (SCSR) as examples to study the level of regional cooperation in the marine plastic waste cleanup initiative. We designed a cooperation model to investigate the “cleanup system” from the Ocean Cleanup initiative to reduce marine plastic pollution. The non-cooperative game theory was applied to regional cooperation. The simulation results indicate that the plastic waste cleanup cooperation in the SCSR is related to the plastic trade network structure, the influence parameters of the Experience-Weighted Attraction learning model, and the economic effects. The results suggest that regional cooperation in the cleanup system in the SCSR is feasible, and it could create a significantly larger investment in the cleanup project than the current project attracts. Therefore, countries in the SCSR should adjust their laws and policies to make a good cooperative environment and to maximize the contribution to the marine plastic waste cleanup.

The impact of hydrogen refueling station subsidy strategy on China's hydrogen fuel cell vehicle market diffusion

Lack of hydrogen refueling stations (HRSs) has hindered the diffusion of hydrogen fuel cell vehicles (HFCVs) in the Chinese transport market. By combining the agent-based model (ABM) and the experience weighted attraction (EWA) learning algorithm, this paper explores the impact of government subsidy strategy for HRSs on the market diffusion of HFCVs. The actions of the parties (government, HRS planning department and consumers) and their interactions are taken into account. The new model suggests dynamic subsidy mode based on EWA algorithm yields better results than static subsidy mode: HFCV purchases, HRS construction effort, total number of HRSs and expected HRS planning department profits all outperform static data by around 27%. In addition, choosing an appropriate initial subsidy strategy can increase the sales of HFCVs by nearly 40%. Early investment from government to establish initial HRSs can also increase market diffusion efficiency by more than 76.7%.

Multi-agent simulation for strategic bidding in electricity markets using reinforcement learning

In this paper, Multi-agent Simulation (MAS) and Reinforcement Learning (RL) are studied, a theoretical framework of MAS is proposed for strategic bidding in electricity markets using reinforcement learning, which consists of two parts: one is a MAS system used to simulate the competitive bidding of the actual electricity market; the other is an adaptive learning strategy bidding system used to provide agents with more intelligent bidding strategies. An Experience-Weighted Attraction (EWA) reinforcement learning algorithm (RLA) is applied to the MAS model and a new MAS method is presented for strategic bidding in electricity markets using a new Improved EWA (IEWA). From both qualitative and quantitative perspectives it is compared with other three MAS methods using the Roth-Erev (RE), Q-learning and EWA. The result shows that the performance of the MAS method using IEWA is proved to be better than the others. The four MAS models using four RLAs are built for strategic bidding in electricity markets. Through running the four MAS models, the rationality and correctness of the four MAS methods are verified for strategic bidding in electricity markets using reinforcement learning.

Learning and dropout in contests: an experimental approach

We design an experiment to study investment behavior in different repeated contest settings, varying the uncertainty of the outcomes and the number of participants in contests. We find decreasing over-expenditures and a higher rate of ‘dropout’ in contests with high uncertainty over outcomes (winner-take-all contests), while we detect a quick convergence toward equilibrium predictions and a near to full participation when this type of uncertainty vanishes (proportional-prize contests). These results are robust to changes in the number of contestants. A learning parameter estimation using the experience-weighted attraction (EWA) model suggests that subjects adopt different learning modes across different contest structures and helps to explain expenditure patterns deviating from theoretical predictions.

A Hierarchical Bayesian Implementation of the Experience-Weighted Attraction Model.

Social and decision-making deficits are often the first symptoms of neuropsychiatric disorders. In recent years, economic games, together with computational models of strategic learning, have been increasingly applied to the characterization of individual differences in social behavior, as well as their changes across time due to disease progression, treatment, or other factors. At the same time, the high dimensionality of these data poses an important challenge to statistical estimation of these models, potentially limiting the adoption of such approaches in patients and special populations. We introduce a hierarchical Bayesian implementation of a class of strategic learning models, experience-weighted attraction (EWA), that is widely used in behavioral game theory. Importantly, this approach provides a unified framework for capturing between- and within-participant variation, including changes associated with disease progression, comorbidity, and treatment status. We show using simulated data that our hierarchical Bayesian approach outperforms representative agent and individual-level estimation methods that are commonly used in extant literature, with respect to parameter estimation and uncertainty quantification. Furthermore, using an empirical dataset, we demonstrate the value of our approach over competing methods with respect to balancing model fit and complexity. Consistent with the success of hierarchical Bayesian approaches in other areas of behavioral science, our hierarchical Bayesian EWA model represents a powerful and flexible tool to apply to a wide range of behavioral paradigms for studying the interplay between complex human behavior and biological factors.

An agent-based learning-embedded model (ABM-learning) for urban land use planning: A case study of residential land growth simulation in Shenzhen, China

A forward-looking urban land use plan is crucial to a city’s sustainability, which requires a deep understanding of human-environment interactions between different domains, and modelling them soundly. One of the key challenges of modelling these interactions is to understand and model how human individuals make and develop their location decisions by learning that then shape urban land-use patterns. To investigate this issue, we have constructed an extended experience-weighted attraction learning model to represent the human agents’ learning when they make location decisions. Consequently, we propose and have developed an agent-based learning-embedded model (ABM-learning) for residential land growth simulation that incorporates a learning model, a decision-making model, a land use conversion model and the constraint of urban land use master plan. The proposed model was used for a simulation of the residential land growth in Shenzhen city, China. By validating the model against empirical data, the results showed that the site-specific accuracy of the model has been improved when embedding learning model. The analysis on the simulation accuracies has proved the argument that modelling individual-level learning matters in the agent’s decision model and the agent-based models. We also applied the model to predict residential land growth in Shenzhen from 2015 to 2035, and the result can be a reference for land-use allocation in detailed planning of Shenzhen. The ABM-learning is applicable to studying the past urban growth trajectory, aiding in the formulation of detailed residential land and public service facility planning and assessing the land use planning effectiveness.

Market Microstructure Noise, Intraday Stock Market Returns, and Adaptive Learning: Indian Evidence

What drives intraday traders’ sentiments in the stock markets: information or noise? This paper argues that the market microstructure noise (MMN) manifests intraday traders’ aggregate sentiments depicted by chaotic and noisy market returns. It examines if intraday stock market returns, returns’ variances and higher order moments are erratic, noisy and non-normal. It shows that the intraday Bombay Stock Exchange (BSE) Sensex and National Stock Exchange (NSE) Nifty index returns approximate to zero-mean, zero-variance but skewed and leptokurtic in distributions. In exploring the intraday market index returns, standardisation process reveals noises in the BSE market, but it is evened up in the NSE market. Since intraday traders’ market sentiments and decision choices are behavioural, noisy but adaptive, their decision choices need strategies given that those strategies have numerical “attractions” that determine choice probabilities. We explore the adaptive Experience Weighted Attraction (EWA) learning parameters to show persistent MMN in intraday traders’ adaptive learning behaviours.

Reinforcement-Learning- and Belief-Learning-Based Double Auction Mechanism for Edge Computing Resource Allocation

In recent years, we have witnessed the compelling application of the Internet of Things (IoT) in our daily life, ranging from daily living to industrial production. On account of the computation and power constraints, the IoT devices have to offload their tasks to the remote cloud services. However, the long-distance transmission poses significant challenges for latency-sensitive businesses, such as autonomous driving and industrial control. As a remedy, mobile edge computing (MEC) is deployed at the edge of the network to reduce the transmission delay. With the MEC joining in, how to allocate the limited computing resource of MEC is a critical problem to guarantee efficient working of the whole IoT system. In this article, we formulate the resource management among MEC and IoT devices as a double auction game. Also, for searching the Nash equilibrium, we introduce the experience-weighted attraction (EWA) algorithm performing behind each participant. With this AI method, auction participants acquire and accumulate experience by observing others' behavior and doing introspection, which accelerates the trading policy's learning process of each agent in such an opaque environment. Some simulation results are presented to evaluate the convergence and correctness of our architecture and algorithm.

Reinforcement Learning with Foregone Payoff Information in Normal Form Games

This paper studies the reinforcement learning of Erev and Roth with foregone payoff information in normal form games: players observe not only the realised payoffs but also foregone payoffs, the ones which they could have obtained if they had chosen the other actions. We provide conditions under which the reinforcement learning process almost surely converges to a regular quantal response equilibrium (Goeree et al. 2005). We also show that the reinforcement learning model and an adaptive learning model which nests experience-weighted attraction learning, payoff assessment learning and stochastic fictitious play learning models share the same asymptotic behaviour under the linear choice rule of the reinforcement learning model. In addition, we provide conditions under which the reinforcement learning process under the logit choice rule almost surely converges to a Nash equilibrium.

The Strategy Evolution in Double Auction Based on the Experience-Weighted Attraction Learning Model

The double auction is a widely applicable trading mechanism used to converge to competitive equilibrium in different markets from which multiple equilibriums and incomplete information may arise. Therefore, different learning models have been applied to facilitate bidding strategies for buyers and sellers in the market. However, due to the existence of problems in double auction markets such as individual bounded rationality and information incompleteness, it is still necessary to explore a more general learning model to depict the learning mechanism in double auction markets and predict the evolution processes of bidding strategies for both sides. Therefore, this paper aims at introducing the use of the experience-weighted attraction (EWA) model for double auction because it combines reinforcement learning with belief learning that then converts EWA in a suitable and interesting learning model for describing and improving individuals’ learning behavior. It can become an effective learning model for bidding strategies in the double auction. In addition to the use of the EWA for strategy evolution in the double auction, the impact of its different bidding strategy performance parameters will be also analyzed and compared with other learning models.