Cumulative Payoffs Research Articles

Modeling interactions in a dynamic heuristic business network

This article presents a novel model for understanding the structure and dynamics of business networks, emphasizing the role of propensities to connect and cooperate as key drivers. The model incorporates behavioral elements and imperfect information updates, departing from traditional rational actor approaches. Starting from the theoretical background, several propositions are outlined, such as dynamism, connection choices, costs, strategy selection, information update, and the update based on experiences. Through simulations, the study successfully demonstrates that the proposed model effectively captures essential characteristics of business networks, including reciprocity, complexity, adaptation, and cooperation. The findings highlight the significance of propensities to connect and cooperate in shaping network structure, evolution, and stability. Particularly, higher propensities to cooperate and connect lead to denser and more cohesive networks, fostering reciprocity, stability, and network performance. The increase only in connection propensities does not have the same result. The lower cooperation propensities result in scale-free networks and asymmetrical distribution of cumulative payoffs. This highlights a crucial insight: different levels of cooperation lead to distinct network properties. Practical implications, increasingly relevant with the rise of digital platforms and metaverse, suggest targeted interventions to enhance network effectiveness, such as incentivizing cooperation, reducing relationship costs, and promoting a culture of trust and collaboration. While providing valuable insights, certain limitations exist, such as not considering the influence of preexisting economic or social structures, equalizing costs and payoffs among actors, and overlooking specific reasons for network connections. Future research should address these refinements and explore their effects on network structure and process characteristics.

The Online Saddle Point Problem and Online Convex Optimization with Knapsacks

We study the online saddle point problem, an online learning problem where at each iteration, a pair of actions needs to be chosen without knowledge of the current and future (convex-concave) payoff functions. The objective is to minimize the gap between the cumulative payoffs and the saddle point value of the aggregate payoff function, which we measure using a metric called saddle point regret (SP-Regret). The problem generalizes the online convex optimization framework, but here, we must ensure that both players incur cumulative payoffs close to that of the Nash equilibrium of the sum of the games. We propose an algorithm that achieves SP-Regret proportional to [Formula: see text] in the general case, and [Formula: see text] SP-Regret for the strongly convex-concave case. We also consider the special case where the payoff functions are bilinear and the decision sets are the probability simplex. In this setting, we are able to design algorithms that reduce the bounds on SP-Regret from a linear dependence in the dimension of the problem to a logarithmic one. We also study the problem under bandit feedback and provide an algorithm that achieves sublinear SP-Regret. We then consider an online convex optimization with knapsacks problem motivated by a wide variety of applications, such as dynamic pricing, auctions, and crowdsourcing. We relate this problem to the online saddle point problem and establish [Formula: see text] regret using a primal-dual algorithm.

A novel collaborative iterative greedy algorithm for hybrid flowshop scheduling problem with batch processing machines and variable sublots

ABSTRACT Lot streaming technology enables continuous overlapping operations, which is of great significance in shortening production cycles, reducing unnecessary waiting time, and increasing production capacity. However, the capacity constraint of batch processing machines may lead to inevitable variations in sublots. Therefore, the key focus of our research is to control variations of sublot for maximising benefits. In view of this, we investigate a hybrid flowshop scheduling problem (HFSP) with batch processing machines and variable sublots (HFSP-BVS) integrating sequence-dependent setup times and transportation times. To address HFSP-BVS, a MILP model is first established, and a novel collaborative iterative greedy (NCIG) algorithm is proposed to optimise the cumulative payoffs associated with delivery dates. In NCIG, a collaborative initialisation method by extracting good information from an archive is proposed, and a specific destruction-reconfiguration strategy is designed to control the variations of sublots in the batch processing stage. Furthermore, a dynamic acceptance criterion is designed to balance the algorithm's exploitation and exploration capabilities. Lastly, we conduct comparisons between the NCIG algorithm and five other metaheuristic algorithms on 100 test instances. The results show that NCIG outperforms them by 1.89% and 61.42% on average in terms of the total penalty and RPI values, respectively.

Dynamic Learning in Large Matching Markets

We study a sequential matching problem faced by large centralized platforms where "jobs" must be matched to "workers" subject to uncertainty about worker skill proficiencies. Jobs arrive at discrete times (possibly in batches of stochastic size and composition) with "job-types" observable upon arrival. To capture the "choice overload" phenomenon, we posit an unlimited supply of workers where each worker is characterized by a vector of attributes (aka "worker-types") sampled from an underlying population-level distribution. The distribution as well as mean payoffs for possible workerjob type-pairs are unobservables and the platform's goal is to sequentially match incoming jobs to workers in a way that maximizes its cumulative payoffs over the planning horizon. We establish lower bounds on the regret of any matching algorithm in this setting and propose a novel rate-optimal learning algorithm that adapts to aforementioned primitives online. Our learning guarantees highlight a distinctive characteristic of the problem: achievable performance only has a second-order dependence on worker-type distributions; we believe this finding may be of interest more broadly.

The distributed flowshop scheduling problem with delivery dates and cumulative payoffs

In the classic distributed permutation flowshop scheduling problem (DPFSP), there are more studies on the minimization of makespan, total flow time, total tardiness, etc. This paper studies a new problem with a new optimization goal, the DPFSP with delivery dates and cumulative payoffs. It is a variation of the DPFSP with job release dates that maximizes the total payoff with a stepwise job objective function. The main contributions are summarized as follows. (1) A mathematical model is built to formulate the new problem. (2) The characteristics of the problem are explored, and the upper and lower bounds of the problem are given. Based on the problem-specific knowledge, an algorithm named Insert-Pruning is proposed to improve the efficiency of search. (3) Nine heuristic algorithms are proposed, including DRI, DRA, DEI, DEA, DNI, DNA, DII, DIA and DFF. (4) Combined with the characteristics of the problem, some modifications and improvements have been made to the IG algorithm to solve it, including the destruction method, the local search method and the acceptance criterion. (5) The experimental results show that the presented algorithm significantly outperforms the existing algorithms in the literature. In comparison with other competing algorithms in different dimensions, our algorithm has shown better performance, which verifies the effectiveness of this algorithm.

Open-Loop Equilibrium Strategies for Dynamic Influence Maximization Game Over Social Networks

We consider the problem of budget allocation for competitive influence maximization over social networks. In this problem, multiple competing parties (players) want to distribute their limited advertising resources over a set of social individuals to maximize their long-run cumulative payoffs. It is assumed that the individuals are connected via a social network and update their opinions based on the classical DeGroot model. The players must decide on the budget distribution among the individuals at a finite number of campaign times to maximize their overall payoff as a function of individuals’ opinions. Under some assumptions, we show that i) the optimal investment strategy for a single player can be found in polynomial time by solving a concave program, and ii) the open-loop equilibrium strategies for the multiplayer dynamic game can be computed efficiently by following natural regret-minimization dynamics. Our results extend earlier work on the static version of the problem to a dynamic multistage game.

Wealth-based rule favors cooperation in costly public goods games when individual selection is inevitable

Individual selection, as an effective mechanism, is often used in spatial evolutionary games to promote cooperation. Previous research assumes that, individual selection usually occurs with people who fail to meet a certain criterion. However, individual selection is usually inevitable, regardless of whether players in population cooperate or defect. This paper studies the effects of wealth-based rule in costly public goods games when individual selection is inevitable. Specifically, we assume that only the top V individuals with relatively high cumulative payoffs in each group can be selected for costly PGG. The results show that when V is large, the increase of participation cost has slight inhibitory effects on the evolution of cooperation, but it alleviates the polarization of individuals. However, when V is small, the increase of participation cost within a certain range promotes cooperation prosperity, but it also causes an increase in the proportion of polarized individuals and a widening of the wealth gap between rich and poor individuals.

Avoiding the bullies: The resilience of cooperation among unequals

Can egalitarian norms or conventions survive the presence of dominant individuals who are ensured of victory in conflicts? We investigate the interaction of power asymmetry and partner choice in games of conflict over a contested resource. Previous models of cooperation do not include both power inequality and partner choice. Furthermore, models that do include power inequalities assume a static game where a bully's advantage does not change. They have therefore not attempted to model complex and realistic properties of social interaction. Here, we introduce three models to study the emergence and resilience of cooperation among unequals when interaction is random, when individuals can choose their partners, and where power asymmetries dynamically depend on accumulated payoffs. We find that the ability to avoid bullies with higher competitive ability afforded by partner choice mostly restores cooperative conventions and that the competitive hierarchy never forms. Partner choice counteracts the hyper dominance of bullies who are isolated in the network and eliminates the need for others to coordinate in a coalition. When competitive ability dynamically depends on cumulative payoffs, complex cycles of coupled network-strategy-rank changes emerge. Effective collaborators gain popularity (and thus power), adopt aggressive behavior, get isolated, and ultimately lose power. Neither the network nor behavior converge to a stable equilibrium. Despite the instability of power dynamics, the cooperative convention in the population remains stable overall and long-term inequality is completely eliminated. The interaction between partner choice and dynamic power asymmetry is crucial for these results: without partner choice, bullies cannot be isolated, and without dynamic power asymmetry, bullies do not lose their power even when isolated. We analytically identify a single critical point that marks a phase transition in all three iterations of our models. This critical point is where the first individual breaks from the convention and cycles start to emerge.

Best response dynamics improve sustainability and equity outcomes in common-pool resources problems, compared to imitation dynamics

Shared resource extraction among profit-seeking individuals involves a tension between individual benefit and the collective well-being represented by the persistence of the resource. Many game theoretic models explore this scenario, but these models tend to assume either best response dynamics (where individuals instantly switch to better paying strategies) or imitation dynamics (where individuals copy successful strategies from neighbours), and do not systematically compare predictions under the two assumptions. Here we propose an iterated game on a social network with payoff functions that depend on the state of the resource. Agents harvest the resource, and the strategy composition of the population evolves until an equilibrium is reached. The system is then repeatedly perturbed and allowed to re-equilibrate. We compare model predictions under best response and imitation dynamics. Compared to imitation dynamics, best response dynamics increase sustainability of the system, the persistence of cooperation while decreasing inequality and debt corresponding to the Gini index in the agents’ cumulative payoffs. Additionally, for best response dynamics, the number of strategy switches before equilibrium fits a power-law distribution under a subset of the parameter space, suggesting the system is in a state of self-organized criticality. We find little variation in most mean results over different network topologies; however, there is significant variation in the distributions of the raw data, equality of payoff, clustering of like strategies and power-law fit. We suggest the primary mechanisms driving the difference in sustainability between the two strategy update rules to be the clustering of like strategies as well as the time delay imposed by an imitation processes. Given the strikingly different outcomes for best response versus imitation dynamics for common-pool resource systems, our results suggest that modellers should choose strategy update rules that best represent decision-making in their study systems.

Comparison of competing market mechanisms with reinforcement learning in a carpooling scenario

In this paper a multi-agent simulation was implemented to analyze the dynamics of different market mechanisms with a Reinforcement Learning algorithm in the context of a carpooling market. The agents in the simulation, car owners (COs) and non car owners (NCOs), had to sell or buy a car seat for multiple rounds by picking one of two possible mechanisms: Dutch Auction or Fixed Price. In the beginning of the simulation the agents have no information about the efficiency of these mechanisms and they are chosen with the same probability. In the course of the simulation a Reinforcement Learning algorithm alters the agents' preferences for the two mechanisms depending on their cumulative payoffs. The key finding is that sellers have a clear preference for the Dutch auction mechanism with differing degrees dependent on the seller/buyer ratio. Buyers on the other hand have no significant preference for any mechanism. If these results are replicable, they suggest that an increased utilization of the Dutch auction could lead to an expansion of the carpooling market, increasing its impact as an alternative means of transportation.

A simheuristic algorithm for the stochastic permutation flow‐shop problem with delivery dates and cumulative payoffs

AbstractThis paper analyzes the permutation flow‐shop problem with delivery dates and cumulative payoffs (whenever these dates are met) under uncertainty conditions. In particular, the paper considers the realistic situation in which processing times are stochastic. The main goal is to find the permutation of jobs that maximizes the expected payoff. In order to achieve this goal, the paper first proposes a biased‐randomized heuristic for the deterministic version of the problem. Then, this heuristic is extended into a metaheuristic by encapsulating it into a variable neighborhood descent framework. Finally, the metaheuristic is extended into a simheuristic by incorporating Monte Carlo simulations. According to the computational experiments, the level of uncertainty has a direct impact on the solutions provided by the simheuristic. Moreover, a risk analysis is performed using two well‐known metrics: the value‐at‐risk and conditional value‐at‐risk.

Long deliberation times promote cooperation in the prisoner’s dilemma game

Some individuals could be quite cautious when making decisions and do not update their strategy frequently in the prisoner’s dilemma game. That is, instead of updating their strategy immediately after the game, they make decisions only after a certain period of deliberation. Moreover, individuals’ strategy update is based on the cumulative payoffs they got from their prior update. Since individuals in a group could be homogeneous or heterogeneous, their strategy can depend on two aspects: the initial time of the update and the length of its delay time. In this paper, three different cases that may lead to the delay of strategy update are taken into consideration. We find that introducing a deliberation mechanism can promote cooperation, that this effect could be greater when the delay time is extended, and that heterogeneity among individuals also has a positive effect on cooperation.

An Online Saddle Point optimization algorithm with Regularization

This paper presents an online saddle-point optimization algorithm (OSP) to solve the optimal decision-making problem in economic games. In this setting, the two mutual competing players choose a pair of optimal decisions (Nash equilibrium) at each iteration. Firstly, the Follow the Leader (FTL) algorithm is proposed to update the decisions, and the regularization term is added to stabilize the Nash equilibrium for both players. Secondly, the saddle-point regret (SP-Regret) is used to measure the gap between the cumulative payoffs and the saddle point value of the aggregate payoff functions. To this end, this paper aims to minimize it. Finally, the simulation results show that, under the proposed OSP algorithm, the SP-Regret can still be sublinear with regularization and the decision variables of both players can be constrained to fluctuate within a certain range by adding regularization, which can effectively make the Nash equilibrium stable.

Heuristics for a flowshop scheduling problem with stepwise job objective function

In this work, we introduce the Flowshop Scheduling Problem with Delivery Dates and Cumulative Payoffs. This problem is a variation of the flowshop scheduling problem with job release dates that maximizes the total payoff with a stepwise job objective function. This paper contributes towards proposing a mathematical formulation for this new problem and an original constructive heuristic. Additionally, we develop a new benchmark of 300 hard instances which are available in a public repository. Besides, we provide primal and dual bounds for them. Extensive computational experiments were carried out taking into account classical constructive heuristics and hybrid local searches. Results show the merit of the FF heuristic when compared to other classical heuristics for flowshop scheduling problems. Additionally, we compare an Iterated Local Search (ILS), an Iterated Greedy Search (IGS), and a Biased Random-Key Genetic Algorithm (BRKGA), both customized for the studied problem, and a commercial mixed integer programming solver. The comparison between these methods showed that the BRKGA starting with a solution from FF heuristic is able to find the best solutions in a very short period of time. Iterated local Search and the commercial solver presented significantly worse results than the aforementioned methods.

CBRAP: Contextual Bandits with RAndom Projection

Contextual bandits with linear payoffs, which are also known as linear bandits, provide a powerful alternative for solving practical problems of sequential decisions, e.g., online advertisements. In the era of big data, contextual data usually tend to be high-dimensional, which leads to new challenges for traditional linear bandits mostly designed for the setting of low-dimensional contextual data. Due to the curse of dimensionality, there are two challenges in most of the current bandit algorithms: the first is high time-complexity; and the second is extreme large upper regret bounds with high-dimensional data. In this paper, in order to attack the above two challenges effectively, we develop an algorithm of Contextual Bandits via RAndom Projection (CBRAP) in the setting of linear payoffs, which works especially for high-dimensional contextual data. The proposed CBRAP algorithm is time-efficient and flexible, because it enables players to choose an arm in a low-dimensional space, and relaxes the sparsity assumption of constant number of non-zero components in previous work. Besides, we prove an upper regret bound for the proposed algorithm, which is associated with reduced dimensions. By comparing with three benchmark algorithms, we demonstrate improved performance on cumulative payoffs of CBRAP during its sequential decisions on both synthetic and real-world datasets, as well as its superior time-efficiency.

Total Reward Semi-Markov Mean-Field Games with Complementarity Properties

We study a class of dynamic games with a continuum of atomless players where each player controls a semi-Markov process of individual states, while the global state of the game is the aggregation of individual states of all the players. The model differs from standard models of dynamic games with continuum of players known as mean field or anonymous games in that the moments when the decisions are made are discrete, but different for each of the players. As a result, the individual states of each player follow a continuous time Markov chain, but the global state follows an ordinary differential equation. Games of this type were introduced by Gomes et al. (Appl Math Optim 68:99–143, 2013) and received some attention in the literature in last few years. In our paper we introduce a novel model of this type where players maximize their cumulative payoffs over their lifetime. We show that the payoffs of the players using any stationary strategy of a certain class in a game with continuum of players are close to those obtained in n-person counterparts of this game for n large enough. This implies that equilibrium strategies in the anonymous model can well approximate equilibria in related games with large finite number of players. In the rest of the paper we concentrate on a subclass of games where the payoff and transition probability functions exhibit some strategic complementarities between players. In that case we prove that the game possesses a stationary equilibrium. Moreover, largest and smallest equilibrium strategies are nondecreasing in the states. It also turns out that these equilibria can be well approximated using a distributed iterative procedure.

Imitation dynamics with payoff shocks

We investigate the impact of payoff shocks on the evolution of large populations of myopic players that employ simple strategy revision protocols such as the “imitation of success”. In the noiseless case, this process is governed by the standard (deterministic) replicator dynamics; in the presence of noise however, the induced stochastic dynamics are different from previous versions of the stochastic replicator dynamics (such as the aggregate-shocks model of Fudenberg and Harris in J Econ Theory 57(2):420–441, 1992). In this context, we show that strict equilibria are always stochastically asymptotically stable, irrespective of the magnitude of the shocks; on the other hand, in the high-noise regime, non-equilibrium states may also become stochastically asymptotically stable and dominated strategies may survive in perpetuity (they become extinct if the noise is low). Such behavior is eliminated if players are less myopic and revise their strategies based on their cumulative payoffs. In this case, we obtain a second order stochastic dynamical system where non-equilibrium states are no longer attracting and dominated strategies become extinct (a.s.), no matter the noise level.

Group payoffs as public signals

We study experimentally the effect on individual behavior of comparative, but payoff-irrelevant, information in a near-minimal group setting. Specifically, in each round, group members see the groups’ cumulative payoffs, which consist of an aggregation of the earnings of each member of the group in the round. Our novel experimental design incorporates two games (the Trust game and the Dictator game) whose payoffs are carefully chosen to ensure cross-game comparability. In the baseline, no comparative information is displayed; the sessions are otherwise identical.Our first set of results shows that the display is sufficient to induce an in-group bias, which can neither be attributed to mere categorization of subjects into groups nor to a stronger sense of group identity as a result of the display. Moreover, we corroborate existing results, which find that, relative to the baseline, the display is welfare reducing in the Trust game. Our second set of results shows that when comparing the allocators’ decisions across the two games, a first mover’s trust is reciprocated by the second mover independently of group identity.

Learning in Games via Reinforcement and Regularization

We investigate a class of reinforcement learning dynamics where players adjust their strategies based on their actions’ cumulative payoffs over time—specifically, by playing mixed strategies that maximize their expected cumulative payoff minus a regularization term. A widely studied example is exponential reinforcement learning, a process induced by an entropic regularization term which leads mixed strategies to evolve according to the replicator dynamics. However, in contrast to the class of regularization functions used to define smooth best responses in models of stochastic fictitious play, the functions used in this paper need not be infinitely steep at the boundary of the simplex; in fact, dropping this requirement gives rise to an important dichotomy between steep and nonsteep cases. In this general framework, we extend several properties of exponential learning, including the elimination of dominated strategies, the asymptotic stability of strict Nash equilibria, and the convergence of time-averaged trajectories in zero-sum games with an interior Nash equilibrium.

Strategies generalization and payoff fluctuation optimization in the iterated ultimatum game

An iterated version of ultimatum game, based on generalized probabilistic strategies, which are mathematically modeled by accepting proposal functions is presented. These strategies account for the behavior of the players by mixing levels of altruism and greed. We obtained analytically the moments of the payoff of the players under such a generalization. Our analysis is divided into two cases: (i) no memory players, where players do not remember previous decisions, and (ii) one-step memory players, where the offers depend on players’ last decision. We start considering the former case. We show that when the combination of the proposer’s altruism and responder’s greed levels balances the proposer’s greedy and responder’s altruism levels, the average and variance of the payoff of both players are the same. Our analysis is carried out considering that the acceptance of an offer depends on: (a) a fixed probability p or (b) the value offered. The combination of cases (i) and (a) shows that there exists a p value that maximizes the cumulative gain after n iterations. Moreover, we show n×p diagrams with ïso-average” and ïso-variance” of the cumulative payoff. Our analytical results are validated by Monte Carlo simulations. For the latter case, we show that when players have no memory (i), there are cutoff values, which the variance of the proposer’s cumulative payoff presents local maximum and minimum values, while for the responder, the same amount presents a global maximum. Case (b) combined with one-step memory players (ii), we verified, via MC simulations that, for the same number of iterations, the responder obtains different cumulative payoffs by setting different cutoff values. This result composes an interesting pattern of stripes in the cutoff per n diagrams. Simultaneously, looking at variance of this amount, for the responder player in a similar diagram, we observe regions of iso-variance in non trivial patterns which depend on initial value of the proposal. Our contributions detailed by analytical and MC simulations are useful to design new experiments in the ultimatum game in stochastic scenarios.