Regret Minimization Research Articles

Automatically designing counterfactual regret minimization algorithms for solving imperfect-information games

Strategic decision-making in imperfect-information games is an important problem in artificial intelligence. Counterfactual regret minimization (CFR), a family of iterative algorithms, has been the workhorse for solving these types of games since its inception. In recent years, a series of novel CFR variants have been proposed, significantly improving the convergence rate of vanilla CFR. However, most of these new variants are hand-designed by researchers through trial and error, often based on different motivations, which generally requires a tremendous amount of effort and insight. This work proposes AutoCFR, a systematic framework that meta-learns novel CFR algorithms through evolution, easing the burden of manual algorithm design. We first design a search language that is rich enough to represent various CFR variants. We then exploit a scalable regularized evolution algorithm with a set of acceleration techniques to efficiently search over the combinatorial space of algorithms defined by this language. The learned novel CFR algorithm can generalize to new imperfect-information games not seen during training and performs on par with or better than existing state-of-the-art CFR variants. In addition to superior empirical performance, we also theoretically show that the learned algorithm converges to an approximate Nash equilibrium. Extensive experiments across diverse imperfect-information games highlight the scalability, extensibility, and generalizability of AutoCFR, establishing it as a general-purpose framework for solving imperfect-information games.

Monte-Carlo Regret Minimization for Adversarial Team Games

We study equilibrium approximation in extensive-form adversarial team games, in which two teams of rational players compete in a zero-sum interaction. The suitable solution concept in these settings is the Team-Maxmin Equilibrium with Correlation (TMECor), which naturally arises when the team players play ex-ante correlated strategies. While computing such an equilibrium is APX-hard, recent techniques show that scalability beyond toy instances is possible. However, even compact representations of the team’s strategy space, such as that exploiting Directed Acyclic Graphs (DAGs), have exponential size prohibiting solving large instances. In the present paper, we show that Monte Carlo sampling for regret minimization in adversarial team games can provide an important advancement. In particular, we design a DAG Monte Carlo Counterfactual Regret Minimization algorithm that performs outcome sampling with O ( d ) time complexity per iteration, where d is the depth of the DAG, and with a convergence rate bound of O ( b kd ) , where b is the branching factor and k is the maximum number of private states in each public state of the team. We empirically evaluate our algorithms with a standard testbed of games, showing their performance when approximating equilibria. We investigate both cases in which only one team is composed of multiple players and cases in which both teams are composed of multiple players. Our empirical results show that our algorithms can provide a rough approximation to equilibrium strategies in all game instances considered, trading off a lower precision in the computed equilibrium for lower resource utilization.

Exploring tourists’ preferences and willingness to pay for national park recreation improvements based on regret and utility comparison

Research on the improvement of national park recreation policies has attracted much attention to discrete choice experiments to obtain tourists’ preferences and willingness to pay. However, individual choice behavior is extremely complex, and the single Random Utility Maximization (RUM) model ignores anticipated regret and is insufficient to explain individuals’ actual choice behavior. To investigate whether regret influences tourists’ choices regarding the improvement of national park recreation attributes, this study introduces the Random Regret Minimization (RRM) model and explores the performance of polynomial logit models and hybrid latent class models in analyzing discrete choice models based on utility and regret. By constructing a hybrid utility-regret model, we examine how tourists trade off between attributes such as vegetation coverage, water clarity, amount of litter, and level of crowding in national park recreation. Results indicate that the RRM model has better goodness-of-fit and predictive ability than the RUM model, indicating that regret is a significant choice paradigm, and the hybrid model better explains respondents’ choices. Specifically, 62.5% of tourists’ choices are driven by regret, and regret-driven respondents are more inclined to increase vegetation coverage and improve water clarity, while utility-driven respondents are more inclined to reduce litter and crowding. This study not only provides a reference for managers to develop more optimal recreation improvement strategies but also offers theoretical insights for national park recreation improvement policies.

A fast strategy-solving method for adversarial team games utilizing warm starting

Adversarial team games (ATGs) have garnered significant attention in recent years, leading to the emergence of various solutions such as linear programming algorithms, multi-agent reinforcement learning, and game tree transformation. The ATGs involve large-scale game trees, resulting in higher time costs for computation. In this paper, we focus on expediting the solution of team-maxmin equilibrium with correlation (TMECor), which can be considered the equilibrium that maximizes the team’s payoff. To address this, we propose a transformation with seed strategies (TSS). TSS leverages reinforcement learning to calculate player strategies. We initialize the strategies of all players, referred to as seed strategies, and incorporate them into the multi-agent game tree during the transformation process. These seed strategies serve as the starting strategies for counterfactual regret minimization (CFR). CFR initializes the strategies and cumulative regret of all players based on the seed strategy. By warm starting the whole process, our method accelerates the solving of TMECor. We conducted nine experiments using Kuhn poker and Leduc Hold’em poker. The results demonstrated that TSS improved the solving speed of TMECor.

Pre-disaster evacuation network design with uncertain demand and behavioural choices: A bi-criteria generalized cost perspective

Pre-disaster evacuation network design is critical to improving evacuation efficiency and reducing the loss of life and property caused by disasters. However, evacuation network design is complicated by the uncertainty of pre-disaster evacuation demand and the heterogeneity of evacuees’ route choices when faced with a disaster scenario. Therefore, how to design efficient evacuation networks considering these uncertainties has become an important research topic. In this paper, we study the pre-disaster flood evacuation network design problem, define a bi-criteria generalized cost metric based on Evacuation Travel Time (ETT) and Evacuation Network Congestion Degree (ENCD), and develop a model that considers uncertain demand and evacuation behavioural choices to minimize the generalized cost in the evacuation network. For the uncertain behavioural choice, we propose an Expanded Stochastic User Equilibrium (ESUE) model based on the Random Regret Minimization (RRM) principle and Boundedly Rational User Equilibrium (BRUE) conditions to describe it. For the uncertain demand, we design a Distributionally Robust Optimization (DRO) method based on Mean Absolute Deviation (MAD) to deal with this uncertain model. This paper designs an Improved Method of Successive Averages Combined with Genetic Algorithm (IMSA-GA) to solve the model and uses the Nguyen-Dupuis test network to verify the applicability and feasibility of the model and algorithm. In addition, we also conduct an evacuation network design in the Hawkesbury-Nepean region of Australia. The larger the threshold value based on the regret utility ratio, the smaller the generalized cost value, which indicates the behavioural choices of the evacuees have an impact on the evacuation network design, and the generalized cost decreases by 6.12% when the threshold value is increased from 0.1 to 0.9. In addition, the generalized cost value obtained by IMSA-GA will be 0.0025 lower than Improved Method of Successive Averages Combined with Simulated Annealing (IMSA-SA) and 0.0005 lower than Improved Method of Successive Averages Combined with Taboo Search (IMSA-TS). The theoretical contribution of this paper is to find an approach to the uncertain evacuation network design problem, especially to improve the practicality of the model by fully considering evacuation behaviour. The practical contribution is that our findings can provide insights for government authorities to design efficient and reliable evacuation networks that balance evacuation travel time and evacuation network congestion. In the future, if evacuation behavioural choices can be analysed based on actual surveys, the estimation of behavioural parameters based on survey data will be more instructive for the evacuation process.

Breast cancer pre-diagnosis based on incomplete picture fuzzy multi-granularity three-way decisions

PurposeThree-way decision (3WD) and probabilistic rough sets (PRSs) are theoretical tools capable of simulating humans' multi-level and multi-perspective thinking modes in the field of decision-making. They are proposed to assist decision-makers in better managing incomplete or imprecise information under conditions of uncertainty or fuzziness. However, it is easy to cause decision losses and the personal thresholds of decision-makers cannot be taken into account. To solve this problem, this paper combines picture fuzzy (PF) multi-granularity (MG) with 3WD and establishes the notion of PF MG 3WD.Design/methodology/approachAn effective incomplete model based on PF MG 3WD is designed in this paper. First, the form of PF MG incomplete information systems (IISs) is established to reasonably record the uncertain information. On this basis, the PF conditional probability is established by using PF similarity relations, and the concept of adjustable PF MG PRSs is proposed by using the PF conditional probability to fuse data. Then, a comprehensive PF multi-attribute group decision-making (MAGDM) scheme is formed by the adjustable PF MG PRSs and the VlseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) method. Finally, an actual breast cancer data set is used to reveal the validity of the constructed method.FindingsThe experimental results confirm the effectiveness of PF MG 3WD in predicting breast cancer. Compared with existing models, PF MG 3WD has better robustness and generalization performance. This is mainly due to the incomplete PF MG 3WD proposed in this paper, which effectively reduces the influence of unreasonable outliers and threshold settings.Originality/valueThe model employs the VIKOR method for optimal granularity selections, which takes into account both group utility maximization and individual regret minimization, while incorporating decision-makers' subjective preferences as well. This ensures that the experiment maintains higher exclusion stability and reliability, enhancing the robustness of the decision results.

Fitting mixed random regret minimization models using maximum simulated likelihood

Fitting mixed random regret minimization models using maximum simulated likelihood

Regret optimal control for uncertain stochastic systems

We consider control of uncertain linear time-varying stochastic systems from the perspective of regret minimization. Specifically, we focus on the problem of designing a feedback controller that minimizes the loss relative to a clairvoyant optimal policy that has foreknowledge of both the system dynamics and the exogenous disturbances. In this competitive framework, establishing robustness guarantees proves challenging as, differently from the case where the model is known, the clairvoyant optimal policy is not only inapplicable, but also impossible to compute without knowledge of the system parameters. To address this challenge, we embrace a scenario optimization approach, and we propose minimizing regret robustly over a finite set of randomly sampled system parameters. We prove that this policy optimization problem can be solved through semidefinite programming, and that the corresponding solution retains strong probabilistic out-of-sample regret guarantees in face of the uncertain dynamics. Our method naturally extends to include satisfaction of safety constraints with high probability. We validate our theoretical results and showcase the potential of our approach by means of numerical simulations.

Effective multi-attribute group decision-making approach to study astronomy in the probabilistic linguistic q-rung orthopair fuzzy VIKOR framework

This study employs a novel fuzzy logic-based framework to address multi-attribute group decision-making problems commonly encountered in modern astronomy. Our approach utilizes the probabilistic linguistic q-rung orthopair fuzzy set (PLq-ROFS) to handle the inherent uncertainties associated with astronomical data. The PLq-ROFS offers significant advantages over existing fuzzy sets like probabilistic hesitant, linguistic intuitionistic, and linguistic Pythagorean fuzzy sets, which comprise both stochastic and non-stochastic uncertainties simultaneously. To aggregate the probabilistic linguistic decision information effectively, we propose two novel operators: the PLq-ROF weighted power average (PLq-ROFWPA) and the PLq-ROF weighted power geometric (PLq-ROFWPG). These operators form the foundation of a novel method within the PLq-ROF environment. Furthermore, this study integrates the PLq-ROF framework with the VIseKriterijumska Optimizacija I Kompromisno Resenje (VIKOR) model, a widely used decision-making (DM) tool known for its ability to balance group utility maximization with individual regret minimization. This integration leads to the PLq-ROF-VIKOR model, a novel approach for ranking alternative solutions based on the subjective preferences of decision-makers. The effectiveness of the proposed method is demonstrated through a real-world case study in astronomy, accompanied by both parameter and comparative analyses. These analyses highlight the efficiency and accuracy of the PLq-ROF-VIKOR model, ultimately leading to the conclusion that cosmology is the most optimal key finding in this case study.

Random Regret Minimization Approach to Commuting Mode Choice in São Paulo, Brazil

The random regret minimization (RRM) approach has been widely used in transport literature, but its application in the Global South is still marginal. In this paper we discuss individual commuting mode choice in the city of São Paulo (Brazil) from the perspective of the RRM modeling approach and its variants found in the literature. We estimated several multinomial logit models (random utility maximization [RUM], classical RRM, [Formula: see text]RRM, and hybrid formulations of RUM-RRM models) and explored regret scale and decision rule heterogeneities using latent class models with specific [Formula: see text] parameters. The results showed that the RRM approach outperformed its RUM counterpart in relation to model fit and suggested that it better captured the mode choice behavior of individuals in the analyzed context. We also found that accounting for heterogeneity in scale and decision rules improved the results of the models, and the specific [Formula: see text] parameters indicated that individuals displayed different regret behavior for travel time and travel cost attributes.

Understanding the route choice behavior of metro passenger using the smartphone applications

Understanding the distribution of passenger flow is one of the crucial prerequisites for improving operational efficiency and implementing travel demand management strategies. This paper proposes an improved Random Regret Minimization (i-RRM) model based on the regret theory, where the passenger’s regret on selecting routes is minimized. Considering the growing trend of passengers relying on smartphone apps for route planning and navigation, three factors are investigated, namely, travel time, the number of transfer stations, and the total number of stations traveled. Automatic Fare Collection (AFC) is used to get the proportion of each route, and mobile phone signaling data is used to obtain the ground truth of passengers’ routes proportions for parameter calibration and model validation. The results show that, compared to the ground truth data, the average errors of the proposed i-RRM model, the classic RRM model, and the direct path matching algorithm are 6.03%, 8.96%, and 10.45%, respectively.

The impacts of COVID-19 on route choice with guidance information in urban rail transit of megacities

The outbreak of COVID-19 has caused unprecedented decline of ridership in urban rail transit and changed passenger travel habits, which greatly challenges subway operations. Therefore, it is necessary to better understand and quantify the impact of COVID-19 on passenger travel behavior, specifically route choice. Thus, we collected automatic fare collection data and 2060 random samples through a web-based survey in Beijing on passengers’ route choice behavior during the COVID-19 pandemic. This study utilizes an initial dataset to conduct an analysis and introduces an improved Generalized Random Regret Minimization model (GRRM) aimed at understanding passengers' route choice adjustments in response to COVID-19 guidance information. This improved GRRM accounts for two decision-making criteria, namely, maximum utility and minimum regret, and considers passenger heterogeneity. This marks the first instance of capturing the heterogeneity shift effect in route choice perception during the COVID-19 pandemic. The results show that the improved model has the best fitting result with adjusted Rho square of 0.536, demonstrating that the attributes related to guidance information (i.e., information push/time to receive traffic information/perceived route COVID-19 risk) indeed enhance the model’s fit. Furthermore, the research employs Value of Information Time to quantify the preference of passengers for information in various groups. Compared with the usual scenario, women, young and non-commuter passengers are more likely to receive an early information update to plan their trips in advance. Finally, the perceived risk of COVID-19 on routes is examined in relation to passengers’ personal attributes. It is observed that the elderly and students exhibit heightened sensitivity to the epidemic at all stages, while young passengers and commuters are particularly sensitive only during the small-scale epidemic. These findings offer valuable insights for managers to implement targeted strategies, thereby enhancing passenger flow control and encouraging increased subway ridership.

Communication-Efficient Collaborative Regret Minimization in Multi-Armed Bandits

In this paper, we study the collaborative learning model, which concerns the tradeoff between parallelism and communication overhead in multi-agent multi-armed bandits. For regret minimization in multi-armed bandits, we present the first set of tradeoffs between the number of rounds of communication between the agents and the regret of the collaborative learning process.

Learning Not to Regret

The literature on game-theoretic equilibrium finding predominantly focuses on single games or their repeated play. Nevertheless, numerous real-world scenarios feature playing a game sampled from a distribution of similar, but not identical games, such as playing poker with different public cards or trading correlated assets on the stock market. As these similar games feature similar equilibra, we investigate a way to accelerate equilibrium finding on such a distribution. We present a novel ``learning not to regret'' framework, enabling us to meta-learn a regret minimizer tailored to a specific distribution. Our key contribution, Neural Predictive Regret Matching, is uniquely meta-learned to converge rapidly for the chosen distribution of games, while having regret minimization guarantees on any game. We validated our algorithms' faster convergence on a distribution of river poker games. Our experiments show that the meta-learned algorithms outpace their non-meta-learned counterparts, achieving more than tenfold improvements.

No Internal Regret with Non-convex Loss Functions

Internal regret is a measure of performance of an online learning algorithm, which measures the change in performance by substituting every occurrence of a given action i by an alternative action j. Algorithms for minimizing internal regret are known for the finite experts setting, including a general reduction to the problem of minimizing external regret for this case. The reduction however crucially depends on the finiteness of the action space. In this work we approach the problem of minimizing internal regret for a continuous action space. For the full information setting, we show how to obtain O(sqrt(T)) internal regret for the class of Lipschitz functions, as well as non-Lipschitz dispersed functions, i.e. the non-Lipschitzness may not concentrate in a small region of the action space. We also consider extensions to partial feedback settings, and again obtain sublinear internal regret. Finally we discuss applications of internal regret minimization over continuous spaces to correlated equilibria in pricing problems and auction design, as well as to data-driven hyperparameter tuning.

Comparative analysis of Sliding Window UCB and Discount Factor UCB in non-stationary environments: A Multi-Armed Bandit approach

The Multi-Armed Bandit (MAB) problem is a well-studied topic within stationary environments, where the reward distributions remain consistent over time. Nevertheless, many real-world applications often fall within non-stationary contexts, where the rewards from each arm can evolve. In light of this, our research focuses on examining and contrasting the effectiveness of two leading algorithms tailored for these shifting environments: the Sliding Window Upper Confidence Bound (SW-UCB) and the Discount Factor UCB (DF-UCB). By harnessing both simulated and real-world datasets, our evaluation encompasses adaptability, computational efficiency, and the potential for regret minimization. Our findings reveal that the SW-UCB is adept at swiftly adjusting to sudden shifts, whereas the DF-UCB emerges as the more resource-efficient option amidst gradual transitions. Notably, when pitted against conventional UCB algorithms within non-stationary contexts, both contenders exhibit substantial advancements. Such insights bear significant relevance to fields like online advertising, healthcare, and finance, where the capacity to nimbly adapt to dynamic environments is paramount.

Code and Data Repository for Regret Minimization and Separation in Multi-Bidder, Multi-Item Auctions

This repository aims to showcase the effectiveness of our proposed regret-minimizing mechanism against various benchmark mechanisms.

Regret Minimization and Separation in Multi-Bidder, Multi-Item Auctions

Regret Minimization and Separation in Multi-Bidder, Multi-Item Auctions

Adapting bandit algorithms for settings with sequentially available arms

Many real-world applications involve a sequential decision-making process where the options presented simultaneously. However, other applications, such as, Internet campaign management and environmental monitoring, the available options are presented sequentially to the decision-maker who, at each time, is asked to select the proposed option or not. This scenario is defined as the Sequential Pull/No-Pull setting The present study aims at developing a meta-algorithm, namely Sequential Pull/No-pull for MAB (Seq), to adapt any classical MAB (Multi-Armed Bandit) policy for this setting both in the case of regret minimization (RM) and best-arm identification (BAI) problems. This is achieved by exploting the sequential nature of the these settings allowing to select multiple arms and gather more information compared to classical policies. The proposed Seq meta-algorithm provides the same theoretical guarantees as the MAB policy employed, but was shown to provide improved performance compared to several classical MAB policies in RM and BAI problems employing real-world data. In particular, in the RM scenario regarding Internet advertising optimization, Seq-adapted algorithm resulted, on average, in ≈10% lower regret during the whole time horizon than using classical MAB policies. When tested in a BAI problem involving the identification of the time of the day characterized by the highest concentration of pollutants in a water monitoring scenario, Seq identified the correct time in less than 4 days and 28 measurement.

On the impact of decision rule assumptions in experimental designs on preference recovery: An application to climate change adaptation measures

Efficient experimental designs aim to maximise the information obtained from stated choice data to estimate discrete choice models' parameters statistically efficiently. Almost without exception efficient experimental designs assume that decision-makers use a Random Utility Maximisation (RUM) decision rule. When using such designs, researchers (implicitly) assume that the decision rule used to generate the design has no impact on respondents' choice behaviour. This study investigates whether the decision rule assumption underlying an experimental design affects respondents' choice behaviour. We use four stated choice experiments on coastal adaptation to climate change: Two are based on experimental designs optimised for utility maximisation and two are based on experimental designs optimised for a mixture of RUM and Random Regret Minimisation (RRM). Generally, we find that respondents place value on adaptation measures (e.g., dykes and beach nourishments). We evaluate the models' fits and investigate whether some choice tasks particularly invoke RUM or RRM decision rules. For the latter, we develop a new sampling-based approach that avoids the confounding between preference and decision rule heterogeneity. We find no evidence that RUM-optimised designs invoke RUM-consistent choice behaviour. However, we find a relationship between some of the attributes and decision rules, and compelling evidence that some choice tasks invoke RUM consistent behaviour while others invoke RRM consistent behaviour. This implies that respondents’ choice behaviour and choice modelling outcomes are not exogenous to the choice tasks, which can be particularly critical when information on preferences is used to inform actual decision-making on a sensitive issue of common interest as climate change.