Towards a moving target defense based on stochastic games and honeypots

Variants of Harsanyi’s tracing procedures for selecting a perfect stationary equilibrium in stochastic games

ABSTRACTBuilding on the macroscopic market making framework as a control problem, this paper investigates its extension to stochastic games. In the context of price competition, each agent is benchmarked against the best quote offered by the others. We begin with the linear case. While constructing the solution directly, the ordering property and the dimension reduction in the equilibrium are revealed. For the nonlinear case, we extend the decoupling approach by introducing a multidimensional characteristic equation to analyze the well‐posedness of the forward–backward stochastic differential equations. Properties of the coefficients in this characteristic equation are derived using tools from nonsmooth analysis. Several new well‐posedness results are presented. Finally, we discuss applications to price impacts and the optimal execution problem.

Risk-sensitive continuous-time stochastic games with the average criterion and a compact state space

CIL: Cyber security index level of cyber-physical systems: A robust stochastic game approach using MITRE ATT&CK framework

Unmanned aerial vehicles (UAVs) used as aerial base stations (ABS) can provide economical, on-demand wireless access. This research investigates dynamic resource allocation in multi-UAV-enabled communication systems with the aim of maximizing long-term rewards. More specifically, without exchanging information with other UAVs, every UAV chooses its communicating users, power levels, and sub-channels to establish communication with a ground user. In the proposed work, the dynamic scheme-based resource allocation is investigated of communication networks made possible by many UAVs to achieve the highest possible performance level over time. Specifically, each UAV selects its connected users, battery power, and communication channel independently, without exchanging information across multiple UAVs. This allows each UAV to connect with ground users. To model the unpredictability of the environment, we present the problem of long-term allocation of system resources as a stochastic game to maximize the anticipated reward. Each UAV in this game plays the role of a learnable agent, and the system solution for resource allocation matches the actions made by the UAV. Afterward, we built a framework called reward-based multi-agent learning (RMAL), in which each agent uses learning to identify its best strategies based on local observations. RMAL is an acronym for ″reward-based multi-agent learning″. We specifically offer an agent-independent strategy where each agent decides algorithms separately but cooperates on a common Q-learning-based framework. The performance of the suggested RMAL-based resource allocation method may be enhanced by employing the right development and exploration parameters, according to the simulation findings. Secondly, the proposed RMAL algorithm provides acceptable performance over full information exchange between UAVs. Doing so achieves a satisfactory compromise between the increase in performance and the additional burden of information transmission.

Dense, microbial communities are shaped by local interactions between cells. Both the nature of interactions, spanning antagonistic to cooperative, and the strength of interactions vary between and across microbial species and strains. These local interactions can influence the emergence and maintenance of microbial diversity. However, it remains challenging to link features of local interactions with spatially mediated coexistence dynamics given the significant variation in the microscopic mechanisms involved in cell-to-cell feedback. Here, we explore how microbial interactions over a broad range of ecological contexts spanning antagonism to cooperation can enable coexistence as spatially explicit domains emerge. To do so, we introduce and analyze a family of stochastic coordination games, where individuals do better when playing (i.e., interacting) with individuals of the same type than when playing with individuals of a different type. Using this game-theoretic framework, we show that the population dynamics for coordination games is governed by a double-well shaped interaction potential. We find that in a spatial setting this double-well potential induces phase separation, facilitating coexistence. Moreover, we show that for microbes engaged in symmetric coordination games, phase separation takes on a universal scaling that follows ‘Model A’ coarsening, consistent with prior experimental observations for Vibrio cholerae mutual killers. Finally, we derive a PDE equivalent of the spatial stochastic game, confirming both the double-well nature of spatial coordination games and the universality of phase separation. Altogether, this work extends prior findings on the link between microbial interactions and population structure and suggests generic mechanisms embedded in local interactions that can enable coexistence.

ABSTRACT This study introduces a continuous-time stochastic game model to analyze the strategic dynamics of economic sanctions within the global monetary system, focusing on the interplay between a sanctioning player (e.g. Western countries) and a sanctioned player (e.g. Russia). Employing a four-node framework – provocation, sanctioning, reinforcement, and continuation – the model captures the recursive, uncertain interactions exemplified by Russia’s 2022 SWIFT exclusion, Libya’s 2003 strategic retreat, and the 1979 Iranian hostage crisis. By integrating game theory with stochastic differential equations, it formalizes the cost-benefit calculus of sanctions, revealing emergent patterns of sustained conflict or de-escalation driven by economic pressures, geopolitical resilience, and stochastic shocks. Monte Carlo simulations quantify how variations in sanction sensitivity, costs, and benefits shape strategic outcomes, validated against empirical data (e.g. Russia’s 20–45 billion USD/year trade losses). The model’s contributions are threefold: it advances theoretical rigor by embedding stochastic processes in game-theoretic analysis, provides practical insights for predicting sanction efficacy and unintended consequences (e.g. de-dollarization, trade realignments), and offers policymakers a versatile tool to navigate global financial volatility. This framework redefines sanctions as dynamic processes, ensuring lasting relevance for understanding economic statecraft amid deepening global interdependence.

Fuzzy Dynamic Event-Triggered Control for Constrained Stochastic Game Systems

Ad hoc networks are self-organizing systems that operate without a centralized controller or orchestration mechanism. As a result, it is not possible to apply allocation methods designed for centralized systems, which typically require complete information and aim to optimize overall system performance without accounting for the individual interests of network members. To address this challenge, we propose a computational justice model for dynamic resource allocation, drawing on socially inspired computing and agent-based modeling. The model integrates stochastic games, the concept of social institutions, principles of distributive justice, and adaptive strategies to design an allocation mechanism guided by fairness and cooperation. A central contribution of this work is the conceptual integration of these components into a unified framework that supports dynamic resource allocation in decentralized environments. We evaluated our proposal through simulation and compared its performance with previous works. The results show that the proposed model ensures the endurance of available resources and maintains cooperative behavior among network members, even in the presence of selfish behaviors. These findings suggest that the proposed model is a potential solution for addressing dynamic allocation problems in ad hoc networks.

We present a solution to the complex design-automation problem of scheduling test operations in a validation laboratory or production facility. Our goal is to maximize the utilization of a fleet of test stations and minimize the overall test time for a set of products. We consider the realistic scenario where tests can have dependency graphs, implying that some tests must be completed and passed before others can proceed. We also consider a mix of product types that require different kinds of tests and a mix of testers, which implies that each product can only be tested only on a specific set of testers. To ensure scalability and flexibility, we have formulated this scheduling problem as a ’partially observable stochastic game’, a multi-agent extension of a partially observable Markov decision process. We have implemented multi-agent reinforcement learning agents to maximize parallelization in a manner that speeds up both training and inferencing. We present scheduling results for synthetic test cases as well as real-life data from a production facility

Fixpoints are ubiquitous in computer science as they play a central role in providing a meaning to recursive and cyclic definitions. Bisimilarity, behavioural metrics, termination probabilities for Markov chains and stochastic games are defined in terms of least or greatest fixpoints. Here we show that our recent work which proposes a technique for checking whether the fixpoint of a function is the least (or the largest) admits a natural categorical interpretation in terms of gs-monoidal categories. The technique is based on a construction that maps a function to a suitable approximation. We study the compositionality properties of this mapping and show that under some restrictions it can naturally be interpreted as a (lax) gs-monoidal functor. This guides the development of a tool, called UDEfix that allows us to build functions (and their approximations) like a circuit out of basic building blocks and subsequently perform the fixpoints checks. We also show that a slight generalisation of the theory allows one to treat a new relevant case study: coalgebraic behavioural metrics based on Wasserstein liftings.

Optimal Doubling Thresholds in Backgammon-Like Stochastic Games

Single-Controller Chance-Constrained Stochastic Games

Corrigendum to “Towards a moving target defense based on stochastic games and honeypots”. [Inf. Sci. 720 (2025) 122488

Zero-sum risk-sensitive continuous-time stochastic games with unbounded reward and transition rates in Borel spaces

Partially observable discrete-time stochastic games under risk probability criterion

Multiagent learning is challenging when agents face mixed-motivation interactions, where conflicts of interest arise as agents independently try to optimize their respective outcomes. Recent advancements in evolutionary game theory have identified a class of "zero-determinant" strategies, which confer an agent with significant unilateral control over outcomes in repeated games. Building on these insights, we present a comprehensive generalization of zero-determinant strategies to stochastic games, encompassing dynamic environments. We propose an algorithm that allows an agent to discover strategies enforcing predetermined linear (or approximately linear) payoff relationships. Of particular interest is the relationship in which both payoffs are equal, which serves as a proxy for fairness in symmetric games. We demonstrate that an agent can discover strategies enforcing such relationships through experience alone, without coordinating with an opponent. In finding and using such a strategy, an agent ("enforcer") can incentivize optimal and equitable outcomes, circumventing potential exploitation. In particular, from the opponent's viewpoint, the enforcer transforms a mixed-motivation problem into a cooperative problem, paving the way for more collaboration and fairness in multiagent systems.

Biological systems, ranging from ant colonies to neural ecosystems, exhibit remarkable self-organizing intelligence. Inspired by these phenomena, this study investigates how bio-inspired computing principles can bridge game-theoretic rationality and multi-agent adaptability. This study systematically reviews the convergence of multi-agent reinforcement learning (MARL) and game theory, elucidating the innovative potential of this integrated paradigm for collective intelligent decision-making in dynamic open environments. Building upon stochastic game and extensive-form game-theoretic frameworks, we establish a methodological taxonomy across three dimensions: value function optimization, policy gradient learning, and online search planning, thereby clarifying the evolutionary logic and innovation trajectories of algorithmic advancements. Focusing on complex smart city scenarios-including intelligent transportation coordination and UAV swarm scheduling-we identify technical breakthroughs in MARL applications for policy space modeling and distributed decision optimization. By incorporating bio-inspired optimization approaches, the investigation particularly highlights evolutionary computation mechanisms for dynamic strategy generation in search planning, alongside population-based learning paradigms for enhancing exploration efficiency in policy refinement. The findings reveal core principles governing how groups make optimal choices in complex environments while mapping the technological development pathways created by blending cross-disciplinary methods to enhance multi-agent systems.

Stochastic two-player games model systems with an environment that is both adversarial and stochastic. The adversarial part of the environment is modeled by a player (Player 2) who tries to prevent the system (Player 1) from achieving its objective. We consider finitary versions of the traditional mean-payoff objective, replacing the long-run average of the payoffs by payoff average computed over a finite sliding window. Two variants have been considered: in one variant, the maximum window length is fixed and given, while in the other, it is not fixed but is required to be bounded. For both variants, we present complexity bounds and algorithmic solutions for computing strategies for Player 1 to ensure that the objective is satisfied with positive probability, with probability 1, or with probability at least $p$, regardless of the strategy of Player 2. The solution crucially relies on a reduction to the special case of non-stochastic two-player games. We give a general characterization of prefix-independent objectives for which this reduction holds. The memory requirement for both players in stochastic games is also the same as in non-stochastic games by our reduction. Moreover, for non-stochastic games, we improve upon the upper bound for the memory requirement of Player 1 and upon the lower bound for the memory requirement of Player 2.