Hierarchical Games Research Articles

Robust hierarchical games of linear discrete-time systems based on off-policy model-free reinforcement learning

An off-policy model-free reinforcement learning (RL) algorithm is proposed for a robust hierarchical game while considering incomplete information and input constraints. The robust hierarchical game exhibits characteristics of a Stackelberg–Nash (SN) game, where equilibrium points are designated as Stackelberg–Nash–Saddle equilibrium (SNE) points. An off-policy method is employed for the RL algorithm, addressing input constraints by using excitation input instead of real-time update polices as control inputs. Moreover, a model-free method is implemented for the off-policy RL algorithm, accounting for the challenge posed by incomplete information. The goal of this paper is to develop an off-policy model-free RL algorithm to obtain approximate SNE polices of the robust hierarchical game with incomplete information and input constraints. Furthermore, the convergence and effectiveness of the off-policy model-free RL algorithm are guaranteed by proving the equivalence of Bellman equation between nominal SNE policies and approximate SNE policies. Finally, a simulation is provided to verify the advantage of the developed algorithm.

Robust Decision-Making for Collaborative Manufacturing in ICPSs via Hierarchical Games

Robust Decision-Making for Collaborative Manufacturing in ICPSs via Hierarchical Games

FORMATION OF A RATIONAL STRUCTURE FOR MANAGING THE DYNAMIC CAPABILITIES OF THE ENTERPRISE'S HUMAN RESOURCES POTENTIAL

The purpose of the study is to substantiate the methodological decision-making tools for the formation of a rational management structure of the dynamic capabilities of the company's personnel potential. It has been scientifically substantiated and proven that the formation of a rational management structure for the dynamic capabilities of the company's personnel potential is based on the development of a dynamic model of their assessment. The model is presented taking into account the integration of the dynamic capabilities of the enterprise, its personnel and processes business environment’s transformation.The research uses the methods of system analysis and logical generalizations, the statistical concept of dynamic capabilities, the methods of the theory of active systems, the theory of hierarchical games, financial mathematics, and the theory of optimal control of continuous and discrete systems. Updated directions of the managing formation process of a rational structure of the company's personnel potential.The proposed model for assessing the dynamic capabilities of the company's personnel potential allows for optimizing the utility function of personnel resources. Optimizing the usefulness of human resources takes place on the basis of determining the level of their dynamic capabilities to integrate, create and change the configuration of internal and external competencies to achieve sustainable competitiveness of the enterprise.The model for assessing the dynamic capabilities of the company's personnel resources determines the competencies of personnel, business systems and strategies for its development. The use of the proposed model to determine the dynamic capabilities of the enterprise regarding the reconfiguration of available human resources allows us to focus on the relevant behavioural aspects of enterprise managers and limitations, taking into account the exchange between the internal and external environment.

On the computation of equilibria in monotone and potential stochastic hierarchical games

We consider a class of noncooperative hierarchical $${\textbf{N}}$$ -player games where the ith player solves a parametrized stochastic mathematical program with equilibrium constraints (MPEC) with the caveat that the implicit form of the ith player’s in MPEC is convex in player strategy, given rival decisions. Few, if any, general purpose schemes exist for computing equilibria, motivating the development of computational schemes in two regimes: (a) Monotone regimes. When player-specific implicit problems are convex, then the necessary and sufficient equilibrium conditions are given by a stochastic inclusion. Under a monotonicity assumption on the operator, we develop a variance-reduced stochastic proximal-point scheme that achieves deterministic rates of convergence in terms of solving proximal-point problems in monotone/strongly monotone regimes with optimal or near-optimal sample-complexity guarantees. Finally, the generated sequences are shown to converge to an equilibrium in an almost-sure sense in both monotone and strongly monotone regimes; (b) Potentiality. When the implicit form of the game admits a potential function, we develop an asynchronous relaxed inexact smoothed proximal best-response framework, requiring the efficient computation of an approximate solution of an MPEC with a strongly convex implicit objective. To this end, we consider an $$\eta $$ -smoothed counterpart of this game where each player’s problem is smoothed via randomized smoothing. In fact, a Nash equilibrium of the smoothed counterpart is an $$\eta $$ -approximate Nash equilibrium of the original game. Our proposed scheme produces a sequence and a relaxed variant that converges almost surely to an $$\eta $$ -approximate Nash equilibrium. This scheme is reliant on resolving the proximal problem, a stochastic MPEC whose implicit form has a strongly convex objective, with increasing accuracy in finite-time. The smoothing framework allows for developing a variance-reduced zeroth-order scheme for such problems that admits a fast rate of convergence. Numerical studies on a class of multi-leader multi-follower games suggest that variance-reduced proximal schemes provide significantly better accuracy with far lower run-times. The relaxed best-response scheme scales well with problem size and generally displays more stability than its unrelaxed counterpart.

On Nash–Stackelberg–Nash games under decision-dependent uncertainties: Model and equilibrium

In this paper, we discuss a class of two-stage hierarchical games with multiple leaders and followers, which is called Nash–Stackelberg–Nash (N–S–N) games. Particularly, we consider N–S–N games under decision-dependent uncertainties (DDUs). DDUs refer to the uncertainties that are affected by the strategies of decision-makers and have been rarely addressed in game equilibrium analysis. In this paper, we first formulate the N–S–N games with DDUs of complete ignorance, where the interactions between the players and DDUs are characterized by uncertainty sets that depend parametrically on the players’ strategies. Then, a rigorous definition for the equilibrium of the game is established by consolidating generalized Nash equilibrium and Pareto-Nash equilibrium. Afterward, we prove the existence of the equilibrium of N–S–N games under DDUs by applying Kakutani’s fixed-point theorem. Finally, an illustrative example is provided to show the impact of DDUs on the equilibrium of N–S–N games.

Hierarchical Games with Feedback under the Assumption of Benevolence of the Lower-Level Player

Hierarchical Games with Feedback under the Assumption of Benevolence of the Lower-Level Player

Entropic Regularization in Hierarchical Games

Entropic Regularization in Hierarchical Games

Stochastic sequential supply chain management system: with a solution approach using the systematic sampling evolutionary method

Supply chain management describes a complex sequence of strategies implemented by multiple decision makers to transform raw materials into products and deliver to the market. Mathematical formulations of such problems involve hierarchical games with some form of stochastic properties in the problem definition. Such kind of mathematical problems are generally known to be NP-hard and are challenging to solve. This paper considers a general form of supply chain management problem with various forms of model formulations and analysis. Moreover, a solution approach based on a systematic sampling evolutionary method is also proposed to solve any form of such problem definitions to obtain a Stackelberg equilibrium or Stackelberg-Nash equilibrium solution. The convergence of the solution approach is shown. The reliability of the proposed method is checked. In addition to this, the algorithm is implemented on carefully constructed stochastic supply chain management problems and solutions to these problems are presented.

Hierarchical games with feedback on the assumption of a lower-level player's benevolence

A new optimality principle is proposed that generalizes the Stackelberg equilibrium principle. Its connection with the classical definition is investigated. The technique of working with the new definition is discussed. As an example, solutions are found in two hierarchical games with feedback.

Multi-leader-follower potential games

In this paper, we discuss a particular class of Nash games, where the participants of the game (the players) are divided into two groups (leaders and followers) according to their position or influence on the other players. Moreover, we consider the case, when the leaders’ and/or the followers’ game can be described as a potential game. This is a subclass of Nash games that has been introduced by Monderer and Shapley in 1996 and has beneficial properties to reformulate the bilevel Nash game. We develope necessary and sufficient conditions for Nash equilibria and present existence and uniqueness results. Furthermore, we discuss some Examples to illustrate our results. In this paper, we discussed analytical properties for multi-leader follower potential games, that form a subclass of hierarchical Nash games. The application of these theoretical results to various fields of applications are a future research topic. Moreover, they are meant to serve as a starting point for the developement of efficient numerical solution methods for multi-leader-follower games.

CONSTRUCTION OF COMPLEX SCHEDULES FOR EXECUTION OF TASK PACKAGES AT FORMING SETS IN SPECIFIED DIRECTIVE TERMS

The current state with the solution of the problem complex planning of the execution of task packets in multistage system is characterized by the absence of universal methods of forming decisions on the composition of packets, the presence of restrictions on the dimension of the problem and the impossibility of guaranteed obtaining effective solutions for various values of its input parameters, as well the impossibility of registration the condition of the formation of sets from the results. The solution of the task of planning the execution of task packets in multistage systems with the formation of sets of results within the specified deadlines has been realized of authors in article. To solve the planning problem, the generalized function of the system was decomposed into a set of hierarchically interrelated subfunctions. The use of decomposition made it possible to use a hierarchical approach for planning the execution of task packets in multistage systems, which involves defining solutions based on the composition of packets at the top level of the hierarchy and scheduling the execution of packages at the bottom level of the hierarchy. The theory of hierarchical games is used to optimize solutions for the compositions of task packets and schedules for their execution is built, which is a system of criteria at the decision-making levels. Evaluation of the effectiveness of decisions by the composition of packets at the top level of the hierarchy is ensured by the distribution of the results of task execution by packets in accordance with the formed schedule. To evaluate the effectiveness of decisions on the composition of packets, method for ordering the identifiers of the types of sets with registration of the deadlines and a method for distributing the results of the tasks performed by packets has been formulated, which calculates the moments of completion of the formation of sets and delays with their formation relative to the specified deadlines. The studies of planning the process of the executing task packages in multistage systems have been carried out, provided that the sets are formed within specified deadlines. On their basis, conclusions, regarding the dependence of the planning efficiency from the input parameters of the problem, were formulated.

Solution Concepts in Hierarchical Games Under Bounded Rationality With Applications to Autonomous Driving

With autonomous vehicles (AV) set to integrate further into regular human traffic, there is an increasing consensus of treating AV motion planning as a multi-agent problem. However, the traditional game theoretic assumption of complete rationality is too strong for the purpose of human driving, and there is a need for understanding human driving as a bounded rational activity through a behavioral game theoretic lens. To that end, we adapt three metamodels of bounded rational behavior; two based on Quantal level-k and one based on Nash equilibria with quantal errors. We formalize the different solution concepts that can be applied in the context of hierarchical games, a framework used in multi-agent motion planning, for the purpose of creating game theoretic models of driving behavior. Furthermore, based on a contributed dataset of human driving at a busy urban intersection with a total of ~4k agents and ~44k decision points, we evaluate the behavior models on the basis of model fit to naturalistic data, as well as their predictive capacity. Our results suggest that among the behavior models evaluated, modeling driving behavior as pure strategy Nash equilibria with quantal errors at the level of maneuvers with bounds sampling of actions at the level of trajectories provides the best fit to naturalistic driving behavior, and there is a significant impact of situational factors on the performance of behavior models.

Hierarchy and the size of a firm

In this note, we apply weighted hierarchical games of cooperative game theory to the problem of optimal firm size of the firm. In particular, we analyze the influence of production technology on the size of the firm. Our note enhances previous approaches using a permission structure with equally strong relationships between predecessor and direct successors.

Methods of Solving the Theoretic Game Models for Coordinating Interests in Regulating the Fishery Industry

Dynamic theoretic game models for the coordination of private and social interests of agents in the concept of the sustainable development (SD) of the dynamic system controlled by them are investigated. Within this concept, the hierarchical control mechanisms—methods of administrative control and inducement—are formalized as solutions of hierarchical differential games with the phase constraints reflecting the requirements for the state of the controlled dynamic system providing the conditions for SD. Administrative control involves the impact of the leading player (the subject of managing SD) on the set of admissible controls for the followers (the subject of the impact on the controlled dynamic system); and the incentive, on his payoff function. Mechanisms of administrative and economic management are formalized as computer imitation scenarios. The dynamic models considered in the article are the development of the models for the coordination of private and social interests proposed by Yu.B. Germeyer and I.A. Vatel. Numerical calculations are carried out and a comparative analysis of the effectiveness of these control mechanisms for the fisheries’ model is fulfilled.

Finding the parameters of a franchising agreement in the production sphere by methods of theory of hierarchical games

Finding the parameters of a franchising agreement in the production sphere by methods of theory of hierarchical games

Risk Management in Hierarchical Games with Random Factors

A game-theoretic model of the Principal-agent type is considered, in which the result of an agent’s activity depends not only on his/her choice but also on some random factor. The Principal is assumed to choose the total probability of all negative events that he/she will exclude from consideration; in the other respects, he/she is cautious. The structure of the Principal’s optimal strategies is found. Two models differing by the Principal’s awareness of the partner’s actions are studied.

Modeling the mechanisms of the functioning of the logistics distribution system

Currently, the use of the concept of logistics in inventory management is considered by enterprises as one of the reserves to achieve a competitive advantage. In theory, inventory control deficit arises due to insufficient supply, unreliable source of supply, and because of inaccurate information provided by customers to suppliers. The solution to this problem consists in finding a compromise between the cost of creation of material stockpiles and storage costs, and the creation of the mechanisms of management of active systems, elements of which are suppliers and customers. The implementation of the control mechanisms of the two-level system supplier-consumer, we have considered the example of the game-theoretic model. For the consistency of the idea considered in the article, it should be emphasized that in the supply chain-each customer-consumer is also a supplier in an orderly sequence of suppliers and consumers, as long as the product does not reach the end user. In this article, the simulation game tool in the context of hierarchical games is used to simulate the simplest scheme of the logistics distribution system. In the process of realization of finished products the model of distribution logistics between the supplier and the consumer enterprises is considered. The case of deficit is simulated when the total volume of customers ‘ orders for early delivery exceeds the production capacity of the supplier. The main purpose of the article is to give a clear idea of the functioning mechanisms used in the management of the distribution system of logistics, as well as to provide distribution logistics specialists with the possibility of effective solutions in the process of distribution of finished products in a deficit. Key words: distribution logistics, functioning mechanisms, game-theoretic model, supplier, consumer.

From the Calculation of Combat Effectiveness Indices to the Operations Research Theory and Non-Zero-Sum Games: Scientific Legacy of Professor Yurii Borisovich Germeier

The main scientific results of Professor Yurii B. Germeier are presented, including the results of calculations of combat effectiveness indices of aviation systems and theories of non-zero-sum games, hierarchical games, reliability, and operations research. Milestones of his scientific, teaching, and life paths are listed. Directions for the further development of the Moscow school of operations research created by him are given.

Numerical simulation of fractured-vuggy porous media based on gamblets

Numerical simulation of a fractured-vuggy porous medium is a challenging problem. One reason is the coexistence of matrix, fractures and vugs on multiple scales that need to be coupled, and the other reason is that the high-resolution fractured-vuggy model may contain up to several millions of gridcells in applications, which brings severe computational challenges into the numerical methods. Therefore, the requirement for accurate and efficient technique is widely increasing. Fractured-vuggy porous medium is generally represented by triple-continuum model in which the matrix system, fracture system and vug system each are treated as a parallel continuous system. Although triple-continuum model is widely used because of its easy-implementation and high efficiency, it fails to capture the detailed flow patterns of reservoir with disconnected long fractures. Discrete fracture-vug network (DFVN) model can precisely model the fluid flow in fractures and vugs. However, the simulation of this model is deemed intractable even with the advent of supercomputers because of the large amount of calculation. In view of the fact that the multigrid method is now well known as one of the fastest method of solving elliptic problems, in this paper we introduce a nearly linear complexity multiresolution decomposition method for fluid flow in a fractured-vuggy reservoir. The detailed flow patterns are described by combing the advantages of continuum model and discrete model. That is, the homogenization theory is used to construct an equivalent permeability in each coarse grid block in which the vugs and small-scale fractures are represented by discrete fracture-vug network model. We decompose the solution space into several subspaces and then we compute the corresponding solutions of heterogeneous discrete fracture network model in each subspace. Gamblets are constructed and they are elementary solutions of hierarchical information games associated with the process of computing with partial information and limited resources. These gamblets have a natural Bayesian interpretation under the mixed strategy emerging from the game theoretic formulation. This method could realize its fast simulation by decomposing the solution space into a direct sum of linear subspaces that are orthogonal to each other. Finally, the pressure difference distribution of fractured-vuggy porous medium is obtained by combing the DFVN solutions of all subspaces. Numerical results are presented to demonstrate the accuracy and efficiency of the proposed multiresolution decomposition method. The results show that this method is a promising method of numerically simulating the fractured-vuggy porous medium.

Stackelberg–Nash equilibrium, social welfare and optimal structure in hierarchical continuous Public Goods game

This paper studies Stackelberg–Nash equilibrium, social welfare and optimal structure of hierarchical Public Goods game in which the players have continuous action spaces. There can be multiple leaders, followers or levels and the leaders are assumed to take linear strategies with respect to the followers’ actions. Explicit forms of Stackelberg–Nash equilibriums of the game under different structures are solved from which social welfare index emerges with good properties. Based on these the optimal 2-level structure for any given population is proved and constructed in steps less than the population size. Moreover, equilibrium under the simplest three-level structure is computed, which implies a bound of hierarchy for theoretical analysis. Results in this paper are rigorous and show interesting phenomena in hierarchical games, which may help decision-making in practice especially in hierarchical systems.