Continuous Dynamic Game Research Articles

The strategic interaction between business mode and store brand introduction in a platform-based supply chain

The phenomenon of store brands introduced by large-scale platforms is becoming common, and manufacturers should carefully choose the business mode of selling national brand (NB) products via platforms. Considering big-data marketing, we examine the sales mode selection in a platform-based supply chain based on the strategic interaction between the business mode decision and store brand decision. By continuous dynamic game theory, the strategies and performance under different modes are solved. We find that given the store brand (SB) decision, the business mode adopted by the manufacturer depends only on the commission rate, which is high for reselling and low for agency selling. Given the business mode, under reselling, whether the platform introduces SB depends only on the brand preference, i.e., SB is introduced when the preference for NB is low. While under agency, he introduces SB in the general or relatively passive situation, depending both on the commission rate and the brand preference. In addition, four equilibrium sales modes are obtained based on the strategic interaction. In order to achieve a win-win-win situation in profitability among the manufacturer, the platform and the platform-based supply chain, the manufacturer should adopt agency selling and the platform forgoes introducing SB when both the commission rates and NB preference are low.

Online and offline learning of player objectives from partial observations in dynamic games

Robots deployed to the real world must be able to interact with other agents in their environment. Dynamic game theory provides a powerful mathematical framework for modeling scenarios in which agents have individual objectives and interactions evolve over time. However, a key limitation of such techniques is that they require a priori knowledge of all players’ objectives. In this work, we address this issue by proposing a novel method for learning players’ objectives in continuous dynamic games from noise-corrupted, partial state observations. Our approach learns objectives by coupling the estimation of unknown cost parameters of each player with inference of unobserved states and inputs through Nash equilibrium constraints. By coupling past state estimates with future state predictions, our approach is amenable to simultaneous online learning and prediction in receding horizon fashion. We demonstrate our method in several simulated traffic scenarios in which we recover players’ preferences, for, e.g. desired travel speed and collision-avoidance behavior. Results show that our method reliably estimates game-theoretic models from noise-corrupted data that closely matches ground-truth objectives, consistently outperforming state-of-the-art approaches.

Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere

It is well known that time-dependent Hamilton–Jacobi–Isaacs partial differential equations (HJ PDEs) play an important role in analyzing continuous dynamic games and control theory problems. An important tool for such problems when they involve geometric motion is the level set method (Osher and Sethian in J Comput Phys 79(1):12–49, 1988). This was first used for reachability problems in Mitchell et al. (IEEE Trans Autom Control 50(171):947–957, 2005) and Mitchell and Tomlin (J Sci Comput 19(1–3):323–346, 2003). The cost of these algorithms and, in fact, all PDE numerical approximations is exponential in the space dimension and time. In Darbon (SIAM J Imaging Sci 8(4):2268–2293, 2015), some connections between HJ PDE and convex optimization in many dimensions are presented. In this work, we propose and test methods for solving a large class of the HJ PDE relevant to optimal control problems without the use of grids or numerical approximations. Rather we use the classical Hopf formulas for solving initial value problems for HJ PDE (Hopf in J Math Mech 14:951–973, 1965). We have noticed that if the Hamiltonian is convex and positively homogeneous of degree one (which the latter is for all geometrically based level set motion and control and differential game problems) that very fast methods exist to solve the resulting optimization problem. This is very much related to fast methods for solving problems in compressive sensing, based on $$\ell _1$$ optimization (Goldstein and Osher in SIAM J Imaging Sci 2(2):323–343, 2009; Yin et al. in SIAM J Imaging Sci 1(1):143–168, 2008). We seem to obtain methods which are polynomial in the dimension. Our algorithm is very fast, requires very low memory and is totally parallelizable. We can evaluate the solution and its gradient in very high dimensions at $$10^{-4}$$ – $$10^{-8}$$ s per evaluation on a laptop. We carefully explain how to compute numerically the optimal control from the numerical solution of the associated initial valued HJ PDE for a class of optimal control problems. We show that our algorithms compute all the quantities we need to obtain easily the controller. In addition, as a step often needed in this procedure, we have developed a new and equally fast way to find, in very high dimensions, the closest point y lying in the union of a finite number of compact convex sets $$\Omega $$ to any point x exterior to the $$\Omega $$ . We can also compute the distance to these sets much faster than Dijkstra type “fast methods,” e.g., Dijkstra (Numer Math 1:269–271, 1959). The term “curse of dimensionality” was coined by Bellman (Adaptive control processes, a guided tour. Princeton University Press, Princeton, 1961; Dynamic programming. Princeton University Press, Princeton, 1957), when considering problems in dynamic optimization.

Developing a Non-Discrete Dynamic Game Model and Corresponding Monthly Collocation Solution Considering Variability in Reservoir Inflow

So far a huge number of dynamic game models, both discrete and continuous, have been developed for reservoir operation. Most of them seek to overcome potential conflicts while allocating limited water among various users in reservoir systems. A number of them have successfully provided efficient approaches to tackle the problem of optimal water allocation. There are, however, still weaknesses in determining optimal policies. By taking account of the random nature of the inflow while producing monthly operating policies for a reservoir in a dynamic non-discrete framework, many of these deficiencies can be eliminated. In this study, considering the randomness in reservoir inflow, a continuous dynamic game model for water allocation in a reservoir system was developed. A corresponding monthly-basis solution based on collocation was then structured. This collocation method does not rely on first and second degree approximations. Instead, in order to evaluate the uncertainty triggered by the random variable, a discrete approximant was applied to quantify the random variable in the state transition function. A case study was carried out at the Zayandeh-Rud river basin in central Iran to identify the efficiency of the proposed method and evaluate the effect of uncertainty on decision variables. The results of the presented model (i.e., monthly-basis operating rules and value functions) proved to be more practical and reliable than similar continuous dynamic game models working on an annual basis. Moreover, the results show that bringing the stochasticity associated with inflow into the equation has an impact on the value functions and operating polices. Indeed, applying the mean of the random variable of inflow (deterministic form) reduce consistancy in monthly reliability indices throughout the year.

Hamilton–Jacobi Formulation for Reach–Avoid Differential Games

A new framework for formulating reachability problems with competing inputs, nonlinear dynamics, and state constraints as optimal control problems is developed. Such reach-avoid problems arise in, among others, the study of safety problems in hybrid systems. Earlier approaches to reach-avoid computations are either restricted to linear systems, or face numerical difficulties due to possible discontinuities in the Hamiltonian of the optimal control problem. The main advantage of the approach proposed in this paper is that it can be applied to a general class of target-hitting continuous dynamic games with nonlinear dynamics, and has very good properties in terms of its numerical solution, since the value function and the Hamiltonian of the system are both continuous. The performance of the proposed method is demonstrated by applying it to a case study, which involves the target-hitting problem of an underactuated underwater vehicle in the presence of obstacles.

Two solution methods for dynamic game in reservoir operation

During the last decade, a number of models have been developed to consider the conflict in dynamic reservoir operation. Most of these models are discrete dynamic models which are developed based on game theory. In this study, a continuous model of dynamic game and its corresponding solutions are developed for reservoir operation. Two solution methods are used to solve the model of continuous dynamic game, namely the Ricatti equations and collocation methods. The Ricatti equations method is a closed form solution, requiring less computational efforts compared with discrete models. The collocation solution method applies Newton's method or a quasi-Newton method to find the problem solution. These approaches are able to generate operating policies for dynamic reservoir operation. The Zayandeh-Rud river basin in central Iran is used as a case study and the results are compared with alternative water allocation models. The results show that the proposed solution methods are quite capable of providing appropriate reservoir operating policies, while requiring rather short computational times due to continuous formulation of state and decision variables. Reliability indices are used to compare the overall performance of the proposed models. Based on the results from this study, the collocation method leads to improved values of the reliability indices for total reservoir system and utility satisfaction of water users, compared to the Ricatti equations method. This is attributed to the flexible structure of the collocation model. When compared to alternative water allocation models, lower values of reliability indices are achieved by the collocation method.

A time-dependent Hamilton-Jacobi formulation of reachable sets for continuous dynamic games

We describe and implement an algorithm for computing the set of reachable states of a continuous dynamic game. The algorithm is based on a proof that the reachable set is the zero sublevel set of the viscosity solution of a particular time-dependent Hamilton-Jacobi-Isaacs partial differential equation. While alternative techniques for computing the reachable set have been proposed, the differential game formulation allows treatment of nonlinear systems with inputs and uncertain parameters. Because the time-dependent equation's solution is continuous and defined throughout the state space, methods from the level set literature can be used to generate more accurate approximations than are possible for formulations with potentially discontinuous solutions. A numerical implementation of our formulation is described and has been released on the web. Its correctness is verified through a two vehicle, three dimensional collision avoidance example for which an analytic solution is available.

Overapproximating Reachable Sets by Hamilton-Jacobi Projections

In earlier work, we showed that the set of states which can reach a target set of a continuous dynamic game is the zero sublevel set of the viscosity solution of a time dependent Hamilton-Jacobi-Is...

Closed-loop strategies in continuous dynamic game with multi-level of hierarchy

The derivation is given of the closed-loop Stackelberg strategy for a class of continuous three-player non-zero-sum differential games using the idea of the team optimal Stackelberg strategy. The game systems are described by linear state dynamics and quadratic objective functions. First, a definition of the three-player hierarchical equilibrium is given, then a general theorem which was developed by Basar (1981) is examined and applied to the game under consideration to deduce some sufficient conditions for the solution of the game to exist. A simple example is given.

Algorithm for overcoming the curse of dimensionality for certain non-convex Hamilton–Jacobi equations, projections and differential games

Abstract : In this paper, we develop a method for solving a large class of non-convex Hamilton-Jacobi partial differential equations (HJ PDE). The method yields decoupled subproblems, which can be solved in an embarrassingly parallel fashion. The complexity of the resulting algorithm is polynomial in the problem dimension; hence, it overcomes the curse of dimensionality [1, 2]. We extend previous work in[6] and apply the Hopf formula to solve HJ PDE involving non-convex Hamiltonians. We propose an ADMM approach for finding the minimizer associated with the Hopf formula. Some explicit formulae of proximal maps, as well as newly-defined stretch operators, are used in the numerical solutions of ADMM subproblems. Our approach is expected to have wide applications in continuous dynamic games, control theory problems, and elsewhere.