A reinforcement learning approach to stochastic business games

Abstract

The Internet revolution has resulted in increased competition among providers of goods and services to lure customers by tearing down the barriers of time and distance. For example, a home buyer shopping for a mortgage loan through the Internet is now a potential customer for a large number of lending institutions throughout the world. The lenders (players, in generic game theory nomenclature) seeking to capture this customer are involved in a nonzero-sum stochastic game. Stochastic games are among the least studied and understood of the management science problems, and no computationally tractable solution technique is available for multi-player nonzero-sum stochastic games. We now develop a computer-simulation-based machine learning algorithm that can be used to solve nonzero-sum stochastic game problems that are modeled as competitive Markov decision processes. The methodology based on this algorithm is implemented on a supply chain inventory planning problem with a limited state space. The equilibrium reward obtained from the stochastic game problem is compared with a logical upper bound obtained from the corresponding Markov decision problem in which a single decision maker (player) is substituted for all the competing players in the game. Several numerical versions of the problem are studied to assess the performance of the methodology. The results obtained from our methodology for the inventory planning problems are within 0.8% of the upper bound.