Jogos markovianos alternados sob incerteza

Abstract

FRANCO, F. O. Alternating Markov Games under uncertainty. 2012. 85 f. Dissertacao (Mestrado) Instituto de Matematica e Estatistica, Universidade de Sao Paulo, Sao Paulo, 2012. An Alternating Markov Game (AMG) is an extension of a Markov Decision Process (MDP) for multiagent environments. This model is used on sequencial decision making for n agents when we know the state transition probabilities of actions being taken by each agent. In this work we are interested in AMGs with imprecise probabilities on state transition function, for example, when they are given by probabilities intervals. We present a new AMG model, which we call Alternating Markov Game with Imprecise Probabilities (AMG-IP) that allows imprecision on state transition probabilities given by parameters subject to linear constraints that extend previous works which the imprecision is given by probabilities intervals (AMG-INTERVAL). We say that the imprecision represents the Nature choices. The imprecision of these models implies the game value is given by interval function. There are several ways to calculate the solution of the game, that depend on the behavior of the Nature and the preference criteria of the players on the choices of Nature. Therefore, in this work we discuss various solutions to AMG-IP and AMG-INTERVAL. Also from our study on the relationship among the MDPs and AMGs, we propose a new model called Alternating Markov Game with Set-valued Transition (AMG-ST), that can be used to model the uncertainty of an MDP-ST (Markovian Decision Process with Set-valued Transition) as a result of the match between the agent and the Nature, i.e., a game where the Nature is seen as one of the players.