Approximate regret based elicitation in Markov decision process

Abstract

Consider a decision support system (DSS) designed to find optimal strategies in stochastic environments, on behalf of a user. To perform this computation, the DSS will need a precise model of the environment. Of course, when the environment can be modeled as a Markov decision process (MDP) with numerical rewards (or numerical penalties), the DSS can compute the optimal strategy in polynomial time. But in many real-world cases, rewards are unknown. To compensate this missing information, the DSS may query the user for its preferences among some alternative policies. Based on the user's answers, the DSS can step-by-step compute the user's preferred policy. In this work, we describe a computational method based on minimax regret to find optimal policy when rewards are unknown. Then we present types of queries on feasible set of rewards by using preference elicitation approaches. When user answers these queries based on her preferences, we will have more information about rewards which will result in more desirable policies.