On a Notion of Resilience for Markov Decision Processes with Reachability Objectives

Abstract

We propose and study a notion of resilience for Markov decision processes (MDPs) with the almost-sure reachability objective to action losses. Given an MDP with an initial state and a set of target states, we define the resilience degree of the MDP as the minimum number of actions that must be removed so that the target states cannot be reached almost surely from the initial state. This notion measures the level of tolerance of an MDP to action losses under the reachability objective. We first preprocess the MDP to remove irrelevant states and actions and construct a reduced transition diagram. Then, we show that computing the resilience degree is an NP-hard problem and provide an exact solution based on the mixed-integer linear programming.