Discrete type shock semi-markov decision processes with borel state space

Abstract

This paper investigates discrete type shock semi-Markov decision processes (SMDP for short) with Borel state and action space. The discrete type shock SMDP describes a system which behaves like a discrete type SMDP, except that the system is subject to random shocks from its environment. Following each shock, an instantaneous state transition occurs and the parameters of the SMDP are changed. After presenting the model, we transform the discrete type shock SMDP into an equivalent discrete time Markov decision process under the condition that one of the assumptions P, N, D, holds. So the most results from discrete time Markov decision processes can be generalized directly to hold for the discrete type shock SMDP.