Uncountable State Research Articles

A Markovian Decision Process with hidden states and hidden costs

A Markovian Decision Process (MDP) is considered in which it is not permitted to observe the state at any observation point as well as the associated cost.It is shown that for a particular class of a MDP with uncountable state space and finite action space the Howard Policy Improvement Routine (HPIR) cannot be used for finding an optimal policy. Some immediate results out of this model are presented.

On Dynamic Programming with Unbounded Rewards

Using the technique employed by the author in an earlier paper, the existence of an optimal stationary policy that can be obtained from the usual functional equation is again established in the presence of a bound (not necessarily polynomial) on the one-period reward of a semi-Markov decision process. This is done for both the discounted and the average cost case. In addition to allowing an uncountable state space, the law of motion of the system is rather general in that we permit any state to be reached in a single transition. There is, however, a bound on a weighted moment of the next state reached. Finally, we indicate the applicability of these results.