Control Limit Policy for Partially Observable Markov Decision Process Based on Stochastic Increasing Ordering

Abstract

We investigated an optimal maintenance problem for a condition monitoring system that is formulated as a partially observable Markov decision process. In almost all previous research, the optimal control limit policy with respect to totally positive of order 2 (TP2) ordering of the state probability vectors was derived using the property of posterior probability. We call this approach “Approach PP.” Attempts to achieve an optimal control limit policy based on stochastic increasing (SI) ordering, which is weaker than TP2 ordering, ran into a difficulty. That is, the SI property of the posterior probability vectors could not be obtained, and this property is needed to derive results expected using Approach PP. We investigated the conditions for the SI property of posterior probability vectors and derived a necessary and sufficient condition that cannot be further relaxed when we try to derive an optimal control limit policy based on SI ordering using Approach PP. This condition is that the conditional probability matrix of the monitor observation given the state of the system is given as Type 1), the probabilities of the same monitoring output are the same whatever the true state, or Type 2), an identity matrix. Type 1) means that the monitoring output is independent of the true state.