Optimal maintenance policies for three-states POMDP with quality measurement errors

Abstract

Partially Observed Markov Decision Process (POMDP) has been used to model decision making under uncertainty in several areas. A few areas of application include: manufacturing, healthcare, business and military applications. In the POMDP context, systems are considered as multi-state systems with hidden states. The common thing among all POMDP models is the existence of measurements utilized to infer about the actual hidden state of the system on hand. However, measurements, in general, are not error free. The impact of measurement errors on the POMDP optimal decision polices is formulated and studied for a three-state deteriorating machine with two quality outcomes and possible quality measurement errors. The decision making problem is modeled as a Three-Layers Hidden Markov Decision Process (TLHMDP). The objective function of the POMDP problem is shown to be a piecewise linear convex one. The impact of measurement errors in the POMDP context is demonstrated by numerical example.