Conditional Decision Processes with Recursive Function

Abstract

We consider stochastic optimization of not necessarily additive but recursive functions over multistage decision processes. Without assuming any monotonicity, we optimize a regular process by a direct dynamic programming approach. On the regular decision process, we propose two related conditional decision processes: an a posteriori conditional decision process and an a priori. When the Markov transition law degenerates into a deterministic dynamics, the two conditional processes reduce to the same deterministic decision process. The conditional processes with monotonicity are optimized by the usual backward dynamic programming. We show that under additional convexity the regular process dominates the a priori in maximum value function and the a priori does the a posteriori. We show that the a posteriori process illustrates Kreps and Porteus's dynamic choice problem. The numerical example also verifies the dominance relation in three optimal value functions.