Optimal halting policies in Markov population decision chains with constant risk posture

Abstract

This paper concerns infinite-horizon Markov population decision chains with finite state-action space, where one exerts control on a population of individuals in different states by assigning an action to each individual in the system in each period. In every transition, each individual earns a random reward and generates a random progeny vector. The objective of the decision maker is to maximize expected (infinite-horizon) system utility under the following assumptions: (i) The utility function exhibits constant risk posture, (ii) the (random) progeny vectors of distinct individuals are independent, and (iii) the progeny vectors of individuals in a state who take the same action are identically distributed. The paper deals with the problem of finding an optimal stationary halting policy and shows that this problem can be solved efficiently using successive approximations with the original state-action space without enlarging it to include information about the population in each state or any other aspect of the system history in a state. The proposed algorithm terminates after a finite number of iterations with an optimal stationary halting policy or with proof of nonexistence.