Optimal switching problem for countable Markov chains: average reward criterion

Abstract

Optimal switching we consider is the following generalization of optimal stopping: (i) there are a reward function and a cost function on the state space of a Markov chain; (ii) a controller selects stopping times sequentially; (iii) at those times the controller receives rewards and pays costs in an alternating order. In this paper we treat the case of a positive recurrent countable Markov chain and the average per unit time criterion. We find an optimal strategy and the maximal average gain in terms of the solution of a variational problem with two obstacles, known also in connection with Dynkin games.