Explicit Solution of the Average-Cost Optimality Equation for a Pest-Control Problem

Epaminondas G Kyriakidis

doi:10.1155/2011/617812

Epaminondas G Kyriakidis

Open Access

https://doi.org/10.1155/2011/617812

Copy DOI

Abstract

We introduce a Markov decision process in continuous time for the optimal control of a simple symmetrical immigration-emigration process by the introduction of total catastrophes. It is proved that a particular control-limit policy is average cost optimal within the class of all stationary policies by verifying that the relative values of this policy are the solution of the corresponding optimality equation.

Highlights

The term “Markov decision process” was introduced by Bellman 1 for the description of a stochastic process controlled by a sequence of actions
We introduce a Markov decision process in continuous time for the optimal control of a simple symmetrical immigration-emigration process by the introduction of total catastrophes
We present a Markov decision process in continuous time for the optimal control of a symmetrical immigration-emigration process through total catastrophes that annihilate the population size

Summary

Introduction

The term “Markov decision process” was introduced by Bellman 1 for the description of a stochastic process controlled by a sequence of actions. During the last fifty years, the Markov decision process has been the subject of remarkable research activity. It is called discrete-time Markov decision process or semi-Markov decision process if the times between consecutive decision epochs are equal or random, respectively. The Markov decision process in continuous time is a special semi-Markov decision process if the times between consecutive decision epochs are exponentially distributed. Collections of results with some emphasis to the theoretical aspects of Markov decision processes are given in the books of Derman 2 , Ross 3 , Whittle 4, 5 , Puterman 6 , and Sennott 7. The computational aspects of Markov decision processes can be found in detail in the books of Puterman 6 and Tijms 8

Objectives

Results

Conclusion