Nondiscounted Continuous Time Markovian Decision Process with Countable State Space

Abstract

Previous article Nondiscounted Continuous Time Markovian Decision Process with Countable State SpacePrasadarao KakumanuPrasadarao Kakumanuhttps://doi.org/10.1137/0310016PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] Kai Lai Chung, Markov chains with stationary transition probabilities, Second edition. Die Grundlehren der mathematischen Wissenschaften, Band 104, Springer-Verlag New York, Inc., New York, 1967xi+301 MR0217872 0146.38401 Google Scholar[2] Cyrus Derman, Denumerable state Markovian decision processes—average cost criterion, Ann. Math. Statist., 37 (1966), 1545–1553 MR0207138 0144.43102 CrossrefISIGoogle Scholar[3] Ronald A. Howard, Dynamic programming and Markov processes, The Technology Press of M.I.T., Cambridge, Mass., 1960viii+136 MR0118514 0091.16001 Google Scholar[4] P. Kakumanu, Continuous time Markov decision models with applications to optimization problems, Tech. Rep., 63, Cornell University, Ithaca, N.Y., 1969 Google Scholar[5] Prasadarao Kakumanu, Continuously discounted Markov decision model with countable state and action space, Ann. Math. Statist., 42 (1971), 919–926 MR0282651 0234.93027 CrossrefISIGoogle Scholar[6] Bruce L. Miller, Finite state continuous time Markov decision processes with an infinite planning horizon, J. Math. Anal. Appl., 22 (1968), 552–569 10.1016/0022-247X(68)90194-7 MR0228251 0157.50301 CrossrefISIGoogle Scholar[7] Sheldon M. Ross, Non-discounted denumerable Markovian decision models, Ann. Math. Statist., 39 (1968), 412–423 MR0226933 0157.50504 CrossrefISIGoogle Scholar[8] Howard M. Taylor, III, Markovian sequential replacement processes, Ann. Math. Statist., 36 (1965), 1677–1694 MR0189839 0139.37802 CrossrefISIGoogle Scholar[9] D. V. Widder, The Laplace Transform, Princeton University Press, Princeton, N.J., 1946 Google Scholar Previous article FiguresRelatedReferencesCited byDetails Ergodic BSDEs Driven by Markov ChainsSamuel N. Cohen and Ying Hu28 October 2013 | SIAM Journal on Control and Optimization, Vol. 51, No. 5AbstractPDF (350 KB)Linear Programming and Constrained Average Optimality for General Continuous-Time Markov Decision Processes in History-Dependent PoliciesXianping Guo, Yonghui Huang, and Xinyuan Song3 January 2012 | SIAM Journal on Control and Optimization, Vol. 50, No. 1AbstractPDF (300 KB)Optimal Control of Ergodic Continuous-Time Markov Chains with Average Sample-Path RewardsXianping Guo and Xi-Ren Cao26 July 2006 | SIAM Journal on Control and Optimization, Vol. 44, No. 1AbstractPDF (207 KB)On Homogeneous Markov Models with Continuous Time and Finite or Countable State SpaceA. A. Yushkevic and E. A. Fainberg17 July 2006 | Theory of Probability & Its Applications, Vol. 24, No. 1AbstractPDF (653 KB)Controlled Markov Models with Countable State Space and Continuous TimeA. A. Yushkevich17 July 2006 | Theory of Probability & Its Applications, Vol. 22, No. 2AbstractPDF (1900 KB) Volume 10, Issue 1| 1972SIAM Journal on Control History Submitted:26 May 1970Published online:18 July 2006 InformationCopyright © 1972 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/0310016Article page range:pp. 210-220ISSN (print):0036-1402Publisher:Society for Industrial and Applied Mathematics