Markov Processes in Continuous Time

Abstract

In this chapter we review some important aspects of Markov processes in continuous time. Their study involves much more analytic work than in the discrete-time setting. However, there is also a lot of common structure. The basic definition of a Markov process was already given in Chap. 4 . Clearly, one-step transition kernels no longer make sense and we need to look for the appropriate analogue of the generator of the process. Section 5.1 takes a brief look at Markov jump processes. Section 5.2 lists a few basic properties of Brownian motion. Section 5.3 gives the definition of general Markov processes via generators and semigroups, and focusses on a special class called Feller-Dynkin processes, emphasising the central rôle of the strong Markov property. Section 5.4 introduces and studies the so-called martingale problem, which is a powerful way to construct general Markov processes. Section 5.5 gives a brief summary on Itō calculus needed for Sect. 5.6, which introduces and studies stochastic differential equations. Section 5.7 looks at stochastic partial differential equations.