Chapter Four - STOCHASTIC PROCESSES

Abstract

A family of random variables {X(t) t є T} is called a stochastic process. For each t є T, where T is the index set of the process, X(t) is a random variable. An element t of T is usually referred to as a time parameter and t is often referred to as time. The state space of the process is the set of all possible values that the random variables X(t) can assume. Each of these values is called a state of the process. Stochastic processes are classified in a number of ways, such as by the index set and by the state space. This chapter reviews stochastic processes with emphasis on those that have immediate application to computer science problems. It considers the Poisson process, which is fundamental to computer science and to all the areas of applied probability. The birth-and-death process is a key element in queueing theory. The chapter discusses Markov chains. These are used in queueing theory and in many other computer science applications. A Markov process is called a Markov chain if its state space is discrete. The chapter presents a classification of Markov processes.