Stochastic Processes. An Introduction

Abstract

Some Background on Probability Introduction Probability Conditional probability and independence Discrete random variables Continuous random variables Mean and variance Some standard discrete probability distributions Some standard continuous probability distributions Generating functions Conditional expectation Some Gambling Problems Gambler's ruin Probability of ruin Some numerical simulations Duration of the game Some variations of gambler's ruin Random Walks Introduction Unrestricted random walks The probability distribution after n steps First returns of the symmetric random walk Markov Chains States and transitions Transition probabilities General two-state Markov chains Powers of the transition matrix for the m-state chain Gambler's ruin as a Markov chain Classification of states Classification of chains Poisson Processes Introduction The Poisson process Partition theorem approach Iterative method The generating function Variance in terms of the probability generating function Arrival times Summary of the Poisson process Birth and Death Processes Introduction The birth process Birth process: Generating function equation The death process The combined birth and death process General population processes Queues Introduction The single-server queue The stationary process Queues with multiple servers Queues with fixed service times Classification of queues A general approach to the M(lambda)/G/1 queue Reliability and Renewal Introduction The reliability function Exponential distribution and reliability Mean time to failure Reliability of series and parallel systems Renewal processes Expected number of renewals Branching and Other Random Processes Introduction Generational growth Mean and variance Probability of extinction Branching processes and martingales Stopping rules The simple epidemic An iterative solution scheme for the simple epidemic Computer Simulations and Projects Answers and Comments on End-of-Chapter Problems Appendix References and Further Reading Index Problems appear at the end of each chapter.