A Review of Probability and Stochastic Processes

Abstract

The seminal studies about probability go back to the 17th century with Blaise Pascal (1623–1662), Pierre de Fermat (1601–1665), Jacques Bernoulli (1654–1705) and Abraham de Moivre (1667–1754). Today, probability and random processes (or stochastic processes) are the basis for the study of many areas, including Electrical Engineering and, particularly, communications theory. That is the reason for including disciplines on the subject in the regular curriculum of such courses. Moreover, in communication systems, most of the signals are of random nature, demanding the use of probabilistic tools for their characterization and for supporting system design and assessment. In this chapter, mostly inspired in the book by Alberto Leon-Garcia [9], we review basic concepts about probability and random processes. The chapter is intended to allow for the reader to have prompt access to these concepts when studying specific subjects on digital transmission. We start by reviewing some concepts and properties of the set theory, aiming at using them to define probability and to help with the solutions of problems. In the sequel, basic combinatorial theory is presented as an additional tool for probability calculations based on counting methods. An important rule of probabilities, known as the Bayes rule is then presented. Random variables and averages of random variables are also discussed, followed by the central limit theorem. Confidence interval analysis is then studied. Its use is related to the analysis of the precision associated to averages obtained from sample data. Random number generation is also discussed as a means for computer simulating random phenomena that frequently appear in digital transmission systems. Random processes are defined and the analysis of averages and filtering of stochastic processes are presented. The chapter ends with the investigation of the thermal noise, one of the main random processes encountered in communication systems.