Finitely additive Markov chains

Abstract

In this paper we develop the theory of Markov chains with stationary transition probabilities, where the transition probabilities and the initial distribu- tion are assumed only to be finitely additive. We prove a strong law of large numbers for recurrent chains. The problem of existence and uniqueness of finitely additive stationary initial distributions is studied and the ergodicity of recurrent chains under a stationary initial distribution is proved. 1. Introduction. In this paper we start the study of Markov chains with stationary finitely additive transition probabilities and an initial distribution that need only be a finitely additive probability. The first serious attempt to study stochastic processes in a finitely additive setting was made by Dubins and Savage (8) who studied gambling with finitely additive probabilities. Given any strategy, they obtained a finitely additive probability on all clopen subsets of the infinite product in a natural way. This enabled them to state and prove weak versions of various convergence theorems. Dubins (7) showed how this measure can be extended to the open sets. Purves and Sudderth (13) then showed that every Borel set can be squeezed between an open set and a closed set. Therefore a strategy induces a finitely additive probability on all Borel sets of the product space, unique subject to some regularity conditions. Purves and Sudderth (13) then formulated and proved a special case of the strong law of large numbers for the i.i.d. situation and a version of the martingale convergence theorem. This was followed by a systematic study of almost sure convergence for independent strategies and martingales by Chen ((2) and (3)). In the same spirit we take up here the study of Markov chains. In §2, after setting up the basic framework, we state some of the results from earlier work in the subject, which we shall need in the sequel. In §3, we obtain the strong Markov property which is fundamental for our theory. The classification of states is studied in §4 and the notions of recurrence and transience are examined in detail. §5 contains the very important blocks theorem which enables us to use in our theory known results on almost sure convergence in an i.i.d. setting. In §6 we study positive recurrence. §7 contains the strong law of large numbers for Markov chains and a few other related results on almost sure convergence. §8 shows the existence of finitely additive stationary initial distributions for any Markov chain. It needs to be remarked that in the countably additive case, a countably additive