The strong law of large numbers for sequential decisions under uncertainty

Abstract

Combines optimization and ergodic theory to characterize the optimum long-run average performance that can be asymptotically attained by nonanticipating sequential decisions. Let {X/sub t/} be a stationary ergodic process, and suppose an action b/sub t/ must be selected in a space /spl Bscr/ with knowledge of the t-past (X/sub 0/, /spl middot//spl middot//spl middot/, X/sub t-1/) at the beginning of every period t/spl ges/0. Action b/sub t/ will incur a loss l(b/sub t/, X/sub t/) at the end of period t when the random variable X/sub t/ is revealed. The author proves under mild integrability conditions that the optimum strategy is to select actions that minimize the conditional expected loss given the currently available information at each step. The minimum long-run average loss per decision can be approached arbitrarily closely by strategies that are finite-order Markov, and under certain continuity conditions, it is equal to the minimum expected loss given the infinite past. If the loss l(b, x) is bounded and continuous and if the space /spl Bscr/ is compact, then the minimum can be asymptotically attained, even if the distribution of the process {X/sub t/} is unknown a priori and must be learned from experience.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>