Search for the maximum of a random walk

Abstract

This paper examines the efficiency of various strategies for searching in an unknown environment. The model is that of the simple random walk, which can be taken as a representation of a function with a bounded derivative that is difficult to compute. Let X1, X2+,. be independent and identically distributed with Prob(Xj = 1) = Prob(Xj = -1) = 1/2, and let Sk=X1+X2 + + Xk. Thus Sk is the position of a symmetric random walk on the line after k steps. The number of the Sk that have to be examined to determine their maximum Mn = max{S0, Sn} is ˜n/2 as n→ ∞, but that is a worst case result. Any algorithm that determines Mn with certainty must examine at least (c0 + o(l))n1/2 of the Sk on average for a certain constant co > 0, if all random walks with n steps are considered equally likely. There is also an algorithm that on average examines only (co + o(l))n1/2 of the Sk to determine Mn. Different results are obtained when one allows a nonzero probability of error, or else asks only for an estimate of Mn. It is also shown that a global search (one that can ask for any value Sk at any time) for the exact maximum is faster by a factor of log n (when comparing average running times) than a linear sequential one that can skip through some values but cannot go back. © 1995 John Wiley & Sons, Inc.