A Scaled Stochastic Approximation Algorithm

Abstract

Consider a stochastic system of such complexity that its performance can only be evaluated by using simulation or direct experimentation. To optimize the expected performance of such systems as a function of several continuous input parameters (decision variables), we present a “scaled” stochastic approximation algorithm for finding the zero (root) of the gradient of the response function. In each iteration of the scaled algorithm, two independent gradient estimates are sampled at the current estimate of the optimal input-parameter vector to compute a scale-free estimate of the next search direction. We establish sufficient conditions to ensure strong consistency and asymptotic normality of the resulting estimator of the optimal input-parameter vector. Strong consistency is also established for a variant of the scaled algorithm with Kesten's acceleration. An experimental performance comparison of the scaled algorithm and the classical Robbins-Monro algorithm in two simple queueing systems reveals some of the practical advantages of the scaled algorithm.