An Algorithm for Gradient-Free Simulation Optimization using Sampling Control

Abstract

This article considers a class of continuous-variable optimization problems in which the objective function cannot be computed exactly but must instead be estimated by using, typically, a simulation. As the simulation output is stochastic, iterative optimization algorithms these problems are often augmented with noise-removal features to ensure convergence to an optimal solution. Two broad noise-removal approaches are considered: stepsize-control, which involves a decreasing stepsize, or sampling-control, which involves an increasing sample size. Of these two, stepsize-control predates sampling-control by several decades and is the most commonly used approach, although, as we show, sampling-control has some advantages over stepsize-control. Our particular focus is on optimization problems for which direct gradient estimates are not available, and that instead must be approximated using estimates of the objective function. The classic Kiefer-Wolfowitz algorithm, using stepsize-control, is one such algorithm that estimates a divided difference approximation of the gradient. This article presents a sampling-controlled version of this algorithm that also uses divided difference estimates and has the benefit of being easily parallelizable. A convergence proof and some simulation results are also included. Note that one of the advantages of the sampling-controlled method is that it is better suited to estimators that have small-sample bias but are asymptotically unbiased: because the sampling rate increases, estimator bias is gradually decreased as the number of samples increases.