Simulation Optimization: Integrating Research and Practice

Abstract

The feature article by Fu (2002) identifies several areas within the field of simulation optimization that need further attention, including the development of good stopping criteria, providing improved measures of goodness for the estimated optimal solution, and addressing optimization problems with noisy constraints. These are all important areas for future research. Fu (2002) also describes the differences between research and software developments in the area of simulation optimization. In particular, the research literature is focused on rigorous simulation optimization algorithms (with provable performance guarantees), while algorithms implemented in commercial simulation software are generally heuristic in nature. This presumably reflects a belief that the simulation optimization approaches developed by researchers either require too much special structure to be implemented in a general purpose simulation language, or they are too slow to be used to solve large and complex simulation optimization problems. The main thesis of this article is that it is possible to develop simulation optimization algorithms that are efficient, rigorous, and suitable for implementation in simulation languages and that the development and implementation of such algorithms should be a common goal for both researchers and software developers. The desirability of having efficient optimization approaches that are suitable for implementation in commercial simulation software packages seems obvious given the extensive use of simulation software for design and analysis of complex systems. Similarly, among algorithms that are comparable in terms of applicability and efficiency, it seems evident that the choices that have provable performance guarantees are preferable over choices that are purely heuristic. However, many simulationists appear to have the perception that one can either use a heuristic algorithm with a fast empirical convergence rate, or use a rigorous method with a slow empirical convergence rate, but that it is not possible to use a simulation optimization approach that is both efficient and rigorous. When evaluating this perceived tradeoff between a fast empirical convergence rate and provable performance guarantees, it is not surprising that software developers have chosen to implement heuristic approaches that appear to work well in practice on a wide range of problems. The perceived tradeoff described in the previous paragraph may be due to the frequent use of provable performance guarantees in the research literature that severely restrict the flexibility of algorithm developers (in that these guarantees only hold under highly restrictive conditions). However, there are several different types of provable performance guarantees that a simulation optimization algorithm may have. Fu (2002) discusses a few of these (see also Banks et al. 2000), and additional discussion is provided below. An ideal performance guarantee would be to know that an algorithm stops and returns an estimated optimal solution whose corresponding objective function value differs by less than a prespecified amount from the optimal objective function value. In other words, if it is of interest to solve the optimization problem min ∈ J , if ∗ ∈ is a (global) optimal solution, and if
∈ is the estimated solution returned by the optimization algorithm, then ideally, we would want to know either that J
≤ J ∗ + or that J
≤