Selecting good enough simulated designs

Abstract

This paper studies the problem of selecting a subset of good designs from a finite set of simulated designs. We develop an approach to select r good enough designs instead of the exact top r designs from k alternatives, where good enough designs are defined as the top g designs (r ≤ g < k). Our approach aims to improve the selection efficiency while ensuring the performance of the selected designs in an acceptable range. Using the optimal computing budget allocation (OCBA) framework, we formulate the problem as that of maximizing the probability of correctly selecting r good enough designs under a simulation budget constraint. Based on the approximate measure of the probability of correct selection, we derive an asymptotically optimal selection procedure for selecting a good enough design subset. The proposed method demonstrates good empirical performance on some typical selection problems, including a practical inventory system problem.