Trust region based stochastic optimization with adaptive restart: A family of global optimization algorithms

Abstract

The field of simulation optimization has seen algorithms proposed for local optimization, drawing upon different locally convergent search methods. Similarly, there are numerous global optimization algorithms with differing strategies to achieve convergence. In this paper, we look specifically into meta-model based algorithms that stochastically drive global search through an optimal sampling criteria evaluated over a constructed meta-model of the predicted response considering the uncertainty of the response. We propose Trust Region Based Optimization with Adaptive Restart (TBOAR), a family of algorithms that dynamically restarts a trust region based search method via an optimal sampling criteria derived upon a meta-model based global search approach. Additionally, we propose a new sampling criteria to reconcile undesirable adaptive restart trajectories. This paper presents preliminary results showing the advantage of the proposed approach over the benchmark Efficient Global Optimization algorithm, focusing on a deterministic black box simulator with a d-dimensional input and a one-dimensional response.