Extremum Seeking Over a Discrete Action Space

Abstract

Extremum Seeking is a black box optimization/-control technique that utilizes perturbations to system inputs in order to optimize outputs. In this work, we propose an extension to the family of Extremum Seeking for model-free optimization of convex functions over a discrete action space. We refer to this method as Discrete Action Extremum Seeking (DA-ES). In this setting the DA-ES controller perturbs system inputs by visiting neighboring discrete actions, and then estimates a gradient from the resulting output signal. The DA-ES then uses the gradient estimate to select the best action to optimize the objective (e.g. the system output), and the perturbation process repeats. In this paper we outline the DA-ES algorithm, and derive convergence criteria for local minima of convex functions. Simulation results demonstrate the effectiveness of DA-ES in optimizing convex functions over a space of discrete actions.