A Derivative-Free Optimization Method With Application to Functions With Exploding and Vanishing Gradients

Abstract

In this letter, we propose a bio-inspired derivative-free optimization algorithm capable of minimizing objective functions with vanishing or exploding gradients. The proposed method searches for improvements by leveraging a PCA-based strategy similar to fish foraging. The strategy does not require explicit gradient computation or estimation and is shown in simulation to require few function evaluations. Additionally, our analysis proves that the proposed algorithm’s search direction converges to the gradient direction everywhere outside of small neighborhoods around local minima. Applications to a data-driven LQR problem and noisy Rosenbrock optimization problem are demonstrated. Empirical results show the proposed method exhibits fast convergence and is able to find the LQR gains for any controllable system, including unstable systems, and is robust to noisy function evaluations.