Non-smooth optimization based on resilient backpropagation search for unconstrained and simply bounded problems

Abstract

This paper describes a non-smooth optimization method based on a backpropagation search method. More specifically, the Resilient backPROPagation (RPROP) algorithm, used extensively for neural network training, is employed here. RPROP is recast in terms of numerical optimization and used as a step-finding method. The result is the fast search algorithm RPROP, which avoids expensive line searches and performs one function and one gradient evaluation per iteration. Furthermore, only the gradient's sign is used rather than its value. RPROP is applied to a set of 10 test problems, which have unconstrained and simply bounded constrained versions. The results are discussed and assessed against a set of reported results based on bundle methods. It is shown that RPROP is able to deal efficiently with very large non-smooth problems.