A quasi-local Levenberg-Marquardt algorithm for neural network training

Abstract

Although the Levenberg-Marquardt algorithm has been extensively used as a neural network training method, it suffers from being very expensive, both in memory and number of operations required, when the network to be trained has a significant number of adaptive weights. In this work we propose a modification of this method that considers the concept of neural neighbourhoods. It is shown that, by performing a Levenberg-Marquardt step to a single neighbourhood at each iteration, significant savings in computing effort and memory occupation are obtained, without efficiency loss.