A Neural Network Training Algorithm Based on Collinear Scaling Quasi-Newton Method

Abstract

Based on collinear scaling and local quadratic approximation, quasi-Newton methods have improved for function value is not fully used in the Hessian matrix. As collinear scaling factor in paper may appear singular, this paper, a new collinear scaling factor is studied. Using local quadratic approximation, an improved collinear scaling algorithm to strengthen the stability is presented, and the global convergence of the algorithm is proved. In addition, numerical results of training neural network with the improved collinear scaling algorithm shown the efficiency of this algorithm is much better than traditional ones.