Back-propagation algorithm with variable adaptive momentum

Abstract

In this paper, we propose a novel machine learning classifier by deriving a new adaptive momentum back-propagation (BP) artificial neural networks algorithm. The proposed algorithm is a modified version of the BP algorithm to improve its convergence behavior in both sides, accelerate the convergence process for accessing the optimum steady-state and minimizing the error misadjustment to improve the recognized patterns superiorly. This algorithm is controlled by the learning rate parameter which is dependent on the eigenvalues of the autocorrelation matrix of the input. It provides low error performance for the weights update. To discuss the performance measures of this proposed algorithm and the other supervised learning algorithms such as k-nearest neighbours (k-NN), Naive Bayes (NB), linear discriminant analysis (LDA), support vector machines (SVM), BP, and BP with adaptive momentum (PBPAM) have been compared in term of speed of convergence, Sum of Squared Error (SSE), and accuracy by implementing benchmark problem - XOR and seven datasets from UCI repository.