Performance Analysis of Learning Rate Parameter on Prediction of Signal Power Loss for Network Optimization and Better Generalization

Abstract

This research work explores the neural network learning capabilities by using a multi-layer perceptron artificial neural network to predict signal power loss by means of dataset from long term evolution network. The analysis of the effect of the learning rate parameter and the adoption of early stopping method during network training have been executed by using varied values of learning rate to ascertain the best learning rate during the neural network training. Also, there were neural network training without the application of learning rate and early stopping method and comparison have been made with the output results as shown in different tables. Output results comparisons have been performed using training regression and performance mean squared error. Two back propagation training algorithms, the Levenberg–Marquardt and the Bayesian Regularization algorithms were employed for the network training and comparison of their prediction abilities examined using same values of learning rates and on application of early stopping method as well as without learning rate and without early stopping method. The result shows an optimal performance of the neural network model on application of 0.005 learning rate and using 75%:15%:15% early stopping method with training regression 0.99267 and performance mean squared error 2.47 using Levenberg–Marquardt and training regression 0.99488 and performance mean squared error of 1.910 using Bayesian Regularization algorithms, respectively. Without application of learning rate and early stopping method, training the network using Levenberg–Marquardt algorithm gives training regression of 0.97111 and performance mean squared error of 7.38 using Levenberg–Marquart algorithm and training regression of 0.99248 and performance mean squared error of 4.42 using Bayesian Regularization algorithm. The margin between the two output results demonstrates the impact and importance of learning rate parameter as well as adopting early stopping method for neural network training for network optimization and better network generalization.