Learning Rate Optimisation of an Image Processing Deep Convolutional Neural Network

Abstract

This paper proposes the use of a mathematical model to identify an optimal learning rate (OLR) for an image processing deep convolutional neural network (DCNN). An OLR may be realized by defining the relationship between the learning rate and model performance. This relationship is meant to resolve the problem of arbitrarily selecting the initial learning rate. The benefit of an OLR includes improved training stability and reduced computational resources. An algorithm is developed to analyze an inputted DCNN model and subsequently render output parameters that may be used to aid in the selection of an OLR. The results rendered by the OLR algorithm proposes that an optimal learning rate improves model performance. The mathematical model is also capable of approximating the model performance to a high degree of accuracy averaging at 91%. Furthermore, a model validation graph is also extrapolated, which will illustrate the mathematical model accuracy and the region of interest (ROI); the ROI defines a region in the learning rate spectrum with a positive effect on model performance.