Early Stopping Criteria for Levenberg-Marquardt Based Neural Network Training Optimization

Azizah Suliman,Batyrkhan Omarov

doi:10.14419/ijet.v7i4.36.25382

Abstract

In this research we train a direct distributed neural network using Levenberg-Marquardt algorithm. In order to prevent overtraining, we proposed correctly recognized image percentage based on early stop condition and conduct the experiments with different stop thresholds for image classification problem. Experiment results show that the best early stop condition is 93% and other increase in stop threshold can lead to decrease in the quality of the neural network. The correct choice of early stop condition can prevent overtraining which led to the training of a neural network with considerable number of hidden neurons.

Highlights

Analysis of literatures shows that neural networks are effectively used in crucial applications such as pattern recognition [1], image classification [2, 8], speech recognition, natural language processing [1 - 3]
Each neural network was tested on the same test together
We can conclude, that quality of the resulting neural network depends on the size of the “stop threshold”

Summary

Introduction

Analysis of literatures shows that neural networks are effectively used in crucial applications such as pattern recognition [1], image classification [2, 8], speech recognition, natural language processing [1 - 3]. ANN should include the existing object characteristics (observable data) to one or more specific classes. One of the main challenges in implementing ANN is the significant amount of time needed in the training phase especially when solving complex problems. Depending on the growth of a number of hidden layers and neurons, the required time for ANN learning process and new instance assessment time, grows by leaps and bounds. The rate of successful classification depends on the growth of a number of hidden layer and neurons. The challenge remains in improving the ANN by improving the training algorithms, selecting the best network topology, determining the number of hidden layers neurons, interpretation of weighting coefficients and bias, and their evaluation of optimality, etc

Objectives

Results

Conclusion