An Efficient Algorithm for Multi Class Classification in Deep Neural Network

Abstract

AbstractIn comparison to other machine learning techniques, deep neural networks are effective in classifying non-linearly separable data. Because of its simplicity, contemporary gradient-based algorithms such as momentum Stochastic Gradient Descent (SGD) are commonly employed in Deep Neural Networks (DNN). However, the process of convergence is slowed by the choice of an appropriate learning rate and the local minima problem. To address these issues, this research proposes a unique approach for training DNNs called Simulated Annealing Based Gradient Descent (SAGD), which involves optimizing weights and biases. The SAGD technique optimizes the function by combining gradient information with the simulated annealing notion. The learning rate does not need to be manually adjusted with this method. Instead, using the simulated annealing approach, the learning rate is modified automatically for each epoch. The approach is tested utilizing VGG16, ResNet 18 and InceptionV3 architectures on typical multi-class classification data sets such as Iris, MNIST, and CIFAR10. The performance of SAGD and other state-of-the-art gradient descent optimization methods is compared, and it is demonstrated that SAGD performs comparably to existing gradient descent methods.KeywordsDNNClassificationSimulated annealingOptimizationGradient descent