In-Depth Case Study on Artificial Neural Network Weights Optimization Using Meta-Heuristic and Heuristic Algorithmic Approach

Abstract

The Meta-heuristic and Heuristic algorithms that have been introduced for deep neural network optimization is in this paper. Artificial Intelligence, and also the most used Deep Learning methods are all growing in popularity these days, thus we need faster optimization strategies for finding the results of future activities. Neural Network Optimization with Particle Swarm Optimization, Backpropagation (BP), Resilient Propagation (Rprop), and Genetic Algorithm (GA) is used for numerical analysis of different datasets and comparing each other to find out which algorithms work better for finding optimal solutions by reducing training loss. Genetic algorithm and also bio-inspired Particle Swarm Optimization is introduced in this paper. Besides, Resilient Propagation and Conventional Backpropagation algorithms which are application-specific algorithms have also been introduced. Meta-heuristic algorithms GA and PSO are a higher-level formula and problem-independent technique that may be used to a diverse number of challenges. The characteristic of Heuristic algorithms has extremely specific features that vary depending on the problem. The conventional Backpropagation (BP) based optimization, the Particle Swarm Optimization methodology, and Resilient Propagation (Rprop) are all fully presented, and how to apply these procedures in Artificial Deep Neural networks Optimization is also thoroughly described. Applied numerical simulation over several datasets proves that the Meta-heuristic algorithm Particle Swarm Optimization and also Genetic Algorithm performs better than the conventional heuristic algorithm like Backpropagation and Resilient Propagation.