Evolution of Optimal MLFFNN Design: A Novel VLC-GA Approach

Abstract

AbstractRecent advancements in neural networks demonstrate that various tasks viz. classification/regression, computer vision and object tracking are performed very accurately using Multilayer Feed Forward Neural Networks (MLFFNN), Convolutional Neural Networks (CNN) and variants of Recurrent Neural Networks (RNN). Presently, the Neural Network architectures (NNA) are designed by human expertise and trial and error fashion. Therefore, NNA design is both manual and time consuming process. The problem of finding optimal neural network design (number of hidden layers and number neurons in each hidden layer) is a challenging and less explored problem. Genetic Algorithms (GA) solves optimization problems based on natural evolution and selection. Simple GAs use fixed length chromosomes for representing problem variable domain and does not change subsequently. Therefore, SGAs are not suitable for solving optimization problems having too many and dynamically changing decision variables viz. structural topology and path optimizations. Variable length chromosomes (VLC) are applied for solving graph based problems wherein the problem variable domain space is known apriori. But, in case of optimal neural network design, problem variable domain is not known apriori. Therefore, we propose a Novel VLC-GA approach for Evolving Optimal Neural Network Architecture Design (ONNAD) of MLFFNNs. In this approach, each NNA is represented using a VLC consisting of hidden layer number and number of neurons in a layer. Designing optimal architecture is a two-stage process. Stage1, Evolves the network architecture from population of Individual architectures by natural selection and genetics using VLC-GA. The generated NNAs are trained using back propagation algorithm. The sub-optimal architecture of stage1 is used to generate initial population for stage2. Stage 2, adaptively refine the domain space and increase in chromosome length based on average fitness to achieve fine level accuracy for the defined task. We also compared the performance accuracy (PA) and convergence Rate (CR) of our method with existing empirically. In future, we apply this approach for learning CNN design automatically.KeywordsANNCNNGAVLCSAGAAGAMLFFNNNNNNAPACRDSRIGLONNADRNN