An Experiment on the Use of Genetic Algorithms for Topology Selection in Deep Learning

Fernando Mattioli,Daniel Caetano,Eduardo Naves,Alexandre Cardoso,Edgard Lamounier

doi:10.1155/2019/3217542

Fernando Mattioli, Daniel Caetano + Show 3 more

Open Access

https://doi.org/10.1155/2019/3217542

Copy DOI

Abstract

The choice of a good topology for a deep neural network is a complex task, essential for any deep learning project. This task normally demands knowledge from previous experience, as the higher amount of required computational resources makes trial and error approaches prohibitive. Evolutionary computation algorithms have shown success in many domains, by guiding the exploration of complex solution spaces in the direction of the best solutions, with minimal human intervention. In this sense, this work presents the use of genetic algorithms in deep neural networks topology selection. The evaluated algorithms were able to find competitive topologies while spending less computational resources when compared to state-of-the-art methods.

Highlights

Many digital signals, including images, present an hierarchical nature, in which higher level features are obtained through composition of lower level ones [1]
Deep neural networks (DNN) are a particular class of artificial neural networks (ANN), composed by stacked layers which explore this hierarchical behavior through automatic feature extraction [2]
E design of neural network topologies depends on previous domain knowledge and expertise

Summary

Introduction

Many digital signals, including images, present an hierarchical nature, in which higher level features are obtained through composition of lower level ones [1]. Deep learning allows computational models with multiple computing layers to process data in different levels of abstraction [1]. Training a deep learning model from scratch requires two main steps: selection of the model topology and presentation of input/output data. E result of this recombination process is an offspring of new solutions, in which individuals inherit different characteristics from their parents [9]. E chosen individuals are subjected to the genetic operators of crossover and mutation to create an offspring of new solutions (generation). Elements of selected chromosomes are exchanged, creating new solutions inheriting characteristics of their parents. E solutions in the new generation will be evaluated and will replace the previous population, optionally keeping some of its best individuals through elitism [11, 17] An element of the chromosome is randomly changed, producing a new solution. e solutions in the new generation will be evaluated and will replace the previous population, optionally keeping some of its best individuals through elitism [11, 17]

Methods

Results

Conclusion