Abstract
Neuroevolution is the field of study that uses evolutionary computation in order to optimize certain aspect of the design of neural networks, most often its topology and hyperparameters. The field was introduced in the late-1980s, but only in the latest years the field has become mature enough to enable the optimization of deep learning models, such as convolutional neural networks. In this paper, we rely on previous work to apply neuroevolution in order to optimize the topology of deep neural networks that can be used to solve the problem of handwritten character recognition. Moreover, we take advantage of the fact that evolutionary algorithms optimize a population of candidate solutions, by combining a set of the best evolved models resulting in a committee of convolutional neural networks. This process is enhanced by using specific mechanisms to preserve the diversity of the population. Additionally, in this paper, we address one of the disadvantages of neuroevolution: the process is very expensive in terms of computational time. To lessen this issue, we explore the performance of topology transfer learning: whether the best topology obtained using neuroevolution for a certain domain can be successfully applied to a different domain. By doing so, the expensive process of neuroevolution can be reused to tackle different problems, turning it into a more appealing approach for optimizing the design of neural networks topologies. After evaluating our proposal, results show that both the use of neuroevolved committees and the application of topology transfer learning are successful: committees of convolutional neural networks are able to improve classification results when compared to single models, and topologies learned for one problem can be reused for a different problem and data with a good performance. Additionally, both approaches can be combined by building committees of transferred topologies, and this combination attains results that combine the best of both approaches.
Highlights
Deep Learning encompasses a broad set of techniques that are able to infer “deep” models to solve diverse machine learning problems
convolutional neural networks (CNNs) are a type of neural network that most typically comprises two different parts: first, convolutional layers are in charge of automatically extracting relevant features from the input; fullyconnected layers are responsible for performing supervised learning
For the sake of clarity and economy, throughout this section, we will be more exhaustive when describing the results obtained using genetic algorithms (GA), which are slightly better, with a very small difference, than those attained by grammatical evolution (GE)
Summary
Deep Learning encompasses a broad set of techniques that are able to infer “deep” models to solve diverse machine learning problems From these techniques, convolutional neural networks (CNNs) are probably the most well-known, extensively studied and widely used. CNNs are a type of neural network that most typically comprises two different parts: first, convolutional layers are in charge of automatically extracting relevant features from the input; fullyconnected layers are responsible for performing supervised learning. The most common CNN architecture comprises two distinct parts: first, a sequence of convolutional layers is in charge of automatically extracting relevant features from input data This stage is known as “feature learning” or “representation learning” and replaces the procedure in which an expert or group of experts perform manual feature engineering to convert some unstructured information into a set of valuable features. The backpropagation mechanism is used both for learning the parameters of the dense layers (as in classical neural networks) and for the convolutional layers, in order to minimize a loss function defined over the output of the neural network and the expected classification output
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
