Abstract
The development of deep learning has led to a dramatic increase in the number of applications of artificial intelligence. However, the training of deeper neural networks for stable and accurate models translates into artificial neural networks (ANNs) that become unmanageable as the number of features increases. This work extends our earlier study where we explored the acceleration effects obtained by enforcing, in turn, scale freeness, small worldness, and sparsity during the ANN training process. The efficiency of that approach was confirmed by recent studies (conducted independently) where a million-node ANN was trained on non-specialized laptops. Encouraged by those results, our study is now focused on some tunable parameters, to pursue a further acceleration effect. We show that, although optimal parameter tuning is unfeasible, due to the high non-linearity of ANN problems, we can actually come up with a set of useful guidelines that lead to speed-ups in practical cases. We find that significant reductions in execution time can generally be achieved by setting the revised fraction parameter (zeta ) to relatively low values.
Highlights
The effort to simulate the human brain behaviour is one of the top scientific trends today
The goal is to demonstrate the effectiveness of the sparse evolutionary training (SET) approach, aiming at lower revised fraction values, in the context of the multilayer perceptron (MLP) supervised model
Supervised learning involves observing several samples of a given dataset, which will be divided into ‘training’ and ‘test’ samples. While the former is used to train the neural network, the latter works as a litmus test, as it is compared with the artificial neural networks (ANNs) predictions
Summary
The effort to simulate the human brain behaviour is one of the top scientific trends today. Deep learning strategies pave the way to many new applications, thanks to their ability to manage complex architectures. MLP is a feed-forward ANN composed by several hidden layers, forming a deep network, as shown in Fig.. Because of the intra-layer links flow, an MLP can be seen as a fully connected directed graph between the input and output layers. Supervised learning involves observing several samples of a given dataset, which will be divided into ‘training’ and ‘test’ samples. While the former is used to train the neural network, the latter works as a litmus test, as it is compared with the ANN predictions. One can find further details on deep learning in LeCun et al (2015); Goodfellow et al (2016)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
