A Novel Algorithm for Neural Network Architecture Generation, Parameter Optimization and Feature Selection

Abstract

In this paper, we propose ParFeatArch Generator, a new algorithm for generating Neural Network architectures with optimal features and parameters through Particle Swarm Optimization. Selecting the best architecture for a Neural Network is usually done through a trial and error process, in which the number of layers is selected usually based on previous experience and then the network is trained and tested. When using Neural Networks as classifiers in feature selection algorithms, usually the number of layers in the Neural Network is selected prior to using the Neural Network as a classifier to the feature selection algorithm. In this work we propose a new generative algorithm called ParFeatArch Generator, which is based on PSO and combines the feature selection process with the Neural Network architecture selection process and parameter optimization in one algorithm which generates the Neural Network topology with optimal parameters while at the same time performs feature selection and evaluates the Neural Network topology to determine its quality. With the proposed algorithm, given a dataset, it is possible to end up with the optimal features on the dataset and with an optimal Neural Network classifier with optimal parameters for such features.