This paper presents a heuristic-based convolutional neural network architecture optimization scheme (HeuCNN). First, CNN architectures are encoded as particles in optimization space with two objectives test error and number of parameters. Then, the effectiveness of CNN particles is represented by a heuristic in optimization space. It is mathematically proven and experimentally shown that the proposed heuristic assigns the best values for CNNs with higher performances. Uniform and nonuniform distribution of CNN particles in optimization space are analyzed and behavior of CNNs moving towards higher test error and higher number of parameters are also mathematically proven. CNN architecture search is proposed based on particle swarm optimization (PSO) by new algorithms for particle difference and velocity computation as well as particle update rules. Limitations of HeuCNN are explained and primary insights are presented to address its challenges as future works. Evaluation of HeuCNN on 9 publicly available datasets and comparing with 38 constant and optimized CNN peer competitive models reveal that HeuCNN achieves the best test error and the lowest number of parameters simultaneously. Source codes, data and software associated with this research will be available online after the publication of the paper.
Read full abstract