Multiobjective visual evolutionary neural network and related convolutional neural network optimization

Abstract

Biological visual systems intrinsically include a variety of direction-selective neurons capable of detecting changes in visual motion. Some of the neurons have been used to construct computational models for problem-specific engineering applications. However, it remains unclear how these neurons' visual response mechanisms can serve an interdisciplinary topic - visual evolutionary neural networks. In this study, we draw inspiration from swarm evolution and the fly's visual response and attention mechanisms to develop a fast multiobjective visual evolutionary neural network for solving multiobjective optimization problems, specifically focusing on convolutional neural network optimization. Our approach involves designing a multi-channel input visual neural network that outputs global and local learning rates for state transition, proposing a new population-like state update strategy to move current states toward the Pareto front as quickly as possible, and exploiting a point-to-plane distance model to control the size of the external archive set. The computational complexity of the evolutionary neural network mainly depends on the size of the external archive set and the number of fitness evaluations. Comparative experiments demonstrate that our network is an extremely competitive optimizer, effectively solving multiple kinds of benchmark examples. Especially, when optimizing the 8,248 parameters of a conventional convolutional neural network, our approach quickly achieves large-scale optimization.