Adaptive Cauchy Differential Evolution with Strategy Adaptation and Its Application to Training Large-Scale Artificial Neural Networks

Abstract

Artificial neural networks are a computational system, and usually, backpropagation algorithm is used for learning a task, because of its simplicity. However, backpropagation algorithm is likely to converge to a local minimum or saddle point, so that a global minimum may not be found. Differential Evolution (DE) is a simple yet powerful global optimization algorithm for solving multi-dimensional continuous functions. In this paper, we propose a new DE algorithm by combining two excellent DE algorithms, Adaptive Cauchy DE (ACDE) and Self-adaptive DE (SaDE). ACDE shows promising performance by using the Cauchy distribution based on control parameter adaptation. However, ACDE uses only one mutation strategy. SaDE adapts mutation strategies automatically, which shows its effectiveness. Therefore, we extend ACDE with the strategy adaptation of SaDE for enhancing the global optimization performance. The result indicates that the extended ACDE performs better than standard DE not only on conventional benchmark problems but also for training neural networks.