Hopfield neural network: The hyperbolic tangent and the piecewise-linear activation functions

Abstract

This paper reports two-dimensional parameter-space plots for both, the hyperbolic tangent and the piecewise-linear neuron activation functions of a three-dimensional Hopfield neural network. The plots obtained using both neuron activation functions are compared, and we show that similar features are present on them. The occurrence of self-organized periodic structures embedded in chaotic regions is verified for the two cases.