On Activation Functions for Complex-Valued Neural Networks — Existence of Energy Functions —

Abstract

Recently models of neural networks that can directly deal with complex numbers, complex-valued neural networks, have been proposed and several studies on their abilities of information processing have been done. One of the important factors to characterize behavior of a complex-valued neural network is its activation function which is a nonlinear complex function. This paper discusses the properties of activation functions from the standpoint of existence of an energy function for complex-valued neural networks. Two classes of complex functions which are widely used as activation functions in the models of complex-valued neural networks are considered. We investigate the properties of activation functions which assure existence of energy functions and discuss about how to find out complex functions which satisfy the properties.