Some remarks on activation function design in complex extreme learning using Schwarz lemma

Abstract

Processing of complex valued data has become a challenge issue in classification problems where artificial neural networks are used as the classifier. This issue particularly arises in design of complex valued activation functions. To address this problem, a complex valued activation function which is obtained by using Schwarz lemma is proposed in this study and a complex-valued extreme learning classifier is utilized to analyse its classification performance. Accordingly, three inequalities have been presented first by considering the different versions of the boundary Schwarz lemma for Nα class and then, the proposed activation function has been obtained by performing extremal analyses of these inequalities. During simulations, complex extreme learning machine has been used to compare the classification performances of the proposed and other frequently-used activation functions. In classification step, three multi-class and four binary-class datasets have been utilized. In addition, proposed activation function has been considered for two exemplary function approximation problems. According to simulation results, proposed activation function outperforms other activation functions in term of classification accuracy for all considered datasets. It has also been observed that the proposed activation function gives a lower root mean square error than other trigonometric functions in function approximation problem.