Performance analysis of activation function in higher order logic programming

Abstract

Higher-order logic programming in Hopfield neural network is a vital paradigm to solve numerous combinatorial optimization problem and pattern recognition. Hence, activation function can be integrated as catalyst or accelerating techniques of doing higher order logic programming in Hopfield network. Obviously, the McCulloch-Pitts learning rule is widely used in higher order logic programming. Hereby, we proposed the Bipolar sigmoid and Hyperbolic activation function trained by Wan Abdullah’s method by integrating energy minimization scheme in order to speed up the training process. Computer simulations are carried out to authenticate the performance of Hyperbolic activation function, Bipolar sigmoid activation function and McCulloch-Pitts function (Logistic Function) in higher order Hopfield network. We used Microsoft Visual C++ 2013 as a platform of simulating, training and testing the network. Therefore, evaluations are made between these activation functions to see which one is superior in the aspects of global solutions, hamming distance, complexity and computation time. It was proven by the computer simulations that Hyperbolic activation function outperformed Bipolar sigmoid activation function and McCulloch-Pitts function.