An Adaptive Slope Sigmoidal Function Cascading Neural Networks Algorithm

Abstract

Cascade 2 algorithm is a variant of Cascade-Correlation algorithm that is a well-known and widely used constructive neural networks algorithm. We propose an adaptive slope sigmoidal function cascading neural networks algorithm (ASCNNA) in this paper. The proposed algorithm emphasizes on architectural adaptation and functional adaptation during training. This algorithm is a constructive approach of building cascading architecture and uses gradient descent method in sequential mode as the weight update rule of individual hidden node. To achieve functional adaptation, the slope of the sigmoidal function is adapted during learning. The algorithm determines not only the optimum number of hidden layers' node, as also the optimum value of the slope parameter of sigmoidal function for nonlinear nodes. One simple variant derived from ASCNNA is where the slope parameter of sigmoidal function is fixed. Both the variants are compared to each other on five function approximation tasks. Simulation results reveal that adaptive slope sigmoidal function presents several advantages over traditional fixed shape sigmoidal function, resulting in increased flexibility, smoother learning, and better convergence and generalization performance.