Fusion of Chaotic Activation Functions in training neural network

Abstract

Multilayer feed-forward neural network is trained with a supervised algorithm which is loosely connected with biological learning. Bio-inspired system development is recently a challenging topic in intelligent system design. To make the learning biologically plausible, we propose `Fusion of Chaotic Activation Functions' (FCAF) in which multiple chaotic activation functions (AFs) are used to compute final activation. It is to investigate whether FCAF can enable the learning to be faster. Validity of the proposed method is examined by performing simulations on challenging ten real benchmark classification and time series prediction problems. The FCAF has been applied to 2-bit, 3-bit and 4-bit parity, the breast cancer, Diabetes, Heart disease, Glass, Flare, credit card and thyroid problems. The algorithm is shown to work better than other AFs used independently in BP such as sigmoid (SIG), arctangent (ATAN), logarithmic (LOG), and that of jointly such as fusion of activation function (FAF).