A neural nonlinear adaptive filter with a trainable activation function

Abstract

The normalized nonlinear gradient descent learning algorithm (NNGD) for a class of nonlinear finite impulse response (FIR) adaptive filters (dynamical perceptron) is extended to the case where the amplitude of the nonlinear activation function is made gradient adaptive. This makes the adaptive amplitude normalized nonlinear gradient descent (AANNGD) algorithm. The AANNGD is suitable for processing of nonlinear and nonstationary signals with a large dynamical range. Experimental results show that AANNGD outperforms the standard LMS, NGD, NNGD, the fully adaptive (FANNGD) and the sign algorithm on nonlinear input with large dynamics.