A nonlinear neural fir filter with an adaptive activation function

Abstract

An adaptive amplitude normalized nonlinear gradient descent (AANNGD) algorithm for the class of nonlinear finite impulse response (FIR) adaptive filters (dynamical perception) is introduced. This is achieved by making the amplitude of the nonlinear activation function gradient adaptive. The proposed learning algorithm is suitable for processing of nonlinear and nonstationary signals with a large dynamical range, and removes the unwanted effect of saturation nonlinearities. For rigor, sensitivity analysis is performed and the improved performance of the AANNGD algorithm over the standard LMS, NGD, NNGD, the fully adaptive NNGD (FANNGD) and the sign algorithm is verified by simulations on nonlinear and nonstationary inputs with large dynamics.