Kernel width adaptation in information theoretic cost functions

Abstract

This paper presents an algorithm for online adaptation of the kernel width parameter in information theoretic cost functions used for adaptive system training. Training algorithms which optimize information theoretic quantities like entropy involve choosing a kernel size for their sample estimators. The kernel size essentially dictates the nature of the performance surface of the cost function over which the system parameters adapt. This, in turn, governs factors like speed of adaptation and presence of local minima. We show results of using the Minimum Error Entropy (MEE) criterion with the proposed adaptive kernel algorithm for training a time delay neural network. Our simulations show that having an adaptive kernel width results in faster convergence of parameters as compared to having fixed values.