Challenging the Deployment of Fiducial Points in Minimum Error Entropy

Abstract

In this paper, robust linear adaptive filtering in presence of non-Gaussian noise is addressed. More precisely, the well-known algorithm for robust adaptive learning called minimum error entropy with fiducial points (MEEF) is challenged. Error entropy and error correntropy are two information theoretic cost functions that can be used in a supervised learning problem. They incorporate higher-order statistics of the error between labels and system outputs and therefore are comprehensive descriptors of data that show more robustness against non-Gaussianity of the environment compared to the conventional cost functions. This robustness makes them strong substitutions for classical mean square error (MSE) that only considers the variance (second-order moment) of the error. In minimum error entropy (MEE) specifically, error entropy is minimized to extract as much information as possible about the data generating system. However, this minimum entropy can also occur for other error PDFs not located at the origin inasmuch as entropy is shift-invariant. In these cases, an undesired estimate of the system parameters is obtained. Therefore, an extra step must be taken to concentrate error samples around the origin. The most celebrated approach towards that end is MEEF, in which some external and artificial zero error samples, called fiducial points (not generated by the underlying system), are added to the cost function as the reference points to force actual error samples to get concentrated around them. Using these fiducial points translates MEEF into a weighted combination of MEE and maximum correntropy criterion (MCC). In this paper, it is shown that incorporating these fiducial points into MEE can even degrade the steady state misalignment and/or convergence speed.