Kernel Adaptive Filtering Research Articles

A family of split kernel adaptive filtering algorithms for nonlinear stereophonic acoustic echo cancellation

A family of split kernel adaptive filtering algorithms for nonlinear stereophonic acoustic echo cancellation

Quantized kernel least inverse hyperbolic sine adaptive filtering algorithm

In the last few decades, the kernel method has been successfully used in the field of adaptive filtering to solve nonlinear problems. Mercer kernel is used to map data from input space to reproducing kernel Hilbert space (RKHS) by kernel adaptive filter (KAF). In regenerated kernel Hilbert spaces, the inner product can be easily calculated by computing the so-called kernel trick. The Kernel adaptive filtering algorithm is superior to common adaptive filtering algorithm in solving nonlinear problems and nonlinear channel equalization. For nonlinear problems, a robust kernel least inverse hyperbolic sine (KLIHS) algorithm is proposed by combining the kernel method with the inverse of hyperbolic sine function.The main disadvantage of KAF is that the radial-basis function (RBF) network grows with every new data sample, which increases the computational-complexity and requires more momories. The vector quantization (VQ) has been proposed to address this problem and has been successfully applied to the current kernel adaptive filtering algorithm. The main idea of the VQ method is to compress the input space through quantization to curb the network-size growth. In this paper, vector quantization is used to quantify the input spatial data, and a quantized kernel least inverse hyperbolic sine (QKLIHS) algorithm is constructed to restrain the growth of network scale. The energy conservation relation and convergence condition of quantized kernel least inverse hyperbolic sine algorithm are given. The simulation results of Mackey-Glass short-time chaotic time series prediction and nonlinear channel equalization environment show that the proposed kernel least inverse hyperbolic sine algorithm and quantized kernel least inverse hyperbolic sine algorithm have advantages in convergence speed, robustness and computational complexity.

Prediction of chaotic time series based on Nyström Cauchy kernel conjugate gradient algorithm

Chaotic time series can well reflect the nonlinearity and non-stationarity of real environment changes. The traditional kernel adaptive filter (KAF) with second-order statistical characteristics suffers performance degeneration dramatically for predicting chaotic time series containing noises and outliers. In order to improve the robustness of adaptive filters in the presence of impulsive noise, a nonlinear similarity measure named Cauchy kernel loss (CKL) is proposed, and the global convexity of CKL is guaranteed by the half-quadratic (HQ) method. To improve the convergence rate of stochastic gradient descent and avoid a local optimum simultaneously, the conjugate gradient (CG) method is used to optimize CKL. Furthermore, to address the issue of kernel matrix network growth, the Nyström sparse strategy is adopted to approximate the kernel matrix and then the probability density rank-based quantization (PRQ) is used to improve the approximation accuracy. To this end, a novel Nyström Cauchy kernel conjugate gradient with PRQ (NCKCG-PRQ) algorithm is proposed for the prediction of chaotic time series in this paper. Simulations on prediction of synthetic and real-world chaotic time series validate the advantages of the proposed algorithm in terms of filtering accuracy, robustness, and computational storage complexity.

Functional Bayesian Filter

We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS). By applying the kernel method and the representer theorem to perform linear quadratic estimation in a functional space, we derive a Bayesian recursive state estimator for a general nonlinear dynamical system in the original input space. Unlike existing nonlinear extensions of the Kalman filter where the system dynamics are assumed known, the state-space representation for the Functional Bayesian Filter (FBF) is completely learned online from measurement data in the form of an infinite impulse response (IIR) filter or recurrent network in the RKHS, with universal approximation property. Using a positive definite kernel function satisfying Mercer’s conditions to compute and evolve information quantities, the FBF exploits both the statistical and time-domain information about the signal, extracts higher-order moments, and preserves the properties of covariances without the ill effects due to conventional arithmetic operations. We apply this novel kernel adaptive filtering (KAF) to recurrent network training, chaotic time-series estimation and cooperative filtering using Gaussian and non-Gaussian noises, and inverse kinematics modeling. Simulation results show FBF outperforms existing Kalman-based algorithms.

Kernel least logarithm absolute difference algorithm based on P-norm

The kernel adaptive filtering is an efficient and nonlinear approximation method which is developed in reproducing kernel Hilbert space (RKHS). Kernel function is used to map input data from original space to RKHS space, thus solving nonlinear problems is efficient.Impulse noise and non-Gaussian noise exist in the real application environment, and the probability density distribution of these noise characteristics shows a relatively heavy trailing phenomenon in the statistical sense. α stable distribution can be used to model this kind of non-Gaussian noise well. The kernel least mean square(KLMS) algorithms usually perform well in Gaussian noise, but the mean square error criterion only captures the second-order statistics of the error signal, this type of algorithm is very sensitive to outliers, in other words, it lacks robustness in α stable distribution noise. The kernel least logarithm absolute difference(KLLAD) algorithm can deal with outliers well, but it has the problem of slow convergence.In order to further improve the convergence speed of nonlinear adaptive filtering algorithm in α stable distributed noise background, a new kernel least logarithm absolute difference algorithm based on p-norm (P-KLLAD) is presented in this paper. The algorithm combining least logarithm absolute difference algorithm and p norm, on the one hand, the least logarithm difference criteria is ensure the algorithm to have good robustness in α stable distribution noise environment, and on the other hand, add p norm on the absolute value of error.The steepness of the cost function is controlled by p norm and a posititive constant ɑ to improve the convergence speed of the algorithm.The computer simulation results of Mackey-Glass chaotic time series prediction and nonlinear system identification show that this algorithm improves the convergence speed with good robustness,and the convergence speed and robustness better than the kernel least mean square algorithm,the kernel fractional lower power algorithm, the kernel least logarithm absolute difference algorithm and the kernel least mean p-norm algorithm.

Zero-Attracting Kernel Maximum Versoria Criterion Algorithm for Nonlinear Sparse System Identification

Sparsity-induced kernel adaptive filters have emerged as a promising candidate for a nonlinear sparse system identification (SSI) problem. The existing zero-attracting kernel least mean square (ZA-KLMS) algorithm relies on minimum mean square error criterion, which considers only second order statistics of error, thereby resulting in suboptimal performance in the presence of non-Gaussian/impulsive distortions. In this letter, we propose a novel random Fourier features (RFF) based ZA kernel maximum Versoria criterion (ZA-KMVC) algorithm, and their variants, which are robust for nonlinear SSI in the presence of non-Gaussian distortions over both stationary and time-varying environments. Furthermore, the mean-square convergence analysis of the proposed RFF-ZA-KMVC algorithm is performed. It has been observed from the simulation results that the proposed algorithm delivers better convergence performance as compared to the existing state-of-art approaches.

Formula omitted]-norm constraint kernel adaptive filtering framework for precise and robust indoor localization under the internet of things

The mixed noise such as Gaussian noise together with the abrupt noise widely exists in the indoor environment, which always leads to the problem of performance degradation of the positioning system under the Internet of Things (IoT). In this paper, a novel kernel function named generalized Student’s t kernel (GSt) and a resulting sparse generalized Student’s t kernel adaptive filter (SGStKAF) is proposed to attack this problem. The proposed SGStKAF utilizes the kernel mean p-power error criterion (KMPE) with the L1-norm penalty. The proposed SGStKAF has three significant features. Firstly, the generalized Student’s t kernel can suppress the abrupt noise effectively. Secondly, the L1-norm penalty guarantees that the fixed-point sub-iteration is available so that the more precise solution can be obtained in a few iterations. At last, a sparse structure of neural networks for the implementation of the proposed method can also be obtained via the L1 constraint. Three experiments and comparisons are carried out to prove the effectiveness of the proposed positioning framework in terms of accuracy and robustness in both the simulation situation and the real-world indoor environments.

The Generalized Complex Kernel Affine Projection Algorithms

The complex kernel adaptive filter (CKAF) has been widely applied to the complex-valued nonlinear problem in signal processing and machine learning. However, most of the CKAF applications involve the complex kernel least mean square (CKLMS) algorithms, which work in a pure complex or complexified reproducing kernel Hilbert space (RKHS). In this paper, we propose the generalized complex kernel affine projection (GCKAP) algorithms in the widely linear complex-valued RKHS (WL-RKHS). The proposed algorithms have two main notable features. One is that they provide a complete solution for both circular and non-circular complex nonlinear problems and show many performance improvements over the CKAP algorithms. The other is that the GCKAP algorithms inherit the simplicity of the CKLMS algorithm while reducing its gradient noise and boosting its convergence. The second-order statistical characteristics of WL-RKHS have also been developed. An augmented Gram matrix consists of a standard Gram matrix and a pseudo-Gram matrix. This decomposition provides more underlying information when the real and imaginary parts of the signal are correlated and learning is independent. In addition, some online sparsification criteria are compared comprehensively in the GCKAP algorithms, including the novelty criterion, the coherence criterion, and the angle criterion. Finally, two nonlinear channel equalization experiments with non-circular complex inputs are presented to illustrate the performance improvements of the proposed algorithms.

Multivariate and Online Prediction of Closing Price Using Kernel Adaptive Filtering.

This paper proposes a multivariate and online prediction of stock prices via the paradigm of kernel adaptive filtering (KAF). The prediction of stock prices in traditional classification and regression problems needs independent and batch-oriented nature of training. In this article, we challenge this existing notion of the literature and propose an online kernel adaptive filtering-based approach to predict stock prices. We experiment with ten different KAF algorithms to analyze stocks' performance and show the efficacy of the work presented here. In addition to this, and in contrast to the current literature, we look at granular level data. The experiments are performed with quotes gathered at the window of one minute, five minutes, ten minutes, fifteen minutes, twenty minutes, thirty minutes, one hour, and one day. These time windows represent some of the common windows frequently used by traders. The proposed framework is tested on 50 different stocks making up the Indian stock index: Nifty-50. The experimental results show that online learning and KAF is not only a good option, but practically speaking, they can be deployed in high-frequency trading as well.

Kernel Minimum Error Entropy Based Estimator for MIMO Radar in Non-Gaussian Clutter

In this paper, a kernel minimum error entropy (KMEE) based estimator is proposed for the estimation of multiple targets’ direction of departure (DOD), the direction of arrival (DOA), and the Doppler shift with multiple input multiple output radar in the presence of non-Gaussian clutter. Most existing estimation approaches are based on optimization of a complex cost function which often leads to a sub-optimum solution. Therefore, for the accurate estimation of DOD, DOA and Doppler shift, an efficient, kernel adaptive filter (KAF) based estimation approach is proposed. The proposed estimator utilizes the minimum error entropy (MEE) criterion and minimizes the error entropy function. The MEE, being an information-theoretic criterion, optimizes the higher-order statistics of error and thus makes the proposed estimator robust against the effects of outliers like clutter. The KMEE based estimator without any sparsification suffers from a linear increase in computational complexity. Thus, subsequently, the computational complexity of the proposed KMEE based estimator is reduced by incorporation of novelty criterion (NC) based sparsification technique, and the resulting estimator is called KMEE-NC. The performance of the proposed KMEE-NC based estimator is compared with the recently introduced sparse estimators based on kernel maximum correntropy criterion, and kernel minimum mean square error criterion. Additionally, KMEE-NC based estimator is also compared with other existing conventional estimators. Further, for assessing the accuracy of the proposed estimator, the modified Cramer-Rao lower bound is derived using the modified Fisher information matrix.

Robust Cauchy Kernel Conjugate Gradient Algorithm for Non-Gaussian Noises

The Cauchy loss (CL) is a high-order loss function which has been successfully used to overcome large outliers in kernel adaptive filters. The squared error in the CL is then transformed into a reproducing kernel Hilbert space (RKHS) to generate the Cauchy kernel loss (CKL). However, due to its non-convexity, the CKL optimized by the stochastic gradient descent (SGD) suffers from poor performance in the presence of non-Gaussian noise. To improve the performance of CKL, the kernel conjugate gradient (KCG) method combining the half-quadratic (HQ) method twice is used to transform it into a globally convex function. A novel Cauchy kernel loss conjugate gradient (CKCG) algorithm is therefore proposed in the transformed CKL. Simulations on nonlinear system identification in non-Gaussian noises confirm the superiorities of the proposed CKCG from the aspects of robustness and filtering accuracy.

General Cauchy Conjugate Gradient Algorithms Based on Multiple Random Fourier Features

The general Cauchy loss (GCL) criterion has been successfully proposed to improve the performance of the Cauchy loss (CL) criterion for linear adaptive filtering in the presence of complex non-Gaussian noise. However, the commonly used adaptive filtering algorithms based on the GCL criterion utilize the stochastic gradient descent (SGD) method to update their weights with slow convergence rate and poor steady-state performance. To overcome these issues, a general Cauchy-loss conjugate gradient (GCCG) method is first developed by solving the proposed half-quadratic general Cauchy loss (HQGCL) with the conjugate gradient method. To further tackle complex nonlinear issues, novel multiple random Fourier features (MRFF) spaces are then constructed in finite-dimensional features spaces, which is proven effective for approximation of multi-kernel adaptive filter (MKAF), theoretically. The GCCG method is thus applied into the constructed MRFF spaces to generate novel multiple random Fourier features GCCG (MRFGCG) algorithms, curbing the linear growth structures of kernel adaptive filters (KAFs) and MKAFs. The proposed MRFGCG algorithms in fixed networks have lower computational complexity and higher filtering accuracy than sparsification KAFs in non-Gaussian environments. Monte Carlo simulations on the prediction of synthetic and real-world time-series and the identification of nonlinear system confirm the superiorities of the proposed MRFGCG algorithms.

A variable-scale S-type kernel fractional low-power adaptive filtering algorithm

The adaptive kernel algorithms usually achieve a good convergence performance and a tracking performance due to the universal approximator, offering an excellent solution to many problems with nonlinearities. However, as is well known, the convergence rate and steady-state error of adaptive filtering algorithm are a pair of inherent contradictions, and the kernel method is not exceptional. For this problem, a robust kernel adaptive filtering algorithm, called the variable-scaling factor kernel fractional lower power adaptive filtering algorithm based on the Sigmoid function, is developed by creating a new framework of cost function which combines the kernel fractional low power error criterion with the Sigmoid function for system identification of different noise environments. This new cost framework incorporates a scaling factor into the cost function of the Sigmoid kernel fractional lower power adaptive filtering algorithm (VS-SKFLP) in this paper. One of the main features in the new framework is its scaling factor. This scaling factor is used to control the steepness of the Sigmoid function, and the steepness can affect the convergence speed of filtering algorithm. The scaling factor provides a tradeoff between the convergence rate and the steady-state mean square error (MSE), which improves the convergence rate under the same steady-state mean square error. However, it is also an important problem to choose an appropriate scale factor. Therefore, a variable-scale factor SKFLP algorithm is also proposed to improve the convergence rate and steady-state MSE, simultaneously. The proposed variable-scale factor structure consists of a function of error, featuring the adaptive updates of their parameter estimated by making discerning use of the error. In this paper, the nonlinear saturation characteristic of the Sigmoid function and low order norm criterion are used to overcome the performance degradation of training data destroyed by non-Gaussian impulse noise and colored noise. Through the convergence analysis, the parameter estimation sequence of our proposed algorithm proves convergent. Simulation results show that the proposed algorithm (VS-SKFLP) outperforms other kernel adaptive filtering algorithms in system recognition with different noise environments.

Kernel adaptive filtering algorithm based on Softplus function under non-Gaussian impulse interference

Kernel adaptive filters are a class of powerful nonlinear filter developed in reproducing kernel Hilbert space (RKHS).The Gaussian kernel is usually the default kernel in KAF algorithm, because the Gaussian kernel has the universal approximation. However, in previous research the kernel adaptive filtering algorithms were mostly based on mean square error criterion and assumed to be in a Gaussian noise environment. When environmental noise is changed, the performance of conventional kernel adaptive filtering algorithm based on mean square error criterion is seriously reduced to failure due to the interference of non-Gaussian noise and the influence of inappropriate non-Gaussian modeling. Therefore, it is important to develop a new method of suppressing the noise of non-Gaussian signals. In this paper, a new kernel fractional lower power adaptive filtering algorithm is proposed by combining the benefits of the kernel method and a new loss function which is robust against non-Gaussian impulsive interferences and has fast convergence under a similar stability condition. The proposed SP-KFLP algorithm generates a new framework of cost function which combines the Softplus function with the KFLP algorithm by updating its weight vector according to the gradient estimation while nonlinear saturation characteristics of output error are used. Compared with the features of sigmoid function the features of the Softplus function guarantee the SP-KFLP an excellent performance for combatting impulsive interference and speeding up the convergence rate. In the kernel fractional low power criterion the reciprocal of the system error is used as the coefficient of the weight vector update formula, and the method of error burst is used to make the weight vector not update to resist the impulse noise. The mean square convergence analysis for SP-KFLP is conducted, and a sufficient condition for guaranteeing convergence is therefore obtained by using the energy conservation relation. The proposed algorithm is very simple computationally. Simulations in a system identification show that the proposed SP-KFLP algorithm outperforms the kernel least-mean-square algorithm, kernel fractional lower power algorithm, and sigmoid kernel fractional lower algorithm in terms of convergence rate and the robustness of against impulsive interference. The proposed algorithm improves not only the capability of resisting impulsive interference, but also the convergence rate. In other words, the contradiction between convergence and tracking performance stability is well taken into account, and the performance under Gaussian noise is also better than the performance of the traditional kernel adaptive algorithm.

Recursive Minimum Kernel Mixture Mean p-Power Error Algorithm Based on the Nyström Method

In this brief, a novel kernel mixture mean $p$ -power error (KMP) criterion is proposed by combining the mixture of two Gaussian functions into the kernel function of kernel mean $p$ -power error. The mixture correntropy (MC) measure can be viewed as a special case of the KMP with $p = 2$ . And the KMP with an appropriate $p$ can provide better accuracy than MC for robust learning. Some properties of KMP are presented for discussion. The Nystrom method is an efficient method for curbing the growth of network size of kernel adaptive filters (KAFs), and the recursive update form can improve the tracking ability of KAFs. To this end, we apply the Nystrom method and recursive update form to the KMP criterion, generating a novel recursive minimum kernel mixture mean $p$ -power error algorithm based on the Nystrom method (NysRMKMP). Monte Carlo simulations on chaotic time-series prediction illustrate the desirable accuracy and robustness of NysRMKMP.

Projected Kernel Least Mean $p$ -Power Algorithm: Convergence Analyses and Modifications

Sparsified kernel adaptive filters (SKAFs) is an attractive filtering solution with low memory and computational complexity. Most of existing SKAFs are based on the mean square error (MSE) criterion under Gaussian noise assumption for its simplicity and convenience. When the assumption deviates largely from the underlying truth, the performance of these methods could degrade significantly. In this paper, we propose a novel SKAF, named as projected kernel least mean $p$ -power algorithm (PKLMP), based on the mean $p$ -power error (MPE) criterion and vector projection (VP) method. We provide convergence analyses in terms of the stead-state MSE, based on a Taylor expansion method, and derive the lower and upper bounds for the steady-state excess MSE. We also conduct mean convergence analysis for PKLMP, and derive convergence conditions. To exploit the information in the desired outputs, we further derive a modified PKLMP by smoothing the desired signal. Finally, a simple and effective online variable kernel centers strategy is proposed to improve the filtering performance of the proposed KAFs. Simulation results under a static function estimation, a chaotic time-series prediction, and two real-world time-series predictions are conducted and validate the effectiveness of the proposed PKLMP algorithms.

Anisotropic Gaussian kernel adaptive filtering by Lie-group dictionary learning.

The present paper proposes a novel kernel adaptive filtering algorithm, where each Gaussian kernel is parameterized by a center vector and a symmetric positive definite (SPD) precision matrix, which is regarded as a generalization of scalar width parameter. In fact, different from conventional kernel adaptive systems, the proposed filter is structured as a superposition of non-isotropic Gaussian kernels, whose non-isotropy makes the filter more flexible. The adaptation algorithm will search for optimal parameters in a wider parameter space. This generalization brings the need of special treatment of parameters that have a geometric structure. In fact, the main contribution of this paper is to establish update rules for precision matrices on the Lie group of SPD matrices in order to ensure their symmetry and positive-definiteness. The parameters of this filter are adapted on the basis of a least-squares criterion to minimize the filtering error, together with an ℓ1-type regularization criterion to avoid overfitting and to prevent the increase of dimensionality of the dictionary. Experimental results confirm the validity of the proposed method.

The Nyström minimum kernel risk-sensitive loss algorithm with k-means sampling

The minimum kernel risk-sensitive loss (MKRSL) algorithm has been developed to improve the filtering accuracy and robustness of kernel least mean square (KLMS) in non-Gaussian noises. However, the linear growth of network size in MKRSL leads to a huge burden on time consumption and memory requirement. To curb this growth issue, a novel Nystro¨m minimum kernel risk-sensitive loss with k-means sampling (NysMKRSL-KM) algorithm is proposed by using the Nyström method combined with k-means sampling to approximate the kernel matrix of MKRSL in this paper. The proposed NysMKRSL-KM algorithm with low time and storage complexity achieves a comparable performance to kernel adaptive filters (KAFs). In addition, the energy conservation relation and the sufficient condition of NysMKRSL-KM are obtained for performing theoretical analysis and guaranteeing the mean square convergence, respectively. The steady-state excess mean square errors (EMSEs) of NysMKRSL-KM for different noises are therefore derived for evaluating accuracy theoretically. Monte Carlo simulations are conducted to validate the theoretical analysis results and superiorities of the proposed NysMKRSL-KM algorithm.

Multikernel adaptive filtering based on random features approximation

Multikernel adaptive filters (MKAFs) have been proposed to address the selection issue of kernel parameter in single kernel based kernel adaptive filters (KAFs). However, the conventional MKAFs suffer from large memory requirements and computational burdens owing to their growing network structures. To address this issue, a random features approximation for online multikernel adaptive filtering is presented in this paper. Based on this efficient approximation, a novel robust MKAF based on the minimum kernel mean p-power error criterion, namely multiple random features kernel mean p-power (MRFKMP) algorithm, is therefore proposed in a fixed-dimensional random Fourier features space. The mean square steady-state performance of MRFKMP is derived for theoretical analysis in terms of excess mean square error (EMSE). In addition, an extension of MRFKMP is further developed in a recursive form for performance improvement. Monte Carlo simulations are conducted to validate the obtained theoretical results and the superiorities of the proposed algorithms from the aspects of computational efficiency, filtering accuracy, robustness to large outliers.

Stock returns prediction using kernel adaptive filtering within a stock market interdependence approach

Stock returns are continuously generated by different data sources and depend on various factors such as financial policies and national economic growths. Stock returns prediction, unlike traditional regression, requires consideration of both the sequential and interdependent nature of financial time-series. This work uses a two-stage approach, using kernel adaptive filtering (KAF) within a stock market interdependence approach to sequentially predict stock returns. Thus, unlike traditional KAF formulations, prediction uses not only their local models but also the individual local models learned from other stocks, enhancing prediction accuracy. The enhanced KAF plus market interdependence framework has been tested on 24 different stocks from major economies. The enhanced approach obtains higher sharpe ratio when compared with KAF-based methods, long short-term memory, and autoregressive-based models.