Kernel Adaptive Filtering Algorithm Research Articles

A Kernel Fractional Low-Power Chaotic Time Series Prediction Algorithm Based on Lncosh Function

In this paper, a kernel fraction low-power adaptive filtering algorithm based on lncosh function (KFLP-LHCF) is proposed for chaotic time series prediction. The algorithm combines the lncosh function with the kernel fractional low-power algorithm. The nonlinear saturation characteristic of the lncosh function is used to achieve resistance to impulse noise, and the fractional low power is used to suppress the influence of error mutation on the performance of the algorithm. In addition, by adjusting the scale factor, a balance can be achieved between convergence speed as well as steady-state performance. In the alpha noise environment, the proposed algorithm is used to predict and simulate two typical chaotic time series of Lorenz and Mackey–Glass. Experimental results show that the algorithm has better steady-state performance and lower steady-state error than other kernel adaptive filtering algorithms, provided that the convergence speed is similar.

Kernel Least Logarithmic Absolute Difference Algorithm

Kernel adaptive filtering (KAF) algorithms derived from the second moment of error criterion perform very well in nonlinear system identification under assumption of the Gaussian observation noise; however, they inevitably suffer from severe performance degradation in the presence of non-Gaussian impulsive noise and interference. To resolve this dilemma, we propose a novel robust kernel least logarithmic absolute difference (KLLAD) algorithm based on logarithmic error cost function in reproducing kernel Hilbert spaces, taking into account of the non-Gaussian impulsive noise. The KLLAD algorithm shows considerable improvement over the existing KAF algorithms without restraining impulsive interference in terms of robustness and convergence speed. Moreover, the convergence condition of KLLAD algorithm with Gaussian kernel and fixed dictionary is presented in the mean sense. The superior performance of KLLAD algorithm is confirmed by the simulation results.

An Online Kernel Adaptive Filtering-Based Approach for Mid-Price Prediction

The idea of multivariate and online stock price prediction via the kernel adaptive filtering (KAF) paradigm is proposed in this article. The prediction of stock prices is traditionally done with regression and classification, thereby requiring a large set of batch-oriented and independent training samples. This is problematic considering the nonstationary nature of a financial time series. In this research, we propose an online kernel adaptive filtering-based approach for stock price prediction to overcome this challenge. To examine a stock's performance and demonstrate the work's superiority, we use ten different KAF family of algorithms. In this paper, we take on this challenge and propose an approach for predicting stock prices. To analyze a stock's performance and demonstrate the work's superiority, we use ten distinct KAF algorithms. Besides, the results are analyzed on nine-time windows such as one day, sixty minutes, thirty minutes, twenty five minutes, twenty minutes, fifteen minutes, ten minutes, five minutes, and one minute. We are the first to experiment with several time windows for all fifty stocks on the Indian National Stock Exchange, to the best of our knowledge. It should be noted here that the experiments are performed on stocks making up the main index: Nifty-50. In terms of performance and compared to existing methods, we have a 66% probability of correctly predicting a stock's next upward or downward movement. This number clearly shows the edge that the proposed method has in actual deployment. Furthermore, the experimental findings show that KAF is not only a better option for predicting stock prices but that it may also be used as an alternative in high-frequency trading due to its low latency.

A family of split kernel adaptive filtering algorithms for nonlinear stereophonic acoustic echo cancellation

A family of split kernel adaptive filtering algorithms for nonlinear stereophonic acoustic echo cancellation

Multivariate and Online Prediction of Closing Price Using Kernel Adaptive Filtering.

This paper proposes a multivariate and online prediction of stock prices via the paradigm of kernel adaptive filtering (KAF). The prediction of stock prices in traditional classification and regression problems needs independent and batch-oriented nature of training. In this article, we challenge this existing notion of the literature and propose an online kernel adaptive filtering-based approach to predict stock prices. We experiment with ten different KAF algorithms to analyze stocks' performance and show the efficacy of the work presented here. In addition to this, and in contrast to the current literature, we look at granular level data. The experiments are performed with quotes gathered at the window of one minute, five minutes, ten minutes, fifteen minutes, twenty minutes, thirty minutes, one hour, and one day. These time windows represent some of the common windows frequently used by traders. The proposed framework is tested on 50 different stocks making up the Indian stock index: Nifty-50. The experimental results show that online learning and KAF is not only a good option, but practically speaking, they can be deployed in high-frequency trading as well.

A quantized adaptive algorithm based on the q-Rényi kernel function

In this paper, we proposed a novel online kernel-adaptive learning algorithm under the reproducing-kernel-Hilbert space (RKHS), which is called kernel q-Rényi (KqR) algorithm. The KqR algorithm is derived via constructing a q-Rényi kernel function as a cost function into kernel adaptive filtering algorithm (KAF). The vector-quantization (VQ) method is wildly used in the KAF algorithms, which is called quantized KAF. The quantized kernel q-Rényi (QKqR) algorithm is also proposed to significantly reduce computational cost via quantizing input space to curb the size of neural networks growth. The mean-square-convergence analysis of the QKqR algorithm is established, and the theoretical-value of the excess-mean-square-error (EMSE) are presented. The performance of the KqR and QKqR algorithms are investigated and demonstrated via tracking the Mackey-Glass (MG) time-series-prediction and the nonlinear channel equalization (NCE) under the non-Gaussian background interferences.

A variable-scale S-type kernel fractional low-power adaptive filtering algorithm

The adaptive kernel algorithms usually achieve a good convergence performance and a tracking performance due to the universal approximator, offering an excellent solution to many problems with nonlinearities. However, as is well known, the convergence rate and steady-state error of adaptive filtering algorithm are a pair of inherent contradictions, and the kernel method is not exceptional. For this problem, a robust kernel adaptive filtering algorithm, called the variable-scaling factor kernel fractional lower power adaptive filtering algorithm based on the Sigmoid function, is developed by creating a new framework of cost function which combines the kernel fractional low power error criterion with the Sigmoid function for system identification of different noise environments. This new cost framework incorporates a scaling factor into the cost function of the Sigmoid kernel fractional lower power adaptive filtering algorithm (VS-SKFLP) in this paper. One of the main features in the new framework is its scaling factor. This scaling factor is used to control the steepness of the Sigmoid function, and the steepness can affect the convergence speed of filtering algorithm. The scaling factor provides a tradeoff between the convergence rate and the steady-state mean square error (MSE), which improves the convergence rate under the same steady-state mean square error. However, it is also an important problem to choose an appropriate scale factor. Therefore, a variable-scale factor SKFLP algorithm is also proposed to improve the convergence rate and steady-state MSE, simultaneously. The proposed variable-scale factor structure consists of a function of error, featuring the adaptive updates of their parameter estimated by making discerning use of the error. In this paper, the nonlinear saturation characteristic of the Sigmoid function and low order norm criterion are used to overcome the performance degradation of training data destroyed by non-Gaussian impulse noise and colored noise. Through the convergence analysis, the parameter estimation sequence of our proposed algorithm proves convergent. Simulation results show that the proposed algorithm (VS-SKFLP) outperforms other kernel adaptive filtering algorithms in system recognition with different noise environments.

Anisotropic Gaussian kernel adaptive filtering by Lie-group dictionary learning.

The present paper proposes a novel kernel adaptive filtering algorithm, where each Gaussian kernel is parameterized by a center vector and a symmetric positive definite (SPD) precision matrix, which is regarded as a generalization of scalar width parameter. In fact, different from conventional kernel adaptive systems, the proposed filter is structured as a superposition of non-isotropic Gaussian kernels, whose non-isotropy makes the filter more flexible. The adaptation algorithm will search for optimal parameters in a wider parameter space. This generalization brings the need of special treatment of parameters that have a geometric structure. In fact, the main contribution of this paper is to establish update rules for precision matrices on the Lie group of SPD matrices in order to ensure their symmetry and positive-definiteness. The parameters of this filter are adapted on the basis of a least-squares criterion to minimize the filtering error, together with an ℓ1-type regularization criterion to avoid overfitting and to prevent the increase of dimensionality of the dictionary. Experimental results confirm the validity of the proposed method.

Kernel Mixture Correntropy Conjugate Gradient Algorithm for Time Series Prediction.

Kernel adaptive filtering (KAF) is an effective nonlinear learning algorithm, which has been widely used in time series prediction. The traditional KAF is based on the stochastic gradient descent (SGD) method, which has slow convergence speed and low filtering accuracy. Hence, a kernel conjugate gradient (KCG) algorithm has been proposed with low computational complexity, while achieving comparable performance to some KAF algorithms, e.g., the kernel recursive least squares (KRLS). However, the robust learning performance is unsatisfactory, when using KCG. Meanwhile, correntropy as a local similarity measure defined in kernel space, can address large outliers in robust signal processing. On the basis of correntropy, the mixture correntropy is developed, which uses the mixture of two Gaussian functions as a kernel function to further improve the learning performance. Accordingly, this article proposes a novel KCG algorithm, named the kernel mixture correntropy conjugate gradient (KMCCG), with the help of the mixture correntropy criterion (MCC). The proposed algorithm has less computational complexity and can achieve better performance in non-Gaussian noise environments. To further control the growing radial basis function (RBF) network in this algorithm, we also use a simple sparsification criterion based on the angle between elements in the reproducing kernel Hilbert space (RKHS). The prediction simulation results on a synthetic chaotic time series and a real benchmark dataset show that the proposed algorithm can achieve better computational performance. In addition, the proposed algorithm is also successfully applied to the practical tasks of malware prediction in the field of malware analysis. The results demonstrate that our proposed algorithm not only has a short training time, but also can achieve high prediction accuracy.

Variable learning rates kernel adaptive filter with single feedback

In this paper, we propose a novel kernel adaptive filtering algorithm, which called variable learning rates kernel adaptive filter with single feedback (SF-VLRKAF). Based on a feedback structure, the past information can be used to estimate current output to improve the filtering performance, because of a momentum term existing in the weight update equation. A switch ON–OFF normalized variable learning rate is developed to obtain a tradeoff between convergence rate and filtering accuracy of the proposed algorithm. The weights, in the SF-VLRKAF, are updated at each iteration to avoid local minimum, and the analysis of mean square convergence is performed. Furthermore, a sufficient condition ensuring mean square convergence is obtained by applying the energy conservation relation. Moreover, we derive the lower and upper bounds on a theoretical result of the steady-state excess mean square error. Simulations for nonlinear regression, chaotic time-series predictions and real-world applications are presented to illustrate the effectiveness of the new algorithm.

Parsimonious Kernel Recursive Least Squares Algorithm for Aero-Engine Health Diagnosis

Kernel adaptive filtering (KAF) has gained widespread popularity among the machine learning community for online applications due to its convexity, simplicity, and universal approximation ability. However, the network generated by KAF keeps growing with the accumulation of the training samples, which leads to the increasing memory requirement and computational burden. To address this issue, a pruning approach that attempts to restrict the network size to a fixed value is incorporated into a kernel recursive least squares (KRLS) algorithm, yielding a novel KAF algorithm called parsimonious KRLS (PKRLS). The basic idea of the pruning technique is to remove the center with the least importance from the existing dictionary. The importance of a center is quantified by its contribution to minimizing the cost function. The calculation of the importance measure is formulated in an efficient manner, which facilitates its implementation in online settings. Experimental results on the benchmark tasks show that PKRLS obtains a parsimonious network structure with the satisfactory prediction accuracy. Finally, a multi-sensor health diagnosis approach based on PKRLS is developed for identifying the health state of a degraded aero-engine in real time. A case study in a turbofan engine degradation data set demonstrates that PKRLS provides an effective and efficient candidate for modeling the performance deterioration of real complex systems.

Kernel recursive generalized mixed norm algorithm

This work studies the problem of kernel adaptive filtering (KAF) for nonlinear signal processing under non-Gaussian noise environments. A new KAF algorithm, called kernel recursive generalized mixed norm (KRGMN), is derived by minimizing the generalized mixed norm (GMN) cost instead of the well-known mean square error (MSE). A single error norm such as lp error norm can be used as a cost function in KAF to deal with non-Gaussian noises but it may exhibit slow convergence speed and poor misadjustments in some situations. To improve the convergence performance, the GMN cost is formed as a convex mixture of lp and lq norms to increase the convergence rate and substantially reduce the steady-state errors. The proposed KRGMN algorithm can solve efficiently the problems such as nonlinear channel equalization and system identification in non-Gaussian noises. Simulation results confirm the desirable performance of the new algorithm.

A regularized estimation framework for online sparse LSSVR models

Aiming at machine learning applications in which fast online learning is required, we develop a variant of the Least Squares SVR (LSSVR) model that can learn incrementally from data and eventually provide a sparse solution vector. This is possible by incorporating into the LSSVR model the sparsification mechanism used by the kernel RLS (KRLS) model introduced in Engel et al., 2004. The performance of the resulting model, henceforth referred to as the online sparse LSSVR (OS-LSSVR) model, is comprehensively evaluated by computer experiments on several benchmarking datasets (including a large scale one) covering a number of challenging tasks in nonlinear time series prediction and system identification. Convergence, efficiency and error bounds of the OS-LSSVR model are also addressed. The results indicate that the proposed approach consistently outperforms the state of the art in kernel adaptive filtering algorithms, by providing more sparse solutions with smaller prediction errors and smaller norms for the solution vector.

Kernel Least Mean Square with Single Feedback

In this letter, a novel kernel adaptive filtering algorithm, namely the kernel least mean square with single feedback (SF-KLMS) algorithm, is proposed. In SF-KLMS, only a single delayed output is used to update the weights in a recurrent fashion. The use of past information accelerates the convergence rate significantly. Compared with the kernel adaptive filter using multiple feedback, SF-KLMS has a more compact and efficient structure. Simulations in the context of time-series prediction and nonlinear regression show that SF-KLMS outperforms not only the kernel adaptive filter with multiple feedback but also the kernel adaptive filter without feedback in terms of convergence rate and mean square error.

Kernel recursive least-squares tracker for time-varying regression.

In this paper, we introduce a kernel recursive least-squares (KRLS) algorithm that is able to track nonlinear, time-varying relationships in data. To this purpose, we first derive the standard KRLS equations from a Bayesian perspective (including a sensible approach to pruning) and then take advantage of this framework to incorporate forgetting in a consistent way, thus enabling the algorithm to perform tracking in nonstationary scenarios. The resulting method is the first kernel adaptive filtering algorithm that includes a forgetting factor in a principled and numerically stable manner. In addition to its tracking ability, it has a number of appealing properties. It is online, requires a fixed amount of memory and computation per time step, incorporates regularization in a natural manner and provides confidence intervals along with each prediction. We include experimental results that support the theory as well as illustrate the efficiency of the proposed algorithm.