Kernel Recursive Least Squares Algorithm Research Articles

Sparse Sliding-Window Kernel Recursive Least-Squares Channel Prediction for Fast Time-Varying MIMO Systems.

Accurate channel state information (CSI) is important for MIMO systems, especially in a high-speed scenario, fast time-varying CSI tends to be out of date, and a change in CSI shows complex nonlinearities. The kernel recursive least-squares (KRLS) algorithm, which offers an attractive framework to deal with nonlinear problems, can be used in predicting nonlinear time-varying CSI. However, the network structure of the traditional KRLS algorithm grows as the training sample size increases, resulting in insufficient storage space and increasing computation when dealing with incoming data, which limits the online prediction of the KRLS algorithm. This paper proposed a new sparse sliding-window KRLS (SSW-KRLS) algorithm where a candidate discard set is selected through correlation analysis between the mapping vectors in the kernel Hilbert spaces of the new input sample and the existing samples in the kernel dictionary; then, the discarded sample is determined in combination with its corresponding output to achieve dynamic sample updates. Specifically, the proposed SSW-KRLS algorithm maintains the size of the kernel dictionary within the sample budget requires a fixed amount of memory and computation per time step, incorporates regularization, and achieves online prediction. Moreover, in order to sufficiently track the strongly changeable dynamic characteristics, a forgetting factor is considered in the proposed algorithm. Numerical simulations demonstrate that, under a realistic channel model of 3GPP in a rich scattering environment, our proposed algorithm achieved superior performance in terms of both predictive accuracy and kernel dictionary size than that of the ALD-KRLS algorithm. Our proposed SSW-KRLS algorithm with achieved 2 dB NMSE less than that of the ALD-KRLS algorithm with , while the kernel dictionary was about 17% smaller when the speed of the mobile user was 120 km/h.

Management of Distributed Renewable Energy Resources with the Help of a Wireless Sensor Network

Photovoltaic (PV) and wind energy are widely considered eco-friendly renewable energy resources. However, due to the unpredictable oscillations in solar and wind power production, efficient management to meet load demands is often hard to achieve. As a result, precise forecasting of PV and wind energy production is critical for grid managers to limit the impact of random fluctuations. In this study, the kernel recursive least-squares (KRLS) algorithm is proposed for the prediction of PV and wind energy. The wireless sensor network (WSN) typically adopted for data collection with a flexible configuration of sensor nodes is used to transport PV and wind production data to the monitoring center. For efficient transmission of the data production, a link scheduling technique based on sensor node attributes is proposed. Different statistical and machine learning (ML) techniques are examined with respect to the proposed KRLS algorithm for performance analysis. The comparison results show that the KRLS algorithm surpasses all other regression approaches. For both PV and wind power feed-in forecasts, the proposed KRLS algorithm demonstrates high forecasting accuracy. In addition, the link scheduling proposed for the transmission of data for the management of distributed renewable energy resources is compared with a reference technique to show its comparable performance. The efficacy of the proposed KRLS model is better than other regression models in all assessment events in terms of an RMSE value of 0.0146, MAE value of 0.00021, and R2 of 99.7% for PV power, and RMSE value of 0.0421, MAE value of 0.0018, and R2 of 88.17% for wind power. In addition to this, the proposed link scheduling approach results in 22% lower latency and 38% higher resource utilization through the efficient scheduling of time slots.

Random Fourier feature kernel recursive maximum mixture correntropy algorithm for online time series prediction

In the paper, a novel kernel recursive least-squares (KRLS) algorithm named random Fourier feature kernel recursive maximum mixture correntropy (RFF-RMMC) algorithm is proposed, which improves the prediction efficiency and robustness of the KRLS algorithm. Random Fourier feature (RFF) method as well as maximum mixture correntropy criterion (MMCC) are combined and applied into KRLS algorithm afterwards. Using RFF to approximate the kernel function in KRLS with a fixed cost can greatly reduce the computational complexity and simultaneously improve the prediction efficiency. In addition, the MMCC maintains the robustness like the maximum correntropy criterion (MCC). More importantly, it can enhance the accuracy of the similarity measurement between predicted and true values by more flexible parameter settings, and then make up for the loss of prediction accuracy caused by RFF to a certain extent. The performance of the RFF-RMMC algorithm for online time series prediction is verified by the simulation results based on three datasets.

Deep kernel recursive least-squares algorithm

We present a new kernel-based algorithm for modeling evenly distributed multidimensional datasets that does not rely on input space sparsification. The presented method reorganizes the typical single-layer kernel-based model into a deep hierarchical structure, such that the weights of a kernel model over each dimension are modeled over its adjacent dimension. We show that modeling weights in the suggested structure leads to significant computational speedup and improved modeling accuracy.

Quaternion kernel recursive least-squares algorithm

Various kernel-based algorithms have been successfully applied to nonlinear problems in adaptive filters over the last two decades. In this paper, we study a kernel recursive least squares (KRLS) algorithm in the quaternion domain. By the generalized Hamilton-real calculus method, we can apply the kernel trick to calculate the quaternion KRLS filter. In order to show the feasibility of the proposed algorithm, firstly we investigate the quaternion recursive least squares (QRLS) algorithm, and simulations show that the proposed QRLS algorithm has the same steady error as that of the closed-form solution; Secondly, we generalize the QRLS algorithm to the quaternion KRLS algorithm, theoretical analysis show the convergence, and simulations are described demonstrating the performance of the proposed algorithm.

Improved Kernel Recursive Least Squares Algorithm Based Online Prediction for Nonstationary Time Series

We present an improved kernel recursive least squares (KRLS) algorithm for the online prediction of nonstationary time series. In order to adaptively sparsify a selected kernel dictionary for the KRLS algorithm, the approximate linear dependency (ALD) criterion based KRLS algorithm is combined with the quantized kernel recursive least squares algorithm to provide an initial framework. In order to sufficiently track the strongly changeable dynamic characteristics due to nonstationarity, a forgetting factor is further inserted into the proposed combined algorithm. It is shown that our proposed algorithm, referred to as the FFIKRLS algorithm, provides a clearly compatible algorithm structure and can be improved by the existing modeling techniques from both mapping and weights updating perspectives. Numerical simulations using benchmark Lorenz time series in comparison with existing methods have demonstrated that the proposed algorithm has superior performance in terms of both predictive accuracy and kernel dictionary size.

Gait Phase Classification and Assist Torque Prediction for a Lower Limb Exoskeleton System Using Kernel Recursive Least-Squares Method.

The gait phase classification method is a key technique to control an exoskeleton robot. Different people have different gait features while wearing an exoskeleton robot due to the gap between the exoskeleton and the wearer and their operation habits, such as the correspondence between the joint angle and the moment at which the foot contacts the ground, the amplitude of the joint angle and others. In order to enhance the performance of the gait phase classification in an exoskeleton robot using only the angle of hip and knee joints, a kernel recursive least-squares (KRLS) algorithm is introduced to build a gait phase classification model. We also build an assist torque predictor based on the KRLS algorithm in this work considering the adaptation of unique gait features. In this paper, we evaluate the classification performance of the KRLS model by comparing with two other commonly used gait recognition methods—the multi-layer perceptron neural network (MLPNN) method and the support vector machine (SVM) algorithm. In this experiment, the training and testing datasets for the models built by KRLS, MLPNN and SVM were collected from 10 healthy volunteers. The gait data are collected from the exoskeleton robot that we designed rather than collected from the human body. These data depict the human-robot coupling gait that includes unique gait features. The KRLS classification results are in average 3% higher than MLPNN and SVM. The testing average accuracy of KRLS is about 86%. The prediction results of KRLS are twice as good as MLPNN in assist torque prediction experiments. The KRLS performs in a good, stable, and robust way and shows model generalization abilities.

The Kernel Conjugate Gradient Algorithms

Kernel methods have been successfully applied to nonlinear problems in machine learning and signal processing. Various kernel-based algorithms have been proposed over the last two decades. In this paper, we investigate the kernel conjugate gradient (KCG) algorithms in both batch and online modes. By expressing the solution vector of CG algorithm as a linear combination of the input vectors and using the kernel trick, we developed the KCG algorithm for batch mode. Because the CG algorithm is iterative in nature, it can greatly reduce the computations by the technique of reduced-rank processing. Moreover, the reduced-rank processing can provide the robustness against the problem of overlearning. The online KCG algorithm is also derived, which converges as fast as the kernel recursive least squares (KRLS) algorithm, but the computational cost is only a quarter of that of the KRLS algorithm. Another attractive feature of the online KCG algorithm compared with other kernel adaptive algorithms is that it does not require the user-defined parameters. To control the growth of data size in online applications, a simple sparsification criterion based on the angles among elements in reproducing kernel Hilbert space is proposed. The angle criterion is equivalent to the coherence criterion but does not require the kernel to be unit norm. Finally, numerical experiments are provided to illustrate the effectiveness of the proposed algorithms.

Echo state kernel recursive least squares algorithm for machine condition prediction

Abstract Kernel adaptive filter (KAF) has been widely utilized for time series prediction due to its online adaptation scheme, universal approximation capability and convexity. Nevertheless, KAF’s ability to handle temporal tasks is limited, because it is essentially a feed-forward neural network that lacks dynamic characteristics. Traditionally, a sliding widow that contains consecutive data points is constructed to deal with the temporal dependency between data points at neighboring time steps, but the restricted widow length may be incapable of capturing temporal patterns on a larger time scale. To manage this issue, a novel sequential learning approach called echo state KRLS (ES-KRLS) algorithm is proposed by incorporating a dynamic reservoir into kernel recursive least squares (KRLS) algorithm. The reservoir, consisting of a large number of sparsely interconnected hidden units, is treated as a temporal function that transforms the history of the time series into a high-dimensional reservoir state space. Subsequently, the spatial relationship between the reservoir state and the target output is effectively approximated by KRLS algorithm. With the utilization of the fixed reservoir, our novel method not only maintains the simplicity of the learning process but also leads to a significant improvement in the capability of modeling dynamic systems. Numerical results on benchmark tasks demonstrate the excellent performance of the novel method with respect to long-term prediction. Finally, an online prognostic method that combines ES-KRLS and a Bayesian technique is developed for tracking the health status of a degraded system and predicting remaining useful life (RUL). This prognostic method is applied to a turbofan engine degradation dataset to demonstrate its effectiveness.

Multivariate Chaotic Time Series Online Prediction Based on Improved Kernel Recursive Least Squares Algorithm.

Kernel recursive least squares (KRLS) is a kind of kernel methods, which has attracted wide attention in the research of time series online prediction. It has low computational complexity and updates in a recursive form. However, as data size increases, computational complexity of calculating kernel inverse matrix will raise. And it has some difficulties in accommodating time-varying environments. Therefore, we have presented an improved KRLS algorithm for multivariate chaotic time series online prediction. Approximate linear dependency, dynamic adjustment, and coherence criterion are combined with quantization to form our improved KRLS algorithm. In the process of online prediction, it can bring computational efficiency up and adjust weights adaptively in time-varying environments. Moreover, Lorenz chaotic time series, El Nino-Southern Oscillation indexes chaotic time series, yearly sunspots and runoff of the Yellow River chaotic time series online prediction are presented to prove the effectiveness of our proposed algorithm.

An outlier-robust kernel RLS algorithm for nonlinear system identification

The kernel recursive least squares (KRLS), a nonlinear counterpart of the famed RLS algorithm, performs linear regression in a high-dimensional feature space induced by a Mercer kernel. Despite the growing interest in the KRLS for nonlinear signal processing, the presence of outliers in the estimation data causes the resulting predictor’s performance to deteriorate considerably. Bearing this in mind, we introduce an approach to amalgamate the kernel-based learning framework that gives rise to the KRLS algorithm with the robust regression framework of M-estimators with the aim of building an outlier-robust variant for the KRLS. Initially, we develop the theoretical aspects of the proposed algorithm and then analyze its behavior in nonlinear system identification problems using synthetic and real-world datasets (including a large-scale one) contaminated with outliers. The obtained results indicate that the robust variant of the KRLS algorithm consistently outperforms the state-of-the-art in robust adaptive filtering algorithms, as the amount of outliers in the data increases.

Reinforcement Learning with Kernel Recursive Least-Squares Support Vector Machine

A reinforcement learning system based on the kernel recursive least-squares algorithm for continuous state-space is proposed in this paper. A kernel recursive least-squares- support vector machine is used to realized a mapping from state-action pair to Q-value function. An online sparsification process that permits the addition of training sample into the Q-function approximation only if it is approximately linearly independent of the preceding training samples. Simulation result of two-link robot manipulator show that the proposed method has high learning efficiency - better accuracy measured in terms of mean square error, and lesser computation time compare to the least-squares support vector machine. I. I NTRODUCTION Support vector machine (SVM), which is based on Vapnik's structural risk minimization (SRM) (1), has become one of the most popular methods in solving classification and regression problems. Conventional SVMs have properties of global optimization, and good adaptability. However, the optimal solutions are obtained by solving standard quadratic programming which results in high computational cost. In order to reduce the computational cost, a least square support vector (LS-SVM) was proposed in (2) by converting inequality constraints to linear equations. LS-SVM has been successfully applied to reinforcement learning (RL) problems (3). A RL problem is converted into a regression problem, wherein the observed states and actions are considered as inputs and Q-value functions as output. All the training samples may be support vectors in LS-SVM, and thus the support vectors are no longer sparse. It may lead to poor generalization. In addition, with the number of the input training pairs increasing, the number of equations will increase which may result in higher computational cost. Our focus in this paper is on reinforcement learning problems. The objective in hand is to explore the use of a support vector machine with sparse support vectors, low computational cost and satisfactory accuracy. The kernel recursive least-squares (KRLS) - SVM (4), is a strong candidate to achieve this objective. We develop a KRLS-SVM algorithm for reinforcement learning and demonstrate its potential through case study - two-link robot manipulator. The mean square error accuracy, computational

Nonlinear System Identification using a New Sliding-Window Kernel RLS Algorithm

In this paper we discuss in detail a recently proposed kernel-based version of the recursive least-squares (RLS) algorithm for fast adaptive nonlinear filtering. Unlike other previous approaches, the studied method combines a sliding-window approach (to fix the dimensions of the kernel matrix) with conventional ridge regression (to improve generalization). The resulting kernel RLS algorithm is applied to several nonlinear system identification problems. Experiments show that the proposed algorithm is able to operate in a time-varying environment and to adjust to abrupt changes in either the linear filter or the nonlinearity.