Kernel Recursive Least Squares Research Articles

KRLS post-distorter with adaptive kernel width for visible light communications.

As a viable supplement to the fifth generation wireless communication, visible light communications (VLC) with affluent spectrum resources can cater to the ever-increasing high speed data transmission demand. However, the nonlinear characteristics of light emitting diode (LED) can distort the transmitted signal in the VLC link, which damages the communication quality. To mitigate the nonlinear impairments, a reproducing kernel Hilbert space post-distortion scheme is proposed in this paper, which is based on kernel recursive least squares (KRLS) with adaptive kernel width. In this kernel based method, the kernel width will affect the approximation ability of the model. Therefore, in the recursive process of KRLS, Gauss-Newton (GN) algorithm is adopted to update the kernel width. In addition, combined with the enhanced novelty criterion (ENC), the KRLS-GN post-distorter learns the sparse dictionary adaptively according to the input data, which is beneficial to complete the linearization under the limited memory budget constraints. The performance of the proposed KRLS-GN-ENC scheme is verified by simulations, and the results show that KRLS-GN-ENC can achieve a significant improvement over KRLS-ENC. Compared with the schemes based on classical polynomial filtering, KRLS-GN-ENC exhibits better nonlinear compensation performance and faster convergence speed.

Quantized kernel recursive minimum error entropy algorithm

In this paper, we propose a online vector quantization (VQ) method based on the kernel recursive minimum error entropy (KRMEE) algorithm. According to information theoretic learning (ITL), the minimum error entropy criterion (MEE) is robust and can effective resistance to non-Gaussian noise. By combining the kernel recursive least squares (KRLS) algorithm with MEE criterion, KRMEE algorithm has been generated, which has excellent performance in non-Gaussian environments. However, with the size of data increases, the computational complexity will raise. We propose a quantized to solve this problem, the input space of the algorithm is quantized to suppress the linear growth radial basis function (RBF) network in kernel adaptive filtering (KAF). The VQ method is different from novelty criterion (NC), approximate linear dependency (ALD) criterion, and other sparsity methods, the online VQ method need to construct the dictionary, and calculate the distance by Euclidean norm. We propose a novel quantized kernel recursive minimum error entropy (QKRMEE) algorithm by combining VQ method with KRMEE algorithm, and update the solution with a recursive algorithm. In Mackey-Glass time series and a real-world datasets, Monte Carlo simulation experiments show that the proposed algorithm achieves better predictive performance in non-Gaussian noise environment. Meanwhile, the algorithm can restrain the growth of RBF network well, thus reducing the computational complexity and memory consumption effectively.

The Kernel Recursive Maximum Total Correntropy Algorithm

Focusing on the performance deterioration of nonlinear filtering algorithms in the error-in-variable (EIV) model, a nonlinear version of maximum total correntropy (MTC) algorithm called kernel recursive maximum total correntropy (KRMTC) is proposed in this brief, which constructs a nonlinear relationship based on MTC criterion between inputs and outputs in high-dimensional feature space. Compared with existing kernel adaptive filtering (KAF) algorithms based on traditional mean square error (MSE) and maximum correntropy criteria (MCC), the proposed algorithm can handle nonlinear modeling problems in the EIV model well. In addition, an approximate linear dependency (ALD) criterion is applied to reduce the computational complexity of KRMTC. To verify the feasibility of the proposed algorithm, firstly, the convergence of the KRMTC algorithm under some assumptions is discussed and its local convergence condition is given. Secondly, the proposed algorithm is applied to the Lorenz time-series prediction and wind prediction simulations. Simulation results display that the KRMTC algorithm has better performance than kernel recursive least squares (KRLS) and kernel recursive maximum correntropy (KRMC) algorithms in EIV case.

Generalized minimum error entropy for robust learning

The applications of error entropy (EE) are sometimes limited because its shape cannot be flexibly adjusted by the default Gaussian kernel function to adapt to noise variation and thus lowers the performance of algorithms based on minimum error entropy (MEE) criterion. In this paper, a generalized EE (GEE) is proposed by introducing the generalized Gaussian density (GGD) as its kernel function to improve the robustness of EE. In addition, GEE can be further improved to reduce its computational load by the quantized GEE (QGEE). Furthermore, two learning criteria, called generalized minimum error entropy (GMEE) and quantized generalized minimum error entropy (QGMEE), are developed based on GEE and QGEE, and new adaptive filtering (AF), kernel recursive least squares (KRLS), and multilayer perceptron (MLP) based on the proposed criteria are presented. Several numerical simulations indicate that the performance of proposed algorithms performs better than that of algorithms based on MEE.

Design of data-driven model for the pressurizer system in nuclear power plants using a TSK fuzzy neural network

In pressurized water reactors (PWRs), the main role of the pressurizer unit is to maintain the primary circuit pressure of the reactor within preset limits. To ensure this role, the pressure and the water level control systems for the pressurizer system are very significant. Hence, it is very important to monitor and predict them with great accuracy during operation. In this paper, a data-driven model using a TSK fuzzy neural network (TSKFNN) for the prediction of the pressure and the water level inside the pressurizer is developed. The identification task of the TSKFNN model of the pressurizer system is fulfilled in two steps: the structure and the parameters identification. In the first step, the fuzzy C-means clustering method is used to separate the input data into clusters and obtain the rule antecedent parameters of the TSKFNN model. Besides, the initial values of the rule consequent parameters are defined using the kernel ridge regression algorithm. In the second step, the sliding-window Kernel Recursive Least Squares (KRLS) algorithm and the gradient method are developed to adapt TSKFNN model parameters. By using input/ output data collected from the PCTran VVER-1200 simulator, the data-driven model was trained, tested, and evaluated. The simulation results demonstrate that the data-driven model using the TSKFNN structure can effectively predict the pressure and the water level in the pressurizer system. Furthermore, the simulation results also involve a comparison with two other mathematical models to confirm the effectiveness of the developed data-driven model.

Sparse Sliding-Window Kernel Recursive Least-Squares Channel Prediction for Fast Time-Varying MIMO Systems.

Accurate channel state information (CSI) is important for MIMO systems, especially in a high-speed scenario, fast time-varying CSI tends to be out of date, and a change in CSI shows complex nonlinearities. The kernel recursive least-squares (KRLS) algorithm, which offers an attractive framework to deal with nonlinear problems, can be used in predicting nonlinear time-varying CSI. However, the network structure of the traditional KRLS algorithm grows as the training sample size increases, resulting in insufficient storage space and increasing computation when dealing with incoming data, which limits the online prediction of the KRLS algorithm. This paper proposed a new sparse sliding-window KRLS (SSW-KRLS) algorithm where a candidate discard set is selected through correlation analysis between the mapping vectors in the kernel Hilbert spaces of the new input sample and the existing samples in the kernel dictionary; then, the discarded sample is determined in combination with its corresponding output to achieve dynamic sample updates. Specifically, the proposed SSW-KRLS algorithm maintains the size of the kernel dictionary within the sample budget requires a fixed amount of memory and computation per time step, incorporates regularization, and achieves online prediction. Moreover, in order to sufficiently track the strongly changeable dynamic characteristics, a forgetting factor is considered in the proposed algorithm. Numerical simulations demonstrate that, under a realistic channel model of 3GPP in a rich scattering environment, our proposed algorithm achieved superior performance in terms of both predictive accuracy and kernel dictionary size than that of the ALD-KRLS algorithm. Our proposed SSW-KRLS algorithm with achieved 2 dB NMSE less than that of the ALD-KRLS algorithm with , while the kernel dictionary was about 17% smaller when the speed of the mobile user was 120 km/h.

Management of Distributed Renewable Energy Resources with the Help of a Wireless Sensor Network

Photovoltaic (PV) and wind energy are widely considered eco-friendly renewable energy resources. However, due to the unpredictable oscillations in solar and wind power production, efficient management to meet load demands is often hard to achieve. As a result, precise forecasting of PV and wind energy production is critical for grid managers to limit the impact of random fluctuations. In this study, the kernel recursive least-squares (KRLS) algorithm is proposed for the prediction of PV and wind energy. The wireless sensor network (WSN) typically adopted for data collection with a flexible configuration of sensor nodes is used to transport PV and wind production data to the monitoring center. For efficient transmission of the data production, a link scheduling technique based on sensor node attributes is proposed. Different statistical and machine learning (ML) techniques are examined with respect to the proposed KRLS algorithm for performance analysis. The comparison results show that the KRLS algorithm surpasses all other regression approaches. For both PV and wind power feed-in forecasts, the proposed KRLS algorithm demonstrates high forecasting accuracy. In addition, the link scheduling proposed for the transmission of data for the management of distributed renewable energy resources is compared with a reference technique to show its comparable performance. The efficacy of the proposed KRLS model is better than other regression models in all assessment events in terms of an RMSE value of 0.0146, MAE value of 0.00021, and R2 of 99.7% for PV power, and RMSE value of 0.0421, MAE value of 0.0018, and R2 of 88.17% for wind power. In addition to this, the proposed link scheduling approach results in 22% lower latency and 38% higher resource utilization through the efficient scheduling of time slots.

Kernel Recursive Maximum Versoria Criterion Based Post-Distorter for VLC Using Kernel-Width Sampling

Visible light communication (VLC) has emerged as a potential candidate for next generation wireless communication systems. However, nonlinear characteristics of light emitting diode (LED), user-mobility, and DC-bias fluctuations are the major factors that limit the throughput of a VLC link, and makes the overall additive distortion as non-Gaussian distributed. To mitigate this non-Gaussian noise processes encountered in VLC systems due to LED nonlinearity, and user-mobility, recently a random Fourier features (RFF) based kernel recursive maximum Versoria criterion (KRMVC) based post-distortion algorithm is proposed, which delivers better performance as compared to the classical polynomial series, and kernel recursive least squares (KRLS) algorithms due to the incorporation of higher order statistics of error. However, the performance of RFF-KRMVC algorithm is sensitive to the choice of kernel-width, and results in approximation errors due to imperfect choice of kernel-width. This paper proposes a novel RFF-KRMVC algorithm using a kernel-width sampling (KWS) technique called as RFF-KWS-KRMVC, which implements the post-distortion under a hyperparameter-free finite memory budget. Furthermore, analytical expressions for mean square error, and error rate are quantified for the proposed RFF-KWS-KRMVC post-distorter, and corroborated by Monte-Carlo simulations performed over standard VLC channel models.

Random Fourier feature kernel recursive maximum mixture correntropy algorithm for online time series prediction

In the paper, a novel kernel recursive least-squares (KRLS) algorithm named random Fourier feature kernel recursive maximum mixture correntropy (RFF-RMMC) algorithm is proposed, which improves the prediction efficiency and robustness of the KRLS algorithm. Random Fourier feature (RFF) method as well as maximum mixture correntropy criterion (MMCC) are combined and applied into KRLS algorithm afterwards. Using RFF to approximate the kernel function in KRLS with a fixed cost can greatly reduce the computational complexity and simultaneously improve the prediction efficiency. In addition, the MMCC maintains the robustness like the maximum correntropy criterion (MCC). More importantly, it can enhance the accuracy of the similarity measurement between predicted and true values by more flexible parameter settings, and then make up for the loss of prediction accuracy caused by RFF to a certain extent. The performance of the RFF-RMMC algorithm for online time series prediction is verified by the simulation results based on three datasets.

Deep kernel recursive least-squares algorithm

We present a new kernel-based algorithm for modeling evenly distributed multidimensional datasets that does not rely on input space sparsification. The presented method reorganizes the typical single-layer kernel-based model into a deep hierarchical structure, such that the weights of a kernel model over each dimension are modeled over its adjacent dimension. We show that modeling weights in the suggested structure leads to significant computational speedup and improved modeling accuracy.

Prediction of Time Series Empowered with a Novel SREKRLS Algorithm

For the unforced dynamical non-linear statespace model, a new Q1 and efficient square root extended kernel recursive least square estimation algorithm is developed in this article. The proposed algorithm lends itself towards the parallel implementation as in the FPGA systems. With the help of an ortho-normal triangularization method, which relies on numerically stable givens rotation, matrix inversion causes a computational burden, is reduced. Matrix computation possesses many excellent numerical properties such as singularity, symmetry, skew symmetry, and triangularity is achieved by using this algorithm. The proposed method is validated for the prediction of stationary and non-stationary MackeyGlass Time Series, along with that a component in the x-direction of the Lorenz Times Series is also predicted to illustrate its usefulness. By the learning curves regarding mean square error (MSE) are witnessed for demonstration with prediction performance of the proposed algorithm from where it’s concluded that the proposed algorithm performs better than EKRLS. This new SREKRLS based design positively offers an innovative era towards non-linear systolic arrays, which is efficient in developing very-large-scale integration (VLSI) applications with non-linear input data. Multiple experiments are carried out to validate the reliability, effectiveness, and applicability of the proposed algorithm and with different noise levels compared to the Extended kernel recursive least-squares (EKRLS) algorithm.

A Self-Adaptive Multikernel Machine Based on Recursive Least-Squares Applied to Very Short-Term Wind Power Forecasting

Wind power has contributed significantly to the increase in electricity generation, but a decision-making tool capable of dealing with its variability and limited predictability is necessary. For this purpose, a novel self-adaptive approach for kernel recursive least-squares machines named multiple challengers is introduced in this work, which is successfully used to produce very short-term wind power forecasts at eight wind farms in Australia. The proposed method is based on a competitive tracking method, and the algorithm deals with some common difficulties of kernel methods, e.g., the increasing kernel matrix size associated with time and memory complexities and the overfitting problem. The proposed method always considers the new information received by the model, thus identifying changes in the time series, avoiding abrupt loss of information and maintaining a controlled number of examples since there is an adaptive selection of the active kernel. It works with the smallest dictionary possible, reducing the probability of overfitting. Five minute-ahead wind power forecasts are produced and evaluated in terms of point forecast skill scores and calibration. The results of the proposed method are compared with those provided by other kernel-based versions of the recursive least-squares algorithm, an online version of the extreme learning machine method, and the persistence time series model. An increase in the number of kernels used in the ensemble system can lead to better results when compared with those of single-kernel models.

Mixed-Precision Kernel Recursive Least Squares.

Kernel recursive least squares (KRLS) is a widely used online machine learning algorithm for time series predictions. In this article, we present the mixed-precision KRLS, producing equivalent prediction accuracy to double-precision KRLS with a higher training throughput and a lower memory footprint. The mixed-precision KRLS applies single-precision arithmetic to the computation components being not only numerically resilient but also computationally intensive. Our mixed-precision KRLS demonstrates the 1.32, 1.15, 1.29, 1.09, and 1.08× training throughput improvements using 24.95%, 24.74%, 24.89%, 24.48%, and 24.20% less memory footprint without losing any prediction accuracy compared to double-precision KRLS for a 3-D nonlinear regression, a Lorenz chaotic time series, a Mackey-Glass chaotic time series, a sunspot number time series, and a sea surface temperature time series, respectively.

Quaternion kernel recursive least-squares algorithm

Various kernel-based algorithms have been successfully applied to nonlinear problems in adaptive filters over the last two decades. In this paper, we study a kernel recursive least squares (KRLS) algorithm in the quaternion domain. By the generalized Hamilton-real calculus method, we can apply the kernel trick to calculate the quaternion KRLS filter. In order to show the feasibility of the proposed algorithm, firstly we investigate the quaternion recursive least squares (QRLS) algorithm, and simulations show that the proposed QRLS algorithm has the same steady error as that of the closed-form solution; Secondly, we generalize the QRLS algorithm to the quaternion KRLS algorithm, theoretical analysis show the convergence, and simulations are described demonstrating the performance of the proposed algorithm.

A Novel Kalman Filter Formulation for Improving Tracking Performance of the Extended Kernel RLS

In this paper, two contributions are presented. Firstly, a novel variant of the Kalman filter is developed and used to improve the tracking performance of the extended kernel recursive least squares algorithm (Ex-KRLS). Without resorting to the Riccati equation, the proposed formulation of the Kalman filter relies on principles of optimization and convex duality to obtain a stable quadratic form. It allows for performance and stability efficiency by minimizing the estimation error with the manipulation of a single scalar variable. Secondly, the proposed Kalman filter formulation is embedded into the Ex-KRLS-KF algorithm, an extension of the Ex-KRLS, to further improve its tracking performance. For this purpose, the state model of the proposed Ex-KRLS-KF variant is constructed in the original state space, while the hidden state is then estimated using the proposed Kalman filter formulation. The standard KRLS algorithm learns the measurement model used in hidden state estimation. We show through a comprehensive set of computer experiments that the proposed hybrid algorithm has more flexible state and noise models than competing alternative algorithms. We evaluate the proposed algorithm in two benchmarking tasks (nonlinear Rayleigh multipath channel tracking and Lorenz system modeling) and compare its performance with those provided by the state-of-the-art algorithms.

Quantized generalized maximum correntropy criterion based kernel recursive least squares for online time series prediction

With the rapid development of information theoretic learning, the maximum correntropy criterion (MCC) has been widely used in time series prediction area. Especially, the kernel recursive least squares (KRLS) based on MCC is studied recently due to its online recursive form and the ability to resist noise and be robust in non-Gaussian environments. However, it is not always an optimal choice that using the correntropy, which is calculated by default Gaussian kernel function, to describe the local similarity between variables. Besides, the computational burden of MCC based KRLS will raise as data size increases, thus causing difficulties in accommodating time-varying environments. Therefore, this paper proposes a quantized generalized MCC (QGMCC) to solve the above problem. Specifically, a generalized MCC (GMCC) is utilized to enhance the accuracy and flexibility in calculating the correntropy. In order to solve the problem of computational complexity, QGMCC quantizes the input space and upper bounds the network size by vector quantization (VQ) method. Furthermore, QGMCC is applied to KRLS and forming a computationally efficient and precisely predictive algorithm. After that, the improved algorithm named quantized kernel recursive generalized maximum correntropy (QKRGMC) is set up and the derivation process is also given. Experimental results of one benchmark dataset and two real-world datasets are present to verify the effectiveness of the online prediction algorithm.

A Pipeline for Adaptive Filtering and Transformation of Noisy Left-Arm ECG to Its Surrogate Chest Signal

The performance of a low-power single-lead armband in generating electrocardiogram (ECG) signals from the chest and left arm was validated against a BIOPAC MP160 benchtop system in real-time. The filtering performance of three adaptive filtering algorithms, namely least mean squares (LMS), recursive least squares (RLS), and extended kernel RLS (EKRLS) in removing white (W), power line interference (PLI), electrode movement (EM), muscle artifact (MA), and baseline wandering (BLW) noises from the chest and left-arm ECG was evaluated with respect to the mean squared error (MSE). Filter parameters of the used algorithms were adjusted to ensure optimal filtering performance. LMS was found to be the most effective adaptive filtering algorithm in removing all noises with minimum MSE. However, for removing PLI with a maximal signal-to-noise ratio (SNR), RLS showed lower MSE values than LMS when the step size was set to 1 × 10−5. We proposed a transformation framework to convert the denoised left-arm and chest ECG signals to their low-MSE and high-SNR surrogate chest signals. With wide applications in wearable technologies, the proposed pipeline was found to be capable of establishing a baseline for comparing left-arm signals with original chest signals, getting one step closer to making use of the left-arm ECG in clinical cardiac evaluations.

Convergence and performance analysis of kernel regularized robust recursive least squares

Kernel recursive least squares (KRLS) is very sensitive to non-Gaussian noise and hence, robust extensions are proposed using maximum correntropy criterion or generalized maximum correntropy. However, because of the complex form of the model, there is no theoretical analysis on the convergence of these filters. In this paper, we propose a new alternative: Kernel Regularized Robust RLS (KR3LS). It uses half-quadratic technique to simplify the form of the loss function. Our major contribution is then proving the convergence of the filter to the target weights and desired output. The bounds of regularization factor is also obtained. KR3LS is experimentally tested using synthetic and real data and is shown to perform superior compared to other robust alternatives.

Online prediction of noisy time series: Dynamic adaptive sparse kernel recursive least squares from sparse and adaptive tracking perspective

With the deepening of the research on kernel recursive least squares (KRLS), significant researches have been applied to time series online prediction. However, it usually ignores the extraneous and redundant factors in the raw data, which can cause bias in the prediction. In addition, it usually contains both noise and non-stationary characteristics, resulting in deteriorated prediction accuracy and reduced model efficiency. To ease the above two drawbacks of conventional KRLS, this brief presents a dynamic adaptive sparse kernel recursive least squares (DASKRLS) filtering algorithm. It first uses the online vector projection standard and the approximate linear dependence criterion to effectively constrain kernel matrix dimension, and reduce the computational complexity of the model. After that, the regularized maximum correlation entropy criterion to significant process noise-containing data from the perspective of improving generalization ability. Moreover, the adaptive update mechanism can dynamically track the real-time weight of non-stationary signals. The dynamic sparse process is essentially equivalent to a feature selection process that maintains low-dimensional manifold information. Lorenz benchmarking experiments and real data experiments show that DASKRLS achieves better prediction performance in complex systems with noise and nonstationary.

Kernel Recursive Least Squares Algorithm Based on the Nystr${\rm {{\ddot{\bf {o}}}}}$m Method With $k$-Means Sampling

The kernel recursive least squares (KRLS) algorithm is used to improve the convergence rate and filtering accuracy of kernel adaptive filters (KAFs) in the Gaussian noise case. However, the linear growing network size in KRLS poses a huge amount of time and storage consumption. To address this issue, a novel Nyström kernel recursive least squares (NysKRLS) algorithm is proposed by approximating the Gaussian kernel with the Nyström method. In addition, the k-means sampling is adopted in NysKRLS to develop another Nyström kernel recursive least squares with k-means sampling (NysKRLS-KM) algorithm for further improving the approximation accuracy. NysKRLS-KM with a fixed dimensional network structure can achieve significantly better performance than the KAFs based on the stochastic gradient descent (SGD) method, and almost the same performance as KRLS efficiently. Monte Carlo simulations on nonlinear system identification and prediction of real-world data illustrate the superiorities of the proposed NysKRLS-KM algorithm from the aspects of computational and spatial complexity.