Kernel Recursive Research Articles

Application of DBN-based KRLS method for RUL prediction of lithium-ion batteries

The Remaining Useful Life (RUL) of lithium batteries is vital for maintaining and safely operating the batteries, making precise RUL predictions highly significant. This paper introduces a method for predicting the RUL of lithium-ion batteries, utilizing a kernel adaptive filtering algorithm integrated with Deep Belief Networks (DBN). The method constructs a novel prediction model based on the Fixed-Budget Kernel Recursive Least Squares (FB-KRLS) algorithm. In this approach, the DBN extracts features from the original lithium battery data to reduce data complexity. The Square-root Cubature Kalman Filter (SCKF) is integrated with the FB-KRLS algorithm, employing a dual al-ternating learning strategy to improve the model's nonlinear fitting performance. The model was validated using NASA's lithium battery data, showing that the minimum val-ues for the MAPE, RMSE and MAE were 0.102%, 0.0016 and 0.0014, respectively. Therefore, the proposed method demonstrates potential for application in predicting the RUL of lithium-ion batteries.

Design of data-driven model for the pressurizer system in nuclear power plants using a TSK fuzzy neural network

In pressurized water reactors (PWRs), the main role of the pressurizer unit is to maintain the primary circuit pressure of the reactor within preset limits. To ensure this role, the pressure and the water level control systems for the pressurizer system are very significant. Hence, it is very important to monitor and predict them with great accuracy during operation. In this paper, a data-driven model using a TSK fuzzy neural network (TSKFNN) for the prediction of the pressure and the water level inside the pressurizer is developed. The identification task of the TSKFNN model of the pressurizer system is fulfilled in two steps: the structure and the parameters identification. In the first step, the fuzzy C-means clustering method is used to separate the input data into clusters and obtain the rule antecedent parameters of the TSKFNN model. Besides, the initial values of the rule consequent parameters are defined using the kernel ridge regression algorithm. In the second step, the sliding-window Kernel Recursive Least Squares (KRLS) algorithm and the gradient method are developed to adapt TSKFNN model parameters. By using input/ output data collected from the PCTran VVER-1200 simulator, the data-driven model was trained, tested, and evaluated. The simulation results demonstrate that the data-driven model using the TSKFNN structure can effectively predict the pressure and the water level in the pressurizer system. Furthermore, the simulation results also involve a comparison with two other mathematical models to confirm the effectiveness of the developed data-driven model.

Novel cascade filter design of improved sparse low-rank matrix estimation and kernel adaptive filtering for ECG denoising and artifacts cancellation

ElectroCardioGram (ECG) signals are highly vulnerable to disturbances caused by noise and artifact sources which can degrade the ECG signal quality and increase the difficulty in obtaining reliable and accurate clinical interpretations for heart conditions. This paper introduces, for the first time, the Improved Sparse Low-Rank (ISLR) algorithm for suppressing white/colored noises, and the Kernel Recursive Least Squares with Approximate Linear Dependency (ALDKRLS) algorithm for eliminating various artifact sources. A novel automated multi-stage filter is introduced for suppressing artifact components in the first stage using ALDKRLS and eliminating noise sources in the subsequent stage using ISLR. The robustness of the suggested multi-stage filter is demonstrated by eliminating noise and artifact components individually and when both present concurrently using real ECG data. Experimental results elucidate the outstanding accuracy of the suggested framework in eliminating interference sources and keeping the essential and important characteristics of the original ECG data. Also, the application of the suggested framework in practical systems is examined by investigating a new efficient ECG multi-class classification system before and after suppressing noise and artifact interferences. Results show that the suggested framework manages not only to eliminate effectively noise and artifact components, but also to achieve very accurate ECG diagnosis results by maintaining the essential characteristics of the ECG signal that differentiate different heart disorders. This elucidates the usefulness of the proposed multi-stage filter as a promising preprocessing tool for obtaining high-resolution ECG data and consequently enhancing the diagnosis performance of several heart diseases.

Histopathological image-based breast cancer detection employing 3D-convolutional neural network feature extraction and Stochastic Diffusion Kernel Recursive Neural Networks classification

ABSTRACT Among the various cancer categories, breast cancer diagnoses are commonly found in many women, and automatic classification of breast cancer images is a crucial task using computer-aided analysis. This paper proposed a method for the detection of histopathological image-based breast cancer and the main motive of the technique is to employ feature extraction and classification based on DL techniques. The input image is filtered using edge smoothening for noise removal and image resizing on the pre-processing method and used 3D-Convolutional Neural network (3D_CNN) for extracting the features of the image process. Extracted deep features have been given as input for the classification process using Stochastic Diffusion Kernel Recursive Neural Networks (SDKRNN). The suggested model combines the sturdiness of convolutional and kernel blocks and demonstrates performance stability when compared to existing techniques. The experimental results are assessed using a variety of performance metrics and compared to current breast cancer detection methodologies. The result demonstrated the experimental evaluation analyzed for various datasets in terms of accuracy, precision, F-1 score, specificity, and AUC of 98%, 93.8%, 89%, 80%, and 70% and it determined the outperforms in the detection of breast cancer.

Variable step-size evolving participatory learning with kernel recursive least squares applied to gas prices forecasting in Brazil.

A prediction model is an indispensable tool in business, helping to make decisions, whether in the short, medium, or long term. In this context, the implementation of machine learning techniques in time series forecasting models has a notorious relevance, as information processing and efficient and dynamic knowledge uncovering are increasingly demanded. This paper develops a model called Variable step-size evolving Participatory Learning with Kernel Recursive Least Squares, VS-ePL-KRLS, applied to the forecast of weekly prices for S500 and S10 diesel oil, at the Brazilian level, for biweekly and monthly horizons. The presented model demonstrates a better accuracy compared with analogous models in the literature, without loss of computational performance for all time series analyzed.

Kernel Recursive Least Square Approach for Power System Harmonic Estimation

Harmonic frequencies existing in power systems produce harmful effects. Signals containing harmonic effect produce power quality problems such as heating, premature wear, motor pulsations, and so on. Commonly, to cancel the harmonic effects in power systems is based on traditional filtering methods. However, such methodologies tend to fail in the presence of high convexity, which produces undesirable results. Under such circumstances, the appropriate harmonic identification is an important task for removing and reducing the harmonic effects, avoiding the potential damage in electrical power systems. Several published research in the literature has addressed this issue using traditional techniques as Recursive Least Square (RSL), Evolutionary Computation Techniques (ECT), or even the hybridization of both achieving interesting results. However, such techniques present some shortcomings, such as low accuracy and principally high computational time. In this paper, a Kernel Recursive Least Square (KRLS) approach for harmonic estimation is proposed as an alternative methodology. The KRLS uses the least-square formulation in feature space by using kernel mapping for determining the harmonic signals, different experimental simulations demonstrate the high accuracy and robustness, such as the considerably low computational time in different noisy conditions of the proposed method regarding similar approaches.

Mixed-Precision Kernel Recursive Least Squares.

Kernel recursive least squares (KRLS) is a widely used online machine learning algorithm for time series predictions. In this article, we present the mixed-precision KRLS, producing equivalent prediction accuracy to double-precision KRLS with a higher training throughput and a lower memory footprint. The mixed-precision KRLS applies single-precision arithmetic to the computation components being not only numerically resilient but also computationally intensive. Our mixed-precision KRLS demonstrates the 1.32, 1.15, 1.29, 1.09, and 1.08× training throughput improvements using 24.95%, 24.74%, 24.89%, 24.48%, and 24.20% less memory footprint without losing any prediction accuracy compared to double-precision KRLS for a 3-D nonlinear regression, a Lorenz chaotic time series, a Mackey-Glass chaotic time series, a sunspot number time series, and a sea surface temperature time series, respectively.

An enhanced set-membership evolving participatory learning with kernel recursive least squares applied to thermal modeling of power transformers

A factor that directly impacts the lifespan of a power transformer is the hot-spot temperature, and its monitoring is vital to prevent faults, reduce costs, keep the safety, and provide a reliable service to consumers. In this paper, we propose two forecasting models to predict the hot-spot temperature of power transformers. The first is the implementation of Set-Membership in the evolving Participatory Learning with Kernel Recursive Least Squares. And the second is a combination of the evolving Participatory Learning with Kernel Recursive Least Squares and the improved version of the Set-Membership concept, named Enhanced Set-Membership. Both Set-Membership and the Enhanced Set-Membership approaches are implemented to update the rate of change of the arousal index, which is a parameter that controls the creation of rules. A data set collected from an experimental transformer is adopted to evaluate the model’s performance. The obtained results are compared with the performance of the original evolving Participatory Learning with Kernel Recursive Least Squares and with the performance of other classical models suggested in the literature. The proposals have lower errors and a competitive number of final rules, suggesting that the models are efficient approaches to modeling complex data with high accuracy.

Improved Kernel Recursive Least Squares Algorithm Based Online Prediction for Nonstationary Time Series

We present an improved kernel recursive least squares (KRLS) algorithm for the online prediction of nonstationary time series. In order to adaptively sparsify a selected kernel dictionary for the KRLS algorithm, the approximate linear dependency (ALD) criterion based KRLS algorithm is combined with the quantized kernel recursive least squares algorithm to provide an initial framework. In order to sufficiently track the strongly changeable dynamic characteristics due to nonstationarity, a forgetting factor is further inserted into the proposed combined algorithm. It is shown that our proposed algorithm, referred to as the FFIKRLS algorithm, provides a clearly compatible algorithm structure and can be improved by the existing modeling techniques from both mapping and weights updating perspectives. Numerical simulations using benchmark Lorenz time series in comparison with existing methods have demonstrated that the proposed algorithm has superior performance in terms of both predictive accuracy and kernel dictionary size.

Gait Phase Classification and Assist Torque Prediction for a Lower Limb Exoskeleton System Using Kernel Recursive Least-Squares Method.

The gait phase classification method is a key technique to control an exoskeleton robot. Different people have different gait features while wearing an exoskeleton robot due to the gap between the exoskeleton and the wearer and their operation habits, such as the correspondence between the joint angle and the moment at which the foot contacts the ground, the amplitude of the joint angle and others. In order to enhance the performance of the gait phase classification in an exoskeleton robot using only the angle of hip and knee joints, a kernel recursive least-squares (KRLS) algorithm is introduced to build a gait phase classification model. We also build an assist torque predictor based on the KRLS algorithm in this work considering the adaptation of unique gait features. In this paper, we evaluate the classification performance of the KRLS model by comparing with two other commonly used gait recognition methods—the multi-layer perceptron neural network (MLPNN) method and the support vector machine (SVM) algorithm. In this experiment, the training and testing datasets for the models built by KRLS, MLPNN and SVM were collected from 10 healthy volunteers. The gait data are collected from the exoskeleton robot that we designed rather than collected from the human body. These data depict the human-robot coupling gait that includes unique gait features. The KRLS classification results are in average 3% higher than MLPNN and SVM. The testing average accuracy of KRLS is about 86%. The prediction results of KRLS are twice as good as MLPNN in assist torque prediction experiments. The KRLS performs in a good, stable, and robust way and shows model generalization abilities.

An adaptive fuzzy predictive control of nonlinear processes based on Multi-Kernel least squares support vector regression

In this paper, an adaptive fuzzy Generalized Predictive Control (GPC) is proposed for discrete-time nonlinear systems via Takagi–Sugeno system based Multi-Kernel Least Squares Support Vector Regression (TS-LSSVR). The proposed adaptive TS-LSSVR strategy is constructed using a multi-kernel least squares support vector regression where the learning procedure of the proposed TS-LSSVR is achieved in three steps: In the first step, which is an offline step, the antecedent parameters of the TS-LSSVR are initialized using a fuzzy c-means clustering algorithm. The second step, which is an online step, deals with the adaptation of the antecedent parameters which can be implemented using a back-propagation algorithm. Finally, the last online step is to use the Fixed-Budget Kernel Recursive Least Squares algorithm to obtain the consequent parameters. Furthermore, an adaptive generalized predictive control for nonlinear systems is introduced by integrating the proposed adaptive TS-LSSVR into the generalized predictive controller (GPC). The reliability of the proposed adaptive TS-LSSVR GPC controller is investigated by controlling two nonlinear systems: A surge tank and continuous stirred tank reactor (CSTR) systems. The proposed TS-LSSVR GPC controller has demonstrated good results and efficiently controlled the nonlinear plants. Furthermore, the adaptive TS-LSSVR GPC has the ability to deal with disturbances and variations in the nonlinear systems.

Projected Kernel Recursive Maximum Correntropy

In this brief, a different kernel recursive maximum correntropy algorithm is derived using the weighted output information, called KRMC-W. To curb the network growth, we propose a new online sparsification strategy in a feature space, named vector projection (VP) method. Applying the VP to KRMC-W, two novel sparsified kernel adaptive filters, called soft projected KRMC-W and hard projected KRMC-W, respectively, are developed to reduce the average computation complexity. Simulation results under the case of ${\alpha }$ -stable noise are conducted to demonstrate the efficiencies of the proposed algorithms.

Multivariate Chaotic Time Series Online Prediction Based on Improved Kernel Recursive Least Squares Algorithm.

Kernel recursive least squares (KRLS) is a kind of kernel methods, which has attracted wide attention in the research of time series online prediction. It has low computational complexity and updates in a recursive form. However, as data size increases, computational complexity of calculating kernel inverse matrix will raise. And it has some difficulties in accommodating time-varying environments. Therefore, we have presented an improved KRLS algorithm for multivariate chaotic time series online prediction. Approximate linear dependency, dynamic adjustment, and coherence criterion are combined with quantization to form our improved KRLS algorithm. In the process of online prediction, it can bring computational efficiency up and adjust weights adaptively in time-varying environments. Moreover, Lorenz chaotic time series, El Nino-Southern Oscillation indexes chaotic time series, yearly sunspots and runoff of the Yellow River chaotic time series online prediction are presented to prove the effectiveness of our proposed algorithm.

Parsimonious Kernel Recursive Least Squares Algorithm for Aero-Engine Health Diagnosis

Kernel adaptive filtering (KAF) has gained widespread popularity among the machine learning community for online applications due to its convexity, simplicity, and universal approximation ability. However, the network generated by KAF keeps growing with the accumulation of the training samples, which leads to the increasing memory requirement and computational burden. To address this issue, a pruning approach that attempts to restrict the network size to a fixed value is incorporated into a kernel recursive least squares (KRLS) algorithm, yielding a novel KAF algorithm called parsimonious KRLS (PKRLS). The basic idea of the pruning technique is to remove the center with the least importance from the existing dictionary. The importance of a center is quantified by its contribution to minimizing the cost function. The calculation of the importance measure is formulated in an efficient manner, which facilitates its implementation in online settings. Experimental results on the benchmark tasks show that PKRLS obtains a parsimonious network structure with the satisfactory prediction accuracy. Finally, a multi-sensor health diagnosis approach based on PKRLS is developed for identifying the health state of a degraded aero-engine in real time. A case study in a turbofan engine degradation data set demonstrates that PKRLS provides an effective and efficient candidate for modeling the performance deterioration of real complex systems.

A new multi-stage combined kernel filtering approach for ECG noise removal

Electrocardiogram (ECG) signals are contaminated with different artifacts and noise sources which increase the difficulty in analyzing the ECG signals and obtaining accurate diagnosis of heart diseases. In this paper, a new multi-stage combined adaptive filtering design based on Kernel Recursive Least Squares Tracker (KRLST) and Kernel Recursive Least Squares with Approximate Linear Dependency (ALDKRLS) algorithms is proposed for removing artifacts and noise sources, while preserving the low frequency components and the tiny features of the ECG signal. The capability of the proposed approach is demonstrated by investigating several ECG signals from the MIT-BIH database and comparing the results with other adaptive filtering techniques. The results show that the combined ALDKRLS-KRLST approach is much superior in terms of attenuating artifacts components, sensitivity of ECG peak detection, and heart diseases diagnosis. This reveals the effectiveness of the proposed technique as an effective framework for achieving high-resolution ECG from noisy ECG recordings.

Kernel Recursive Least Squares With Multiple Feedback and Its Convergence Analysis

In kernel adaptive filters (KAFs), feedback network can lead to performance improvement from the aspects of estimation accuracy and convergence rate. In this brief, a novel feedback structure is developed and applied to the kernel recursive least squares (KRLS), generating the KRLS with multiple feedback (KRLS-MF). In the proposed KRLS-MF, multiple previous outputs are utilized to update the structure parameters in the recurrent form. The obtained parameters are also proved to be convergent. Compared with other KAFs with and without feedback, KRLS-MF can improve both the filtering accuracy and convergence rate, efficiently.

Kernel-based adaptive learning improves accuracy of glucose predictive modelling in type 1 diabetes: A proof-of-concept study.

This study aims at demonstrating the need for nonlinear recursive models to the identification and prediction of the dynamic glucose system in type 1 diabetes. Nonlinear regression is performed in a reproducing kernel Hilbert space, by the Approximate Linear Dependency Kernel Recursive Least Squares (KRLS-ALD) algorithm, such that a sparse model structure is accomplished. The method is evaluated on seven people with type 1 diabetes in free-living conditions, where a change in glycaemic dynamics is forced by increasing the level of physical activity in the middle of the observational period. The univariate input allows for short-term (≤30 min) predictions with KRLS-ALD reaching an average root mean square error of 15.22±5.95 mgdL-1 and an average time lag of 17.14±2.67 min for an horizon of 30 min. Its performance is considerably better than that of time-invariant (regularized) linear regression models.

A new T-S fuzzy model predictive control for nonlinear processes

A new Takagi-Sugeno system based Kernel ridge regression (TS-KRR) was proposed.The TS-KRR strategy is implemented for both adaptive and offline identification.The TS-KRR was integrated with the GPC to control discrete-time nonlinear systems.The proposed controller showed good results in TS fuzzy GPC with offline modeling.The adaptive TS fuzzy GPC showed good results in dealing with disturbances. In this paper, a novel fuzzy Generalized Predictive Control (GPC) is proposed for discrete-time nonlinear systems via Takagi-Sugeno system based Kernel Ridge Regression (TS-KRR). The TS-KRR strategy approximates the unknown nonlinear systems by learning the Takagi-Sugeno (TS) fuzzy parameters from the input-output data. Two main steps are required to construct the TS-KRR: the first step is to use a clustering algorithm such as the clustering based Particle Swarm Optimization (PSO) algorithm that separates the input data into clusters and obtains the antecedent TS fuzzy model parameters. In the second step, the consequent TS fuzzy parameters are obtained using a Kernel ridge regression algorithm. Furthermore, the TS based predictive control is created by integrating the TS-KRR into the Generalized Predictive Controller. Next, an adaptive, online, version of TS-KRR is proposed and integrated with the GPC controller resulting an efficient adaptive fuzzy generalized predictive control methodology that can deal with most of the industrial plants and has the ability to deal with disturbances and variations of the model parameters. In the adaptive TS-KRR algorithm, the antecedent parameters are initialized with a simple K-means algorithm and updated using a simple gradient algorithm. Then, the consequent parameters are obtained using the sliding-window Kernel Recursive Least squares (KRLS) algorithm. Finally, two nonlinear systems: A surge tank and Continuous Stirred Tank Reactor (CSTR) systems were used to investigate the performance of the new adaptive TS-KRR GPC controller. Furthermore, the results obtained by the adaptive TS-KRR GPC controller were compared with two other controllers. The numerical results demonstrate the reliability of the proposed adaptive TS-KRR GPC method for discrete-time nonlinear systems.

A Class of Weighted Quantized Kernel Recursive Least Squares Algorithms

In this brief, a class of weighted quantized kernel recursive least squares (WQKRLS) algorithms is proposed to efficiently improve the performance of online applications. In the proposed WQKRLS, an online vector quantization with weighted outputs is incorporated into quantized kernel recursive least squares. The resulting desired outputs are smoothed by exponential weights. In addition, the members of the dictionary are updated by the steepest descent method for further performance improvement. Simulations illustrate the superior performance of the proposed WQKRLS.

Kernel Recursive Least Squares Function Approximation in Game Theory Based Control

A game theoretic aspect in reinforcement learning based controller design with kernel recursive least squares algorithm for value function approximation is proposed in this paper. A kernel recursive least-squares-support vector machine is used to realize a mapping from state, controller's action and disturber's action to Q-value function. Online sparsification framework permits the addition of training sample into the Q-function approximation only if it is approximately linearly independent of the preceding training samples. Markov game setup is shown to be one of the important platforms for addressing robustness of direct adaptive optimal control of nonlinear systems. A game against nature strategy shows the strength of state importance in terms of accelerated learning, and better relative stability of the system. Simulation results on two-link robot manipulator show that the proposed method has high learning efficiency—better accuracy measured in terms of mean square error; and lesser computation time, compared to the least-squares support vector machine.