Recursive Least Squares Research Articles

Kernel recursive least squares dictionary learning algorithm

An online dictionary learning algorithm for kernel sparse representation is developed in the current paper. In this framework, the input signal nonlinearly mapped into the feature space is sparsely represented based on a virtual dictionary in the same space. At any instant, the dictionary is updated in two steps. In the first step, the input signal samples are sparsely represented in the feature space, using the dictionary that has been updated based on the previous data. In the second step, the dictionary is updated. In this paper, a novel recursive dictionary update algorithm is derived, based on the recursive least squares (RLS) approach. This algorithm gradually updates the dictionary, upon receiving one or a mini-batch of training samples. An efficient implementation of the algorithm is also formulated. Experimental results over four datasets in different fields show the superior performance of the proposed algorithm in comparison with its counterparts. In particular, the classification accuracy obtained by the dictionaries trained using the proposed algorithm gradually approaches that of the dictionaries trained in batch mode. Moreover, in spite of lower computational complexity, the proposed algorithm overdoes all existing online kernel dictionary learning algorithms.

Application of stochastic filter to three-phase nonuniform transmission lines

ABSTRACT This paper implements the stochastic filters for state and parameter estimation of nonuniform transmission lines (NTL). In general, transmission line (TL) problem is a continuous time and space problem. By taking the line loading noise into account, the TL equations become a stochastic partial differential equation (PDE) rather than a simple set of coupled finite stochastic differential equations (SDE). By transforming the spatial variables into the Fourier domain, the stochastic PDE can be transformed into an infinite sequence of SDE. After truncation to a finite set of Fourier series coefficients, it becomes a finite set of coupled linear SDE, which is the required domain in which extended Kalman filter (EKF) and unscented Kalman filter (UKF) can be applied. For state space equation of EKF and UKF, the voltage and current of periodic NTL are expanded into an infinite set of spatial harmonics. In this way, the voltage and current measurement of NTL become an eigenvalue problem. The NTL is considered as cascade of small infinite NTL and the four distributed primary parameters of the periodic NTL are expressed using Fourier series expansion. Finally, the Kalman filter (KF)-based state estimation and the EKF- and UKF-based parameter estimation have been compared with recursive least squares (RLS) method. The simulation results present the superiority of the method.

Recursive least squares method for training and pruning convolutional neural networks

Convolutional neural networks (CNNs) have shown good performance in many practical applications. However, their high computational and storage requirements make them difficult to deploy on resource-constrained devices. To address this issue, in this paper, we propose a novel iterative structured pruning algorithm for CNNs based on the recursive least squares (RLS) optimization. Our algorithm combines inverse input autocorrelation matrices with weight matrices to evaluate and prune unimportant input channels or nodes in each CNN layer and performs the next pruning operation when the testing loss is tuned down to the last unpruned level. Our algorithm can be used to prune feedforward neural networks (FNNs) as well. The fast convergence speed of the RLS optimization allows our algorithm to prune CNNs and FNNs multiple times in a small number of epochs. We validate its effectiveness in pruning VGG-16 and ResNet-50 on CIFAR-10 and CIFAR-100 and pruning a three-layer FNN on MNIST. Compared with four popular pruning algorithms, our algorithm can adaptively prune CNNs according to the learning task difficulty and can effectively prune CNNs and FNNs with a small or even no reduction in accuracy. In addition, our algorithm can prune the original sample features in the input layer.

Equalizer of Amplitude-Frequency Response of Acoustic Channels and its Adaptive Filter Based on Recursive Least Squares Algorithm with Diagonalized Correlation Matrix

The paper presents an equalizer of the amplitude-frequency response of the acoustic wave propagation channels in the rooms. The considered equalizer uses the so-called modified architecture based on the adaptive filter with the input signal filtering, which weights are calculated using the Recursive Least Squares (RLS) algorithm. The equalizer of the acoustic channels amplitude-frequency response usually requires the usage of the adaptive filters with the large number of weighs that means the large computational complexity of the adaptive filter. The adaptive filters based on the fast RLS algorithms are efficient in the terms of their complexity but often unstable if the number of adaptive filter weights is large. Therefore, the algorithms are poorly suited for usage in the amplitude-frequency response equalizers of the acoustic channels. Adaptive filters based on the RLS algorithm, which use the Matrix Inversion Lemma (MIL), are stable, but computationally complex. The equalizer presented in this paper uses a simplified adaptive filter based on the MIL RLS algorithm with a diagonalized correlation matrix of the filter input signal. In addition, in the filter, each of the square submatrices on the simplified correlation matrix diagonal is inverted using a computationally efficient version of the MIL that takes into account the symmetrical structure of the mentioned submatrices relatively their main diagonals. Computer simulation confirms the performance of the proposed equalizer and demonstrates its efficiency comparing to the unsimplified equalizer. The demo includes the graphs of the equalizer steady-state amplitude-frequency response, the acoustic channel amplitude-frequency responses before and after the equalization, and the power spectral density plots of the equalizer input signal and the signals passed through the equalizer and the acoustic medium. The equalizer can be used in the equipment for the high-quality voice and music reproduction in the conditions of the limited computing resources for the implementation of the equalizer.

Variable Recursive Least Square Algorithm for Lithium-ion Battery Equivalent Circuit Model Parameters Identification

For SOC (state of charge) assessment techniques based on electrical circuit models, the parameters of the model are strongly biased by: battery aging, temperature, causing some errors in the estimation of the SOC. One approach to solve this problem is to update the model parameters constantly. We suggest a new algorithm VRLS (variable recursive least squares) to update the parameters of a 2-resistor-capacitor (RC) network and to estimate the output battery voltage. VRLS is compared to the recursive least squares (RLS) and the adaptive forgetting factor recursive least squares (AFFRLS) algorithms. For algorithm assessment, we utilized real experimental data conducted on the Samsung 18650-20R lithium-ion cell. The tests indicate that compared to RLS and AFFRLS methods, VRLS recorded a low distribution in the high error range, in addition to small predictive performance indicators (RMSE, MAE, and MAPE) in all tests, which implies that VRLS has a good parameter identification ability.

Development of an adaptive finite impulse response filter optimization algorithm using rough set theory

Signal processing is crucial that as one sends information, there is a corresponding process to encode, decode, and clean the signal of unwanted noise and disruptions via use of filters. Due to the environment and how unpredictable it can be and how noise can come from almost anywhere, the typical filter to be used are adaptive filters. Adaptive filters are non-linear filters and have been used regularly regarding adaptive signal processing, this means that the filter changes accordingly and adapts to the environmental noise surrounding it. The world today has numerous applications for adaptive filters such as channel equalization and acoustic noise cancellation. This incentivizes the further development of this specific technology and the constant research that is ongoing. The main component when it comes to adaptive filtering is the algorithms used for the filter. This research compares the least mean square (LMS) and recursive least square (RLS) algorithms concerning their effectiveness in filtering out unwanted acoustic noises. The paper will cover the design and implementation of an optimized rough set based adaptive finite impulse response (FIR) filter for acoustic noise cancellation. The microstrip and bowtie antenna were used to relay the data. The software MATLAB was used for the simulation.

Robot-Assisted Teleoperation Ultrasound System Based on Fusion of Augmented Reality and Predictive Force

This study presents a novel method to realize an augmented reality (AR) and haptic feedback teleoperation ultrasound system, which can reduce the influence of time delay on the operator. The physician uses a haptic device to send commands to a remote site while observing the AR environment. In the virtual environment, the organ model is projected onto the corresponding position according to the human pose, and the UR robot model can respond to the physician's action in advance. Based on force data, the dynamic environmental model is updated by the recursive least-squares method, allowing real-time force feedback. The contact force is computed from the identified parameters together with the position and velocity of the master site. An experimental platform has been built, where experiments have been conducted to evaluate the overall architecture, highlighting the role of AR and contact-force prediction models, and including clinical validation. Results show that the system can correctly perform remote ultrasound scanning tasks and ensure authenticity and synchronization.

Lane Change–Intention Inference and Trajectory Prediction of Surrounding Vehicles on Highways

The behavior prediction of the surrounding vehicles is crucial when planning a minimal-risk path when realizing a collision-avoidance system. Herein, we propose a multiple model–based adaptive estimator (MMAE) that infers the lane-change intention of the surrounding vehicles and then predicts their trajectories. Specifically, first, a path is generated in the form of a cubic spline curve using the Frenet coordinate system, which is robust to changes in road curvatures. Linearized recursive least-squares estimation (LRLSE) method is used to adaptively predict a future trajectory based on the past trajectory of the target vehicle. Preview time is defined as a time-varying parameter that determines the final point of the path, and LRLSE updates it in real time. The MMAE applies LRLSEs to multiple paths and obtains the mode probability for each path, then the lane-change intention is inferred using the mode probability and preview time. The predicted future trajectory is the cubic spline curve determined based on the preview time. Further, we verify the performance of our approach using highD, a naturalistic dataset of vehicle trajectories, and compare it with those of existing methods. The proposed method does not require a large amount of data for training and has a low computational burden and high real-time performance.

Identifiying the PID-N method performance for speed control of BLDC motor propeller on catamaran ships model

Some archipelagic countries must have wide sea. This condition means that there is transportation of goods and people problem. Therefore, an effort has been made to explore renewable energy especially solar energy. In this research, a BLDC (Brush Less Direct Current) motor propeller is applied, and PID-N (Proportional, Integral, and Derivative – Filter Coefficient) method for Catamaran ship speed control. Where PID-N has additional filters such as P, I, and D; however, the advanced performance of this control uses RLS (Recursive Least Square) actuator modeling is the most systematical and flexible design. The experiment includes the speed response of ships with payload and without payload, as well as changes in speed settings. The results show that the ship's speed control using the PID-N has smooth and definite speed control. The PID-N design is systematical, easy and quick for the control designer to change the actuator or sensor performance, if there will be any actuator or sensor replacement with different specifications, which is required in commercial. As it is known, in commercial design, time and method are crucial and has to be effective. The PID-N control method shows its superiority. In detail, the Catamaran speed setting is incremented from 0.4 m/s to 0.6 m/s without payload, the average settling time is 6 seconds with the average error speed 0.071 m/s in the set speed 0.6 m/s. For setting speed increased by 0.4 m/s to 0.6 m/s with payload, settling time gets longer, 7 to 13 seconds with the average error speed 0.01 m/s in the set speed 0.06 m/s. It performs small errors and fasts settling time. The ship model has experimented on an artificial pond, it is related to the proportional small sea wave. The research also shown the ratio of electric power consume toward speed setting is in exponential formula

Research on Direct Lift Carrier-Based Unmanned Aerial Vehicle Landing Control Based on Performance Index Intelligent Optimization/Dynamic Optimal Allocation

High-precision control problems for carrier-based UAVs are challenging due to the requirements for safety, high-performance operation and uncertain ocean environments. To address such problems, this paper proposes a direct lift landing control method that can ensure the safe operation of UAVs using intelligent optimization of control performance indicators and dynamic optimal allocation methods. The direct lift control (DLC) scheme is adopted to improve the operability of the control by introducing flap channels to achieve the fast correction of vertical disturbances. To improve the landing control performance, an intelligent optimization method for DLC gain is proposed, and a neural network relationship between parameter uncertainty and optimal DLC gain is established. In addition, a recursive least-squares identification method is used to estimate the uncertain parameters in real time, so that the established neural network can be used to generate the optimal DLC commands online, and the generated control commands can be assigned to multiple actuators of the carrier-based UAV through a dynamic optimal assignment algorithm. Finally, the superiority of the method is verified using simulation comparison.

A scalable implementation of the recursive least-squares algorithm for training spiking neural networks.

Training spiking recurrent neural networks on neuronal recordings or behavioral tasks has become a popular way to study computations performed by the nervous system. As the size and complexity of neural recordings increase, there is a need for efficient algorithms that can train models in a short period of time using minimal resources. We present optimized CPU and GPU implementations of the recursive least-squares algorithm in spiking neural networks. The GPU implementation can train networks of one million neurons, with 100 million plastic synapses and a billion static synapses, about 1,000 times faster than an unoptimized reference CPU implementation. We demonstrate the code's utility by training a network, in less than an hour, to reproduce the activity of > 66, 000 recorded neurons of a mouse performing a decision-making task. The fast implementation enables a more interactive in-silico study of the dynamics and connectivity underlying multi-area computations. It also admits the possibility to train models as in-vivo experiments are being conducted, thus closing the loop between modeling and experiments.

Long-term deformation safety evaluation method of concrete dams based on the time-varying stability of concrete material

A concrete dam is a kind of large-volume hydraulic structure made of concrete materials. The deformation of the concrete material of the dam body has an obvious time effect. The time-effect component of deformation comprehensively reflects the trend change of the dam deformation and its stability is an important index for evaluating the long-term safety state of concrete dams. Dam safety monitoring theory is generally applied to obtain the time-effect component based on statistical models. The regression parameters of traditional concrete dam deformation statistical models are fixed values, which cannot effectively reflect the dynamic changes in concrete dam deformation characteristics. The variable parameter statistical model is more applicable to practical deformation law of concrete dam based on time-varying parameter. This paper constructs a novel variable parameter statistical model based on the recursive least squares with forgetting factor (FFRLS) method to derive a more precisely concrete dam time-effect component. FFRLS method assigns different weights to the modeling historical data and enhances the impact of new data on the statistical model. To analyze the long-term stability of the time-effect component of concrete deformation, the stability evaluation criterion of the time-effect component is established by its long-term variation characteristics and cusp catastrophe theory. The engineering example shows that the prediction effect of the variable parameter statistical model is better than that of the fixed parameter statistical model and traditional variable parameter statistical model, the mean absolute percentage error of FFRLS model are decreased by 97.39% and 54.63% on average compared with least squares (LS) and recursive least squares (RLS) model. By fusing FFRLS variable parameter statistical model and cusp point mutation model, the safety state of concrete dam after long-term operation can be rapidly and effectively evaluated. The proposed evaluation method has significant engineering application value in concrete dam statistical modeling and safety evaluation.

A Modified Aeromagnetic Compensation Method Robust to the Motion State of Aviation Platform Part

Aeromagnetic compensation is a signal-processing method to suppress the magnetic interference from the airborne platform that far exceeds the target magnetic signal. With the development of measurement technology, the traditional aeromagnetic compensation model in which the motion platform is considered as a rigidly connected whole cannot satisfy the request for high-quality aeromagnetic detection, and the magnetic interference generated by the small relative motion components needs to draw our attention. In this paper, the source of maneuver magnetic interference on the aircraft platform is split into two parts, i.e., the rigidly connected main body part (MBP) and moving part (MP), and their compensation models can be both built by referring to the Tolles–Lawson (TL) model. The key factor in these two models is how to obtain the direction cosine that describe the maneuvering actions of aircraft. In the MBP compensation model, the direction cosine can be directly obtained by the onboard fluxgate magnetometer. However, those in the MP model cannot be directly obtained because there is no any monitoring equipment for it. Therefore, a scheme for calculating the direction cosine of the MP compensation model is proposed. This scheme combines the direction cosine obtained by the onboard fluxgate magnetometer with the relative motion angles. And considering the influence of geomagnetic gradient magnetic interference, its compensation model is brought to form a new aeromagnetic compensation model, herein called the TLGM model. This model is different from the traditional TL model and is nonlinear since the three unknown relative motion angles, so the quantum genetic algorithm (QGA) is introduced to combine with recursive least-squares to estimate those unknowns. Some field experiments were conducted, and the results show that the proposed method reduces the standard deviations and figures of merit by over 42% and 39%, respectively, compared with those using the previous model. Also, the proposed compensation method requires no additional motion monitoring sensors and is more convenient for practical applications.

Parameter and VSI Nonlinearity Hybrid Estimation for PMSM Drives Based on Recursive Least Square

Precise electrical parameters play important roles in the high-performance control of permanent magnet synchronous machines (PMSMs). This paper proposes a novel parameter and voltage source inverter (VSI) nonlinearity hybrid estimation method to accurately estimate stator resistance, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dq</i> -axis inductances, permanent magnet flux linkage, and VSI nonlinearity, in which the effects of magnetic saturation, cross-saturation, and temperature are all considered. The proposed hybrid estimation method consists of two parts: offline and online estimation. In the offline estimation, the four electrical parameters are successively identified by setting different operating conditions, and the identification results are stored in nonvolatile memory in tabular form. In the online estimation, the VSI nonlinearity and the compensation terms of stator resistance and permanent magnet flux linkage related to factors such as temperature and frequency are simultaneously identified by using the recursive least square (RLS) algorithm. Experimental results on a 300 kW PMSM drive system demonstrate that compared to the results achieved with the existing method, the proposed scheme achieves higher estimation accuracy. Consequently, the control performance of the system, such as the output current quality, is efficiently improved.

Transmissibility-Based Fault Detection for Robotic Applications With Time-Varying Parameters

This article investigates transmissibility operators for time-variant systems with bounded nonlinearities. Transmissibility operators are mathematical objects that characterize the relationship between two subsets of responses of an underlying system. The underlying system parameters and nonlinearities are assumed to be unknown. Time-domain transmissibility operators are independent of the system inputs and initial conditions. In this article, we propose an algorithm to identify time-varying transmissibilities using recursive least-squares and noncausal finite impulse response models. The identified transmissibilities are then used for fault detection in three different systems. The first system is an autonomous multirobotic system formulated to emulate connected autonomous vehicle platoons, where the varying parameters are the robot mass and the ground friction coefficient. The second system is a flexible structure with unknown excitation, where the variant parameter is the location of the excitation. Finally, the third system is a robotic manipulator that picks objects with different mass values.

Robustness Enhanced Predictive Control for Dual Three Phase PMSM with Duty Ratio Modulation and Parameter-Free Solution

The classical model predictive control not only relies on the motor model but also is sensitive to the system parameters. To improve the robustness of the model predictive control system, this article proposes a parameter-free predictive control for a dual three-phase PMSM with duty ratio modulation. First, the free-model theory is employed to reconstruct the prediction model of the motor, of which no parameters are needed. So, the problem of parameter mismatch between the motor and the inverter can be handled, and enhance the robustness of the system as well. Moreover, the ultra-local model is optimized with the forgetting factor recursive least-squares algorithm to improve the identification accuracy of the estimated value. Finally, the deadbeat duty ratio modulation solution is introduced to further improve operational performance while ensuring the robustness of the system. Simulation and experiment verify the feasibility and effectiveness of the proposed algorithm.

A Co-Estimation Framework for State of Charge and Parameters of Lithium-Ion Battery With Robustness to Aging and Usage Conditions

Aging and usage conditions affect the battery parameters such as capacity and changes in the open-circuit voltage and internal resistance dependencies on the state of charge. This paper proposes an on-board strategy for the simultaneous estimation of these parameters and their robust evaluation during the battery life. The proposed co-estimation framework consists of a set of interconnected subsystems grounded on the integration of recursive least-squares techniques and a Luenberger-like observer which are independently designed by relying on moving averages of voltage and current measurements. Each subsystem is separately activated through logic variables which select the operating conditions proper for the estimation purposes and allows tracking of model parameters variations. The effectiveness of the solution is shown over experiments with a cylindrical LG M50T INR21700 Li-ion cell with NMC cathode and graphite/silicon anode.

Tracking online low-rank approximations of higher-order incomplete streaming tensors

In this paper, we propose two new provable algorithms for tracking online low-rank approximations of high-order streaming tensors with missing data. The first algorithm, dubbed adaptive Tucker decomposition (ATD), minimizes a weighted recursive least-squares cost function to obtain the tensor factors and the core tensor in an efficient way, thanks to an alternating minimization framework and a randomized sketching technique. Under the canonical polyadic (CP) model, the second algorithm, called ACP, is developed as a variant of ATD when the core tensor is imposed to be identity. Both algorithms are low-complexity tensor trackers that have fast convergence and low memory storage requirements. A unified convergence analysis is presented for ATD and ACP to justify their performance. Experiments indicate that the two proposed algorithms are capable of streaming tensor decomposition with competitive performance with respect to estimation accuracy and runtime on both synthetic and real data.

Receding Horizon D-Optimal Input Design for Identification of Wireless Power Transfer Systems

This paper develops a method of input design for the identification of wireless power transfer (WPT) systems based on receding horizon D-optimization. The dynamic behavior of the system is characterized by a discrete-time, Single-Input, Single-Output (SISO) model, and the associated model parameters are estimated by a recursive least-squares method. To improve the quality of the parameter estimates, the excitation signal of the system is designed based on receding horizon D-optimization. To improve the convergence and efficiency of the input design process, the Sequential Quadratic Programming (SQP) method is employed to solve the receding horizon D-optimization. Numerical and experimental results are presented to demonstrate the effectiveness of the proposed method.

Wave force measurement on a large-scale vertical cylinder using a visual technique

Visual techniques were used to identify the wave force on a large-scale vertical cylinder for the first time in this study by tracking the dynamic waterlines around the cylinder. Experimental validation of the proposed method with two types of waves, regular and irregular, was conducted in a wave flume. Four cameras were used to capture images synchronously, and the dynamic waterlines around the cylinder were identified using a grayscale gradient of pixels. Based on the potential theory, the wave force, including the linear and part of the second- and higher-order force components, was identified using recursive least squares (RLS) in real-time. The identified wave force agreed well with the wave force measured by the load cells, particularly the linear wave forces. The identified results could be significantly improved by considering the nonlinear wave forces. The proposed method has broad applications for laboratory measurements of wave loads on cylindrical structures.