Weighted Recursive Least Squares Research Articles

Global convergence of a blind multichannel identification algorithm

In this paper, we propose an adaptive algorithm for blind identification of single-input multiple-output (SIMO) systems. The algorithm consists of p−1 parallel recursive estimators, where p is the number of system outputs. We analyze the normalized least-mean square (NLMS) estimator, and the weighted recursive least-squares (WRLS) algorithm. It is proved that parameter estimates converge toward a scalar multiple of the true parameters with probability one. The value of the scaling factor is calculated. Numerically simple p−1 parallel NLMS recursions are potential candidate for real-time blind identification applications.

Evaluation of the Suitability of Different Chiller Performance Models for On-Line Training Applied to Automated Fault Detection and Diagnosis (RP-1139)

This paper presents the research results of comparing the suitability of four different chiller performance models to be used for on-line automated fault detection and diagnosis (FDD) of vapor-compression chillers. The models were limited to steady-state performance and included (a) black-box multivariate polynomial (MP) models; (b) artificial neural network (ANN) models, specifically radial basis function (RBF) and multilayer perceptron (MLP); (c) the generic physical component (PC) model approach; and (d) the lumped physical Gordon-Ng (GN) model. All models except for (b) are linear in the parameters. A review of the engineering literature identified the three following on-line training schemes as suitable for evaluation: ordinary recursive least squares (ORLS) under incremental window scheme, sliding window scheme, and weighted recursive least squares (WRLS) scheme, where more weight is given to newer data. The evaluation was done based on five months of data from a 220 ton field-operated chiller from Toronto (a data set of 810 data points) and fourteen days of data from a 450 ton field-operated chiller (a set of about 1120 data points) located on Drexel University campus. The evaluation included a preliminary off-line or batch analysis to gain a first understanding of the suitability of the various models and their particular drawbacks and then to investigate whether the different chiller models exhibit any time variant or seasonal behavior. The subsequent on-line evaluation consisted of assessing the various models in terms of their suitability for model parameter tracking as well as model prediction accuracy (which would provide the necessary thresholds for flagging occurrence of faults). The former assessment suggested that parameter tracking using the GN model parameters could be a viable option for fault detection (FD) implementation, while the black box models were not at all suitable given their high standard errors. The assessment of models in terms of their internal prediction accuracy revealed that the MLP model was best, followed by the MP and GN models. However, the more important test of external predictive accuracy suggests that all models are equally accurate (CV about 2% to 4%) and, hence, comparable within the experimental uncertainty of the data. ORLS with incremental window scheme was found to be the most robust compared to the other computational schemes. The chiller models do not exhibit any time variant behavior since WRLS was found to be poorest. Finally, in terms of the initial length of training data, it was determined—at least with the data sets used that exhibited high autocorrelation—that about 320 and 400 data points would be respectively necessary for the MP and GN model parameter estimates to stabilize at their long-term values. This paper also provides a detailed discussion of the potential advantages that on-line model training can offer and identifies areas of follow-up research.

Quadratic classifier with sliding training data set in robust recursive AR speech analysis

We propose a robust recursive procedure, based on a weighted recursive least squares (WRLS) algorithm with variable forgetting factor (VFF) and a quadratic classifier with sliding training data set, for identification of non-stationary autoregressive (AR) model of speech production system. Experimental evaluation is done using the results obtained by analyzing speech signal with voiced and mixed excitation frames. Experimental results have shown that the proposed robust recursive procedure achieves more accurate AR speech parameter estimates and provides improved tracking performance.

Variable Gain Parameter Estimation Algorithms for Fast Tracking and Smooth Steady State

Based on the ideas of the set-membership identification, we present in this paper variable gain Least Mean Square (LMS) and Weighted Recursive Least Squares (WRLS) algorithms. It is shown that the proposed algorithms possess a fast tracking ability to the variations of the parameter and at the same time are less sensitive to noise leading to a smooth steady-state.

Variable gain parameter estimation algorithms for fast tracking and smooth steady state

Based on the ideas of the set-membership identification, we present in this paper variable gain least mean-square (LMS) and weighted recursive least-squares (WRLS) algorithms. It is shown that the proposed algorithms possess a fast tracking ability to the variations of the parameter and at the same time are less sensitive to noise leading to a smooth steady state.

A statistical pattern recognition approach to robust recursive identification of nonstationary AR model of speech production system

We propose a new robust recursive procedure based on the weighted recursive least squares (WRLS) algorithm with variable forgetting factor (VFF) and frame-based quadratic classifier for identification of nonstationary AR model of speech. Two versions of the frame-based quadratic classifier design procedure are elaborated upon. Experimental results are obtained in analyzing speech signal on voiced and mixed excitation frames.

Stable and fast neurocontroller for robot arm movement

The authors present new learning algorithm schemes using feedback error learning for a neural network model applied to adaptive nonlinear control of a robot arm, namely the QR-WRLS algorithm and its parallel counterpart algorithms. It involves a QR decomposition to transform the system into upper triangular form, and estimation of the neural network weights by a weighted recursive least squares (WRLS) technique. The QR decomposition method, which is known to be numerically stable, is exploited in an algorithm which involves successive applications of a unitary transformation (Givens rotation) directly to the data matrix. The WRLS weight estimation method chosen allows the selection of weighting factors such that each of the linear equations is weighted differently. The QR-WRLS algorithm is shown to provide fast, robust and stable online learning of the dynamic relations necessary for robot control. We show the results of applying these learning schemes with some flexible forgetting strategies to a two-link manipulator. A comparison of their performance with backpropagation algorithm and the recursive prediction error learning algorithm is included.

A real-time implementation of short-term load forecasting for distribution power systems

This paper presents a practical real-time implementation of weather adaptive short-term load forecasting for distribution power utilities. The implementation is accomplished by utilizing a comprehensive load forecasting model consisting of time series, nonlinear load-weather functions and a residual load function represented by an ARMA (auto-regressive moving average) model. The model parameters are estimated and updated online using the WRLS (weighted recursive least squares) algorithm. A variable forgetting factor (VFF) technique is incorporated in the WRLS algorithm for improved model tracking and numerical performance in real-time operation. A software package, STLF, is developed with the proposed implementation method for distribution power utilities. Practical operation of the STLF in several power utilities has demonstrated great success. Offline testing and online operation has consistently shown satisfactory performance with the mean absolute error (MAE) mostly less than 2% for a less than 24-hour ahead forecast and less than 2.5% for a less than 168-hour ahead forecast.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Decimation ratio in subband adaptive identification of a time-varying AR model

This paper presents some conditions to determine decimation ratio in subband adaptive identification of a time-varying system. It has been reported in some experiments that a large decimation ratio is not suitable for subband identification of a time-varying system. However, theoretical conditions for a maximally decimated ratio in a time-varying system have not to date been reported. A short-time invariant model is the starting point for this research. The relation between a forgetting factor used in the recursive weighted least-squares (RWLS) algorithm and a time variation of the time-varying system are derived. Then, new forgetting factors are derived for the multirate processing case. Using a convergence property provided by these forgetting factors, theoretical conditions are obtained having parameters estimated from decimated signals that are as accurate as those from all observed signals. The propriety of these conditions is demonstrated using simulations.

Identification of Continuous-Time Systems with Partially-Known State-Dependent Disturbances

In this paper we present two continuous-time versions (with and without data normalization) of the Exponentially Weighted Recursive Least-squares Algorithm (EW-RLS) developped in [1]. These algorithms are suitable for identifying systems with bounded partially-known disturbances since they explicitly account for these disturbances and ensure parameter boundedness. The paper presents the derivation of these algorithms and the associated convergence issues.

A comparison of recursive weighted least squares estimation and Kalman filtering for source dynamic motion evaluation

The problem of compensating for underwater motion effects (heave component) arises in a number of areas of current interest. Two models of the heave process are discussed. In the first, frequency-response-based higher-order models, including lead-lag factors, are obtained as a prelude to the application of the standard Kalman filter to estimate the heave component. The second model involves the discrete autoregressive moving average with external variables (ARMAX) form. The author proposes the use of the recursive weighted least squares (RWLS) algorithm to solve the filtering problem on the basis of the ARMAX model. The algorithm is simpler than Kalman filtering in terms of the required knowledge of noise statistics, and thus provides an attractive alternative to Kalman filtering. The relation between the RWLS technique and Kalman filtering is explored. Computational results pertaining to the performance of the RWLS technique are given, and the effects of the method's weighting functions are discussed.

Sliding-windowed weighted recursive least-squares method for parameter estimation

This letter presents a modified method of the exponentially weighted recursive least-squares (WRLS) estimation using sliding window. In the method, two windowing techniques are simultaneously used to ensure that the estimator has good parameter tracking property and that the estimated parameters converge the true parameters. Simulation shows that the proposed method tracks rapidly time-varying parameters more effectively than WRLS.

A 'systolic array' formulation of the optimal bounding ellipsoid algorithm

A previously published recursive estimation algorithm (ibid., vol.ASSP-34, p.1331-4, 1986), which updates the parameter vector for a linear system only when incoming data are sufficiently informative, is reformulated to be implementable using well-known systolic array processing schemes. In particular, the optimal bounding ellipsoid algorithm (OBE) for identifying an autoregressive moving-average (ARMA) system is formulated as a conventional weighted recursive least squares (WRLS) estimator with special weights. In this framework, OBE can be implemented using algorithms developed for LS solutions on systolic machines. Adaptation by a sliding window is easily added to this formulation. A simulation example is given to illustrate the results. >