Orthogonal Least-squares Algorithm Research Articles

Modelling of small-angle X-ray scattering data using Hermite polynomials

A new algorithm, called the term-selection algorithm (TSA), is derived to treat small-angle X-ray scattering (SAXS) data by fitting models to the scattering intensity using weighted Hermite polynomials. This algorithm exploits the orthogonal property of the Hermite polynomials and introduces an error-reduction ratio test to select the correct model terms or to determine which polynomials are to be included in the model and to estimate the associated unknown coefficients. With noa prioriinformation about particle sizes, it is possible to evaluate the real-space distribution function as well as three- and one-dimensional correlation functions directly from the models fitted to raw experimental data. The success of this algorithm depends on the choice of a scale factor and the accuracy of orthogonality of the Hermite polynomials over a finite range of SAXS data. An algorithm to select a weighted orthogonal term is therefore derived to overcome the disadvantages of the TSA. This algorithm combines the properties and advantages of both weighted and orthogonal least-squares algorithms and is numerically more robust for the estimation of the parameters of the Hermite polynomial models. The weighting feature of the algorithm provides an additional degree of freedom to control the effects of noise and the orthogonal feature enables the reorthogonalization of the Hermite polynomials with respect to the weighting matrix. This considerably reduces the error in orthogonality of the Hermite polynomials. The performance of the algorithm has been demonstrated considering both simulated data and experimental data from SAXS measurements of dewaxed cotton fibre at different temperatures.

Parameter estimation based on stacked regression and evolutionary algorithms

A new parameter-estimation algorithm, which minimises the cross-validated prediction error for linear-in-the-parameter models, is proposed, based on stacked regression and an evolutionary algorithm. It is initially shown that cross-validation is very important for prediction in linear-in-the-parameter models using a criterion called the mean dispersion error (MDE). Stacked regression, which can be regarded as a sophisticated type of cross-validation, is then introduced based on an evolutionary algorithm, to produce a new parameter-estimation algorithm, which preserves the parsimony of a concise model structure that is determined using the forward orthogonal least-squares (OLS) algorithm. The PRESS prediction errors are used for cross-validation, and the sunspot and Canadian lynx time series are used to demonstrate the new algorithms.

QUALITATIVE VALIDATION OF RADIAL BASIS FUNCTION NETWORKS

In system identification applications of neural networks, the aim is usually to obtain a dynamically valid model of the system which can be used for system analysis and for controller design. In the present study, a cell to a cell mapping procedure is adopted for the global analysis of non-linear systems and the qualitative validation of radial basis function networks. The method is used to graphically display the dynamic properties of non-linear systems in a cell state space, including the fixed points, periodic and aperiodic solutions or chaotic behaviour and the corresponding stability properties. The orthogonal least-squares algorithm (OLS) is then used to train a radial basis function network and the trained network is analysed using the cell mapping framework. In this way, the dynamical properties of the trained network can be qualitatively compared with those of the original system. The effects of overparametrisation and output noise on the dynamic properties of the trained network are investigated using cell map analysis.

On-line identification of nonlinear systems using Volterra polynomial basis function neural networks

An on-line identification scheme using Volterra polynomial basis function (VPBF) neural networks is considered for nonlinear control systems. This comprises a structure selection procedure and a recursive weight learning algorithm. The orthogonal least-squares algorithm is introduced for off-line structure selection and the growing network technique is used for on-line structure selection. An on-line recursive weight learning algorithm is developed to adjust the weights so that the identified model can adapt to variations of the characteristics and operating points in nonlinear systems. The convergence of both the weights and the estimation errors is established using a Lyapunov technique. The identification procedure is illustrated using simulated examples.

Runoff Forecasting Using RBF Networks with OLS Algorithm

This paper illustrates the application of the radial basis function type of artificial neural networks (ANNs) using the orthogonal least-squares (OLS) algorithm to model the rainfall runoff process. Models using this approach differ from the more commonly used ANN models that adopt the back propagation (BP) algorithm, in that the former are linear in the parameters. The OLS algorithm is also capable of synthesizing the suitable network architecture, relieving the user of a time-consuming trial-and-error procedure. The proposed method is then applied to forecast runoff in a small catchment. One-hour predictions using the model are compared with those predicted by an ANN model that uses the BP algorithm, an ARMAX model, and with observed values. Results indicate that the OLS algorithm-based approach produces forecasts of comparable accuracy to those based on the BP algorithm. This approach has the added advantage of requiring less time for model development, and is also readily usable by the hydrologist with little or no background knowledge of ANNs.

Structure Optimization of Takagi-Sugeno Fuzzy Models

A new approach for nonlinear system identification based on Takagi-Sugeno fuzzy models is presented. The premise structure and membership functions are optimized by the LOLIMOT (local linear model tree) algorithm, see [1]. This method is extended by a subset selection technique which automatically determines the structure of the local linear models in the rule consequents. This allows to select the significant input variables for static models and additionally the determination of the dynamic orders and dead times for dynamic models. The utilized subset selection technique is the orthogonal least-squares (OLS) algorithm. It exploits the linear regression structure of the problem and thus is very fast. The applicability of the proposed approach is illustrated by the identification of a transport delay process which has operating point dependent time constants and dead times.

Orthogonal least-squares learning algorithm with local adaptation process for the radial basis function networks

We introduce a local adaptation process in the orthogonal least squares (OLS) learning algorithm for the selection of radial basis function (RBF) networks. Using simulation results, we show that the proposed algorithm can find significantly better subset models than the OLS algorithm.

Two-Pass Orthogonal Least-Squares Algorithm to Train and Reduce the Complexity of Fuzzy Logic Systems

Fuzzy logic systems FLSs can be designed using training data i.e., from M given numerical input/output pairs and supervised learning algorithms. Orthogonal least-squares OLS learning decomposes an FLS into a linear combination of Ms < M nonlinear fuzzy basis functions FBFs, which are optimized during OLS to match the training data. The drawback to OLS is that the resulting system still contains information from all M initial rules, derived from the training points, even though only the most important Ms rules have been established by OLS. This is due to a normalization of the FBFs, and leads to excessive computation times during further processing. Our solution is to construct new FBFs out of the reduced rule base and to run OLS a second time. The resulting system not only is of reduced computational complexity, but is of very similar behavior to the unreduced system. The second run of OLS can be applied to a larger set of training data that greatly improves precision. We illustrate our two-pass OLS algorithm for prediction of the Mackey--Glass chaotic time series. Extensive simulations are provided.

Modelling of Non-Linear Systems using Radial Basis Function Networks

Identification of non-linear dynamical systems is discussed. We show that subject to certain statistical assumptions such systems can be described by the non-linear autoregressive with exogenous input (NARX) model. In this paper, the NARX model is parametrized by the radial basis function neural network. The parameters of the network are estimated with the orthogonal least-squares algorithm the MATLAB implementation of which is included. The algorithm is augmented by a termination criterion to determine the number of basis functions in the network. Modelling of a heating process demonstrates the application of the radial basis function networks to system identification.

Orthogonal least-squares parameter estimation algorithms for non-linear stochastic systems

The derivations of orthogonal least-squares algorithms based on the principle of Hsia's method and generalized least-squares are presented. Extensions to the case of non-linear stochastic systems are discussed and the performance of the algorithms is illustrated with the identification of both simulated systems and linear models of an electric arc furnace and a gas furnace.

Fuzzy basis functions, universal approximation, and orthogonal least-squares learning

Fuzzy systems are represented as series expansions of fuzzy basis functions which are algebraic superpositions of fuzzy membership functions. Using the Stone-Weierstrass theorem, it is proved that linear combinations of the fuzzy basis functions are capable of uniformly approximating any real continuous function on a compact set to arbitrary accuracy. Based on the fuzzy basis function representations, an orthogonal least-squares (OLS) learning algorithm is developed for designing fuzzy systems based on given input-output pairs; then, the OLS algorithm is used to select significant fuzzy basis functions which are used to construct the final fuzzy system. The fuzzy basis function expansion is used to approximate a controller for the nonlinear ball and beam system, and the simulation results show that the control performance is improved by incorporating some common-sense fuzzy control rules.

Orthogonal least-squares algorithm for training multioutput radial basis function networks

A constructive learning algorithm for multioutput radial basis function networks is presented. Unlike most network learning algorithms, which require a fixed network structure, this algorithm automatically determines an adequate radial basis function network structure during learning. By formulating the learning problem as a subset model selection, an orthogonal leastsquares procedure is used to identify appropriate radial basis function centres from the network training data, and to estimate the network weights simultaneously in a very efficient manner. This algorithm has a desired property, that the selection of radial basis function centres or network hidden nodes is directly linked to the reduction in the trace of the error covariance matrix. Nonlinear system modelling and the reconstruction of pulse amplitude modulation signals are used as two examples to demonstrate the effectiveness of this learning algorithm.

Identification of non-linear output-affine systems using an orthogonal least-squares algorithm

Abstract The identification of a non-linear output-affine difference equation model is considered. An orthogonal least-squares algorithm is derived that can determine the term to regress upon, detect significant terms in the model expansion, and provide the final parameter estimates. It is shown that the orthogonal property of the algorithm results in a particularly simple estimation routine and simulated examples are included to illustrate the techniques.