Least-squares Parameter Estimation Research Articles

Threshold ARCH(1) processes: asymptotic inference

This article discusses large sample inference problems for a first-order ARCH(1) process where threshold appears not only in the mean but also in the variance function. Geometric ergodicity of the process is discussed. Least-squares estimators of parameters are derived and relevant limit results are obtained. Also, the uniform local asymptotic normality of the log-likelihood ratio and a class of efficient estimators are briefly discussed. The model is applied to Korean financial time series.

Performance-optimized applied identification of separable distributed-parameter processes

Studies practical algorithms for parametric identification of cross-directional processes from input/output data. Instead of working directly with the original two-dimensional array of the high-resolution profile scans, the proposed algorithms use separation properties of the problem. It is demonstrated that by estimating and identifying in turn cross directional and time responses of the process, it is possible to obtain unbiased least-square error estimates of the model parameters. At each step, a single data sequence is used for identification which ensures high computational performance of the proposed algorithm. A theoretical proof of algorithm convergence is presented. The discussed algorithms are implemented in an industrial identification tool and the note includes a real-life example using paper machine data.

Least-squares estimation of input/output models for distributed linear systems in the presence of noise

This paper addresses least-squares estimation of parameters in digital input/output models of linear time-invariant distributed systems in the presence of white process and sensor noise. The systems of interest have state-space realizations in Hilbert spaces. Both finite-dimensional and infinite-dimensional input/output models are considered. The paper derives a number of new results for recursive least-squares estimation and filtering. The main results characterize the asymptotic values to which parameter estimates converge with increasing amounts of data. The most important result is an equivalence between least-squares parameter estimation on an infinite interval (i.e., with infinitely long data sequences) and linear-quadratic optimal control on a finite interval. Numerical results are presented for a sampled-data version of a wave equation.

A Best-Estimate Reactor Core Monitor Using State Feedback Strategies to Reduce Uncertainties

The development and demonstration of a new algorithm to reduce modeling and state-estimation uncertainty in best-estimate simulation codes has been investigated. Demonstration is given by way of a prototype reactor core monitor. The architecture of this monitor integrates a control-theory-based, distributed-parameter estimation technique into a production-grade best-estimate simulation code. The Kalman Filter-Sequential Least-Squares (KFSLS) parameter estimation algorithm has been extended for application into the computational environment of the best-estimate simulation code RELAP5-3D. In control system terminology, this configuration can be thought of as a “best-estimate” observer. The application to a distributed-parameter reactor system involves a unique modal model that approximates physical components, such as the reactor, by describing both states and parameters by an orthogonal expansion. The basic KFSLS parameter estimation is used to dynamically refine a spatially varying (distributed) parameter. The application of the distributed-parameter estimator is expected to complement a traditional nonlinear best-estimate simulation code by providing a mechanism for reducing both code input (modeling) and output (state-estimation) uncertainty in complex, distributed-parameter systems.

Kriging with nonparametric variance function estimation.

A method for fitting regression models to data that exhibit spatial correlation and heteroskedasticity is proposed. It is well known that ignoring a nonconstant variance does not bias least-squares estimates of regression parameters; thus, data analysts are easily lead to the false belief that moderate heteroskedasticity can generally be ignored. Unfortunately, ignoring nonconstant variance when fitting variograms can seriously bias estimated correlation functions. By modeling heteroskedasticity and standardizing by estimated standard deviations, our approach eliminates this bias in the correlations. A combination of parametric and nonparametric regression techniques is used to iteratively estimate the various components of the model. The approach is demonstrated on a large data set of predicted nitrogen runoff from agricultural lands in the Midwest and Northern Plains regions of the U.S.A. For this data set, the model comprises three main components: (1) the mean function, which includes farming practice variables, local soil and climate characteristics, and the nitrogen application treatment, is assumed to be linear in the parameters and is fitted by generalized least squares; (2) the variance function, which contains a local and a spatial component whose shapes are left unspecified, is estimated by local linear regression; and (3) the spatial correlation function is estimated by fitting a parametric variogram model to the standardized residuals, with the standardization adjusting the variogram for the presence of heteroskedasticity. The fitting of these three components is iterated until convergence. The model provides an improved fit to the data compared with a previous model that ignored the heteroskedasticity and the spatial correlation.

Real-time frequency and harmonic evaluation using artificial neural networks

With increasing harmonic pollution in the power system, real-time monitoring and analysis of harmonic variations have become important. Because of limitations associated with conventional algorithms, particularly under supply-frequency drift and transient situations, a new approach based on nonlinear least-squares parameter estimation has been proposed as an alternative solution for high-accuracy evaluation. However, the computational demand of the algorithm is very high and it is more appropriate to use Hopfield type feedback neural networks for real-time harmonic evaluation. The proposed neural network implementation determines simultaneously the supply-frequency variation, the fundamental-amplitude/phase variation as well as the harmonics-amplitude/phase variation. The distinctive feature is that the supply-frequency variation is handled separately from the amplitude/phase variations, thus ensuring high computational speed and high convergence rate. Examples by computer simulation are used to demonstrate the effectiveness of the implementation. A set of data taken on site was used as a real application of the system.

Another look at the jackknife: further examples of generalized bootstrap

In this paper we have three main results. (a) We show that all jackknife schemes are special cases of generalised bootstrap. (b) We introduce a new generalised bootstrap technique called DBS to estimate the mean-squared error of the least-squares estimate in linear models where the number of parameters tend to infinity with the number of data points, and the error terms are uncorrelated with possibly different variances. Properties of this new resampling scheme are comparable to those of UBS introduced by Chatterjee (1997, Tech. Report No. 2/97, Calcutta). (c) We show that delete- d jackknife schemes belong to DBS or UBS depending on the limit of n −1 d. We also study the second-order properties of jackknife variance estimates of the least-squares parameter estimate in regression.

Consistency property of extended least-squares parameter estimation for stochastic diffusion equation

The extended least-squares parameter estimate for stochastic heat diffusion equations is considered. The unknown parameter is a heat diffusion coefficient which is a function of a spatial variable. Almost sure convergence for the estimated parameter is proved. A numerical example is demonstrated for supporting the theoretical results developed here.

Non-linear effects of action of magnetic fields on models of the tumour growth

Applications of adaptive systems require a certain robustness of the interconnected estimator/controller structure. By restriction to identification based parameter adaptive schemes we prove asymptotic (Ljapunov-) stability of the Least-Squares parameter estimator under persistent excitation and examine the stability of the adaptive loop. However, Ljapunov stability proofs do not allow to evaluate the control performance of the system relative to a certain performance index. In the main part of the contribution we therefore establish time-varying stability margins of parameter estimators by extension of results in sector stability theory of constant gain optimal systems. We will furthermore introduce optimality margins for Riccati-observers and extend the results to time-varying optimality margins of LS-estimators. We analyse the results referring to the estimator’s long time behaviour and robustness. The results are promising with regard to on-line evaluation of local stability and optimality of parameter adaptive systems and contribute to a deeper understanding of the relation between persistent excitation, robustness and adaptation. The results may also be applied for new adaptation strategies in adaptive control of time varying processes.

Genetic Algorithms for Calibrating Water Quality Models

The genetic algorithm (GA) is used as an optimization tool to estimate water quality model parameters in a calibration scenario. The GA is found to be a useful calibration tool, capable of providing least-squares parameter estimates while incorporating field observations as constraints and accumulating useful information about the response surface. Because the GA provides a directed, randomized search using a population of points, a database of information about the response surface, parameter correlation, and objective function sensitivity to model parameters is obtained. Synthetic data with and without error are used initially to investigate the potential of the GA for model calibration applications. A case study is then carried out to confirm GA performance with field data. Constraints are included successfully in the GA search using either a penalty function or a special decoding operation. However, results show that the GA with the penalty function outperforms the GA with the decoder. Furthermore, parameter estimation is found to be improved by the inclusion of multiple-response data. For ill-posed problems, the GA provides several parameter estimates, all performing equally well mathematically.

Parametric Imaging of Ligand-Receptor Binding in PET Using a Simplified Reference Region Model

A method is presented for the generation of parametric images of radioligand–receptor binding using PET. The method is based on a simplified reference region compartmental model, which requires no arterial blood sampling, and gives parametric images of both the binding potential of the radioligand and its local rate of delivery relative to the reference region. The technique presented for the estimation of parameters in the model employs a set of basis functions which enables the incorporation of parameter bounds. This basis function method (BFM) is compared with conventional nonlinear least squares estimation of parameters (NLM), using both simulated and real data. BFM is shown to be more stable than NLM at the voxel level and is computationally much faster. Application of the technique is illustrated for three radiotracers: [ 11C]raclopride (a marker of the D2 receptor), [ 11C]SCH 23390 (a marker of the D1 receptor) in human studies, and [ 11C]CFT (a marker of the dopamine transporter) in rats. The assumptions implicit in the model and its implementation using BFM are discussed.

A Longitudinal Analysis of Height/Diameter Curves

Abstract The paper describes a simultaneous statistical analysis of height/diameter curves in data consisting of several temporal and permanent plots. Two jack pine data sets, one from planted stands and the other from naturally regenerated stands, were analyzed using the same model structure. Parameters of a logarithmic height/diameter curve at a given age in a given stand were decomposed into a trend (an age-dependent population mean), a random stand effect, and a random time effect. The deviation of an observed height from the stand and age specific height/diameter curve was decomposed into a random tree effect and a random residual error which have nonhomogeneous variances. Trend functions for the parameters of the height/diameter curve were estimated using least squares estimates of parameters as dependent variables (generalized least squares would lead to inconsistent estimates). The trend equations describe most of the variation in the height curve parameters. Other stand variables (in addition to age) can explain the variation further, but development over time cannot then be predicted. A less stable description of the height/diameter curves is obtained in terms of dominant height. In applications, height/diameter curves can be calibrated by predicting the random stand and time effects using any combination of height measurements. A simultaneously estimated set of curves will be logical also when there are so few measurements that ordinary least squares curves show erratic fluctuations. For. Sci. 43(4):555-570.

A model of oxygen transport capacity changes for independently living older men and women.

The purpose of the present investigation was to describe, for a subset of a large random survey of men and women, restricted to the ages of 55 to 85 years, the physiological decay pattern for aerobic fitness and contributing factors of cardiovascular and pulmonary function. The time course of the age-related changes in maximal oxygen uptake (VO2max), ventilatory threshold (TVE), maximal ventilation (VEmax), maximal heart rate (HRmax), and O2 pulse (VO2max/HRmax) were examined by fitting the data to a decaying exponential model by use of a least-squares parameter estimation technique. The time constant (tau) was used to describe the rate of decline. The women showed a much slower decline in VO2max (tau = 47.4 years) and TVE (tau = 83.3 years) than the men (tau = 20.8 and 15.4 years, respectively). There was a significant age-related decrease in body weight (0.45 kg.yr-1) in the men, whereas the women showed no change. Pulmonary function did not limit performance based on the very slow decline in VEmax and the normal FEV1.0. The decay in HRmax was better described by a linear model, resulting in an extremely slow tau. Maximal O2 pulse clearly exhibited an exponential decay, with a shorter tau (tau men = 13.5 years; tau women = 28.5 years) than any other variable.

Registering incomplete radar images using the EM algorithm

This paper describes an application of the EM (expectation and maximisation) algorithm to the registration of incomplete millimetric radar images. The data used in this study consists of a series of non-overlapping radar sweeps. Our registration process aims to recover transformation parameters between the radar-data and a digital map. The tokens used in the matching process are fragmented line-segments extracted from the radar images which predominantly correspond to hedge-rows in the cartographic data. The EM technique models data uncertainty using Gaussian mixtures defined over the positions and orientations of the lines. The resulting weighted least-squares parameter estimation problem is solved using the Levenberg-Marquardt method. A sensitivity analysis reveals that the data-likelihood function is unimodal in the translation and scale parameters. In fact, the algorithm is only potentially sensitive to the choice of initial rotation parameter; this is attributable to local sub-optima in the log-likelihood function associated with π 2 orientation ambiguities in the map. By adopting Levenberg-Marquardt optimisation we reduce the local convergence difficulties posed by these local rotation maxima. The method is also demonstrated to be relatively insensitive to random measurement errors on the line-segments.

Multiresponse Parameter Estimation of SVOCs in Activated Sludge Process

Semivolatile organic chemicals (SVOCs) in a treatment system may be dissolved in the liquid phase, adsorbed into the solid phase, become part of the gaseous phase, or be biologically degraded. Experiments on discovering their fate should examine these phases simultaneously. As a result, the experiments produce multiple responses relating the liquid, solid, and gaseous phases of the system. These data are used to estimate parameters (e.g., rate constants) for biodegradation, volatilization, adsorption, and perhaps desorption. This paper explains how data were collected in unsteady-state experiments and how the multiple-response nonlinear least-squares parameter estimation was done on a set of simultaneous differential equations. Some aspects of the model building process, parameter significance and consistency, and model simplification are also discussed.

Extended Horizon Adaptive Model Algorithmic Control

A new, original, robust adaptive control strategy termed Extended Horizon Adaptive Model Algorithmic Control is presented. In EHAMAC, a new, combined, ’single-loop’/’cascade’ adaptive least-squares parameter estimator is coupled with a new, simple but powerful Extended Horizon Model Algorithmic Control so that open-loop stable non-minimum phase systems can be effectively controlled in the time-varying environment. In the new, cascade structure of the ALS estimator, the covariance windup and blowup are totally eliminated. Moreover, the sacramental square-root update of the covariance matrix is no longer needed On the other hand, employing EHMAC facilitates robustness design so that controller parameters can be effectively auto-tuned.

Nonlinear Least-Squares Minimization Applied to Tracer Gas Decay for Determining Airflow Rates in a Two-Zone Building

We developed a method based on tracer gas decay measurements to quantify the airflow rates, including the interzonal airflows, in a two-zone building: different tracer gases were simultaneously pulse-injected into each of the two zones and the evolution of the gas concentrations in each zone was measured; theoretical concentration profiles obtained by solving dynamic material-balance equations for two coupled, well-mixed zones were fit to the experimental data using nonlinear least-squares minimization; and estimates of the airflow rates were iteratively refined until a best fit was achieved between the model and the data. We conducted experiments validating the method in two full-sized rooms of a test house. Airflows were controlled using blowers, and mixing was ensured by the use of fans. Airflow rates inferred by the tracer gas technique agreed with imposed airflow rates within an average absolute error of 8%. Results are also reported for two experiments conducted in the same structure under uncontrolled conditions. Goodness-of-fit tests revealed no statistically significant differences between measured tracer gas concentrations and theoretical concentration profiles constructed using the least-squares parameter estimates.

On critical stability of discrete-time adaptive nonlinear control

In this paper, we examine the global stability and instability problems for a class of discrete-time adaptive nonlinear stochastic control. The systems to be controlled may exhibit chaotic behavior and are assumed to be linear in unknown parameters but nonlinear in output dynamics, which are characterized by a nonlinear function (say, f(x)). It is found and proved that in the scalar parameter case there is a critical stability phenomenon for least squares (LS)-based adaptive control systems. To be specific, let the growth rate of f(x) be f(x)=O(/spl par/x/spl par//sup 6/) with b/spl ges/0, then it is found that b=4 is a critical value for global stability, i.e., the closed-loop adaptive system is globally stable if b<4 and is unstable in general if b/spl ges/4. As a consequence, we find an interesting phenomenon that the linear case does not have: for some LS-based certainty equivalence adaptive controls, even if the LS parameter estimates are strongly consistent, the closed-loop systems may still be unstable. This paper also indicates that adaptive nonlinear stochastic control that is designed based on, e.g., Taylor expansion (or Weierstrass approximation) for nonlinear models, may not be feasible in general.

Collinearity detection in linear regression models

Multicollinearity can seriously affect least-squares parameter estimates. Many methods have been suggested to determine those parameters most involved. This paper, beginning with the contributions of Belsley, Kuh, and Welsch (1980) and Belsley (1991), forges a new direction. A decomposition of the variable space allows the near dependencies to be isolated in one sub-space. And this, in turn, allows a corresponding decomposition of the main statistics, as well as a new one proposed here, to provide better information on the structure of the collinear relations.

Kalman Filter Application for Distributed Parameter Estimation in Reactor Systems

An application of the Kalman filter has been developed for the real-time identification of a distributed parameter in a nuclear power plant. This technique can be used to improve numerical method-based best-estimate simulation of complex systems such as nuclear power plants. The application to a reactor system involves a unique modal model that approximates physical components, such as the reactor, as a coupled oscillator, i.e., a modal model with coupled modes. In this model both states and parameters are described by an orthogonal expansion. The Kalman filter with the sequential least-squares parameter estimation algorithm was used to estimate the modal coefficients of all states and one parameter. Results show that this state feedback algorithm is an effective way to parametrically identify a distributed parameter system in the presence of uncertainties.