Least-squares Estimation Method Research Articles

Kalman Filter—Finite Element Method in Identification

The extended Kalman filter, which is essentially a method of sequential least-squares estimation, has been applied not only for dynamic parameter identification but also for static or dynamic system identification problems. In order to obtain the stable and convergent estimation in dynamic or static parameter identification problems, an extended Kalman filter-weighted local iteration procedure with an objective function (EK-WLI procedure) is previously proposed by writers. This paper investigates the procedure in geotechnical engineering problems, where the EK-WLI procedure is incorporated with the finite element method in order to identify unknown parameters. For the effectiveness of this proposed procedure, parameter identification problems are numerically analyzed for an elastic plain strain field represented by finite element models under several conditions. And numerical examples show the usefulness of this method in parameter identification of a plain strain field.

Test for the choice of approximative models in nonlinear regression when the variance is unknown

Nonlinear regression models are used in many fields. Often the regression for observation-covariate pairs (X(ti), ti) is modeled as X(ti) =f(ti) + [d](fi), i = 1,. .. , n, where f is the continuous possible nonlinear mean function, while the ([d](ti))i=1.....nare zero mean, i.i.d. random errors having finite variance [d]2. The least squares methods for estimation of f are usually based upon a given parametric form of f. In this article we develop two statistical tests, one for testing that f belongs to a given class of functions possibly discontinuous in their first derivative, and another one for comparing two such classes. This is done by introducing an appropriate estimate of the unknown variance [d]2. The numerical results of a simulation study seem satisfactory.

An unbiased and computationally efficient LS estimation method for identifying parameters of 2D noncausal SAR models

An unbiased and computationally efficient modified least squares (LS) estimation method for identifying parameters of two-dimensional noncausal simultaneous autoregressive models is presented. Some intuitive and mathematical proof of the unbiasedness of the method are given, and a recursive in-order fast algorithm to implement it is introduced. Computer simulation results are given to sustain the theoretical analysis. Both the theoretical analysis and the computer simulation show that the method possesses much higher estimation accuracy and lower computational complexity than the conventional LS estimation method. Compared to the approximate maximum-likelihood method of Kashyap and Chellappa (1983), the scheme is much faster, has the same estimation accuracy, and is parallelizable. >

A Study on Parameter Optimization With Numerical Heat Conduction Model For Circumferential Gas Tungsten Arc (GTA) Welding of Thin Pipes

One of the important problems to be solved in welding engineering is to develop a mathematical method for the determination of optimum process parameters. In order to estimate the optimal process parameters in circumferential gas tungsten arc (GTA) welding of thin pipes, an objective was chosen to maintain a uniform bead width over the full circumferential joint, while the constraints consist of the capacity limit of the welding power source and equipment used. A transient three-dimensional finite difference model (FDM) of the heat conduction flow in the circumferential GTA welding of pipes was adopted for calculating the temperature field considering the temperature-dependent thermal properties of the workpiece, and consequently for determining the resultant bead width in circumferential welding of the pipe workpiece. An efficient optimization model for the numerical heat conduction flow was proposed to evaluate the optimal welding current with a given welding velocity for a required bead width. Its solution was obtained by employing the steepest descent method (SDM), where the initial value of the welding current was estimated by using the linear complementary problem (LCP), and the welding currents in the middle part of the pipe were interpolated by the least-squares approximation method of the second order. The experimental results of the bead formation showed that the developed mathematical model can be effectively applied to obtain the optimal welding condition in circumferential welding of thin pipes, especially thin pipes with a small diameter and high thermal conductivity.

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare e...

Estimation of parameter errors from measurement residuals in state estimation (power systems)

Any error of network parameters affects the value of the measurement residuals calculated in state estimation. Explicit mathematical expressions relating the residuals to the parameter errors are derived. A two-step approach is proposed for parameter error estimation. The first step is to estimate a bias vector which combines the effects of parameter errors and the state of the system. A least-square approach using the measurement residuals calculated in each state estimation run is proposed for the first step. After several state estimation runs, a sequence of such bias vectors is obtained. The second step is to estimate the parameter errors from the sequence of bias vectors. A recursive least-square estimation method is proposed for this step. Theoretical and computational issues of the proposed method are addressed. Test results are presented. >

An Introduction to Generalized Linear Models.

Part 1 Background scope notation distributions derived from normal distribution. Part 2 Model fitting: plant growth sample birthweight sample notation for linear models exercises. Part 3 Exponential family of distributions and generalized linear models: exponential family of distributions generalized linear models. Part 4 Estimation: method of maximum likelihood method of least squares estimation for generalized linear models example of simple linear regression for Poisson responses MINITAB program for simple linear regression with Poisson responses GLIM. Part 5 Inference: sampling introduction for scores sampling distribution for maximum likelihood estimators confidence intervals for the model parameters adequacy of a model sampling distribution for the log-likelihood statistic log-likelihood ratio statistic (deviance) assessing goodness of fit hypothesis testing residuals. Part 6 Multiple regression: maximum likelihood estimation least squares estimation log-likelihood ratio statistic multiple correlation coefficient and R numerical example residual plots orthogonality collinearity model selection non-linear regression. Part 7 Analysis of variance and covariance: basic results one-factor ANOVA two-factor ANOVA with replication crossed and nested factors more complicated models choice of constraint equations and dummy variables analysis of covariance. Part 8 Binary variables and logistic regression: probability distributions generalized linear models dose response models general logistic regression maximum likelihood estimation and the log-likelihood ratio statistic other criteria for goodness of fit least squares methods remarks. Part 9 Contingency tables and log-linear models: probability distributions log-linear models maximum likelihood estimation hypothesis testing and goodness of fit numerical examples remarks. Appendices: conventional parametrizations with sum-to-zero constraints corner-point parametrizations three response variables two response variables and one explanatory variable one response variable and two explanatory variables.

Optimizing search-based identification of stochastic time-delay systems

A frequently used identification method of time-delay systems, namely the method of direct least-squares (LS) estimation with step-by-step search, is significantly improved in some aspects. First, a discussion is made on how to design a proper digital filter in order to derive a unimodal criterion function with respect to the time delay. It is then suggested that the golden section algorithm can be implemented to search the time delay optimizingly, while the system parameters are simultaneously estimated by use of the standard LS method. The presented optimizing search-based identification algorithm exhibits superior performance to the step-by-step search-based method, in that it is more widely applicable and less computational demanding. Illustrative examples are also included.

Analysis of survival data: Estimation and specification tests using asymptotic least squares

In this paper we propose an asymptotic least squares method of estimation and test for proportional hazards and accelerated life models with qualitative covariates. We use Nelson-Aalen estimators of the cumulative hazards on the left-hand side of a regression equation whose coefficients are the covariates effects. In this case, asymptotic least squares reduce to generalized least squares on the corresponding regression equation, giving consistent estimates and specification tests. Examples show that asymptotic least squares and other traditional estimators give very close results.

Identification of stochastic time lag systems in the presence of colored noise

This paper is concerned with identification of stochastic time lag systems corrupted by colored noise. Correlation analysis is used for time delay estimation. Some important extensions are made via time series analysis so that the time delay can be estimated accurately even when the input to the identified plant is a common stationary signal and is correlated with the process noise. By applying a new modified least-squares estimation method to eliminating the bias in the system parameters estimates arising from colored noise, an unbiased identification algorithm for jointly estimating the time delay and system parameters can be obtained based on correlation analysis. The proposed identification schemes may be used both in batches and recursively. Numerical examples are given as well.

Optimal magnet design for NMR

The author considers two difficulties inherent to computational methodology of optimal magnet design for NMR (nuclear magnetic resonance): (a) spectral methods of magnetic field analysis that would be highly accurate, fast, and general, and (b) an optimization strategy that would eliminate physically as well as numerically unstable solutions. The least-squares approximation method of zonal harmonics with iterated accuracy improvement is shown to be an effective alternative for the highly accurate spectral analysis of magnetic field in NMR applications. The principle of stability, the averaging of perturbed solutions, and Monte Carlo evolution methods are recommended for optimal magnet design in NMR. Examples of real-world magnet designs are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A bell-shape pulse generator

A simple circuit for forming the function Y=exp(-X/sup 2//2) is described. The original curve is approximated with three quadratic parabolas using the method of least-squares estimation. The approximation function is generated through analog integration of a linear voltage with appropriate shape.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A Recursive Method of Nonlinear System Identification Using Block-oriented Models

The paper deals with recursive identification of nonlinear dynamic systems using block-oriented models. The models consist of various combinations of linear dynamic systems and no-memory nonlinear blocks. Special forms of block-oriented models are presented based on the separation of key terms in general block descriptions. Incorporating of these models into a recursive identification scheme as adjustable models is discussed. The resulting adjustable nonlinear models are linear in all the parameters. This enables simultaneous estimation both the parameters of linear pulse transfer functions and the coefficients of polynomials approximating nonlinear characteristics. The method of recursive least-squares estimation is applied and is extended including sensitivity coefficients.

On-line Modified Least-squares Parameter Estimation of Linear Systems with Input-output Data Polluted by Measurement Noises

It is well known that in practical situations observed inputoutput data of an identified plant are usually polluted by measurement noises. In this case the ordinary least-squares estimates of the system parameters are biased. In order to obtain consistent estimates a new type of modified least-squares (MLS) estimation method is presented in this paper. It is shown that the estimation biases can be determined if the variances of the measurement noises can be obtained accurately. A designed first-order prefilter is connected parallelly to the input of the identified system. On the basis of asymptotic analysis, the noise variances can be estimated correctly by using the processed sampled data. Both batch algorithm and recursive algorithm are presented. It is shown that the presented MLS method gives consistent estimates without a priori knowledge of the input and output noises. The Monte-Carlo stochastic simulation results are presented to support the theoretical discussions.

Unbiased Identification of Linear Time-delay Systems Corrupted by Correlated Noise

Identification of linear systems with unknown time delay, where the systems are corrupted by correlated noise, is studied in this paper. The time delay is estimated by using correlation analysis. Some improvements are made via time series analysis so that the time delay can be accurately estimated even when the system noise is correlated with the input. The ordinary least-squares (LS) estimation method is used to estimate the system parameters. Moreover, combining with a new method for eliminating the estimation biases arising from correlated noise (Zheng and Feng, 1987), an unbiased identification algorithm for jointly estimating the time delay and the system parameters can be obtained on the basis of correlation analysis. The calculation may be made both in batches and recursively. It is also shown that the proposed method can be used to identify systems with unknown time delay in closed loop condition.

Well test interpretation for reservoirs with a single linear no-flow barrier

Sealing faults, which cut formations with or without contrasting rock properties, have a profound impact on transient pressure behavior of nearby wells. New analytical solutions with wellbore storage and skin effects are presented for the interpretation of the wellbore pressure with and without wellbore flow rate (measured simultaneously with the pressure) for well located at a distance from a single fault. In the absence of measured downhole flow rate, both constant wellbore storage and exponential decline flow rate models are considered in this work. These new solutions offer a potential for the estimation of permeability, skin, and wellbore storage coefficient. Additionally, the well-to-fault distance can also be estimated. Application of the new solutions, using the logarithmic convolution, deconvolution, and nonlinear least-squares estimation methods, is demonstrated through examples.

Subpixel edge detection and estimation with a microprocessor-controlled line scan camera

A microprocessor-controlled line scan camera system for measuring edges and lengths of steel strips is described, and the problem of subpixel edge detection and estimation in a line image is considered. The edge image is assumed to change gradually in its intensity, and the true edge location may be between pixels. Detection and estimation of edges are based on measurement of gray values of the line images at a limited number of pixels. A two-stage approach is presented. At the first stage, a computationally simple discrete-template-matching method is used to place the estimated edge point to the nearest pixel value. Three second-stage methods designed for subpixel estimation are examined. The modified Chebyshev polynomial and the three-point interpolation method do not require much knowledge on the shape of the edge intensity. If the functional form of the edge is known, a least-square estimation method may be used for better accuracy. In the case of nonstationary Poisson noise, a recursive maximum-likelihood method for the first-stage edge detection, followed by subpixel estimation, is proposed. >

Parametric identification of nonlinear dynamic systems based on nonlinear crosscorrelation functions

The paper introduces the two-step identification method of least-squares parameter estimation based on correlation functions, for nonlinear dynamic systems with linear parameters. It deals with the identification of the input/ output parameter models working in open- and closed-loops. The relation between the structure of a model, the choice of multipliers and the necessaryshifting time domain are demonstrated. The effect of the noise to the estimation results are shown. The theoretical considerations are illustrated by digital and hybrid simulations and by the identification of a heat exchanger

DNA evolutionary rates in nine-primaried passerine birds.

Differences in single-copy nuclear-DNA sequences among 13 species of passerine birds were measured using DNA-DNA hybridization. A matrix of pairwise dissimilarity values (delta mode distances) was constructed from analysis of fitted thermal dissociation curves. A least-squares method of phylogenetic estimation was used to construct two topologies from the distance matrix, one constraining branch lengths of sister taxa to be equal and the other permitting such lengths to vary. These topologies were identical in the pattern of branching of taxa, and the difference in their sums of squares was not statistically significant, suggesting that rates of DNA evolution in sister groups of nine-primaried oscines are equal. A nonparametric test for nonrandom variation in distances of sister groups to outgroup taxa revealed no statistically significant deviation from random variation that would be expected as a result of measurement error. However, the level of measurement error was such that rates of DNA evolution in sister taxa could vary by as much as 10% without being detected with the statistical methods used here.

Applications of filtering theory for the inversion of temporal chemical systems

This paper presents a method for inverting temporal experimental data from chemical models to obtain estimates of unknown parameters. Most of the models under consideration are deterministic and we assume that the measurements obtained from experimental observations are represented as the solution of a differential equation containing the variables of the model. To incorporate any extraneous laboratory effects that are not included in the model, we assume that these equations are perturbed by a white noise process so that the measurements become time-dependent stochastic variables. A particular measurement is then equivalent to a realization of these variables and applying stochastic estimation theory this realization can be used to obtain estimates of the unknown parameters in the model. As an example of this estimation method, we consider chemical kinetics models with various observational equations and construct an estimator for the unknown reaction rate constants. We also show the estimators for the structural constants in a laser model depending on the representation of the experimental data. In some cases the observations are simulated numerically and we present the parameter estimates as a function of time. The efficiency of the estimation process is calculated as the ratio of the a posteriori variance of the parameter estimator and the Rao–Cramer lower bound. Some issues in the numerical implementation of the filtering equations are discussed and a comparison is made between the minimum least-square estimation method and the filtering method.