Generalized Least-squares Procedure Research Articles

Structural Equation Modeling With Unknown Population Distributions: Ridge Generalized Least Squares

This article proposes 2 classes of ridge generalized least squares (GLS) procedures for structural equation modeling (SEM) with unknown population distributions. The weight matrix for the first class of ridge GLS is obtained by combining the sample fourth-order moment matrix with the identity matrix. The weight matrix for the second class is obtained by combining the sample fourth-order moment matrix with its diagonal matrix. Empirical results indicate that, with data from an unknown population distribution, parameter estimates by ridge GLS can be much more accurate than those by either GLS or normal-distribution-based maximum likelihood; and standard errors of the parameter estimates also become more accurate in predicting the empirical ones. Rescaled and adjusted statistics are proposed for overall model evaluation, and they also perform much better than the default statistic following from the GLS method. The use of the ridge GLS procedures is illustrated with a real data set.

Re-weighting estimation of the coefficients in the varying coefficient model with heteroscedastic errors

ABSTRACTThis article explores the estimation problem of the coefficients in the varying coefficient model with heteroscedastic errors. Specifically, we first present a method for estimating the variance function of the error term and the resulting estimator is proved to be consistent. Then, motivated by the generalized least-squares procedure for dealing with heteroscedasticity in the linear regression literature, we re-weight each squared residual term in the local linear smoother with the inverse of the corresponding estimated error variance to construct estimates of the coefficients. Simulation experiments and practical data analysis conducted demonstrate that the re-weighting approach can improve the accuracy of the coefficient estimates under a finite sample size, especially when the error heteroscedasticity is strong.

A Balanced System of U.S. Industry Accounts and Distribution of the Aggregate Statistical Discrepancy by Industry

This article describes and illustrates a generalized least squares (GLS) method that systematically incorporates all available information on the reliability of initial data in the reconciliation of a large disaggregated system of national accounts. The GLS method is applied to reconciling the 1997 U.S. Input-Output and Gross Domestic Product (GDP)-by-industry accounts with benchmarked GDP estimated from expenditures. The GLS procedure produced a balanced system of industry accounts and distributed the aggregate statistical discrepancy by industry according to the estimated relative reliabilities of initial estimates. The study demonstrates the empirical feasibility and computational efficiency of the GLS method for large accounts reconciliation.

Asymptotics for LS, GLS, and Feasible GLS Statistics in an AR(1) Model with Conditional Heteroskedasticity

This paper considers a first-order autoregressive model with conditionally heteroskedastic innovations. The asymptotic distributions of least squares (LS), infeasible generalized least squares (GLS), and feasible GLS estimators and t statistics are determined. The GLS procedures allow for misspecification of the form of the conditional heteroskedasticity and, hence, are referred to as quasi-GLS procedures. The asymptotic results are established for drifting sequences of the autoregressive parameter and the distribution of the time series of innovations. In particular, we consider the full range of cases in which the autoregressive parameter rho_{n} satisfies (i) n(1 - rho_{n}) approaching infinity and (ii) n(1 - rho_{n}) approaching h in [0,infinity) as n approaching infinity, where n is the sample size. Results of this type are needed to establish the uniform asymptotic properties of the LS and quasi-GLS statistics.

Asymptotics for LS, GLS, and feasible GLS statistics in an AR(1) model with conditional heteroskedasticity

We consider a first-order autoregressive model with conditionally heteroskedastic innovations. The asymptotic distributions of least squares (LS), infeasible generalized least squares (GLS), and feasible GLS estimators and t statistics are determined. The GLS procedures allow for misspecification of the form of the conditional heteroskedasticity and, hence, are referred to as quasi-GLS procedures. The asymptotic results are established for drifting sequences of the autoregressive parameter ρn and the distribution of the time series of innovations. In particular, we consider the full range of cases in which ρn satisfies n(1−ρn)→∞ and n(1−ρn)→h1∈[0,∞) as n→∞, where n is the sample size. Results of this type are needed to establish the uniform asymptotic properties of the LS and quasi-GLS statistics.

An exploratory analysis of error components in hydrological regression modeling

The use of the generalized least squares (GLS) technique for estimation of hydrological regression models has become good practice in hydrology. Through a regression model, a simple link between a particular hydrological variable and a set of catchment descriptors can be established. The regression residuals can be treated as the sum of sampling errors in the hydrological variable and errors in the regression model. This article presents a method for exploring and developing a parameterized form for the cross‐correlation between the regression model errors. Given an initial GLS analysis, a reweighted set of regression residuals is defined such that the covariance of these residuals is essentially similar to that of the model errors. The cross‐products of the reweighted regression residuals, pooled within bins, are then used to identify a structure and to fit a parameterized form for the cross‐correlations of the regression errors. These estimated cross‐correlations are then used to inform improved GLS and reweighting steps, leading to a recursive procedure. The main advantage of the recursive GLS procedure is that it allows for a simple demonstration of the actual existence of model error correlation as well as for exploring suitable models for the correlation. However, while the procedure for identifying a parametric description for the model error correlation is reasonable, estimation of the parameters within the recursive procedure is affected by the data‐binning step. Thus it is suggested that once a structure for the correlation has been decided, further data analysis—such as final decisions about variables included in the regression model and final estimation of parameters—should be undertaken within a maximum likelihood framework. The procedure has been tested on annual maximum flow data from 602 catchments located throughout the United Kingdom. A set of Monte Carlo experiments further confirmed the ability of the recursive GLS procedure to correctly identify and estimate the true model error correlation.

DFT quartic force field of acetonitrile by using a generalized least-squares procedure

Our least-squares fitting procedure, which jointly uses the energy and gradient data obtained from DFT calculations, is applied to construct the anharmonic force field of acetonitrile. The quartic force field performed at B3LYP/cc-pVTZ level of theory from 101 simplex-sum energy and gradient data show very good agreement in comparison with the classical 710 energy calculations. Vibrational energy transitions obtained from the two force fields in the range of 300–3000 cm −1 are consistent with each other.

Fitting the Linear Model of Coregionalization by Generalized Least Squares

In geostatistical studies, the fitting of the linear model of coregionalization (LMC) to direct and cross experimental semivariograms is usually performed with a weighted least-squares (WLS) procedure based on the number of pairs of observations at each lag. So far, no study has investigated the efficiency of other least-squares procedures, such as ordinary least squares (OLS), generalized least squares (GLS), and WLS with other weighing functions, in the context of the LMC. In this article, we compare the statistical properties of the sill estimators obtained with eight least-squares procedures for fitting the LMC: OLS, four WLS, and three GLS. The WLS procedures are based on approximations of the variance of semivariogram estimates at each distance lag. The GLS procedures use a variance–covariance matrix of semivariogram estimates that is (i) estimated using the fourth-order moments with sill estimates (GLS1), (ii) calculated using the fourth-order moments with the theoretical sills (GLS2), and (iii) based on an approximation using the correlation between semivariogram estimates in the case of spatial independence of the observations (GLS3). The current algorithm for fitting the LMC by WLS while ensuring the positive semidefiniteness of sill matrix estimates is modified to include any least-squares procedure. A Monte Carlo study is performed for 16 scenarios corresponding to different combinations of the number of variables, number of spatial structures, values of ranges, and scale dependence of the correlations among variables. Simulation results show that the mean square error is accounted for mostly by the variance of the sill estimators instead of their squared bias. Overall, the estimated GLS1 and theoretical GLS2 are the most efficient, followed by the WLS procedure that is based on the number of pairs of observations and the average distance at each lag. On that basis, GLS1 can be recommended for future studies using the LMC.

Comparative methods for the analysis of continuous variables: geometric interpretations.

This study is concerned with statistical methods used for the analysis of comparative data (in which observations are not expected to be independent because they are sampled across phylogenetically related species). The phylogenetically independent contrasts (PIC), phylogenetic generalized least-squares (PGLS), and phylogenetic autocorrelation (PA) methods are compared. Although the independent contrasts are not orthogonal, they are independent if the data conform to the Brownian motion model of evolution on which they are based. It is shown that uncentered correlations and regressions through the origin using the PIC method are identical to those obtained using PGLS with an intercept included in the model. The PIC method is a special case of PGLS. Corrected standard errors are given for estimates of the ancestral states based on the PGLS approach. The treatment of trees with hard polytomies is discussed and is shown to be an algorithmic rather than a statistical problem. Some of the relationships among the methods are shown graphically using the multivariate space in which variables are represented as vectors with respect to OTUs used as coordinate axes. The maximum-likelihood estimate of the autoregressive parameter, p, has not been computed correctly in previous studies (an appendix with MATLAB code provides a corrected algorithm). The importance of the eigenvalues and eigenvectors of the connection matrix, W, for the distribution of p is discussed. The PA method is shown to have several problems that limit its usefulness in comparative studies. Although the PA method is a generalized least-squares procedure, it cannot be made equivalent to the PGLS method using a phylogenetic model.

Efficiency analysis of ten estimation procedures for quantitative linear models with autocorrelated errors

Many estimation procedures for quantitative linear models with autocorrelated errors have been proposed in the literature. A number of these procedures have been compared in various ways for different sample sizes and autocorrelation parameters values and for structured or random explanatory vaiables. In this paper, we revisit three situations that were considered to some extent in previous studies, by comparing ten estimation procedures: Ordinary Least Squares (OLS), Generalized Least Squares (GLS), estimated Generalized Least Squares (six procedures), Maximum Likelihood (ML), and First Differences (FD). The six estimated GLS procedures and the ML procedure differ in the way the error autocovariance matrix is estimated. The three situations can be defined as follows: Case 1, the explanatory variable x in the simple linear regression is fixed; Case 2,x is purely random; and Case 3x is first-order autoregressive. Following a theoretical presentation, the ten estimation procedures are compared in a Monte Carlo study conducted in the time domain, where the errors are first-order autoregressive in Cases 1-3. The measure of comparison for the estimation procedures is their efficiency relative to OLS. It is evaluated as a function of the time series length and the magnitude and sign of the error autocorrelation parameter. Overall, knowledge of the model of the time series process generating the errors enhances efficiency in estimated GLS. Differences in the efficiency of estimation procedures between Case 1 and Cases 2 and 3 as well as differences in efficiency among procedures in a given situation are observed and discussed.

COMPARATIVE METHODS FOR THE ANALYSIS OF CONTINUOUS VARIABLES: GEOMETRIC INTERPRETATIONS

This study is concerned with statistical methods used for the analysis of comparative data (in which observations are not expected to be independent because they are sampled across phylogenetically related species). The phylogenetically independent contrasts (PIC), phylogenetic generalized least-squares (PGLS), and phylogenetic autocorrelation (PA) methods are compared. Although the independent contrasts are not orthogonal, they are independent if the data conform to the Brownian motion model of evolution on which they are based. It is shown that uncentered correlations and regressions through the origin using the PIC method are identical to those obtained using PGLS with an intercept included in the model. The PIC method is a special case of PGLS. Corrected standard errors are given for estimates of the ancestral states based on the PGLS approach. The treatment of trees with hard polytomies is discussed and is shown to be an algorithmic rather than a statistical problem. Some of the relationships among the methods are shown graphically using the multivariate space in which variables are represented as vectors with respect to OTUs used as coordinate axes. The maximum-likelihood estimate of the autoregressive parameter, ρ, has not been computed correctly in previous studies (an appendix with MATLAB code provides a corrected algorithm). The importance of the eigenvalues and eigenvectors of the connection matrix, W, for the distribution of ρ is discussed. The PA method is shown to have several problems that limit its usefulness in comparative studies. Although the PA method is a generalized least-squares procedure, it cannot be made equivalent to the PGLS method using a phylogenetic model.Corresponding Editor: T. Garland Jr.

Regional hydrologic analysis: Ordinary and generalized least squares revisited

Generalized least squares (GLS) regional regression procedures have been developed for estimating river flow quantiles. A widely used GLS procedure employs a simplified model error structure and average covariances when constructing an approximate residual error covariance matrix. This paper compares that GLS estimator ( ), an idealized GLS estimator ( ) based on the simplifying assumptions of with true underlying statistics in a region, the best possible GLS estimator ( ) obtained using the true residual error covariarice matrix, and the ordinary least squares estimator ( ). Useful analytic expressions are developed for the variance of , and For previously examined cases the average sampling mean square error (mses) of was the same as the mses of and the mses of usually was larger than the mses of both and The loss in efficiency of was mostly due to estimating streamflow statistics employed in the construction of the residual error covariance matrix rather than the simplifying assumptions in presently employed GLS estimators. The new analytic expressions were used to compare the performance of the OLS and GLS estimators for new cases representing greater model variability across sites as well as the effect return period has on the estimators' relative performance. For a more heteroscedastic model error variance and larger return periods, some increase in the mses of relative to the mses of was observed.

Generalized least squares and empirical bayes estimation in regional partial duration series index‐flood modeling

A regional estimation procedure that combines the index‐flood concept with an empirical Bayes method for inferring regional information is introduced. The model is based on the partial duration series approach with generalized Pareto (GP) distributed exceedances. The prior information of the model parameters is inferred from regional data using generalized least squares (GLS) regression. Two different Bayesian T‐year event estimators are introduced: a linear estimator that requires only some moments of the prior distributions to be specified and a parametric estimator that is based on specified families of prior distributions. The regional method is applied to flood records from 48 New Zealand catchments. In the case of a strongly heterogeneous intersite correlation structure, the GLS procedure provides a more efficient estimate of the regional GP shape parameter as compared to the usually applied weighted regional average. If intersite dependence is ignored, the uncertainty of the regional estimator may be seriously underestimated and erroneous conclusions with respect to regional homogeneity may be drawn. The GLS procedure is shown to provide a general framework for a reliable evaluation of parameter uncertainty as well as for an objective appraisal of regional homogeneity. A comparison of the two different Bayesian T‐year event estimators reveals that generally the simple linear estimator is adequate.

A method for calibrating molecular clocks and its application to animal mitochondrial DNA.

A generalized least-squares procedure is introduced for the calibration of molecular clocks and applied to the complete mitochondrial DNA sequences of 13 animal species. The proposed technique accounts for both nonindependence and heteroscedasticity of molecular-distance data, problems that have not been taken into to account in such analyses in the past. When sequence-identity data are transformed to account for multiple substitutions/site, the molecular divergence scales linearly with time, but with substantially more variation in the substitution rate than expected under a Poisson model. Significant levels of divergence are predicted at zero divergence time for most loci, suggesting high levels of site-specific heterozygosity among mtDNA molecules establishing in sister taxa. For nearly all loci, the baseline heterozygosity is lower and the substitution rate is higher in mammals relative to other animals. There is considerable variation in the evolutionary rate among loci but no compelling evidence that the average rate of mtDNA evolution is elevated with respect to that of nuclear DNA. Using the observed patterns of interspecific divergence, empirical estimates are derived for the mean coalescence times of organelles colonizing sister taxa.

A Bayesian analysis of the generalized least-squares procedure for functional relationships

In this paper we present an exact Bayesian justification of the least-squares estimates for the functional relationships situation (Sprent, 1966) using the Reilly/Palatino-Leal Jeffreys' prior. It is shown that the profile posterior and the marginal posterior do not give conflicting information about the parameter of interest if an orthogonal parametrization (Cox and Reid, 1987) is adopted for the nuisance parameter.

The generated regressor correction: Impacts upon inferences in hypothesis testing

The recent attention to the role of expectations in economic theory has increased the use of empirical models in which a regressor is generated by an “auxiliary” equation specifying the expectations formation rule. Pagan (1984) shows that the widely used two-step ordinary least squares (OLS) procedures for estimating such models produce biased estimates of the covariance matrix of parameter estimates and may lead to erroneous conclusions in hypothesis tests. This research examines the impacts upon test conclusions of a generalized least squares (GLS) correction proposed by Hoffman (1987). The evaluation is based upon an economic model representative of the one used in many of the existing studies utilizing generated regressors. Test conclusions from uncorrected OLS and corrected GLS procedures are compared for four specifications of the auxiliary equation. Results indicate that the impact of the correction procedure is substantial when the specification of the auxiliary equation is relatively simple.

Ringdown of inertial waves during spin-up from rest of a fluid contained in a rotating cylindrical cavity

Time-dependent complex eigenfrequencies for traveling inertial waves in a rotating cylinder of fluid have been measured while the fluid was spun up from rest. The waves were excited by precessing the lid of the cylindrical container at a frequency near a predicted eigenfrequency for a given rotation speed. Time sequences of disturbance pressure differences measured during subsequent free ringdown of the excited wave were inverted to recover inertial eigenfrequency and decay rate using a generalized least-squares procedure.

Regional Hydrologic Analysis, 2, Model‐Error Estimators, Estimation of Sigma and Log‐Pearson Type 3 Distributions

For a number of cases, weighted and generalized least squares regression procedures were shown by Stedinger and Tasker (1985, 1986) to provide better estimators of the parameters of models of hydrologic statistics as a function of drainage basin characteristics. These procedures require estimation of the true model's standard error of prediction. Evaluated here are three model error estimators for weighted least squares (WLS) regression and two for generalized least squares (GLS) regression. The generalized mean square estimator employed by Stedinger and Tasker (1985) was nearly unbiased, and comparable in accuracy to and easier to compute than the maximum likelihood estimator. GLS and WLS regression estimators of the log‐streamflows' standard deviation are shown to be substantially more accurate than ordinary least squares estimators. Finally, the consequences and impact of nonlognormal streamflows were evaluated. In the cases considered, GLS procedures continued to perform well even when the normality assumption was violated.

Regional Hydrologic Analysis: 1. Ordinary, Weighted, and Generalized Least Squares Compared

Streamflow gaging networks provide hydrologic information which is often used to derive relationships between physiographic variables and Streamflow statistics. This paper compares the performance of ordinary, weighted, and generalized least squares estimators of the parameters of such regional hydrologic relationships in situations where the available Streamflow records at gaged sites can be of different and widely varying lengths and concurrent flows at different sites are cross‐correlated. A Monte Carlo study illustrates the performance of an ordinary least squares (OLS) procedure and an operational generalized least squares (GLS) procedure which accounts for and directly estimates the precision of the predictive model being fit. The GLS procedure provided (1) more accurate parameter estimates, (2) better estimates of the accuracy with which the regression model's parameters were being estimated, and (3) almost unbiased estimates of the model error. The OLS approach can provide very distorted estimates of the model's predictive precision (model error) and the precision with which the regression model's parameters are being estimated. A weighted least squares procedure which neglects the cross correlations among concurrent flows does as well as the GLS procedure when the cross correlation among concurrent flows is relatively modest. The Monte Carlo examples also explore the value of Streamflow records of different lengths in regionalization studies.

FIML estimation of dynamic econometric systems from inconsistent data

Economic data are typically inconsistent, contain measurement errors and are sometimes unavailable. The paper therefore proposes a bayesian and a generalized least-squares procedure for adjusting the data to satisfy accounting identities, to incorporate subjective views and to recover unobserved data. The resulting posterior observations may be used to estimate the parameters of a multivariate ARMAX model of the economy by maximizing the appropriate log-likelihood criterion subject to the constraints of the Kalman-Bucy filter (generating the optimal estimates of the states of the economy). The paper proposes a generalized reduced-gradient algorithm, based on a state space realization of the ARMAX model, for the estimation procedure, and discusses a number of special cases where the filter need not be implemented. Extensions to the estimation of continuous-time or non-linear models from noisy data are also provided. When the original data contain systematic errors time-decomposition is not valid, hence the...