Squares Estimates In Regression Models Research Articles

Strong consistency of the least squares estimator in regression models with adaptive learning

This paper looks at the strong consistency of the ordinary least squares (OLS) estimator in linear regression models with adaptive learning. It is a companion to Christopeit & Massmann (2018) which considers the estimator’s convergence in distribution and its weak consistency in the same setting. Under constant gain learning, the model is closely related to stationary, (alternating) unit root or explosive autoregressive processes. Under decreasing gain learning, the regressors in the model are asymptotically collinear. The paper examines, first, the issue of strong convergence of the learning recursion: It is argued that, under constant gain learning, the recursion does not converge in any probabilistic sense, while for decreasing gain learning rates are derived at which the recursion converges almost surely to the rational expectations equilibrium. Secondly, the paper establishes the strong consistency of the OLS estimators, under both constant and decreasing gain learning, as well as rates at which the estimators converge almost surely. In the constant gain model, separate estimators for the intercept and slope parameters are juxtaposed to the joint estimator, drawing on the recent literature on explosive autoregressive models. Thirdly, it is emphasised that strong consistency is obtained in all models although the near-optimal condition for the strong consistency of OLS in linear regression models with stochastic regressors, established by Lai & Wei (1982a), is not always met.

Strong Consistency of the Least Squares Estimator in Regression Models with Adaptive Learning

This paper looks at the strong consistency of the ordinary least squares (OLS) estimator in a stereotypical macroeconomic model with adaptive learning. It is a companion to Christopeit & Massmann (2017, Econometric Theory) which considers the estimator’s convergence in distribution and its weak consistency in the same setting. Under constant gain learning, the model is closely related to stationary, (alternating) unit root or explosive autoregressive processes. Under decreasing gain learning, the regressors in the model are asymptotically collinear. The paper examines, first, the issue of strong convergence of the learning recursion: It is argued that, under constant gain learning, the recursion does not converge in any probabilistic sense, while for decreasing gain learning rates are derived at which the recursion converges almost surely to the rational expectations equilibrium. Secondly, the paper establishes the strong consistency of the OLS estimators, under both constant and decreasing gain learning, as well as rates at which the estimators converge almost surely. In the constant gain model, separate estimators for the intercept and slope parameters are juxtaposed to the joint estimator, drawing on the recent literature on explosive autoregressive models. Thirdly, it is emphasised that strong consistency is obtained in all models although the near-optimal condition for the strong consistency of OLS in linear regression models with stochastic regressors, established by Lai & Wei (1982), is not always met.

Exact Results on the Inadmissibility of the Feasible Generalized Least Squares Estimator in Regression Models with Non-Spherical Disturbances

In various guises, feasible generalized least squares (FGLS) estimation has occupied an important place in regression analysis for more than 35 years. Past studies on the characteristics of the FGLS estimators are largely based on large sample evaluations, and the important issue of admissibility remains unexplored in the case of the FGLS estimator. In this paper, an exact sufficient condition for the dominance of a Stein-type shrinkage estimator over the FGLS estimator in finite samples based on squared error loss is given. In deriving the condition, we assume that the model's disturbance covariance matrix is unknown except for a scalar multiple. Further, for models with AR(1) disturbances, it is observed that the dominance condition reduces to one that involves no unknown parameter. In other words, in the case of AR(1) disturbances and where the condition for risk dominance is met, the FGLS estimator is rendered inadmissible under squared error loss.

Properties of ordinary least squares estimators in regression models with nonspherical disturbances

A general discussion is presented of the properties of the OLS estimator in regression models where the disturbances do not have a scalar identity covariance matrix. Several new and interesting characterizations are provided together with a synthesis of existing results. A distinguishing feature of the paper is the unified treatment of the issues of efficiency and inference. The theoretical material is illustrated by examples covering a wide range of models of interest to applied econometricians.

Consistency of linear and quadratic least squares estimators in regression models with covariance stationary errors

The least squres invariant quadratic estimator of an unknown covariance function of a stochastic process is defined and a sufficient condition for consistency of this estimator is derived. The mean value of the observed process is assumed to fulfil a linear regresion model. A sufficient condition for consistency of the least squares estimator of the regression parameters is derived, too.

An extension of Lai and Wei's law of the iterated logarithm with applications to time series analysis and regression

For a double array a N = ( a Nn ) −∞<n<∞ of nonrandom constants and independent random variables ε n , under some conditions, Lai and Wei (1982, Ann. Statist. 10 320–335) proved the law of the iterated logarithms for S N = Σ n a Nn ε n . This paper extends this law to the case of S ̃ N = Σ na Nnu n , where { u n } is a linear series of the form u n = Σ j=0 ∞ κ j ε n− j . As applications of the theorem we establish the law of the iterated logarithm for finite Fourier transforms in spectral analysis and for least squares estimates in regression models in which the random disturbances form a linear series.

A Linear Programming Proof of Kantorovich's Inequality

Abstract A simple linear programming proof of Kantorovich's inequality is given in this article. This inequality is usually used to compare the variances of the best linear estimates and least squares estimates in regression models.

Partially generalized least squares and two-stage least squares estimators

A class of partially generalized least squares estimators and a class of partially generalized two-stage least squares estimators in regression models with heteroscedastic errors are proposed. By using these estimators a researcher can attain higher efficiency than that attained by the least squares or the two-stage least squares estimators without explicitly estimating each component of the heteroscedastic variances. However, the efficiency is not as high as that of the generalized least squares or the generalized two-stage least squares estimator calculated using the knowledge of the true variances. Hence the use of the term partial.

Convergence systems and strong consistency of least squares estimates in regression models

A recent theorem of T. L. Hai, H. Robbins, and C. Z. Wei ( J. Multivariate Anal. 9 (1979), 343–362) is extended to a more general form which unifies previous results in the literature on the strong consistency of least squares estimates in multiple regression models with nonrandom regressors. In particular the issue of strong consistency of the least squares estimate in the Gauss-Markov model, in the i.i.d. model with infinite second moment, and in general time series models is examined. In this connection, some basic properties of convergence systems are also obtained and are applied to the strong consistency problem.

A Note on Strong Consistency of Least Squares Estimators in Regression Models with Martingale Difference Errors

Conditions for the pointwise consistency of weighted least squares estimators from multivariate regression models with martingale difference errors are given in terms of the relative rates at which certain quadratic forms diverge to infinity.

Weak and strong consistency of the least squares estimators in regression models

Weak and strong consistency of the least squares estimators in regression models

On Asymptotic Distribution Theory in Segmented Regression Problems-- Identified Case

This paper deals with the asymptotic distribution theory of least squares estimators in regression models having different analytical forms in different regions of the domain of the independent variable. An important special case is that of broken line regression, in which each segment of the regression function is a different straight line. The residual sum of squares function has many corners, and so classical least squares techniques cannot be directly applied. It is shown, however, that the problem can be transformed into a new problem in which the sum of squares function is locally smooth enough to apply the classical techniques. Asymptotic distribution theory is discussed for the new problem and it is shown that the results are also valid for the original problem. Results related to the usual normal theory are derived.