Consistency Of Least Squares Estimator Research Articles

On the Almost Sure Convergence of Series of Stationary and Related Nonstationary Variables

Let $\{X_n\}$ be, for example, a weakly stationary sequence or a lacunary system with finite $p$th moment, $1 \leq p \leq 2$, and let $\{a_n\}$ be a sequence of scalars. We obtain here conditions which ensure the almost sure convergence of the series $\sum a_nX_n$. When $\{X_n\}$ is an orthonormal sequence, the classical Rademacher-Menchov theorem is recovered. This is then applied to study the strong consistency of least squares estimates in multiple regression models.

Consistency of least‐squares estimator and its jackknife variance estimator in nonlinear models

AbstractThe purpose of this paper is twofold: (1) We establish the consistency of the least‐squares estimator in a nonlinear modelyi = f(xi,θ) +σiei where the range of the parameter θ is noncompact, the regression function is unbounded, and the σi,'s are not necessarily equal. This extends the results in Jennrich (1969) and Wu (1981). (2) Under the same model, the jackknife estimator of the asymptotic covariance matrix of the least‐squares estimator is shown to be consistent, which provides a theoretical justification of the empirical results in Duncan (1978) and the use of the jackknife method in large‐sample inferences.

Strong consistency of least squares estimates in general ARXd (p, s) system

We establish strong consistency of least squares estimates in the multivariate general ARXd(p, s) system. No stability or regularity assumption on its matrix polynomial A(z) are made. The input signal (UnS) is assumed to have a degree of excitation equal to dp + 5 and an intensity n such that sn/n goes to infinity

Nonlinear Regression With Variance Components

Abstract The nonlinear model with variance components, which combines a nonlinear model for the mean with additive random effects, is applicable to split-plot and nested experiments. We propose two methods of estimation for the parameters of the nonlinear model for the mean: (1) estimated generalized least squares (EGLS), and (2) maximum likelihood (MLE) by the method of scoring. Using a generalization of Klimko and Nelson's theorem on strong consistency of least squares estimators, it is possible to show that both the MLE and the EGLS estimators are strongly consistent, asymptotically normal, and asymptotically efficient.

Nonlinear Regression with Variance Components

Abstract The nonlinear model with variance components, which combines a nonlinear model for the mean with additive random effects, is applicable to split-plot and nested experiments. We propose two methods of estimation for the parameters of the nonlinear model for the mean: (1) estimated generalized least squares (EGLS), and (2) maximum likelihood (MLE) by the method of scoring. Using a generalization of Klimko and Nelson's theorem on strong consistency of least squares estimators, it is possible to show that both the MLE and the EGLS estimators are strongly consistent, asymptotically normal, and asymptotically efficient.

Consistency of linear and quadratic least squares estimators in regression models with covariance stationary errors

The least squres invariant quadratic estimator of an unknown covariance function of a stochastic process is defined and a sufficient condition for consistency of this estimator is derived. The mean value of the observed process is assumed to fulfil a linear regresion model. A sufficient condition for consistency of the least squares estimator of the regression parameters is derived, too.

Consistency Via Type 2 Inequalities: A Generalization of Wu's Theorem

Wu introduced a new technique for proving consistency of least-squares estimators in nonlinear regression. This paper extends his results in three directions. First, we consider the minimization of arbitrary functions (M-estimators instead of least squares). Second, we use an improved type 2 inequality. Third, an extension of Kronecker's lemma yields a more powerful result.

Least squares estimation of the parameters of a stochastic difference equation with polynomial regression component

In this paper, strong consistency of least squares estimates of the coefficients of stochastic difference equations with polynomial regression components, has been established, using martingale arguments, under autoregressive, partially explosive and purely explosive situations. The asymptotic normality of these estimates have also been discussed in this paper.

Strong consistency of least squares estimates in linear regression models driven by semimartingales

Multiple linear regression models with non random regressors in continuous time are considered. The strong consistency of least squares estimates is established under minimal assumptions on the design when the process of errors is a semimartingale satisfying some regularity condition.

Least Squares Regression with Integrated or Dynamic Regressors under Weak Error Assumptions

This paper establishes consistency of least squares estimators in (i) a multiple regression model with integrated regressors and explosive, non-mixing errors, and (ii) a dynamic linear regression model with regressors and errors that may have infinite variances. In the former context, the asymptotic distribution of the least squares estimator also is obtained, in certain cases.

Convergence systems and strong consistency of least squares estimates in regression models

A recent theorem of T. L. Hai, H. Robbins, and C. Z. Wei ( J. Multivariate Anal. 9 (1979), 343–362) is extended to a more general form which unifies previous results in the literature on the strong consistency of least squares estimates in multiple regression models with nonrandom regressors. In particular the issue of strong consistency of the least squares estimate in the Gauss-Markov model, in the i.i.d. model with infinite second moment, and in general time series models is examined. In this connection, some basic properties of convergence systems are also obtained and are applied to the strong consistency problem.

Asymptotic Theory of Nonlinear Least Squares Estimation

For a linear regression model, the necessary and sufficient condition for the asymptotic consistency of the least squares estimator is known. An analogous condition for the nonlinear model is considered in this paper. The condition is proved to be necessary for the existence of any weakly consistent estimator, including the least squares estimator. It is also sufficient for the strong consistency of the nonlinear least squares estimator if the parameter space is finite. For an arbitrary compact parameter space, its sufficiency for strong consistency is proved under additional conditions in a sense weaker than previously assumed. The proof involves a novel use of the strong law of large numbers in $C(S)$. Asymptotic normality is also established.

Strong Consistency of Least Squares Estimators in Regression with Correlated Disturbances

This note considers, under minimal assumptions, the strong consistency of least squares estimates in regression with correlated errors.

A Note on Strong Consistency of Least Squares Estimators in Regression Models with Martingale Difference Errors

Conditions for the pointwise consistency of weighted least squares estimators from multivariate regression models with martingale difference errors are given in terms of the relative rates at which certain quadratic forms diverge to infinity.

Strong Consistency of Least Squares Estimators in Linear Regression Models

For the linear regression model $y = X \beta + u$ with stochastic regressor matrix, strong consistency of the least squares estimator of $\beta$ is proved in the case of martingale difference errors and predetermined regressors and for the case where errors and regressors are orthogonal up to the second order. The results obtained are applied to parameter estimation in autoregressive processes, leading to strong consistency if the errors are quasi-independent up to the fourth order.

Strong consistency of least squares estimates in multiple regression II

The strong consistency of least squares estimates in multiple regression models is established under minimal assumptions on the design and weak dependence and moment restrictions on the errors.

Strong Consistency of Least Squares Estimates in Dynamic Models

The least squares estimate of the parameter matrix $\mathbf{B}$ in the model $\mathbf{y}_t = \mathbf{B'x}_t + \mathbf{u}_t$, where $\mathbf{u}_t$ is an $m$-component vector of unobservable disturbances and $x_t$ is a $p$-component vector, converges to $\mathbf{B}$ with probability one under certain conditions on the behavior of $x_t$ and $\mathbf{u}_t$. When $\mathbf{x}_t$ is stochastic and the conditional expectation of $\mathbf{u}_t$ given $\mathbf{x}_s$ for $s \leqslant t$ and $\mathbf{u}_t$ for $s < t$ is zero, then the least squares estimates are strongly consistent if the inverse of $\mathbf{A}_T = \sigma^T_{t=1} \mathbf{x}_t\mathbf{x}'_t$, where $T$ is the sample size, converges to the zero matrix and if the ratio of the largest to the smallest characteristic root of $\mathbf{A}_T$ is bounded with probability one.

Strong consistency of least squares estimates in multiple regression

The strong consistency of least squares estimates in multiple regression models with independent errors is obtained under minimal assumptions on the design and weak moment conditions on the errors.

Parameter estimation in nonlinear regression models1

Publisher Summary This chapter discusses the concept of parameter estimation in nonlinear regression models. As for linear models, the least squares method occurs as the most important estimation method because many favorable properties known from the linear theory can be at least asymptotically assured. The solution of the corresponding nonlinear optimization problem requires in general iterative methods. In concrete case studies, the selection of an appropriate procedure from the large set of iterative methods proposed in the literature, the choice of starting guesses, incremental change, and step size requires a lot of experience. The examples of nonlinear regression models are empirical growth curves, exponential models, and Coob–Douglas models. The least squares estimation is the main estimation method in nonlinear regression. The formulation as an inadequate least squares problem may be needed because of the fact that the numerical treatment may be much simpler or, looking on statistical efficiency, it could seem to be reasonable to diminish the dimension of parameters. However, the investigation of weighted inadequate least squares approximation (WILSA) is of basic importance, taking into consideration that the model choice problem in the nonlinear case is difficult. The chapter discusses the problem of approximating a nonlinear response function in a given linear set of functions, polynomial approximation, the consistency of least squares estimators, asymptotic distribution of least-squares estimators, asymptotic optimality of GLSE without normality, maximum likelihood estimation, robust nonlinear regression, and confidence regions of the model.

Strong Consistency of Least Squares Estimates in Normal Linear Regression

In the usual linear regression model the sample regression coefficients converge with probability one to the population regression coefficients when the dependent variables are normally distributed and the inverse of the second-order moment matrix of the independent variables converges to the zero matrix.