Restricted Least-squares Estimator Research Articles

Weighted-average least squares (WALS): Confidence and prediction intervals

We extend the results of De Luca et al. (2021) to inference for linear regression models based on weighted-average least squares (WALS), a frequentist model averaging approach with a Bayesian flavor. We concentrate on inference about a single focus parameter, interpreted as the causal effect of a policy or intervention, in the presence of a potentially large number of auxiliary parameters representing the nuisance component of the model. In our Monte Carlo simulations we compare the performance of WALS with that of several competing estimators, including the unrestricted least-squares estimator (with all auxiliary regressors) and the restricted least-squares estimator (with no auxiliary regressors), two post-selection estimators based on alternative model selection criteria (the Akaike and Bayesian information criteria), various versions of frequentist model averaging estimators (Mallows and jackknife), and one version of a popular shrinkage estimator (the adaptive LASSO). We discuss confidence intervals for the focus parameter and prediction intervals for the outcome of interest, and conclude that the WALS approach leads to superior confidence and prediction intervals, but only if we apply a bias correction.

Comparisons of estimators for regression coefficient in a misspecified linear model with elliptically contoured errors

In a misspecified linear regression model with elliptically contoured errors, the exact risks of generalized least squares (GLS), restricted least squares (RLS), preliminary test (PT), Stein-rule (SR) and positive-rule shrinkage (PRS) estimators of regression coefficient are derived. Risk superiority conditions dependent on prior constraint error and the model specification error are obtained. When the model is misspecified and the error terms obey the elliptically contoured distribution, it is shown analytically that the PRS estimator dominates uniformly the SR estimator. However, the PRS and SR estimators do not dominate uniformly the GLS estimator. Furthermore, the dominance of the RLS estimator over the GLS estimator does not hold necessarily even if the model has linear constraint.

Improved Liu-type estimator in partial linear model

In this article, a Liu-type estimation is proposed for the vector-parameter in a partial linear model. This new estimator can be regarded as generalization of the restricted least-squares estimator, the restricted ridge estimator and the restricted Liu estimator. We also obtain the asymptotic distributional bias and risk of these estimators and we also discuss some properties of the new estimator. The selection of the tuning parameter in the proposed estimator is also presented. Finally, a simulation study is presented to explain the performance of the new estimator.

Restricted Two Parameter Ridge Estimator

SummaryIn 2005 Lipovetsky and Conklin proposed an estimator, the two parameter ridge estimator (TRE), as an alternative to the ordinary least squares estimator (OLSE) and the ordinary ridge estimator (RE) in the presence of multicollinearity, and in 2006 Lipovetsky improved the two parameter model. In this paper, we introduce two new estimators, one of which is the modified two parameter ridge estimator (MTRE) defined by following Swindel's paper of 1976. The other one is the restricted two parameter ridge estimator (RTRE) which is derived by setting additional linear restrictions on the parameter vectors. This estimator is a generalization of the restricted least squares estimator (RLSE) and includes the restricted ridge estimator (RRE) proposed by Groß in 2003. A numerical example is provided and a simulation study is conducted for the comparisons of the RTRE with the OLSE, RLSE, RE, RRE and TRE.

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.

More on the restricted ridge regression estimation

Several alternative methods for derivation of the restricted ridge regression estimator (RRRE) are provided. Theoretical comparison and relationship of RRRE with related methods for regression with the multicollinearity problem are described. We also find inter-connections among RRRE, ordinary ridge regression estimator (ORRE), restricted least squares estimator (RLSE), modified ridge regression estimator (MRRE) and restricted modified generalized ridge estimator (RMGRE). Finally, numerical comparison, in addition to theoretical derivation, is also conducted with a Monte Carlo simulation and a real data example.

Ridge regression methodology in partial linear models with correlated errors

In this article, a generalized restricted difference-based ridge estimator is defined for the vector parameter in a partial linear model when the errors are dependent. It is suspected that some additional linear constraints may hold on to the whole parameter space. The estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. The risk functions of the proposed estimators are derived under balanced loss function. Finally, the performance of the new estimators is evaluated by a simulated data set.

Semiparametric Ridge Regression Approach in Partially Linear Models

In this article, we introduce a semiparametric ridge regression estimator for the vector-parameter in a partial linear model. It is also assumed that some additional artificial linear restrictions are imposed to the whole parameter space and the errors are dependent. This estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. Asymptotic distributional bias and risk are also derived and the comparison result is then given.

Un teorema di rappresentazione per lo stimatore vincolato

A recent result in partitioned inversion (Faliva and Zoia, 2000) paves the way to shed some more light on constrained estimation, as the paper shows for the linear model. Indeed, a novel closed-form expression for the restricted least-squares(RLS) estimator is derived, which reads as a mirror image of the well-known Theil's(1961) solution and actually turns out to cover the latter as a special case. This leads eventually to an elegant representation theorem for the RLS estimator which combines classical and new results as well.

On the Restricted Liu Estimator in the Gauss–Markov Model

In this paper, we compare two estimators, the RLE (restricted Liu estimator) and the RLSE (restricted least squares estimator) of parameters in linear models under Gauss–Markov models. Using generalized inverse of matrices, we found some equivalency conditions for the superiority of the RLE with respect to the MSE criterion.

Restricted ridge estimation

In this paper, we introduce a ridge estimator for the vector of parameters in a linear regression model when additional linear restrictions on the parameter vector are assumed to hold. The estimator is a generalization of the well-known restricted least-squares estimator and is confined to the (affine) subspace which is generated by the restrictions. Necessary and sufficient conditions for the superiority of the new estimator over the restricted least-squares estimator are derived. Our new estimator is not to be confounded with the restricted ridge regression estimator introduced by Sarkar (Comm. Statist. Theory Methods 21 (1992) 1987).

On preliminary test and shrinkage estimation in linear models with long-memory errors

This paper discusses the problem of estimating a subset of parameters when the complementary subset is possibly redundant, in a linear regression model when the errors are generated from a long-memory process. Such a model arises due to the overmodelling of a situation involving long-memory data. Along with the classical least-squares estimator and restricted least-squares estimator, preliminary test least-squares estimator and shrinkage least-squares estimator are investigated in an asymptotic set-up and their relative performances are studied under contiguous alternatives. The contiguous alternatives under such dependence are fundamentally different from those under the independent errors case.

A note on comparing the unrestricted and restricted least-squares estimators

The problem of comparing the ordinary least-squares estimator β̂ and the restricted least-squares estimator β∗ with respect to a weighted quadratic risk under the normal linear regression model in which restrictions are not known to be true is approached in the literature either (i) by choosing the weight matrix in a special form such that the corresponding necessary and sufficient conditions become operational or (ii) by replacing these necessary and sufficient conditions with weaker sufficient conditions, which may lead, however, to a region of uncertainty where neither dominance of β∗ over β̂ nor dominance of β̂ over β∗ is demonstrable. This note shows that the two approaches are partially reconciliable, in the sense that the class of weight matrices which are appropriate for approach (i) coincides with the class of weight matrices for which the region of uncertainty disappears in approach (ii). Moreover, a number of alternative characterizations of this class are given, and a useful invariance property is established.

A note on generalized ridge estimators

It is shown that a necessary and sufficient condition derived by Farebrother (1984)for a generalized ridge estimator to dominate the ordinary least-squares estimator with respect to the mean-square-error-matrix criterion in the linear regression model admits a similar interpretation as the well known criterion of Toro-Viz-carrondo and Wallace (1968)for the dominance of a restricted least-squares estimator over the ordinary least-squares estimator. Two other properties of the generalized ridge estimators, referring to the concept of admissibility, are also pointed out.

Mean square error matrix superiority of estimators under linear restrictions and misspecification

In this paper attention is focussed on the mixed regression estimator (MRE), which is based upon incorrect prior information, and on the restricted least squares estimator (RLSE), which uses valid or invalid restrictions. With respect to the mean square error (MSE) matrix criterion we can derive conditions, under which MRE and RLSE are better than the ordinary least squares estimator (OLSE), especially in the case that the regression model is misspecified. Finally we examine the superiority of the MSE-predictor over the OLS-predictor.

Restricted least squares, pre-test, ols and stein rule estimators: Risk comparisons under model misspecification

The predictive squared error risk behavior of a restricted least squares estimator (RLS), pre-test estimator (PT), and Stein rule (SR) estimator are each compared to the behavior of the OLS estimator when relevant variables are omitted. Risk superiority conditions dependent on prior constraint error and this model specification error in a two-dimensional parameter space are exhibited. Among the consequences of this model misspecification, it is found that (i) perfectly correct prior constraints do not ensure risk superiority of any of the alternatives to OLS, (ii) the risk difference between OLS and PT is no longer bounded, and (iii) the SR no longer dominates OLS. The usefulness of risk superiority hypothesis tests based on the doubly non-centralF distribution is examined.

On the admissibility of restricted least squares estimators

This article examines the question of admissibility of the restricted least squares (RLS) estimator applied to the linear model , where . Straightforward proofs based on generalizations of the results of Cohen (1966) demonstrate that RLS estimators are admissible under predictive squared error risk iff the dimension of the β parameter space minus the number of linearly independent restrictions on β is less than three. A Stein-type estimator that dominates the inadmissible RLS estimator is presented. Some extensions of the results are indicated.

Cubic Splines as a Special Case of Restricted Least Squares

Abstract The standard cubic spline regression method is shown to be a special case of the restricted least-squares estimator. The equivalence of the two procedures under a common set of restrictions is proved. The greater flexibility of the restricted least-squares estimator in terms of the number of restrictions and tests of hypotheses that can be utilized is illustrated by an application to a set of data that has been previously analyzed by the spline method.