Stein-like Estimators Research Articles

Stein-like estimators for causal mediation analysis in randomized trials.

Causal mediation analysis aims to estimate natural direct and natural indirect effects under clearly specified assumptions. Traditional mediation analysis based on Ordinary Least Squares assumes an absence of unmeasured causes to the putative mediator and outcome. When these assumptions cannot be justified, instrumental variable estimators can be used in order to produce an asymptotically unbiased estimator of the mediator-outcome link, commonly referred to as a Two-Stage Least Squares estimator. Such bias removal, however, comes at the cost of variance inflation. A Semi-Parametric Stein-Like estimator has been proposed in the literature that strikes a natural trade-off between the unbiasedness of the Two-Stage Least Squares procedure and the relatively small variance of the Ordinary Least Squares estimator. The Semi-Parametric Stein-Like estimator has the advantage of allowing for a direct estimation of its shrinkage parameter. In this paper, we demonstrate how this Stein-like estimator can be implemented in the context of the estimation of natural direct and natural indirect effects of treatments in randomized controlled trials. The performance of the competing methods is studied in a simulation study, in which both the strength of hidden confounding and the strength of the instruments are independently varied. These considerations are motivated by a trial in mental health, evaluating the impact of a primary care-based intervention to reduce depression in the elderly.

Stein-like 2SLS estimator

ABSTRACTMaasoumi (1978) proposed a Stein-like estimator for simultaneous equations and showed that his Stein shrinkage estimator has bounded finite sample risk, unlike the three-stage least square estimator. We revisit his proposal by investigating Stein-like shrinkage in the context of two-stage least square (2SLS) estimation of a structural parameter. Our estimator follows Maasoumi (1978) in taking a weighted average of the 2SLS and ordinary least square estimators, with the weight depending inversely on the Hausman (1978) statistic for exogeneity. Using a local-to-exogenous asymptotic theory, we derive the asymptotic distribution of the Stein estimator and calculate its asymptotic risk. We find that if the number of endogenous variables exceeds 2, then the shrinkage estimator has strictly smaller risk than the 2SLS estimator, extending the classic result of James and Stein (1961). In a simple simulation experiment, we show that the shrinkage estimator has substantially reduced finite sample median squared error relative to the standard 2SLS estimator.

A Stein-like estimator for linear panel data models

In this paper we follow Hansen (2015a) and propose a Stein-like estimator for linear panel data models. Our estimator takes a weighted average of the fixed effects estimator and the random effects estimator using the weights constructed from Hausman’s (1978) testing statistic. We establish the asymptotic distribution of the Stein-like estimator and show its asymptotic risk being strictly smaller than the fixed effects estimator within a local asymptotic framework.

Shrinkage Estimation in the Random Parameters Logit Model

In this paper, we explore the properties of a positive-part Stein-like estimator which is a stochastically weighted convex combination of a fully correlated parameter model estimator and uncorrelated parameter model estimator in the Random Parameters Logit (RPL) model. The results of our Monte Carlo experiments show that the positive-part Stein-like estimator provides smaller MSE than the pretest estimator in the fully correlated RPL model. Both of them outperform the fully correlated RPL model estimator and provide more accurate information on the share of population putting a positive or negative value on the alternative attributes than the fully correlated RPL model estimates. The Monte Carlo mean estimates of direct elasticity with pretest and positive-part Stein-like estimators are closer to the true value and have smaller standard errors than those with fully correlated RPL model estimator.

A Semiparametric Basis for Combining Estimation Problems Under Quadratic Loss

When there is uncertainty concerning the appropriate statistical model-estimator to use in representing the data sampling process, we consider a basis for optimally combining estimation problems. The objective is to produce natural adaptive estimators that are free of subjective choices and tuning parameters. In the context of two competing multivariate linear statistical models–estimators, we demonstrate a semiparametric Stein-like (SPSL) estimator, , that, under quadratic loss, has superior risk performance relative to the conventional least squares estimator. The relationship of the SPSL estimator to the family of Stein estimators is noted, and asymptotic and analytic finite-sample risk properties of the estimator are developed for some special cases. As an application we consider the problem of combining two polar linear models and demonstrate a corresponding SPSL estimator. An extensive sampling experiment is used to investigate the finite-sample performance of the SPSL estimator over a wide range of data sampling designs and symmetric and skewed distributions. Bootstrapping procedures are used to develop confidence sets and serve as a basis for inference.

Estimating multivariate random effects without replication

This paper considers the estimation of multivariate random effects that are measured with error, but for which there are no replications. Using structural simplification of the correlation of the data, separate estimates are generated for the covariance of the random effects and the covariance of the error. An estimator of the random effects based on a truncated eigen structure is defined, and matrix mean squared error and its trace (risk) are analyzed, with comparison to the maximum likelihood estimator (m.l.e) and also to the Stein-like estimator of Efron and Morris (1972). It is shown that the estimator has risk which is smaller than the risk of the maximum likelihood estimator and the Efron-Morris estimator in most cases.

Shrinkage estimation in nonlinear regression The Box-Cox transformation

The risk superiority of the Stein-rule estimator over the maximum likelihood (ML) estimator is known in the context of the normal linear regression model. We present a positive-part Stein-like estimator that dominates the ML and pretest estimators under quadratic loss, weighted by the information matrix, in nonlinear regression. The risk properties of ML, constrained ML, pretest, and shrinkage estimators for the Box-Cox model are discussed. In a Monte Carlo experiment, the shrinkage estimator dominates the ML estimator under the unweighted quadratic loss. However, the ML estimator may have lower risk than the shrinkage estimator under other weighted quadratic loss functions in small samples.

The non-optimality of interval restricted and pre-test estimators under squared error loss

We consider the linear regression model with an interval restriction imposed on the coefficients, and examine the sampling performance of a family of Stein interval restricted and pre-test estimators Tor the coefficient vector. The risk, under squared error loss, of these Stein-like estimators are derived, and the inadmissibility of the maximum likelihood interval restricted and pre-test estimators is demonstrated.

Shrinkage Estimators Applied to Prediction of French Winter Wheat Yield

SUMMARY The paper shows that it is possible to pool departmental information to predict regional wheat yield. In an MSEP (mean squared error of prediction) context, shrinkage estimation in linear agrometeorological models improves regional prediction. Cross-validation is not precise enough to enable comparison between models. The authors calculate the MSEP using the hypothesis of a multinormal distribution in simple cases (the distribution parameters are estimated with data input) and simulations are used for complex cases. Stein-like estimators are shown to be a superior alternative to the conventional estimators usually applied for prediction.

Estimating the variability of the Stein estimator by bootstrap

Abstract Theoretical and empirical results indicate that the bootstrap method provides a viable alternative in estimating the scale parameters of Stein-like estimators.

Risk characteristics of a stein-like estimator for the probit regression model

A shrinkage estimator is proposed for the probit regression model and its risk characteristics are studied in a Monte Carlo experiment.

Estimation of Location Parameters Under Nonnormal Errors and Quadratic Loss

Assuming a statistical model in which the joint distribution of the unobservable errors is drawn from independent univariate Student t's that are identically and symmetrically distributed, the sampling performance of traditional robust estimators and a family of Stein-like estimators are compared and evaluated. These results suggest that under thick-tailed distributions, the relative sampling performances and risk characteristics for a range of nonconventional Stein estimators remains approximately the same as in the case of their normal counterparts. The empirical risk implications of misspecifying the error distribution are investigated.

Some Further Remarks on Multicollinearity and the Minimax Conditions of the Bock Stein-like Estimator

Some Further Remarks on Multicollinearity and the Minimax Conditions of the Bock Stein-like Estimator

Some improved estimators in the case of possible heteroscedasticity

Under a two-sample linear statistical model with a common location vector but possibly different scale parameters and a squared error loss measure of performance, the sampling properties of traditional and Stein-like estimators are compared and analytically evaluated. The corresponding pre-test estimator under possible heteroscedasticity is analyzed and its inadmissibility under squared error loss is demonstrated.

The risk of general Stein-like estimators in the presence of multicollinearity

This paper investigates the behavior of the bounding risk functions for theStein-like estimator that shrinks toward the restricted least squares estimator. First, it is demonstrated that the form and severity of multicollinearity affect the amount of potential risk gains over ordinary least squares (OLS). In general, the less extensive the multicollinearity the smaller the potential risk improvements over OLS. Second, when multicollinearity is present, potential risk gains are increased by using a number of parameter restrictions approximately equal to the number of small characteristic roots ofX′X. Third, potential risk gains are dramatically affected by the degree of shrinkage. Under severe multicollinearity, maximum shrinkage often, but not always, produces the greatest risk gains.

Discussion of Professor Copas's Paper

Discussion of Professor Copas's Paper

Small sample performance of the Stein-Rule in non-orthogonal designs

Abstract The effects of multicollinearity upon the risk improvement provided by Stein-like estimators over the maximum likelihood estimator in the normal regression model are investigated via Monte Carlo methods. Risk gains are found to degenerate rapidly given moderate degrees of multicollinearity.

A Modified Stein-like Estimator for the Reduced Form Coefficients of Simultaneous Equations

In this paper a reduced form estimator is developed which combines the corresponding restricted 3SLS and the unrestricted LS estimators. This estimator is similar to the 'positive part' Stein-like estimators proposed by Baranchik [2] and S. Sclove [16] in the classical multivariate regression context. It is shown that, whereas the restricted (derived) 3SLS and 2SLS reduced form estimates possess no finite moments (hence have unbounded risk), the modified Stein-like reduced form (MSRF) estimator has finite moments of up to order (T n m), where T is the sample size, n and m are the number of the endogenous and the non-stochastic exogenous variables in the system. Furthermore it is argued that, asymptotically, the difference between the MSRF and the 3SLS estimators is negligible.