Stein's Estimate Research Articles

A pair of estimating equations for a mean vector

Consider a class of estimators of a mean vector, indexed by a parameter. We introduce a pair of estimating equations for the parameter and the variance in the normal distribution. The equations provide us with an interpretation of the empirical Bayes method for smoothing and the James–Stein estimator. They can also be applied to various methods such as the S function lowess, the ridge estimator and the method of moving average. An extension to the non-Gaussian case is also discussed.

Non‐parametric Curve Estimation by Wavelet Thresholding with Locally Stationary Errors

An important aspect in the modelling of biological phenomena in living organisms, whether the measurements are of blood pressure, enzyme levels, biomechanical movements or heartbeats, etc., is time variation in the data. Thus, the recovery of a ‘smooth’ regression or trend function from noisy time‐varying sampled data becomes a problem of particular interest. Here we use non‐linear wavelet thresholding to estimate a regression or a trend function in the presence of additive noise which, in contrast to most existing models, does not need to be stationary. (Here, non‐stationarity means that the spectral behaviour of the noise is allowed to change slowly over time). We develop a procedure to adapt existing threshold rules to such situations, e.g. that of a time‐varying variance in the errors. Moreover, in the model of curve estimation for functions belonging to a Besov class with locally stationary errors, we derive a near‐optimal rate for the ‐risk between the unknown function and our soft or hard threshold estimator, which holds in the general case of an error distribution with bounded cumulants. In the case of Gaussian errors, a lower bound on the asymptotic minimax rate in the wavelet coefficient domain is also obtained. Also it is argued that a stronger adaptivity result is possible by the use of a particular location and level dependent threshold obtained by minimizing Stein's unbiased estimate of the risk. In this respect, our work generalizes previous results, which cover the situation of correlated, but stationary errors. A natural application of our approach is the estimation of the trend function of non‐stationary time series under the model of local stationarity. The method is illustrated on both an interesting simulated example and a biostatistical data‐set, measurements of sheep luteinizing hormone, which exhibits a clear non‐stationarity in its variance.

Minimax estimation in linear regression under restrictions

Abstract We consider the problem of estimating the regression coefficients in a linear regression model under ellipsoid constraints on the parameter space. The problem is closely connected to spline smoothing. A minimax estimator under weighted squared error is derived. Special cases include ridge regression, Stein's estimator and principal component regression. The asymptotic risk ratio for ridge regression versus the minimax estimator is computed for a special case and seen to be infinite in some situations. Adaptive estimators based on Mallows’ CL-statistic are suggested when the size of the parameter space is unknown and shown to have the same asymptotic risk as the estimator based on known size. The minimax estimator is compared to ridge regression and principal component regression on real data sets.

Stein estimation—A review

This paper presents an expository development of Stein estimation in several distribution families. Considered are both the point estimation and confidence interval cases. Specific results for linear regression models are added. Emphasis is laid on the chronological history and on recent results.

Future production systems: Influence of self-organization on approaches to quality engineering

Self-organization and cooperation have been introduced into Smart production systems to improve the flexibility and reactivity of OKP systems (one of a kind production). In this paper, we analyze the influence of self-organization and collaborative processing concepts on process and product quality. These concepts have a direct impact on measurement and control processes of the critical components of end-products. We will focus on some new techniques – such as the James–Stein estimators and the Levy stable distribution. To end with, SPC techniques are reviewed in order to implement best of breed solutions with a view of monitoring and improving the performance of the process.

Signal and noise separation: Art and science

The separation of signal and noise is a central issue in seismic data processing. The noise is both random and coherent in nature, the coherent part often masquerading as signal. In this tutorial, we present some approaches to signal isolation, in which stacking is a central concept. Our methodology is to transform the data to a domain where noise and signal are separable, a goal that we attain by means of inversion. We illustrate our ideas with some of our favorite transformations: wavelets, eigenvectors, and Radon transforms. We end with the notion of risk, baseball, and the Stein estimator.

A class of shrinkage estimators in linear regression

AbstractWe consider the problem of using shrinkage estimators that shrink towards subspaces in linear regression, in particular subspaces spanned by principal components. This is especially important when multicollinearity is present and the number of predictors is not small compared to the sample size. New theoretical results about Stein estimation are used to get estimators with lower theoretical risk than standard Stein estimators used by Oman (1991). Application of the techniques to real data is largely successful.

A new general interpretation of the Stein estimate and how it adapts: Applications

In this article, we demonstrate that Stein estimators of the form [1-a/(b+∥X∥ 2 )]X arise under a broad variety of location parameter problems from a natural calculation, wherein one does not need independence of the coordinates, nor symmetry, and the loss function is quite general, including the important absolute error loss. The results are different in character from classic previous works of Brown, Casella, Strawderman, and Shinozaki. Our method of calculation automatically shows how the Stein estimate could be adapted to a specific parent distribution and a specific loss. A number of examples that illustrate this adaptive calculation result in quite remarkable features. In particular, some very specific and new Stein estimates emerge in important cases, specifically for absolute error loss. The estimates are studied with respect to their risk, both theoretically and via simulation, and the evidence suggests that minimaxity can be expected in generality in 4 or more dimensions. In addition, the specific proposed estimate appears to outperform both X and the ordinary James-Stein estimate under absolute error loss for the normal case.

Shrinkage to smooth non-convex cone :Principal component analysis as stein estimation

In Kuriki and Takemura [1997a) we established a general theory of James-Stein type shrinkage to convex sets with smooth boundary. In this paper we show that our results can be generalized to the case where shrinkage is toward smooth non-convex cones. A primary example of this shrinkage is descriptive principal component analysis, where one shrinks small singular values of the data matrix. Here principal component analysis is interpreted as the problem of estimation of matrix mean and the shrinkage of the small singular values is regarded as shrinkage of the data matrix toward the manifold of matrices of smaller rank.

Improved Estimation in Measurement Error Models Through Stein Rule Procedure

This paper examines the role of Stein estimation in a linear ultrastructural form of the measurement errors model. It is demonstrated that the application of Stein rule estimation to the matrix of true values of regressors leads to the overcoming of the inconsistency of the least squares procedure and yields consistent estimators of regression coefficients. A further application may improve the efficiency properties of the estimators of regression coefficients. It is observed that the proposed family of estimators under some constraint on the characterizing scalar dominates the conventional consistent estimator with respect to the criterion of asymptotic risk under a specific quadratic loss function. Then the problem of prediction of the values of the study variable within the sample is considered, and it is found that the predictors based on the proposed family of estimators are always more efficient than the predictors based on the conventional estimator according to asymptotic predictive mean squared error criterion, although both are biased.

Modulation of estimators and confidence sets

An unknown signal plus white noise is observed at $n$ discrete time points. Within a large convex class of linear estimators of $\xi$, we choose the estimator $\hat{\xi}$ that minimizes estimated quadratic risk. By construction, $\hat{\xi}$ is nonlinear. This estimation is done after orthogonal transformation of the data to a reasonable coordinate system. The procedure adaptively tapers the coefficients of the transformed data. If the class of candidate estimators satisfies a uniform entropy condition, then $\hat{\xi}$ is asymptotically minimax in Pinsker’s sense over certain ellipsoids in the parameter space and shares one such asymptotic minimax property with the James–Stein estimator. We describe computational algorithms for $\hat{\xi}$ and construct confidence sets for the unknown signal. These confidence sets are centered at $\hat{\xi}$, have correct asymptotic coverage probability and have relatively small risk as set-valued estimators of $\xi$.

Different Discrete Wavelet Transforms Applied to Denoising Analytical Data

Discrete wavelet transform (DWT) denoising contains three steps: forward transformation of the signal to the wavelet domain, reduction of the wavelet coefficients, and inverse transformation to the native domain. Three aspects that should be considered for DWT denoising include selecting the wavelet type, selecting the threshold, and applying the threshold to the wavelet coefficients. Although there exists an infinite variety of wavelet transformations, 22 orthonormal wavelet transforms that are typically used, which include Haar, 9 daublets, 5 coiflets, and 7 symmlets, were evaluated. Four threshold selection methods have been studied: universal, minimax, Stein's unbiased estimate of risk (SURE), and minimum description length (MDL) criteria. The application of the threshold to the wavelet coefficients includes global (hard, soft, garrote, and firm), level-dependent, data-dependent, translation invariant (TI), and wavelet package transform (WPT) thresholding methods. The different DWT-based denoising m...

New approach in biased regression

Summary An optimization problem which provides a new characterization for ridge regression is discussed. A variant of this optimization problem leads to a new family of biased estimators that includes the Stein estimation method and principal components regression as particular cases. The whole approach is illustrated on the basis of real data sets.

THE VALIDITY AND POWER OF TESTS FOR EQUALITY OF TWO CORRELATED PROPORTIONS

With measurements taken on subjects over time, on matched pairs of subjects or on clusters of subjects, the data often contain pairs of correlated dichotomous responses. McNemar's test is perhaps the best known test to compare two correlated binomial proportions. The salient feature of McNemar's test is that we compute the variance of the contrast estimator under the restriction that the null hypothesis is true. Wald's test, on the other hand, does not require that restriction. As a consequence, Wald's statistic is always greater in magnitude than McNemar's statistic when the marginal proportions are unequal, but there is a problem with the validity of both McNemar's test and Wald's test with small to moderate samples. There have been various modifications suggested for McNemar's test to improve its performance. We propose a modified Wald's test that is valid in small to moderate samples and maintains good power. We also evaluate the performance of McNemar's test and Wald's test with and without modifications to enhance validity as well as the performance of the large sample likelihood ratio test and an exact test of the equality of correlated binomial proportions. In a smaller study, we compare the behaviour of a test based on the James–Stein estimator of the common odds ratio proposed by Liang and Zeger to McNemar's test and Wald's test. © 1997 by John Wiley & Sons, Ltd. Stat. Med., Vol. 16, 1081–1096 (1997).

Bayes and stein estimation under asymmetric loss functions:a numerical risk comparison

We consider the estimation of the scale parameter of the shifted exponential distribution and the variance of the normal distribution when the locations of these distributions are unknown and when loss is measured by invariant asymmetric loss functions. Stein type and Bayesian estimators are derived and compared in terms of risk improvements over the best affine equivariant estimator (BAEE). It is demonstrated that, under asymmetric loss, Bayes estimators provide a much larger degree of improvement over the BAEE than Stein estimators.

A Paradox Concerning Shrinkage Estimators: Should a Known Scale Parameter Be Replaced by an Estimated Value in the Shrinkage Factor?

When estimating, under quadratic loss, the location parameterθof a spherically symmetric distribution with known scale parameter, we show that it may be that the common practice of utilizing the residual vector as an estimate of the variance is preferable to using the known value of the variance. In the context of Stein-like shrinkage estimators, we exhibit sufficient conditions on the spherical distributions for which this paradox occurs. In particular, we show that it occurs fort-distributions when the dimension of the residual vector is sufficiently large. The main tools in the development are upper and lower bounds on the risks of the James–Stein estimators which are exact atθ=0.

Bayesian Methods for Variance Component Models

Abstract An exact Bayesian analysis can be performed for normal theory variance components models, using importance sampling. But remarkably accurate approximations are available using the Laplacian T-approximation introduced by Leonard, Hsu, and Ritter. Instead of maximizing the joint posterior density, conditional upon the parameter of interest, a device described by O'Hagan is used first. The Bayesian estimators are compared to the Lindley—Stein shrinkage estimators and the Lindley—Smith joint modal estimators. It is confirmed that joint modes can overcollapse toward prior hypotheses, when compared with more sensible Bayesian procedures. This is referred to as a “collapsing phenomenon.” A numerical example from the one-way random-effects model is considered, and the risks of the different estimators are simulated under a variety of loss functions. It is concluded that although the Lindley—Stein estimator performs well, a full hierarchical Bayesian analysis performs at least equally well, while permitting more detailed finite-sample inference regarding any parameter of interest.

Bayesian Methods for Variance Component Models

Abstract An exact Bayesian analysis can be performed for normal theory variance components models, using importance sampling. But remarkably accurate approximations are available using the Laplacian T-approximation introduced by Leonard, Hsu, and Ritter. Instead of maximizing the joint posterior density, conditional upon the parameter of interest, a device described by O'Hagan is used first. The Bayesian estimators are compared to the Lindley—Stein shrinkage estimators and the Lindley—Smith joint modal estimators. It is confirmed that joint modes can overcollapse toward prior hypotheses, when compared with more sensible Bayesian procedures. This is referred to as a “collapsing phenomenon.” A numerical example from the one-way random-effects model is considered, and the risks of the different estimators are simulated under a variety of loss functions. It is concluded that although the Lindley—Stein estimator performs well, a full hierarchical Bayesian analysis performs at least equally well, while permitti...

Multiple-Shrinkage Principal Component Regression

SUMMARY Consider the multiple-regression framework where the goal is 'black box' prediction and the explanatory variables are (approximately) multicollinear. For this situation we propose a multiple-shrinkage estimator which adaptively mimics the best principal components shrinkage estimator. The estimator proposed is a variant of the Stein estimator which is also guaranteed to give lower prediction mean-squared error than the least squares estimator, thus avoiding the danger of overshrinking when using principal components regression. The predictive performance of this estimator is illustrated and compared with that of principal components regression on three data sets by using random crossvalidation.

Experimental study of performance of pattern classifiers and the size of design samples

This paper presents results of simulation experiments concerning the two class pattern classification problem assuming a multivariate normal population. Both a quadratic and a linear discriminant function designed by using estimated covariance matrices and mean vectors have limited performance compared to the optimal Bayes decision. Two approximate estimators of the amount of degradation in the recognition rate were proposed by Raudys and Fukunaga, respectively. This paper presents the experimental evaluation of the goodness of these estimators. We show quantitatively how well those estimators work and also confirm that the modified classifier designed by using Stein's estimator achieves better performance than the conventional one.