Stein's Estimate Research Articles

A Sequential Stein's Method for Faster Training of Additive Index Models.

The additive index models (AIMs) can be viewed as a kind of artificial neural networks based on nonparametric activation or so-called ridge functions. Recently, they are shown to achieve enhanced explainability after incorporating various interpretability constraints. However, the training of AIMs by either the backfitting algorithm or the joint stochastic optimization is known to be very slow for especially high dimensional inputs. In this article, we propose a novel sequential approach based on the celebrated Stein's lemma. The proposed SeqStein method can successfully decouple the training of AIMs into two separable steps, namely, the following: 1) Stein's estimation of the projection indices and 2) nonparametric estimation of ridge functions using the smoothing splines. We show through numerical experiments that the SeqStein algorithm is not only more efficient for training AIMs, but also inclined to produce more interpretable models that have smooth ridge functions with sparse and nearly orthogonal projection indices.

Stein estimation in the Conway-Maxwell Poisson model with correlated regressors

Stein estimation in the Conway-Maxwell Poisson model with correlated regressors

Bayesian inference of the mean power of several Gaussian data

The uniform prior probability density for the means of normal data leads to inconsistent Bayesian inference of their mean power and jeopardizes the possibility of selecting among different models that explain the data. We reinvestigated the problem avoiding delivering unrecognised information and looking at it in a novel way. Namely, to consider a finite power, we used a normal prior minimally diverging from the uniform one, hyperparameterised by the mean and variance, and left the data to choose the most supported parameters. We also obtained an extended James–Stein estimator averaging the hyper-parameters and avoiding empirical Bayes techniques.Graphical abstract

Special feature: Stein estimation and statistical shrinkage methods

Special feature: Stein estimation and statistical shrinkage methods

Enhancing model predictions through the fusion of stein estimator and principal component regression

This research aims to enhance model predictions by introducing a novel approach that combines the Stein estimator technique with principal component regression (PCR) within the linear regression context. The study recognizes the challenges posed by multicollinearity in predictive modelling and addresses them by leveraging the strengths of the Stein estimator and PCR. We compared the novel estimator with other commonly used regression techniques to showcase its superiority theoretically. we investigate the impact of different parameter settings and sample sizes on the performance of the enhanced model predictions via simulation design. We analyzed two real-life data to validate the theoretical and simulation results. The mean squared error was employed to measure the performance gain achieved by the fusion of these techniques. The results show the benefits of combining the Stein estimator and PCR. It also provides valuable insights into advanced techniques for enhancing model predictions.

Performance of Alternative Estimators in the Poisson-Inverse Gaussian Regression Model: Simulation and Application

Abstract The study proposed and compared the biased estimators for the Poisson-Inverse Gaussian regression model to deal with correlated regressors. The limitations of each biased estimator are also discussed. Additionally, some biasing parameters for the Stein estimator are proposed. The performance of estimators is evaluated with the help of a simulation study and a real-life application based on the minimum mean squared error criterion. The simulation and application findings favor the ridge estimator with specific biasing parameters because it provides less variation than others.

Retracted: On Some Classes of Estimators Derived from the Positive Part of James–Stein Estimator

Retracted: On Some Classes of Estimators Derived from the Positive Part of James–Stein Estimator

Shrinkage estimation with logarithmic penalties

In this paper, we have developed a novel approach for deriving shrinkage estimators of means without assuming normality. Our method is based on the equation of the first-order condition with a logarithmic penalty, and it introduces both one-step and two-step shrinkage estimators. The one-step estimator closely resembles the James–Stein estimator, while the differentiable two-step estimator exhibits similar performance to the positive-part Stein estimator. Although the latter does not satisfy Baranchik’s conditions, both estimators can be demonstrated to be minimax under normality assumptions. Furthermore, we have extended this method to handle cases involving an unknown scale. We have successfully applied this approach to the simultaneous estimation of Poisson means.

Machine learning and the James–Stein estimator

It is now 62 years since the publication of James and Stein’s seminal article on the estimation of a multivariate normal mean vector. The paper made a spectacular first impression on the statistical community through its demonstration of inadmissability of the maximum likelihood estimator. It continues to be influential, but not for the initial reasons. Empirical Bayes shrinkage estimation, now a major topic, found its early justification in the James–Stein formula. Less obvious downstream topics include Tweedie’s formula and Benjamini and Hochberg’s false discovery rate algorithm. This is a short and mainly non-technical review of the James–Stein rule and its effects on the machine learning era of statistical innovation.

Robust-stein estimator for overcoming outliers and multicollinearity

Linear regression models with correlated regressors can negatively impact the performance of ordinary least squares estimators. The Stein and ridge estimators have been proposed as alternative techniques to improve estimation accuracy. However, both methods are non-robust to outliers. In previous studies, the M-estimator has been used in combination with the ridge estimator to address both correlated regressors and outliers. In this paper, we introduce the robust Stein estimator to address both issues simultaneously. Our simulation and application results demonstrate that the proposed technique performs favorably compared to existing methods.

On Some Classes of Estimators Derived from the Positive Part of James–Stein Estimator

This work consists of developing shrinkage estimation strategies for the multivariate normal mean when the covariance matrix is diagonal and known. The domination of the positive part of James–Stein estimator (PPJSE) over James–Stein estimator (JSE) relative to the balanced loss function (BLF) is analytically proved. We introduce a new class of shrinkage estimators which ameliorate the PPJSE, and then we construct a series of polynomial shrinkage estimators which improve the PPJSE; also, any estimator of this series can be ameliorated by adding to it a new term of higher degree. We end this paper by simulation studies which confirm the performance of the suggested estimators.

The Stein effect for Fréchet means

The Fréchet mean is a useful description of location for a probability distribution on a metric space that is not necessarily a vector space. This article considers simultaneous estimation of multiple Fréchet means from a decision-theoretic perspective, and in particular, the extent to which the unbiased estimator of a Fréchet mean can be dominated by a generalization of the James–Stein shrinkage estimator. It is shown that if the metric space satisfies a nonpositive curvature condition, then this generalized James–Stein estimator asymptotically dominates the unbiased estimator as the dimension of the space grows. These results hold for a large class of distributions on a variety of spaces, including Hilbert spaces and, therefore, partially extend known results on the applicability of the James–Stein estimator to nonnormal distributions on Euclidean spaces. Simulation studies on phylogenetic trees and symmetric positive definite matrices are presented, numerically demonstrating the efficacy of this generalized James–Stein estimator.

Weighted shrinkage estimators of normal mean matrices and dominance properties

In the estimation of the mean matrix in a multivariate normal distribution, the Efron–Morris estimator and the James–Stein estimator are two well-known minimax procedures, where the former is matricial shrinkage and the latter is scalar shrinkage. The methods for combining the two estimators with random weight functions are addressed. For deriving weight functions, the paper suggests the two methods. One is the minimization of a part of the unbiased estimator of the risk function, and the other is the empirical Bayes approach. The resulting weighted shrinkage estimators are shown to be minimax, and the extension to the case of an unknown covariance matrix is developed.

Improved shrinkage estimators in the beta regression model with application in econometric and educational data

Although beta regression is a very useful tool to model the continuous bounded outcome variable with some explanatory variables, however, in the presence of multicollinearity, the performance of the maximum likelihood estimates for the estimation of the parameters is poor. In this paper, we propose improved shrinkage estimators via Liu estimator to obtain more efficient estimates. Therefore, we defined linear shrinkage, pretest, shrinkage pretest, Stein and positive part Stein estimators to estimate of the parameters in the beta regression model, when some of them have not a significant effect to predict the outcome variable so that a sub-model may be sufficient. We derived the asymptotic distributional biases, variances, and then we conducted extensive Monte Carlo simulation study to obtain the performance of the proposed estimation strategy. Our results showed a great benefit of the new methodologies for practitioners specifically in the applied sciences. We concluded the paper with two real data analysis from economics and education.

An Empirical Bayes Approach to Shrinkage Estimation on the Manifold of Symmetric Positive-Definite Matrices

ABSTRACT The James–Stein estimator is an estimator of the multivariate normal mean and dominates the maximum likelihood estimator (MLE) under squared error loss. The original work inspired great interest in developing shrinkage estimators for a variety of problems. Nonetheless, research on shrinkage estimation for manifold-valued data is scarce. In this article, we propose shrinkage estimators for the parameters of the Log-Normal distribution defined on the manifold of N × N symmetric positive-definite matrices. For this manifold, we choose the Log-Euclidean metric as its Riemannian metric since it is easy to compute and has been widely used in a variety of applications. By using the Log-Euclidean distance in the loss function, we derive a shrinkage estimator in an analytic form and show that it is asymptotically optimal within a large class of estimators that includes the MLE, which is the sample Fréchet mean of the data. We demonstrate the performance of the proposed shrinkage estimator via several simulated data experiments. Additionally, we apply the shrinkage estimator to perform statistical inference in both diffusion and functional magnetic resonance imaging problems. Supplementary materials for this article are available online.

On homogeneous James–Stein type estimators

The shrinkage factor of the James–Stein estimators of a multivariate normal involves the power of the squared norm of X. We show powers of the norm greater than 2 also give improvement, while lesser powers do not.

The Global Approximations of the Coverage Probability for a Confidence Set Centered at the Positive Part James–Stein Estimator and Their Applications

The Global Approximations of the Coverage Probability for a Confidence Set Centered at the Positive Part James–Stein Estimator and Their Applications

A new Stein estimator for the zero‐inflated negative binomial regression model

AbstractThe Zero‐inflated negative binomial (ZINB) regression models are mainly applied for count data that shows over‐dispersion and extra zeros. Multicollinearity is considered to be a significant problem in the estimation of parameters in the ZINB regression model. Thus, in order to alleviate the serious effects of multicollinearity, a new estimator is proposed which is called ZINB Stein estimator (ZINBSE). We also proposed various biasing parameters for the ZINBSE. A theoretical comparison is also conducted with some existing estimators in the literature. A Monte Carlo simulation study has been considered in order to judge the superiority of the proposed and other estimators, where the estimated mean squared error and mean absolute error are the evaluation criterion. An empirical application is also considered for illustration purposes. Based on the simulation and application results, it is observed that the new ZINBSE with proposed biasing parameters are superior over the other competitors' estimators.

Bayesian optimality and intervals for Stein‐type estimates

We provide a novel Bayesian decision‐theoretic motivation for Stein‐type estimates, producing them as an adaptive choice between standard point estimation and estimation that rewards proximity to the origin. Unlike conventional approaches, our arguments provide shrunken estimates under any sampling model or prior. They also lead naturally to a form of credible interval, describing uncertainty about the underlying parameters yet focusing on the shrunken estimate. One specific method for constructing intervals provides a close Bayesian analogue of Samworth (2005)'s approximate confidence intervals around Stein estimates. Several examples are given, showing how shrinkage's focus on regions remote to the high‐support areas of the posterior can lead to substantially larger credible sets.

James–Stein estimation of the first principal component

The Stein paradox has played an influential role in the field of high‐dimensional statistics. This result warns that the sample mean, classically regarded as the “usual estimator,” may be suboptimal in high dimensions. The development of the James–Stein estimator that addresses this paradox has by now inspired a large literature on the theme of “shrinkage” in statistics. In this direction, we develop a James–Stein‐type estimator for the first principal component of a high‐dimension and low‐sample size data set. This estimator shrinks the usual estimator, an eigenvector of a sample covariance matrix under a spiked covariance model, and yields superior asymptotic guarantees. Our derivation draws a close connection to the original James–Stein formula so that the motivation and recipe for shrinkage is intuited in a natural way.