Shrinkage Estimators Research Articles

High dimensional discriminant rules with shrinkage estimators of the covariance matrix and mean vector

Linear discriminant analysis (LDA) is a typical method for classification problems with large dimensions and small samples. There are various types of LDA methods that are based on the different types of estimators for the covariance matrices and mean vectors. In this paper, we consider shrinkage methods based on a non-parametric approach. For the precision matrix, methods based on the sparsity structure or data splitting are examined. Regarding the estimation of mean vectors, Non-parametric Empirical Bayes (NPEB) methods and Non-parametric Maximum Likelihood Estimation (NPMLE) methods, also known as f-modeling and g-modeling, respectively, are adopted. The performance of linear discriminant rules based on combined estimation strategies of the covariance matrix and mean vectors are analyzed in this study. Particularly, the study presents a theoretical result on the performance of the NPEB method and compares it with previous studies. Simulation studies with various covariance matrices and mean vector structures are conducted to evaluate the methods discussed in this paper. Furthermore, real data examples such as gene expressions and EEG data are also presented.

Long-term software fault prediction with wavelet shrinkage estimation

Wavelet shrinkage estimation received considerable attentions to estimate stochastic processes such as a non-homogeneous Poisson process in a non-parametric way, and was applied to software reliability estimation/prediction. However, it lacks the prediction ability for unknown future patterns in long term and penalizes assessing the software reliability in practice. In this paper, we focus on the long-term prediction of the number of software faults detected in the testing phase and propose many novel long-term prediction methods based on the wavelet shrinkage estimation. The fundamental idea is to adopt both the denoised fault-count data and prediction values, and to minimize several kinds of loss functions to make effective predictions. We also develop an automated wavelet-based software reliability assessment tool, W-SRAT2, which is a drastic extension of the existing tool, W-SRAT, by adding those prediction algorithms. In numerical experiments with 6 actual software development project data, we investigate the predictive performance of our long-term prediction approaches, which consist of 2,640 combinations, and compare them with the common software reliability growth models with the maximum likelihood estimation. It is shown that our wavelet shrinkage estimation/prediction methods outperform the existing software reliability growth models.

High-dimensional covariance matrices under dynamic volatility models: Asymptotics and shrinkage estimation

High-dimensional covariance matrices under dynamic volatility models: Asymptotics and shrinkage estimation

Doubly Sparse estimator for high-dimensional covariance matrices

The classical sample covariance estimator lacks desirable properties such as consistency and suffers from eigenvalue spreading in high-dimensional settings. Improved estimators have been proposed that shrink sample eigenvalues but retain the eigenvectors of the sample covariance estimator. In high dimensions, however, sample eigenvectors are generally strongly inconsistent, rendering eigenvalue shrinkage estimators suboptimal. A Doubly Sparse Covariance Estimator (DSCE) is developed that goes beyond mere eigenvalue shrinkage: a covariance matrix is decomposed into a signal part, where sparse eigenvectors are estimated via truncation, and an idiosyncratic part, estimated via thresholding. It is shown that accurate estimation is possible if the leading eigenvectors are sufficiently sparse affecting proportionately less than p of the variables. DSCE fills the gap for empirical applications that fall in-between fully sparse settings and conditionally sparse settings: DSCE takes advantage of conditional sparsity implied by factor models while allowing only a subset of variables to load on factors, which relaxes pervasiveness assumption of traditional factor models. An empirical application to the constituents of the S&P 1500 illustrates that DSCE-based portfolios outperform competing methods in terms of Sharpe ratio, maximum drawdown, and cumulative return for monthly and daily data.

A unified Bayesian framework for bias adjustment in multiple comparisons from clinical trials.

In clinical trials, multiple comparisons arising from various treatments/doses, subgroups, or endpoints are common. Typically, trial teams focus on the comparison showing the largest observed treatment effect, often involving a specific treatment pair and endpoint within a subgroup. These findings frequently lead to follow-up pivotal studies, many of which do not confirm the initial positive results. Selection bias occurs when the most promising treatment, subgroup, or endpoint is chosen for further development, potentially skewing subsequent investigations. Such bias can be defined as the deviation in the observed treatment effects from the underlying truth. In this article, we propose a general and unified Bayesian framework to address selection bias in clinical trials with multiple comparisons. Our approach does not require a priori specification of a parametric distribution for the prior, offering a more flexible and generalized solution. The proposed method facilitates a more accurate interpretation of clinical trial results by adjusting for such selection bias. Through simulation studies, we compared several methods and demonstrated their superior performance over the normal shrinkage estimator. We recommended the use of Bayesian Model Averaging estimator averaging over Gaussian Mixture Models as the prior distribution based on its performance and flexibility. We applied the method to a multicenter, randomized, double-blind, placebo-controlled study investigating the cardiovascular effects of dulaglutide.

Comparison of some Bayesian estimation methods for type-I generalized extreme value distribution with simulation

The Weibull distribution is considered one of the Type-I Generalized Extreme Value (GEV) distribution, and it plays a crucial role in modeling extreme events in various fields, such as hydrology, finance, and environmental sciences. Bayesian methods play a strong, decisive role in estimating the parameters of the GEV distribution due to their ability to incorporate prior knowledge and handle small sample sizes effectively. In this research, we compare several shrinkage Bayesian estimation methods based on the squared error and the linear exponential loss functions. They were adopted and compared by the Monte Carlo simulation method. The performance of these methods is assessed based on their accuracy and computational efficiency in estimating the scale parameter of the Weibull distribution. To evaluate their performance, we generate simulated datasets with different sample sizes and varying parameter values. A technique for pre-estimation shrinkage is suggested to enhance the precision of estimation. Simulation experiments proved that the Bayesian shrinkage estimator and shrinkage pre-estimation under the squared loss function method are better than the other methods because they give the least mean square error. Overall, our findings highlight the advantages of shrinkage Bayesian estimation methods for the proposed distribution. Researchers and practitioners in fields reliant on extreme value analysis can benefit from these findings when selecting appropriate Bayesian estimation techniques for modeling extreme events accurately and efficiently.

Cross validation based transfer learning for cross-sectional non-linear shrinkage: A data-driven approach in portfolio optimization

Enhanced covariance estimation approaches, such as (non-)linear shrinkage, are well established in the literature. Non-linear shrinkage estimators generally minimize a certain loss function regarding statistical assumptions about the future covariance matrix. At the same time, the problem of covariance estimation is traditionally considered from a rather restrictive view since the only available data to determine the estimation parameters is given by the return history of the actual portfolio constituents. In this study, we propose a novel and purely data-driven perspective on covariance estimation. We present a non-linear shrinkage estimator that determines the estimation parameters using cross validation to be historically optimal on a disjoint dataset of assets according to the given objective, such as minimum variance or maximum risk-adjusted return. We then transfer the historically optimal estimation parameters learned on the disjoint dataset to the actual covariance estimation problem. Thereby, the sample eigenvalues are corrected in a purely data-driven way, agnostic to theoretically derived parameters. Another benefit of focusing on disjoint data is that we address the problem of limited data availability in high-dimensional estimation problems when the number of assets exceeds the history length. Our empirical evaluation, based on a total of six stock market indices and various problem dimensions, shows that our approach outperforms existing cross-sectional estimators in minimizing variance and maximizing risk-adjusted return. While our study is limited to the cross-section, the method of parameter selection using cross validation and transfer learning can also be combined with other estimators, such as time-series methods.

A novel prognostic signature based on cancer stemness and metabolism-related genes for cervical squamous cell carcinoma and endocervical adenocarcinoma.

CESC is the second most commonly diagnosed gynecological malignancy. Given the pivotal involvement of metabolism-related genes (MRGs) in the etiology of multiple tumors, our investigation aims to devise a prognostic risk signature rooted in cancer stemness and metabolism. The stemness index based on mRNA expression (mRNAsi) of samples from the TCGA dataset was computed using the One-class logistic regression (OCLR) algorithm. Furthermore, potential metabolism-related genes related to mRNAsi were identified through weighted gene co-expression network analysis (WGCNA). We construct a stemness-related metabolic gene signature through shrinkage estimation and univariate analysis, thereby calculating the corresponding risk scores. Moreover, we selected corresponding DEGs between groups with high- and low-risk score and conducted routine bioinformatic analyses. Furthermore, we validated the expression of four hub genes at the protein level through immunohistochemistry (IHC) in samples obtained from our patient cohort. According to the findings, it was found that six genes-AKR1B10, GNA15, ALDH1B1, PLOD2, LPCAT1, and GPX8- were differentially expressed in both TCGA-CSEC and GEO datasets among 23 differentially expressed metabolism-related genes (DEMRGs). mRNAsi exhibited a notable association with the extent of key oncogene mutation. The results showed that the AUC values for forecasting survival at 1, 3, and 5 years are 0.715, 0.689, and 0.748, individually. We observed a notable association between the risk score and different immune cell populations, along with enrichment in crucial signaling pathways in CESC. Four genes differentially expressed between different risk score groups were validated by IHC to be highly expressed in the CESC samples at the protein level. The current investigation indicated that a 3-gene signature based on stemness-related metabolic and 4 hub genes with differential expression between high and low-risk score subgroups may serve as valuable prognostic markers and potential therapeutic targets in CESC.

Wavelet Shrinkage Estimation for Mean Matrix of Matrix-Variate Elliptically Contoured Distributions

Wavelet Shrinkage Estimation for Mean Matrix of Matrix-Variate Elliptically Contoured Distributions

Single Stage Shrinkage Estimation Methods for Reliability Function of Generalized Exponential Distribution Using Simulation

The reliability in stress–strength model (s–s) is estimated in this article when the system has parallel and series components subject to one of the stresses and it follows generalized exponential distribution by taking via different estimation methods, namely the maximum likelihood estimation, unbiased estimation, moment estimation and shrinkage estimation. Also, the Mont Carlo method is used to compare these methods under mean squared error.

Multilateral indices in official price statistics and a new additive splicing method

A substitution of the traditional data sources allows National Statistical Institutes to collect large samples of product items with corresponding prices recorded over more frequent time periods. For some product categories, a frequent replacement of product items requires a dynamic approach to define a product sample that can be achieved by a monthly update of the sample. In a month-on-month comparison of price changes, bilateral indices must be chained that can lead to a chain drift bias. Multilateral methods are designed to reduce the chain drift, albeit not completely eliminate it. A disadvantage of multilateral methods is its ambiguous interpretability and a revision problem. The main objective of this paper is to show an implementation of multilateral methods for official price statistics in practice. The multilateral GEKS method is decomposed into elementary components that identifies the effect of successive price relatives used in its compilation. An importance of the decomposition is shown when interpreting the GEKS index in the context of official price statistics. We demonstrate that it can lead to an inaccurate public dissemination of the consumer price index. Moreover, we propose a new method for the splice extension that is required to avoid the revision of previously published indices. The method uses a Stein-type shrinkage estimator for the splice extension that is applied additively to the last published index. In the empirical study, we use web scraped data, which are obtained from the Slovak market, for comparison of bilateral and multilateral indices.

Low and high dimensional wavelet thresholds for matrix-variate normal distribution

The matrix-variate normal distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables. In this paper, we introduce a wavelet shrinkage estimator based on Stein’s unbiased risk estimate (SURE) threshold for matrix-variate normal distribution. We find a new SURE threshold for soft thresholding wavelet shrinkage estimator under the reflected normal balanced loss function in low and high dimensional cases. Also, we obtain the restricted wavelet shrinkage estimator based on non-negative sub matrix of the mean matrix. Finally, we present a simulation study to test the validity of the wavelet shrinkage estimator and two real examples for low and high dimensional data sets.

Aiding the well celebrated Kuk's randomized response technique through auxiliary and prior information

Asking direct questions in face to face surveys about sensitive traits is an intricate issue. One of the solutions to this issue is the randomized response technique (RRT). Being the most widely used indirect questioning technique to obtain truthful data on sensitive traits in survey sampling RRT has been applied in a variety of fields including behavioral science, socio-economic, psychological, epidemiology, biomedical, criminology, data masking, public health engineering, conservation studies, ecological studies and many others. This paper aims at exploring the methods to subsidize the randomized response technique through additional information relevant to the parameter of interest. Specifically, we plan to contribute by proposing more efficient hybrid estimators compared to existing estimator based on (Kuk, 1990) [31] family of randomized response models. The proposed estimators are based on the methodology of incorporating the pertinent information, available on the basis of either historical records or expert opinion. Specifically, in case of availability of auxiliary information, the regression-cum-ratio estimator is found to be the best to further enhance the estimation through (Kuk, 1990) [31] model while the (Thompson, 1968) [49] shrinkage estimation is observed to be yielding more precise and accurate estimator of sensitive proportion. The findings in this study signify the importance of the proposed methodology. Additionally, to support the mathematical findings, a detailed numerical investigation to evaluate the comparative performances is also conducted. Based on performance analysis, overwhelming evidences are witnessed in the favor of proposed strategies.

Stein-Like Shrinkage Estimators for Coefficients of a Single-Equation in Simultaneous Equation Systems

Two stein-like shrinkage estimators are introduced to modify the 2SLS and the LIML estimators for coefficients of a single equation in a simultaneous system of equations. The proposed estimators are weighted averages of the 2SLS/LIML estimators and the OLS estimator. The shrinkage weight depends on the Wu-Hausman misspecification test statistic which evaluates the null of exogeneity against the alternative hypothesis of endogeneity. The approximate finite sample bias, mean squared errors, and density functions of the Stein-like shrinkage estimators are obtained using small-disturbance approximations. The dominance conditions of the Stein-like shrinkage estimators over the 2SLS/LIML estimator under the mean squared error and the concentration probability are obtained. The proposed method is further illustrated by simulation studies which demonstrate the good finite sample performance of the method, and is also applied to an empirical application of returns to education.

Information criteria for structured parameter selection in high-dimensional tree and graph models

Parameter selection in high-dimensional models is typically fine-tuned in a way that keeps the (relative) number of false positives under control. This is because otherwise the few true positives may be dominated by the many possible false positives. This happens, for instance, when the selection follows from a naive optimisation of an information criterion, such as AIC or Mallows's Cp. It can be argued that the overestimation of the model comes from the optimisation process itself changing the statistics of the selected variables, in a way that the information criterion no longer reflects the true divergence between the selected model and the data generating process. Using lasso, the overestimation can also be linked to the shrinkage estimator, which makes the selection too tolerant of false positive selections. For these reasons, this paper works on refined information criteria, carefully balancing false positives and false negatives, for use with estimators without shrinkage. In particular, the paper develops corrected Mallows's Cp criteria for structured selection in trees and graphical models.

Multiple observers ranked set samples for shrinkage estimators

Ranked set sampling (RSS) is used as a powerful data collection technique for situations where measuring the study variable requires a costly and/or tedious process while the sampling units can be ranked easily (e.g. osteoporosis research). In this paper, we develop ridge and Liu-type shrinkage estimators under RSS data from multiple observers to handle the collinearity problem in estimating coefficients of linear regression, stochastic restricted regression and logistic regression. Through extensive numerical studies, we show that shrinkage methods with the multi-observer RSS result in more efficient coefficient estimates. The developed methods are finally applied to bone mineral data for analysis of bone disorder status of women aged 50 and older.

G.ridge: An R Package for Generalized Ridge Regression for Sparse and High-Dimensional Linear Models

Ridge regression is one of the most popular shrinkage estimation methods for linear models. Ridge regression effectively estimates regression coefficients in the presence of high-dimensional regressors. Recently, a generalized ridge estimator was suggested that involved generalizing the uniform shrinkage of ridge regression to non-uniform shrinkage; this was shown to perform well in sparse and high-dimensional linear models. In this paper, we introduce our newly developed R package “g.ridge” (first version published on 7 December 2023) that implements both the ridge estimator and generalized ridge estimator. The package is equipped with generalized cross-validation for the automatic estimation of shrinkage parameters. The package also includes a convenient tool for generating a design matrix. By simulations, we test the performance of the R package under sparse and high-dimensional settings with normal and skew-normal error distributions. From the simulation results, we conclude that the generalized ridge estimator is superior to the benchmark ridge estimator based on the R package “glmnet”. Hence the generalized ridge estimator may be the most recommended estimator for sparse and high-dimensional models. We demonstrate the package using intracerebral hemorrhage data.

Stein’s identities and the related topics: an instructive explanation on shrinkage, characterization, normal approximation and goodness-of-fit

The Stein-type identities are widely recognized for their substantial utility and potency in deriving shrinkage estimators improving on crude estimators in normal, gamma, Poisson, and negative binomial distributions. Additionally, these identities serve to characterize these distributions themselves. The Stein identities are also used to demonstrate normal approximation. Moreover, they are instrumental in constructing statistical tests to assess the goodness-of-fit for normality, exponentiality, and Poissonity of distributions. This article offers an instructive and comprehensive explanation of the applications of Stein-type identities in the aforementioned contexts.

Asymptotics of coverages of HD confidence sets and recentering at shrinkage estimates: phase transitions, large deviations

Asymptotics of coverages of HD confidence sets and recentering at shrinkage estimates: phase transitions, large deviations

Target selection in shrinkage estimation of covariance matrix: A structural similarity approach

The shrinkage estimator of a high-dimensional covariance matrix relies on a preassigned target matrix during data processing. This paper provides an adaptive approach for selecting the optimal Toeplitz target matrix. We discover a sufficient and necessary condition for characterizing the two kinds of target matrices with the Toeplitz structure, and we propose an adaptive selection algorithm by measuring the similarity between the data and the Toeplitz structure. Numerical simulations and an empirical study on monetary funds verify the effectiveness of the selection approach.