Empirical Bayes Approach Research Articles

Moderated Test Statistics to Detect Differential Deuteration in Hydrogen/Deuterium Exchange Mass Spectrometry Experiments.

With differential hydrogen/deuterium exchange, differences in the structure and dynamics of protein states can be studied. Detecting statistically significant differentially deuterated peptides is crucial to draw meaningful conclusions about the distinct conformations and dynamics of the protein under study. Here, we introduced a linear model in combination with an empirical Bayes approach to detect differentially deuterated peptides. Using a linear model allows one to test for differences in deuteration between two (two-sample t-test) or more groups (F-statistic), while potentially controlling for the effects of other variables that are not of interest. The empirical Bayes approach improves the estimation of deuteration-level variances, especially in experiments with a low number of replicates. As a consequence, the two sample t-tests and the F-statistic become moderated, resulting in a lower number of false positive and false negative findings. Furthermore, we introduce a thresholded-moderated t-statistic to test if the observed deuteration differences are larger than a specified, biologically relevant difference. Finally, we underline the importance of having a sufficient number of replicates, and the effect of the number of replicates on the power of the statistical significance tests. The R-code for the proposed moderated test statistics is available upon request.

SC-MEB: spatial clustering with hidden Markov random field using empirical Bayes.

Spatial transcriptomics has been emerging as a powerful technique for resolving gene expression profiles while retaining tissue spatial information. These spatially resolved transcriptomics make it feasible to examine the complex multicellular systems of different microenvironments. To answer scientific questions with spatial transcriptomics and expand our understanding of how cell types and states are regulated by microenvironment, the first step is to identify cell clusters by integrating the available spatial information. Here, we introduce SC-MEB, an empirical Bayes approach for spatial clustering analysis using a hidden Markov random field. We have also derived an efficient expectation-maximization algorithm based on an iterative conditional mode for SC-MEB. In contrast to BayesSpace, a recently developed method, SC-MEB is not only computationally efficient and scalable to large sample sizes but is also capable of choosing the smoothness parameter and the number of clusters. We performed comprehensive simulation studies to demonstrate the superiority of SC-MEB over some existing methods. We applied SC-MEB to analyze the spatial transcriptome of human dorsolateral prefrontal cortex tissues and mouse hypothalamic preoptic region. Our analysis results showed that SC-MEB can achieve a similar or better clustering performance to BayesSpace, which uses the true number of clusters and a fixed smoothness parameter. Moreover, SC-MEB is scalable to large ‘sample sizes’. We then employed SC-MEB to analyze a colon dataset from a patient with colorectal cancer (CRC) and COVID-19, and further performed differential expression analysis to identify signature genes related to the clustering results. The heatmap of identified signature genes showed that the clusters identified using SC-MEB were more separable than those obtained with BayesSpace. Using pathway analysis, we identified three immune-related clusters, and in a further comparison, found the mean expression of COVID-19 signature genes was greater in immune than non-immune regions of colon tissue. SC-MEB provides a valuable computational tool for investigating the structural organizations of tissues from spatial transcriptomic data.

Abstract 9855: Bayesian Multivariate Genetic Analysis of Lipids Improves Translational Insight

Introduction: Genome-wide association studies (GWAS) of blood lipids have yielded several translational insights for coronary artery disease. Hypothesis: We tested the hypothesis that novel Bayesian methods leveraging genetic correlation can improve insights from lipid genetics analyses. Methods: We used published lipids GWAS results from 297,626 participants of the Million Veterans Program (MVP), and generated summary statistics for 276,096 participants of the UK Biobank. We employed a multivariate adaptive shrinkage (mashR) model to each dataset. MashR allows the effect of a SNP to be modeled as a mixture of multivariate normal distributions across lipid levels (LDL-C, HDL-C, Total Cholesterol, and triglycerides) and an empirical Bayes approach to appropriately nudge the posterior estimates of the effect in accordance with the overall patterns observed in the data. Results: In addition to reproducing all 318 loci significantly associated with lipids in MVP, we identified 4,689 novel loci associated with lipids using mashR. We observed a high degree of agreement between MVP and UK Biobank, with over 95% of UKBB replicated in MVP and an additional 1246 identified. In genetic prioritization analyses, mashR shows improvement in the detection of known Mendelian genes and the consistency of ranks among subgroups compared with univariate results. Our approach improves the estimate of enrichment parameters of known genomic features similar to established GWAS variants. Finally, combining these joint estimates with tools for incorporating the genetic correlation structure, mashR improves the proportion of variation explained by lipid polygenic risk scores by up to 58% (se 0.015) when compared to univariate substrates across lipid traits. This improvement holds across both European and non-European individuals. We show that much of this gain in power arises from improved precision of the posterior estimates where the ratio of original standard error over posterior marginal variance reflects an increased ‘effective sample size’. Conclusions: Our framework shows that Bayesian multivariate genetic analysis markedly improves power for genomic discovery, better identifies causal genes, and improves polygenic risk prediction for complex traits.

Bayesian Meta-Prior Learning Using Empirical Bayes

Adding domain knowledge to a learning system is known to improve results. In multiparameter Bayesian frameworks, such knowledge is incorporated as a prior. On the other hand, the various model parameters can have different learning rates in real-world problems, especially with skewed data. Two often-faced challenges in operation management and management science applications are the absence of informative priors and the inability to control parameter learning rates. In this study, we propose a hierarchical empirical Bayes approach that addresses both challenges and that can generalize to any Bayesian framework. Our method learns empirical meta-priors from the data itself and uses them to decouple the learning rates of first-order and second-order features (or any other given feature grouping) in a generalized linear model. Because the first-order features are likely to have a more pronounced effect on the outcome, focusing on learning first-order weights first is likely to improve performance and convergence time. Our empirical Bayes method clamps features in each group together and uses the deployed model’s observed data to empirically compute a hierarchical prior in hindsight. We report theoretical results for the unbiasedness, strong consistency, and optimal frequentist cumulative regret properties of our meta-prior variance estimator. We apply our method to a standard supervised learning optimization problem as well as an online combinatorial optimization problem in a contextual bandit setting implemented in an Amazon production system. During both simulations and live experiments, our method shows marked improvements, especially in cases of small traffic. Our findings are promising because optimizing over sparse data is often a challenge. This paper was accepted by Hamid Nazerzadeh, Management Science Special Section on Data-Driven Prescriptive Analytics.

Poisson Counts, Square Root Transformation and Small Area Estimation: Square Root Transformation.

The paper intends to serve two objectives. First, it revisits the celebrated Fay-Herriot model, but with homoscedastic known error variance. The motivation comes from an analysis of count data, in the present case, COVID-19 fatality for all counties in Florida. The Poisson model seems appropriate here, as is typical for rare events. An empirical Bayes (EB) approach is taken for estimation. However, unlike the conventional conjugate gamma or the log-normal prior for the Poisson mean, here we make a square root transformation of the original Poisson data, along with square root transformation of the corresponding mean. Proper back transformation is used to infer about the original Poisson means. The square root transformation makes the normal approximation of the transformed data more justifiable with added homoscedasticity. We obtain exact analytical formulas for the bias and mean squared error of the proposed EB estimators. In addition to illustrating our method with the COVID-19 example, we also evaluate performance of our procedure with simulated data as well.

Bayesian Approaches for Poisson Distribution Parameter Estimation

The Bayesian approach, a non-classical estimation technique, is very widely used in statistical inference for real world situations. The parameter is considered to be a random variable, and knowledge of the prior distribution is used to update the parameter estimation. Herein, two Bayesian approaches for Poisson parameter estimation by deriving the posterior distribution under the squared error loss or quadratic loss functions are proposed. Their performances were compared with frequentist (maximum likelihood estimator) and Empirical Bayes approaches through Monte Carlo simulations. The mean square error was used as the test criterion for comparing the methods for point estimation; the smallest value indicates the best performing method with the estimated parameter value closest to the true parameter value. Coverage Probabilities (CPs) and average lengths (ALs) were obtained to evaluate the performances of the methods for constructing confidence intervals. The results reveal that the Bayesian approaches were excellent for point estimation when the true parameter value was small (0.5, 1 and 2). In the credible interval comparison, these methods obtained CP values close to the nominal 0.95 confidence level and the smallest ALs for large sample sizes (50 and 100), when the true parameter value was small (0.5, 1 and 2). Doi: 10.28991/esj-2021-01310 Full Text: PDF

Identifying priority and bright spot areas for improving diabetes care: a geospatial approach

The objective of this study was to describe a novel geospatial methodology for identifying poor-performing (priority) and well-performing (bright spot) communities with respect to diabetes management at the ZIP Code...

Application of regularized Savitzky–Golay filters to identification of time-varying systems

Savitzky–Golay (SG) filtering is a classical signal smoothing technique based on the local least squares approximation of the analyzed signal by a linear combination of known functions of time (originally — powers of time, which corresponds to polynomial approximation). It is shown that the regularized version of the SG algorithm can be successfully applied to identification of time-varying finite impulse response (FIR) systems. Such a solution is possible owing to the recently proposed preestimation technique, which converts the problem of identification of a time-varying FIR system into the problem of smoothing of the appropriately generated preestimates of system parameters. The resulting fast regularized local basis function estimators, optimized using the empirical Bayes approach, have very good parameter tracking capabilities, favorably comparing with the state-of-the-art in terms of accuracy, computational complexity and numerical robustness.

Empirical Bayes Analysis on the Power Law Process with Natural Conjugate Priors

The power law process has been used extensively in software reliability models, reliability growth models and more generally reliable systems. In this paper we work on the Power Law Process via empirical Bayes (EB) approach. Based on a two-hyperparameter natural conjugate prior and a more generalized three-hyperparameter natural conjugate prior, which was stated in Huang and Bier (1998), we work out an empirical Bayes (EB) procedure and provide statistical inferences based on the natural conjugate priors. Given past experience about the parameters of the model, the empirical Bayes (EB) approach uses the observed data to estimate the hyperparamters of priors and then proceeds as though the prior were known.

Least square and Empirical Bayes Approaches for Estimating Random Change Points

Here we develop methods for applications where random change points are known to be present a priori and the interest lies in their estimation and investigating risk factors that influence them. A simple least square method estimating each individual’s change point based on one’s own observations is first proposed. An easy-to-compute empirical Bayes type shrinkage is then proposed to pool information from separately estimated change points. A method to improve the empirical Bayes estimates is developed. Simulations are conducted to compare least-square estimates and Bayes shrinkage estimates. The proposed methods are applied to the Berkeley Growth Study data to estimate the transition age of the puberty height growth.

Least square and Empirical Bayes Approaches for Estimating Random Change Points

Here we develop methods for applications where random change points are known to be present a priori and the interest lies in their esti- mation and investigating risk factors that influence them. A simple least- square method estimating each individual's change point based on one's own observations is first proposed. An easy-to-compute empirical Bayes type shrinkage is then proposed to pool information from separately esti- mated change points. A method to improve the empirical Bayes estimates is developed. Simulations are conducted to compare least-square estimates and Bayes shrinkage estimates. The proposed methods are applied to the Berkeley Growth Study data to estimate the transition age of the puberty height growth.

A Bayesian Approach to Successive Comparisons

The present article discusses and compares multiple testing procedures (MTPs) for controlling the family wise error rate. Machekano and Hubbard (2006) have proposed empirical Bayes approach that is a resampling based multiple testing procedure asymptotically controlling the familywise error rate. In this paper we provide some additional work on their procedure, and we develop resampling based step-down procedure asymptotically controlling the familywise error rate for testing the families of one-sided hypotheses. We apply these procedures for making successive comparisons between the treatment effects under a simple-order assumption. For example, the treatment means may be a sequences of increasing dose levels of a drug. Using simulations, we demonstrate that the proposed step-down procedure is less conservative than the Machekano and Hubbard’s procedure. The application of the procedure is illustrated with an example.

SMALL AREA ESTIMATION METHOD WITH EMPIRICAL BAYES BASED ON BETA BINOMIAL MODEL IN GENERATED DATA

Small Area Estimation is one of the methods that can be used to estimate parameters in an area that has a small population. This study aims to estimate the value of the binary data parameter using the direct estimation method and an indirect estimation method by using the Empirical Bayes approach. To illustrate the method, we consider three conditions: direct estimator, empirical Bayes (EB) with auxiliary variables, and empirical Bayes without auxiliary variables. The smaller value of Mean Square Error is used to determine the better method. The results showed that the indirect estimation methods (EB method) gave the parameter value that was not much different from the direct estimation value. Then, the MSE values of indirect estimation with an auxiliary variable are smaller than the direct estimation method.

A fast and calibrated computer model emulator: an empirical Bayes approach

Mathematical models implemented on a computer have become the driving force behind the acceleration of the cycle of scientific processes. This is because computer models are typically much faster and economical to run than physical experiments. In this work, we develop an empirical Bayes approach to predictions of physical quantities using a computer model, where we assume that the computer model under consideration needs to be calibrated and is computationally expensive. We propose a Gaussian process emulator and a Gaussian process model for the systematic discrepancy between the computer model and the underlying physical process. This allows for closed-form and easy-to-compute predictions given by a conditional distribution induced by the Gaussian processes. We provide a rigorous theoretical justification of the proposed approach by establishing posterior consistency of the estimated physical process. The computational efficiency of the methods is demonstrated in an extensive simulation study and a real data example. The newly established approach makes enhanced use of computer models both from practical and theoretical standpoints.

Consistency of empirical Bayes and kernel flow for hierarchical parameter estimation

Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success of this methodology, in a wide range of applications to complex and real-world problems, is hierarchical modeling and learning of hyperparameters. The purpose of this paper is to study two paradigms of learning hierarchical parameters: one is from the probabilistic Bayesian perspective, in particular, the empirical Bayes approach that has been largely used in Bayesian statistics; the other is from the deterministic and approximation theoretic view, and in particular the kernel flow algorithm that was proposed recently in the machine learning literature. Analysis of their consistency in the large data limit, as well as explicit identification of their implicit bias in parameter learning, are established in this paper for a Mat\'ern-like model on the torus. A particular technical challenge we overcome is the learning of the regularity parameter in the Mat\'ern-like field, for which consistency results have been very scarce in the spatial statistics literature. Moreover, we conduct extensive numerical experiments beyond the Mat\'ern-like model, comparing the two algorithms further. These experiments demonstrate learning of other hierarchical parameters, such as amplitude and lengthscale; they also illustrate the setting of model misspecification in which the kernel flow approach could show superior performance to the more traditional empirical Bayes approach.

Improving target price calculations in Medicare bundled payment programs.

To compare the predictive accuracy of two approaches to target price calculations under Bundled Payments for Care Improvement-Advanced (BPCI-A): the traditional Centers for Medicare and Medicaid Services (CMS) methodology and an empirical Bayes approach designed to mitigate the effects of regression to the mean. Medicare fee-for-service claims for beneficiaries discharged from acute care hospitals between 2010 and 2016. We used data from a baseline period (discharges between January 1, 2010 and September 30, 2013) to predict spending in a performance period (discharges between October 1, 2015 and June 30, 2016). For 23 clinical episode types in BPCI-A, we compared the average prediction error across hospitals associated with each statistical approach. We also calculated an average across all clinical episode types and explored differences by hospital size. We used a 20% sample of Medicare claims, excluding hospitals and episode types with small numbers of observations. The empirical Bayes approach resulted in significantly more accurate episode spending predictions for 19 of 23 clinical episode types. Across all episode types, prediction error averaged $8456 for the CMS approach versus $7521 for the empirical Bayes approach. Greater improvements in accuracy were observed with increasing hospital size. CMS should consider using empirical Bayes methods to calculate target prices for BPCI-A.

Alleviating confounding in spatio-temporal areal models with an application on crimes against women in India

Assessing associations between a response of interest and a set of covariates in spatial areal models is the leitmotiv of ecological regression. However, the presence of spatially correlated random effects can mask or even bias estimates of such associations due to confounding effects if they are not carefully handled. Though potentially harmful, confounding issues have often been ignored in practice leading to wrong conclusions about the underlying associations between the response and the covariates. In spatio-temporal areal models, the temporal dimension may emerge as a new source of confounding, and the problem may be even worse. In this work, we propose two approaches to deal with confounding of fixed effects by spatial and temporal random effects, while obtaining good model predictions. In particular, restricted regression and an apparently—though in fact not—equivalent procedure using constraints are proposed within both fully Bayes and empirical Bayes approaches. The methods are compared in terms of fixed-effect estimates and model selection criteria. The techniques are used to assess the association between dowry deaths and certain socio-demographic covariates in the districts of Uttar Pradesh, India.

Learning linear modules in a dynamic network using regularized kernel-based methods

In order to identify one system (module) in an interconnected dynamic network, one typically has to solve a Multi-Input-Single-Output (MISO) identification problem that requires identification of all modules in the MISO setup. For application of a parametric identification method this would require estimating a large number of parameters, as well as an appropriate model order selection step for a possibly large scale MISO problem, thereby increasing the computational complexity of the identification algorithm to levels that are beyond feasibility. An alternative identification approach is presented employing regularized kernel-based methods. Keeping a parametric model for the module of interest, we model the impulse response of the remaining modules in the MISO structure as zero mean Gaussian processes (GP) with a covariance matrix (kernel) given by the first-order stable spline kernel, accounting for the noise model affecting the output of the target module and also for possible instability of systems in the MISO setup. Using an Empirical Bayes (EB) approach the target module parameters are estimated through an Expectation–Maximization (EM) algorithm with a substantially reduced computational complexity, while avoiding extensive model structure selection. Numerical simulations illustrate the potentials of the introduced method in comparison with the state-of-the-art techniques for local module identification.

Optimal post-selection inference for sparse signals: a nonparametric empirical Bayes approach

Summary Many recently developed Bayesian methods focus on sparse signal detection. However, much less work has been done on the natural follow-up question: how does one make valid inferences for the magnitude of those signals after selection? Ordinary Bayesian credible intervals suffer from selection bias, as do ordinary frequentist confidence intervals. Existing Bayesian methods for correcting this bias produce credible intervals with poor frequentist properties. Further, existing frequentist approaches require sacrificing the benefits of shrinkage typical in Bayesian methods, resulting in confidence intervals that are needlessly wide. We address this gap by proposing a nonparametric empirical Bayes approach to constructing optimal selection-adjusted confidence sets. Our method produces confidence sets that are as short as possible on average, while both adjusting for selection and maintaining exact frequentist coverage uniformly over the parameter space. We demonstrate an important consistency property of our procedure: under mild conditions, it asymptotically converges to the results of an oracle-Bayes analysis in which the prior distribution of signal sizes is known exactly. Across a series of examples, the method is found to outperform existing frequentist techniques for post-selection inference, producing confidence sets that are notably shorter, but with the same coverage guarantee.

Using space-time cube to analyze trends in adverse birth outcomes and maternal characteristics in Massachusetts, USA.

Rates of preterm births (< 37 gestational weeks) and low birthweight (≤ 2500 g) are rising throughout the United States. This study uses singleton live birth data, Empirical Bayes approach, space–time cube and Mann–Kendall statistic to evaluate temporal trends in these adverse birth outcomes (ABO) and maternal characteristics over 15 years (2000–2014) at the census tract level for non-Hispanic white and black women in Massachusetts. In addition to analyzing trends for each variable individually, the study analyzes spatial coincidence of trends to determine which maternal characteristics exhibited trends that most strongly correlated with the ABO trends. The 15-year average rate of ABO was 7.34% for white women, and 12.05% for black women. Results show that more census tracts exhibited an increasing trend than decreasing trend in birth outcomes and in several maternal characteristics for both races (gestational and chronic hypertension, gestational diabetes, and previous preterm birth). Study identified 52 census tracts concurrently experiencing an increasing trend in ABO and in four maternal characteristics for black women, indicating that multiple negative trends in health outcomes are concentrated at the same location creating a potential for even more adverse outcomes in the future. This study provides a novel, spatially explicit analytical framework based on Empirical Bayes rates and space–time cube, which could be extended to analyze trends in other health outcomes at various spatial scales.Supplementary InformationThe online version supplementary material available at 10.1007/s10708-021-10382-w.