Constructing Confidence Intervals Research Articles

Reinforcement Learning in Latent Heterogeneous Environments

Reinforcement Learning holds great promise for data-driven decision-making in various social contexts, including healthcare, education, and business. However, classical methods that focus on the mean of the total return may yield misleading results when dealing with heterogeneous populations typically found in large-scale datasets. To address this issue, we introduce the K-Value Heterogeneous Markov Decision Process, a framework designed to handle sequential decision problems with latent population heterogeneity. Within this framework, we propose auto-clustered policy evaluation for estimating the value of a given policy and auto-clustered policy iteration for estimating the optimal policy within a parametric policy class. Our auto-clustered algorithms can automatically identify homogeneous subpopulations while simultaneously estimating the action value function and the optimal policy for each subgroup. We establish convergence rates and construct confidence intervals for the estimators. Simulation results support our theoretical findings, and an empirical study conducted on a real medical dataset confirms the presence of value heterogeneity and validates the advantages of our novel approach. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Selecting Invalid Instruments to Improve Mendelian Randomization with Two-Sample Summary Data.

Mendelian randomization (MR) is a widely-used method to estimate the causal relationship between a risk factor and disease. A fundamental part of any MR analysis is to choose appropriate genetic variants as instrumental variables. Genome-wide association studies often reveal that hundreds of genetic variants may be robustly associated with a risk factor, but in some situations investigators may have greater confidence in the instrument validity of only a smaller subset of variants. Nevertheless, the use of additional instruments may be optimal from the perspective of mean squared error even if they are slightly invalid; a small bias in estimation may be a price worth paying for a larger reduction in variance. For this purpose, we consider a method for "focused" instrument selection whereby genetic variants are selected to minimise the estimated asymptotic mean squared error of causal effect estimates. In a setting of many weak and locally invalid instruments, we propose a novel strategy to construct confidence intervals for post-selection focused estimators that guards against the worst case loss in asymptotic coverage. In empirical applications to: (i) validate lipid drug targets; and (ii) investigate vitamin D effects on a wide range of outcomes, our findings suggest that the optimal selection of instruments does not involve only a small number of biologically-justified instruments, but also many potentially invalid instruments.

A parametric bootstrap approach for computing confidence intervals for genetic correlations with application to genetically determined protein-protein networks

Genetic correlation refers to the correlation between genetic determinants of a pair of traits. When using individual-level data, it is typically estimated based on a bivariate model specification where the correlation between the two variables is identifiable and can be estimated from a covariance model that incorporates the genetic relationship between individuals, e.g., using a pre-specified kinship matrix. Inference relying on asymptotic normality of the genetic correlation parameter estimates may be inaccurate when the sample size is low, when the genetic correlation is close to the boundary of the parameter space, and when the heritability of at least one of the traits is low. We address this problem by developing a parametric bootstrap procedure to construct confidence intervals for genetic correlation estimates. The procedure simulates paired traits under a range of heritability and genetic correlation parameters, and it uses the population structure encapsulated by the kinship matrix. Heritabilities and genetic correlations are estimated using the close-form, method of moment, Haseman-Elston regression estimators. The proposed parametric bootstrap procedure is especially useful when genetic correlations are computed on pairs of thousands of traits measured on the same exact set of individuals. We demonstrate the parametric bootstrap approach on a proteomics dataset from the Jackson Heart Study.

Importance sampling method for Feldman-Cousins confidence intervals

In various high-energy physics contexts, such as neutrino-oscillation experiments, several assumptions underlying the typical asymptotic confidence interval construction are violated, such that one has to resort to computationally expensive methods like the Feldman-Cousins method for obtaining confidence intervals with proper statistical coverage. By construction, the computation of intervals at high confidence levels requires fitting millions or billions of pseudoexperiments, while wasting most of the computational cost on overly precise intervals at low confidence levels. In this work, a simple importance sampling method is introduced that reuses pseudoexperiments produced for all tested parameter values in a single mixture distribution. This results in a significant error reduction on the estimated critical values, especially at high confidence levels, and simultaneously yields a correct interpolation of these critical values between the parameter values at which the pseudoexperiments were produced. The theoretically calculated performance is demonstrated numerically using a simple example from the analysis of neutrino oscillations. The relationship to similar techniques applied in statistical mechanics and p-value computations is discussed. Published by the American Physical Society 2024

Constructing Confidence Intervals for BIFSG Disparity Estimates

This paper describes a method to construct confidence intervals for estimates of disparities in outcomes between racial/ethnic groups when race and ethnicity are estimated using Bayesian Improved First Name and Surname Geocoding (BIFSG). Standard errors produced by regression packages understate the uncertainty in these estimates because they fail to incorporate noise arising from estimating race and ethnicity. This can produce false confidence when assessing the direction and magnitude of disparities. Conversely, the bootstrapping approach introduced in this paper incorporates uncertainty arising from the use of BIFSG. However, this procedure does not correct for any statistical bias associated with using BIFSG.

Jackknife empirical likelihood for the correlation coefficient with multiplicative distortion measurement errors

In this paper, we consider the estimation problem of a correlation coefficient between two unobserved variables of interest that are distorted in a multiplicative way by some unobserved confounding variable. We investigate the direct plug-in estimator of the correlation coefficient. We propose using jackknife empirical likelihood (JEL) and its variations to construct confidence intervals for the correlation coefficient based on the estimator. The proposed JEL statistic is shown to be asymptotically a standard chi-squared distribution. We compare our methods to the previous empirical likelihood (EL) techniques of Zhang et al. (2014, ‘A Revisit to Correlation Analysis for Distortion Measurement Error Data’, Journal of Multivariate Analysis, 124, 116–129) and show the JEL possesses better small sample properties. Simulation studies are conducted to examine the performance of the proposed estimator, and we also use our proposed methods to analyse the Boston housing data for illustration.

On Sample Size Needed for Block Bootstrap Confidence Intervals to Have Desired Coverage Rates

Block bootstrap is widely used in constructing confidence intervals for parameters estimated from stationary time series. Theoretically, the method should provide valid confidence intervals as the length of the time series goes to infinity. In practice, however, it is necessary to know how large of a finite sample is required for block bootstrap confidence intervals to work well. This study aims to answer this question in a simple simulation setting where the data are generated from a first-order autoregressive process. The empirical coverage rates of several commonly used bootstrap confidence intervals for the mean, standard deviation, and the lag-1 autocorrelation coefficient are compared. A quite large sample is found necessary for the intervals to have the right coverage rates even when estimating a simple parameter like the mean. Some block bootstrap methods could fail when estimating the lag-1 autocorrelation. It is surprising that the coverage property even deteriorates as the sample size increases with some commonly used block bootstrap confidence intervals including the percentile intervals and bias-corrected intervals. KEYWORDS: Autocorrelation; Bias-Correction; Centering; Dependent Data; Percentile; Resampling; Simulation; Time Series

Effectiveness of Constant Stepsize in Markovian LSA and Statistical Inference

In this paper, we study the effectiveness of using a constant stepsize in statistical inference via linear stochastic approximation (LSA) algorithms with Markovian data. After establishing a Central Limit Theorem (CLT), we outline an inference procedure that uses averaged LSA iterates to construct confidence intervals (CIs). Our procedure leverages the fast mixing property of constant-stepsize LSA for better covariance estimation and employs Richardson-Romberg (RR) extrapolation to reduce the bias induced by constant stepsize and Markovian data. We develop theoretical results for guiding stepsize selection in RR extrapolation, and identify several important settings where the bias provably vanishes even without extrapolation. We conduct extensive numerical experiments and compare against classical inference approaches. Our results show that using a constant stepsize enjoys easy hyperparameter tuning, fast convergence, and consistently better CI coverage, especially when data is limited.

Resampling-based confidence intervals and bands for the average treatment effect in observational studies with competing risks

The g-formula can be used to estimate the treatment effect while accounting for confounding bias in observational studies. With regard to time-to-event endpoints, possibly subject to competing risks, the construction of valid pointwise confidence intervals and time-simultaneous confidence bands for the causal risk difference is complicated, however. A convenient solution is to approximate the asymptotic distribution of the corresponding stochastic process by means of resampling approaches. In this paper, we consider three different resampling methods, namely the classical nonparametric bootstrap, the influence function equipped with a resampling approach as well as a martingale-based bootstrap version, the so-called wild bootstrap. For the latter, three sub-versions based on differing distributions of the underlying random multipliers are examined. We set up a simulation study to compare the accuracy of the different techniques, which reveals that the wild bootstrap should in general be preferred if the sample size is moderate and sufficient data on the event of interest have been accrued. For illustration, the resampling methods are further applied to data on the long-term survival in patients with early-stage Hodgkin’s disease.

Confidence intervals for functions of signal-to-noise ratio with application to economics and finance

PurposeConfidence intervals play a crucial role in economics and finance, providing a credible range of values for an unknown parameter along with a corresponding level of certainty. Their applications encompass economic forecasting, market research, financial forecasting, econometric analysis, policy analysis, financial reporting, investment decision-making, credit risk assessment and consumer confidence surveys. Signal-to-noise ratio (SNR) finds applications in economics and finance across various domains such as economic forecasting, financial modeling, market analysis and risk assessment. A high SNR indicates a robust and dependable signal, simplifying the process of making well-informed decisions. On the other hand, a low SNR indicates a weak signal that could be obscured by noise, so decision-making procedures need to take this into serious consideration. This research focuses on the development of confidence intervals for functions derived from the SNR and explores their application in the fields of economics and finance.Design/methodology/approachThe construction of the confidence intervals involved the application of various methodologies. For the SNR, confidence intervals were formed using the generalized confidence interval (GCI), large sample and Bayesian approaches. The difference between SNRs was estimated through the GCI, large sample, method of variance estimates recovery (MOVER), parametric bootstrap and Bayesian approaches. Additionally, confidence intervals for the common SNR were constructed using the GCI, adjusted MOVER, computational and Bayesian approaches. The performance of these confidence intervals was assessed using coverage probability and average length, evaluated through Monte Carlo simulation.FindingsThe GCI approach demonstrated superior performance over other approaches in terms of both coverage probability and average length for the SNR and the difference between SNRs. Hence, employing the GCI approach is advised for constructing confidence intervals for these parameters. As for the common SNR, the Bayesian approach exhibited the shortest average length. Consequently, the Bayesian approach is recommended for constructing confidence intervals for the common SNR.Originality/valueThis research presents confidence intervals for functions of the SNR to assess SNR estimation in the fields of economics and finance.

Confidence Intervals for Parameters of Unobserved Events

Consider a finite sample from an unknown distribution over a countable alphabet. Unobserved events are alphabet symbols which do not appear in the sample. Estimating the probabilities of unobserved events is a basic problem in statistics and related fields, which was extensively studied in the context of point estimation. In this work we introduce a novel interval estimation scheme for unobserved events. Our proposed framework applies selective inference, as we construct confidence intervals (CIs) for the desired set of parameters. Interestingly, we show that obtained CIs are dimension-free, as they do not grow with the alphabet size. Further, we show that these CIs are (almost) tight, in the sense that they cannot be further improved without violating the prescribed coverage rate. We demonstrate the performance of our proposed scheme in synthetic and real-world experiments, showing a significant improvement over the alternatives. Finally, we apply our proposed scheme to large alphabet modeling. We introduce a novel simultaneous CI scheme for large alphabet distributions which outperforms currently known methods while maintaining the prescribed coverage rate. Supplementary materials for this article are available online including a standardized description of the materials available for reproducing the work.

Inference on the reliability of inverse Weibull with multiply Type-I censored data

Inverse Weibull (IW) distribution is widely used due to its non-monotonic hazard function. For the IW distribution, existing research on statistical inference has mostly focused on censored data, but there has been no study on multiply Type-I censoring until now as far as we know. The multiple Type-I censoring, acted as an extended form of Type-I censoring, is frequently encountered in industry and medicine research, which also needs to be paid attention. Thus, this study conducts detailed analyses on the reliability of IW distribution with multiply Type-I censored data, including point estimation and confidence interval (CI) construction. Four methods, including the maximum likelihood estimation (MLE), least square estimation (LSE), and two Bayesian estimations, namely MCMC and Lindley, are adopted in point estimation. Results show that the Lindley method performs best with small mission times, while MLE is optimal for large mission times. Specifically, for the CIs construction, we proposed a pivotal quantity based on LSE to construct the CIs of reliability and compare it with two popular methods, Fisher information matrix (FIM) derivation and MCMC algorithm. Our proposed method shows competitive performance as the MCMC and outperforms FIM. Finally, those estimation methods are applied to an example for illustration.

Adjusting Incidence Estimates with Laboratory Test Performances: A Pragmatic Maximum Likelihood Estimation-Based Approach.

Understanding the incidence of disease is often crucial for public policy decision-making, as observed during the COVID-19 pandemic. Estimating incidence is challenging, however, when the definition of incidence relies on tests that imperfectly measure disease, as in the case when assays with variable performance are used to detect the SARS-CoV-2 virus. To our knowledge, there are no pragmatic methods to address the bias introduced by the performance of labs in testing for the virus. In the setting of a longitudinal study, we developed a maximum likelihood estimation-based approach to estimate laboratory performance-adjusted incidence using the expectation-maximization algorithm. We constructed confidence intervals (CIs) using both bootstrapped-based and large-sample interval estimator approaches. We evaluated our methods through extensive simulation and applied them to a real-world study (TrackCOVID), where the primary goal was to determine the incidence of and risk factors for SARS-CoV-2 infection in the San Francisco Bay Area from July 2020 to March 2021. Our simulations demonstrated that our method converged rapidly with accurate estimates under a variety of scenarios. Bootstrapped-based CIs were comparable to the large-sample estimator CIs with a reasonable number of incident cases, shown via a simulation scenario based on the real TrackCOVID study. In more extreme simulated scenarios, the coverage of large-sample interval estimation outperformed the bootstrapped-based approach. Results from the application to the TrackCOVID study suggested that assuming perfect laboratory test performance can lead to an inaccurate inference of the incidence. Our flexible, pragmatic method can be extended to a variety of disease and study settings.

A maximum likelihood and regenerative bootstrap approach for estimation and forecasting of INAR(p) processes with zero-inflated innovations

In this work, we study a class of p-order non-negative integer-valued autoregressive (INAR(p)) processes, with innovations following zero-inflated (ZI) distributions called ZI-INAR(p) processes. Based on the EM algorithm, we present an estimation procedure of parameters model. We also develop a regenerative bootstrap method to construct confidence intervals for the parameters as well as to estimate the forecasting distributions for future values. We discuss asymptotic properties of the regenerative bootstrap method. The performance of the proposed methods is evaluated considering the analysis of two simulation studies and a real dataset.

Classical and Bayesian inference of inverted modified Lindley distribution based on progressive type-II censoring for modeling engineering data

This paper investigates statistical inferences for product lifetimes following the inverted modified Lindley distribution, utilizing progressive Type-II censored data. The estimation of model parameters employs the maximum likelihood method, complemented by the construction of approximate confidence intervals. Bayesian estimates are also explored, incorporating squared error and linear exponential loss functions with noninformative priors. To approximate Bayes estimates, the proposal presents Gibbs sampling based on the MCMC algorithm. This results in the generation of the greatest posterior density credible intervals for the parameters. A real data analysis is conducted to validate the accuracy of all the models and methods discussed. Finally, computational studies using Monte Carlo simulations are presented to compare the suggested estimators.

BOOTSTRAP CONFIDENCE INTERVALS IN LINEAR MODELS: CASE OF OUTLIERS

Confidence interval estimations in linear models have been of large interest in social science. However, traditional approach of building confidence intervals has a set of assumption including dataset having no extreme outliers. In this study, we discuss presence of severe outliers in linear models and suggest bootstrap approach as an alternative way to construct confidence intervals. We conclude that bootstrap confidence intervals can outperform traditional confidence intervals in presence of outliers when sample size is small or population distribution is not normal. Lastly, we encourage researchers to run a computer simulation to evaluate conclusions of this study.

Learning likelihood ratios with neural network classifiers

The likelihood ratio is a crucial quantity for statistical inference in science that enables hypothesis testing, construction of confidence intervals, reweighting of distributions, and more. Many modern scientific applications, however, make use of data- or simulation-driven models for which computing the likelihood ratio can be very difficult or even impossible. By applying the so-called “likelihood ratio trick,” approximations of the likelihood ratio may be computed using clever parametrizations of neural network-based classifiers. A number of different neural network setups can be defined to satisfy this procedure, each with varying performance in approximating the likelihood ratio when using finite training data. We present a series of empirical studies detailing the performance of several common loss functionals and parametrizations of the classifier output in approximating the likelihood ratio of two univariate and multivariate Gaussian distributions as well as simulated high-energy particle physics datasets.

A Comparison of Statistical Methods to Construct Confidence Intervals and Fiducial Intervals for Measures of Health Disparities.

Health disparities are differences in health status across different socioeconomic groups. Classical methods, e.g., the Delta method, have been used to estimate the standard errors of estimated measures of health disparities and to construct confidence intervals for these measures. However, the confidence intervals constructed using the classical methods do not have good coverage properties for situations involving sparse data. In this article, we introduce three new methods to construct fiducial intervals for measures of health disparities based on approximate fiducial quantities. Through a comprehensive simulation study, We compare the empirical coverage properties of the proposed fiducial intervals against two Monte Carlo simulation-based methods-utilizing either a truncated Normal distribution or the Gamma distribution-as well as the classical method. The findings of the simulation study advocate for the adoption of the Monte Carlo simulation-based method with the Gamma distribution when a unified approach is sought for all health disparity measures.

Confidence intervals for heterogeneity in meta-analysis of the rare binary events based on empirical likelihood-type methods

In meta-analysis, heterogeneity between independent studies is one of the important reference indicators for comprehensive analyses. It is usually described in terms of the variance between groups of the random-effect model. The existing methods are used to construct their confidence intervals based on the assumption of the normal distribution, but in reality, this assumption is often violated, especially in rare binary events. Recently, the method of the jackknife empirical likelihood was proposed to build confidence intervals; however, it tends to perform not as good as the parametric methods when the number of studies involved is small. Therefore, we propose to use the adjusted jackknife empirical likelihood, transformed jackknife empirical likelihood, and transformed adjusted jackknife empirical likelihood to construct confidence intervals for heterogeneity of the rare binary event. We investigate the performance of the proposed methods through the simulations based on different distributions, especially for skewed distributions and the small sample sizes. Applications to real data with the proposed methods are also provided.

Simultaneous Confidence Intervals for Ratios of the Percentiles of the Delta-Lognormal Distribution

Statistical percentiles are extremely important and valuable measurement tools in various contexts. They offer significant advantages when it comes to estimating a wide range of values across different fields, making them essential in statistical analysis and research. This importance motivates researchers to study the construction of simultaneous confidence intervals for ratios of percentiles of the delta-lognormal distribution. This research introduces four distinct methods for constructing such confidence intervals: the fiducial generalized confidence interval, the Bayesian-based highest posterior density credible interval using Jeffreys prior, Jeffreys rule prior, and Uniform prior. All methods are subjected to evaluation and comparison through coverage probabilities and expected lengths in Monte Carlo simulations. The simulation study shows that, overall, the Bayesian-based highest posterior density credible interval works better and more accurately than the fiducial generalized confidence interval. To further validate the findings of the simulation study, these methods are applied to real-world daily rainfall data collected from a river basin in Thailand.