Small Sample Power Research Articles

Two-sample t α -test for testing hypotheses in small-sample experiments.

It has been reported that about half of biological discoveries are irreproducible. These irreproducible discoveries were partially attributed to poor statistical power. The poor powers are majorly owned to small sample sizes. However, in molecular biology and medicine, due to the limit of biological resources and budget, most molecular biological experiments have been conducted with small samples. Two-sample t-test controls bias by using a degree of freedom. However, this also implicates that t-test has low power in small samples. A discovery found with low statistical power suggests that it has a poor reproducibility. So, promotion of statistical power is not a feasible way to enhance reproducibility in small-sample experiments. An alternative way is to reduce type I error rate. For doing so, a so-called t α -test was developed. Both theoretical analysis and simulation study demonstrate that t α -test much outperforms t-test. However, t α -test is reduced to t-test when sample sizes are over 15. Large-scale simulation studies and real experiment data show that t α -test significantly reduced type I error rate compared to t-test and Wilcoxon test in small-sample experiments. t α -test had almost the same empirical power with t-test. Null p-value density distribution explains why t α -test had so lower type I error rate than t-test. One real experimental dataset provides a typical example to show that t α -test outperforms t-test and a microarray dataset showed that t α -test had the best performance among five statistical methods. In addition, the density distribution and probability cumulative function of t α -statistic were given in mathematics and the theoretical and observed distributions are well matched.

Correcting the Bias Correction for the Bootstrap Confidence Interval in Mediation Analysis.

The bias-corrected bootstrap confidence interval (BCBCI) was once the method of choice for conducting inference on the indirect effect in mediation analysis due to its high power in small samples, but now it is criticized by methodologists for its inflated type I error rates. In its place, the percentile bootstrap confidence interval (PBCI), which does not adjust for bias, is currently the recommended inferential method for indirect effects. This study proposes two alternative bias-corrected bootstrap methods for creating confidence intervals around the indirect effect: one originally used by Stine (1989) with the correlation coefficient, and a novel method that implements a reduced version of the BCBCI's bias correction. Using a Monte Carlo simulation, these methods were compared to the BCBCI, PBCI, and Chen and Fritz (2021)'s 30% Winsorized BCBCI. The results showed that the methods perform on a continuum, where the BCBCI has the best balance (i.e., having closest to an equal proportion of CIs falling above and below the true effect), highest power, and highest type I error rate; the PBCI has the worst balance, lowest power, and lowest type I error rate; and the alternative bias-corrected methods fall between these two methods on all three performance criteria. An extension of the original simulation that compared the bias-corrected methods to the PBCI after controlling for type I error rate inflation suggests that the increased power of these methods might only be due to their higher type I error rates. Thus, if control over the type I error rate is desired, the PBCI is still the recommended method for use with the indirect effect. Future research should examine the performance of these methods in the presence of missing data, confounding variables, and other real-world complications to enhance the generalizability of these results.

Churn and catastrophe: insurance loss and high spending among patients with gynecologic cancer in the United States in the era of the Affordable Care Act

<h3>Objectives:</h3> Although often considered a static characteristic in oncology research, insurance coverage is dynamic in reality, potentially changing month-to-month. The Affordable Care Act (ACA) was passed in 2010 with aims of stabilizing and expanding insurance markets, and reducing financial burden, particularly for the poor and sick. We sought to assess insurance churn and catastrophic health expenditures (CHE) among patients with gynecologic cancer in the era of the ACA. <h3>Methods:</h3> Retrospective study of 2006-2017 Medical Expenditure Panel Survey (MEPS) respondents under age 65 reporting care in the given year related to a gynecologic cancer diagnosis. MEPS is an annual survey representative of the civilian, non-institutionalized US population. Weights were applied to estimate national rates of insurance churn (loss/change of coverage) and CHE (family out-of-pocket health spending >10% family income), and differences between subgroups and over time were assessed by the adjusted Wald test. <h3>Results:</h3> A total of 684 MEPS respondents under age 65 reported care related to a gynecologic cancer diagnosis in 2006-2017, representing an estimated average annual population of 533,000 persons (95% CI 462,000-603,000), which was majority White (87.0%) and non-Hispanic (85.5%). Relative to the overall under 65 US population, patients with gynecologic cancer reported higher rates of low-income (family income ≤250% federal poverty level; 45.1% vs 32.2%; <i>p</i><0.001) and employment disruption (15.2% job change/loss, 55.3% part-year unemployment, 38.6% full-year unemployment, vs 10.5%, 44.1%, and 32.4% respectively; <i>p</i><0.05 for all). In the era of the ACA since becoming law in 2010, patients with gynecologic cancer reported high annual rates of insurance churn: 8.8% (95% CI 5.2-12.5) with loss of insurance, 18.7% (95% CI 13.6-23.8) with change in insurance, 21.7% (95% CI 16.5-26.9) uninsured for at least one month, and 8.4% (95% CI 5.7-11.0) uninsured for entire year. The gynecologic cancer population also experienced high rates of catastrophic spending, with 12.8% (95% CI 8.9-16.7) reporting CHE by out-of-pocket expenses alone, and 28.0% (95% CI 21.6-34.4) when including premium spending. Non-white and Hispanic patients collectively reported elevated rates of insurance churn (25.9% insurance change, 30.2% any uninsurance, vs 16.3% and 18.7% respectively for non-Hispanic whites, <i>p</i><0.05 for both), but no difference in rates of CHE. Patients from low-income families faced significantly higher risk of CHE (22.7% vs 3.0% expenses alone, <i>p</i><0.001; 35.3% vs 20.8% including premiums, <i>p</i>=0.01). Full-year Medicaid coverage was more protective from CHE compared with full-year private coverage (overall 15.3% vs 31.3%, <i>p</i>=0.02; low-income 11.5% vs 62.1%, <i>p</i><0.001). In assessment for impact of the ACA comparing full ACA implementation in 2014-17 with pre-ACA years 2006-09, we observed trends towards lower rates of CHE among low-income gynecologic cancer patients after implementation, but no significant differences. <h3>Conclusions:</h3> Gynecologic cancer patients face high rates of insurance churn, with over one in five reporting uninsurance for least one month every year, and catastrophic health spending, with over one in four reporting CHE annually. Low-income patients face the highest risks of CHE, and are better protected with Medicaid coverage as compared to private insurance. There was no significant impact of the ACA, but small sample size limited estimate precision and power to detect small changes.

On Bootstrap Tests of Symmetry About an Unknown Median

It is important to examine the symmetry of an underlying distribution before applying some statistical procedures to a data set. For example, in the Zuni School District case, a formula originally developed by the Department of Education trimmed 5% of the data symmetrically from each end. The validity of this procedure was questioned at the hearing by Chief Justice Roberts. Most tests of symmetry (even nonparametric ones) are not distribution free in finite sample sizes. Hence, using asymptotic distribution may not yield an accurate type I error rate or/and loss of power in small samples. Bootstrap resampling from a symmetric empirical distribution function fitted to the data is proposed to improve the accuracy of the calculated p-value of several tests of symmetry. The results show that the bootstrap method is superior to previously used approaches relying on the asymptotic distribution of the tests that assumed the data come from a normal distribution. Incorporating the bootstrap estimate in a recently proposed test due to Miao, Gel and Gastwirth (2006) preserved its level and shows it has reasonable power properties on the family of distribution evaluated.

Triangulating evidence on the role of perceived versus objective experiences of childhood adversity in psychopathology

Triangulating evidence on the role of perceived versus objective experiences of childhood adversity in psychopathology

A Lagrange-Multiplier Test for Large Heterogeneous Panel Data Models

This paper introduces a new test for error cross-sectional independence in large panel data models with exogenous regressors having heterogenous slope coefficients. The proposed statistic, LM_{RMT}, is based on the Lagrange Multiplier (LM) principle and the sample correlation matrix R_{N} of the model's residuals. Since in large panels R_{N} poorly estimates its population counterpart, results from Random Matrix Theory are used to establish the high-dimensional limiting distribution of LM_{RMT} under heteroskedastic normal errors and assuming that both the panel size N and the sample size T grow to infinity in comparable magnitude. Simulation results support our theoretical findings, with LM_{RMT} being correctly sized (except for some small values of N and T). Further, the small sample size and power outcomes show robustness of our statistic to deviations from the assumptions of normality for the error terms and regressors, of strict exogeneity for the regressors, as well as of heterogeneity for their slope coefficients. The test has comparable small sample properties to related tests in the literature which have been developed under different asymptotic theory.

Cerebral Palsy: Sport and Exercise Considerations.

Cerebral palsy (CP) is a group of debilitating motor disorders that carries a wide array of clinical presentations ranging from isolated physical or cognitive impairment to global loss of function. Despite the prevalence of CP, recommendations and benefits of physical activity/exercise have historically not been clearly defined. The research on the subject has several limitations, including small sample size, power, standardized measures/outcomes, and poor classification regarding severity of the disease. Nonetheless, the general consensus and new research shows that individuals with CP who participate in sports and exercise regimens, even at reduced frequency and intensity, exhibit improvements in health care benefits, including cardiorespiratory endurance, gross motor function, gait stability, and reduction in pain. These regimens can be prescribed safely and individualized by health care providers to improve morbidity and mortality in patients suffering from CP.

On Robust Inference for Consumption-based Asset Pricing

Kleibergen and Zhan (Robust Inference for Consumption-based Asset Pricing, Journal of Finance, 2020) propose a new approach to testing consumption-based asset pricing models that is robust to useless factors, i.e. factors uncorrelated with the test assets. They find that recently proposed factors do not pass their test. I provide extensive simulation experiments of empirically relevant cases, which show that their method has almost no power to detect useful factors with a significant return correlation of 0.45+. In contrast, the traditional Fama-MacBeth method comes with good size and considerably higher power in small samples. Bootstrap methods can be utilized to evaluate accuracy in larger samples. I argue that the problem of useless factors is hugely exaggerated to motivate a new “robust” methodology that is actually not more “robust” than traditional methods but comes at the cost of lacking power to detect useful factors in empirically relevant cases.

Evaluation of eczema, asthma, allergic rhinitis and allergies among the grade-7 children of Iqaluit

BackgroundLittle is known about the prevalence of asthma, allergic rhinitis, eczema and allergies among Canadian Inuit children, especially those living in the arctic and subarctic areas.MethodsA cross-sectional study among grade seven students attending schools in Iqaluit, the capital city of Nunavut, was conducted during the 2016/2017 school year. We used the International Study of Allergy and Asthma in Children (ISAAC) questionnaire with added questions relevant to the population. In addition, skin prick tests (SPT) were conducted to test for sensitization to common food and environmental allergens.ResultsThe prevalence of current asthma is 5.2%, all of them were males and 2/3 of them were Inuit and all had a previous respiratory hospitalization. Past asthma prevalence is 8.6%, 60% males and 60% Inuit. There was an inverse relationship to crowdedness possibly as a confounding factor because of getting a higher prevalence among the non-Inuit who usually live in less crowded houses. Current allergic rhinitis prevalence is 8.6%, 60% of the cases were among the mixed Inuit/Caucasian ethnicity while no cases among the non-Inuit, there was a female predominance 3:2. Past history of allergic rhinitis prevalence is 10.3%, half of the cases were among the mixed ethnicity (5.2% of that ethnicity) followed by Inuit (3.4%) and non-Inuit (1.7%), female: Male ratio 1:1. Current eczema prevalence was 27.6%, with half of the cases among the mixed ethnicity (13.8% of that group), followed by Inuit (8.6%). There was a female predominance with protective effect of exclusive breastfeeding. Past eczema prevalence 34.5%, with half of the cases were among the mixed ethnicity (17.2% of that group), followed by Inuit (10.3%). There was a female predominance. We noted a high rate of sensitization to Cat at 29.2%, most of the cases were among the mixed ethnicity, while absent sensitization to other common inhalant allergens.ConclusionWhile being cautious about firm conclusions due to the small sample size and power, the noticed variations in the prevalence and risk factors of asthma, allergic rhinitis and eczema among different ethnicities living at the same subarctic environment might be related to several possible explanations like genetic, gene-environment interaction and/or lifestyle factors, it was out of the scope of this study to determine the causality of such variation in prevalence, which emphasizes the need for further investigation.

Testing Part of a DSGE Model by Indirect Inference

AbstractWe propose a way of testing a subset of equations of a DSGE model. The test draws on statistical inference for limited information models and the use of indirect inference to test DSGE models. Using the numerical small sample distribution of our test for two subsets of equations of the Smets–Wouters model we show that the test has accurate size and good power in small samples, and better power than using asymptotic distribution theory. In a test of the Smets–Wouters model on US Great Moderation data, we reject the specification of the wage‐price but not the expenditure sector. This points to the wage‐price sector as the source of overall model rejection.

Proposed Distance-Based Test for Testing Multivariate Multiple Regression Coefficients under Restricted Alternatives

In constructing estimation and hypothesis testing procedures, it is important that all available information such as sign of parameter is used in order to maximize power of the test. Often prior information are known about the sign of regression coefficients (parameter) under test, the best example being that variances cannot be negative. Ignoring information about the signs of regression parameters can lead to loss of power in small samples. With this problem in mind, this paper concerned with developing restricted estimation and hypothesis testing approach in the context of multivariate multiple regression model. Developing the technique of estimating constraint regression coefficients and testing restricted parameters with the aid of information theoretic distance are the main contribution of this paper. The distribution of the existing two-sided test follows central chi-square distribution whereas the test statistic of our proposed distance-based one-sided test follows weighted mixture of chi-square distribution. Monte Carlo simulation indicates that our newly proposed test performs better than existing tests.

High dimensional efficiency with applications to change point tests

This paper rigourously introduces the asymptotic concept of high dimensional efficiency which quantifies the detection power of different statistics in high dimensional multivariate settings. It allows for comparisons of different high dimensional methods with different null asymptotics and even different asymptotic behavior such as extremal-type asymptotics. The concept will be used to understand the power behavior of different test statistics as the performance will greatly depend on the assumptions made, such as sparseness or denseness of the signal. The effect of misspecification of the covariance on the power of the tests is also investigated, because in many high dimensional situations estimation of the full dependency (covariance) between the multivariate observations in the panel is often either computationally or even theoretically infeasible. The theoretic quantification by the theory is accompanied by simulation results which confirm the theoretic (asymptotic) findings for surprisingly small samples. The development of this concept was motivated by, but is by no means limited to, high-dimensional change point tests. It is shown that the concept of high dimensional efficiency is indeed suitable to describe small sample power.

Comparing different data descriptors in Indirect Inference tests on DSGE models

Indirect inference testing can be carried out with a variety of auxiliary models. Asymptotically these different models make no difference. However, in small samples power can differ. We explore small sample power with three different auxiliary models: a VAR, average Impulse Response Functions and Moments. The latter corresponds to the Simulated Moments Method. We find that in a small macro model there is no difference in power. But in a large complex macro model the power with Moments rises more slowly with increasing misspecification than with the other two which remain similar.

A Nonparametric Test for Granger Causality in Distribution With Application to Financial Contagion

This article introduces a kernel-based nonparametric inferential procedure to test for Granger causality in distribution. This test is a multivariate extension of the kernel-based Granger causality test in tail event. The main advantage of this test is its ability to examine a large number of lags, with higher-order lags discounted. In addition, our test is highly flexible because it can be used to identify Granger causality in specific regions on the distribution supports, such as the center or tails. We prove that the test converges asymptotically to a standard Gaussian distribution under the null hypothesis and thus is free of parameter estimation uncertainty. Monte Carlo simulations illustrate the excellent small sample size and power properties of the test. This new test is applied to a set of European stock markets to analyze spillovers during the recent European crisis and to distinguish contagion from interdependence effects.

Empirical Tests of Asset Pricing Models with Individual Assets: Resolving the Errors-in-Variables Bias in Risk Premium Estimation

To attenuate an inherent errors-in-variables bias, portfolios are widely employed for risk premium estimation; but portfolios might diversify away and thus mask relevant risk- or return-related features of individual assets. We propose a resolution that allows the use of individual assets while avoiding the bias. It hinges on specific instrumental variables, factor sensitivities (β’s) calculated from alternate observations. Closed-form asymptotics are provided for large cross-sections and time-series. Simulations indicate that the IV method delivers unbiased risk premium estimates and well-specified tests with adequate power in small samples. Empirical implementation finds some evidence of significant risk premiums for the size and book-to-market. However, when controlling for non-β characteristics, estimated risk premiums are insignificant for the CAPM, size, book-to-market, investments, profitability, and the liquidity-adjusted CAPM.

Simple Granger Causality Tests for Mixed Frequency Data

This paper presents simple Granger causality tests applicable to any mixed frequency sampling data setting, which feature remarkable power properties even with a relatively small sample size. Our tests are based on a seemingly overlooked, but simple, dimension reduction technique for regression models. If the number of parameters of interest is large then in small or even large samples any of the trilogy test statistics may not be well approximated by their asymptotic distribution. A bootstrap method can be employed to improve empirical test size, but this generally results in a loss of power. A shrinkage estimator can be employed, including Lasso, Adaptive Lasso, or Ridge Regression, but these are valid only under a sparsity assumption which does not apply to Granger causality tests. The procedure, which is of general interest when testing potentially large sets of parameter restrictions, involves multiple parsimonious regression models where each model regresses a low frequency variable onto only one individual lag or lead of a high frequency series, where that lag or lead slope parameter is necessarily zero under the null hypothesis of non-causality. Our test is then based on a max test statistic that selects the largest squared estimator among all parsimonious regression models. Parsimony ensures sharper estimates and therefore improved power in small samples. We show via Monte Carlo simulations that the max test is particularly powerful for causality with a large time lag.

Asymptotic efficiency and small sample power of a locally most powerful linear rank test for the log-logistic distribution

Assume that two independent random samples are distributed according to a log-logistic distribution (LLD). In this study, the score functions for the locally most powerful rank test were derived for the location and scale parameters. The Wilcoxon rank-sum test was shown to be locally most powerful rank test for the LLD. The asymptotic efficiency of the Wilcoxon rank-sum test was derived and compared with that of the modified Wilcoxon rank-sum test for the LLD.

When the test of mediation is more powerful than the test of the total effect.

Although previous research has studied power in mediation models, the extent to which the inclusion of a mediator will increase power has not been investigated. To address this deficit, in a first study we compared the analytical power values of the mediated effect and the total effect in a single-mediator model, to identify the situations in which the inclusion of one mediator increased statistical power. The results from this first study indicated that including a mediator increased statistical power in small samples with large coefficients and in large samples with small coefficients, and when coefficients were nonzero and equal across models. Next, we identified conditions under which power was greater for the test of the total mediated effect than for the test of the total effect in the parallel two-mediator model. These results indicated that including two mediators increased power in small samples with large coefficients and in large samples with small coefficients, the same pattern of results that had been found in the first study. Finally, we assessed the analytical power for a sequential (three-path) two-mediator model and compared the power to detect the three-path mediated effect to the power to detect both the test of the total effect and the test of the mediated effect for the single-mediator model. The results indicated that the three-path mediated effect had more power than the mediated effect from the single-mediator model and the test of the total effect. Practical implications of these results for researchers are then discussed.

Adaptive permutation tests for serial independence

Permutation tests for serial independence using three different statistics based on empirical distributions are proposed. These tests are shown to be consistent under the alternative of m‐dependence and are all simple to perform in practice. A small simulation study demonstrates that the proposed tests have good power in small samples. The tests are then applied to Canadian gross domestic product (GDP data), corroborating the random‐walk hypothesis of GDP growth.

A NOVEL ROBUST AND EFFICIENT TOOL FOR DETECTING HETEROSCEDASTICITY

One of the greatest values of Quantile Regression ( QR) is that it provides a good procedure in the sense that QR could be much more efficient and sometimes arbitrarily more efficient in recovering the mean function than the Least Squares (LS) even when without moment conditions. However, heteroscedasticity definitely causes conditional va riances of parametric or nonparametric estimates of mean functions to be large, sometimes this may lead to a great loss of efficiency of estimators and affect the goodness-of-fit test substantially and p ratically conditional variance of data is of more concerned in statistical analysis these days, thus detecting heteroscedasticity before further analysi s becomes essential. The virtue of QR as well as the limitation of LS motivates us to develop a new robust detecting tool for heteroscedasticity. Main contributions of this study include three aspects: First of all, a new Dynamic Quantile Regression (DQ R) is introduced. Based on this method estimators for mean function, heteroscedastic function and the error distribution can be obtained simultaneously. Second, a novel diagnostic tool is developed for ch ecking heteroscedasticity by employing the hybrid of QR and DQR. Theoretical properties of the proced ure are investigated and we also demonstrate the performance of the new tool on small sample power properties. Third, further estimator of the conditional variance can be obtained based on impro ved DQR, when heteroscedasticity is detected. Finally these methods are illustrated with some sim ulated examples. Compared with the classical testing procedures, Monte Carlo simulations indicat e that the new tool is more effective, powerful and easy to implement. Applications to a real data anal ysis is also discussed.