Sample Size Estimation Methods Research Articles

A comparison of methods for sample size estimation for non-inferiority studies with binary outcomes

Non-inferiority trials are motivated in the context of clinical research where a proven active treatment exists and placebo-controlled trials are no longer acceptable for ethical reasons. Instead, active-controlled trials are conducted where a treatment is compared to an established treatment with the objective of demonstrating that it is non-inferior to this treatment. We review and compare the methodologies for calculating sample sizes and suggest appropriate methods to use. We demonstrate how the simplest method of using the anticipated response is predominantly consistent with simulations. In the context of trials with binary outcomes with expected high proportions of positive responses, we show how the sample size is quite sensitive to assumptions about the control response. We recommend when designing such a study that sensitivity analyses be performed with respect to the underlying assumptions and that the Bayesian methods described in this article be adopted to assess sample size.

Power and sample size estimation in microarray studies

BackgroundBefore conducting a microarray experiment, one important issue that needs to be determined is the number of arrays required in order to have adequate power to identify differentially expressed genes. This paper discusses some crucial issues in the problem formulation, parameter specifications, and approaches that are commonly proposed for sample size estimation in microarray experiments. Common methods for sample size estimation are formulated as the minimum sample size necessary to achieve a specified sensitivity (proportion of detected truly differentially expressed genes) on average at a specified false discovery rate (FDR) level and specified expected proportion (π1) of the true differentially expression genes in the array. Unfortunately, the probability of detecting the specified sensitivity in such a formulation can be low. We formulate the sample size problem as the number of arrays needed to achieve a specified sensitivity with 95% probability at the specified significance level. A permutation method using a small pilot dataset to estimate sample size is proposed. This method accounts for correlation and effect size heterogeneity among genes.ResultsA sample size estimate based on the common formulation, to achieve the desired sensitivity on average, can be calculated using a univariate method without taking the correlation among genes into consideration. This formulation of sample size problem is inadequate because the probability of detecting the specified sensitivity can be lower than 50%. On the other hand, the needed sample size calculated by the proposed permutation method will ensure detecting at least the desired sensitivity with 95% probability. The method is shown to perform well for a real example dataset using a small pilot dataset with 4-6 samples per group.ConclusionsWe recommend that the sample size problem should be formulated to detect a specified proportion of differentially expressed genes with 95% probability. This formulation ensures finding the desired proportion of true positives with high probability. The proposed permutation method takes the correlation structure and effect size heterogeneity into consideration and works well using only a small pilot dataset.

A mixture model approach to sample size estimation in two-sample comparative microarray experiments

BackgroundChoosing the appropriate sample size is an important step in the design of a microarray experiment, and recently methods have been proposed that estimate sample sizes for control of the False Discovery Rate (FDR). Many of these methods require knowledge of the distribution of effect sizes among the differentially expressed genes. If this distribution can be determined then accurate sample size requirements can be calculated.ResultsWe present a mixture model approach to estimating the distribution of effect sizes in data from two-sample comparative studies. Specifically, we present a novel, closed form, algorithm for estimating the noncentrality parameters in the test statistic distributions of differentially expressed genes. We then show how our model can be used to estimate sample sizes that control the FDR together with other statistical measures like average power or the false nondiscovery rate. Method performance is evaluated through a comparison with existing methods for sample size estimation, and is found to be very good.ConclusionA novel method for estimating the appropriate sample size for a two-sample comparative microarray study is presented. The method is shown to perform very well when compared to existing methods.

Power Approximation for the Van Elteren Test Based on Location-Scale Family of Distributions

The van Elteren test, as a type of stratified Wilcoxon-Mann-Whitney test for comparing two treatments accounting for stratum effects, has been used to replace the analysis of variance when the normality assumption was seriously violated. The sample size estimation methods for the van Elteren test have been proposed and evaluated previously. However, in designing an active-comparator trial where a sample of responses from the new treatment is available but the patient response data to the comparator are limited to summary statistics, the existing methods are either inapplicable or poorly behaved. In this paper we develop a new method for active-comparator trials assuming the responses from both treatments are from the same location-scale family. Theories and simulations have shown that the new method performs well when the location-scale assumption holds and works reasonably when the assumption does not hold. Thus, the new method is preferred when computing sample sizes for the van Elteren test in active-comparator trials.

Sample size estimation for the van Elteren test—a stratified Wilcoxon–Mann–Whitney test

The van Elteren test is a type of stratified Wilcoxon-Mann-Whitney test for comparing two treatments accounting for strata. In this paper, we study sample size estimation methods for the asymptotic version of the van Elteren test, assuming that the stratum fractions (ratios of each stratum size to the total sample size) and the treatment fractions (ratios of each treatment size to the stratum size) are known in the study design. In particular, we develop three large-sample sample size estimation methods and present a real data example to illustrate the necessary information in the study design phase in order to apply the methods. Simulation studies are conducted to compare the performance of the methods and recommendations are made for method choice. Finally, sample size estimation for the van Elteren test when the stratum fractions are unknown is also discussed.

Sample‐size calculations for studies with correlated ordinal outcomes

Correlated ordinal response data often arise in public health studies. Sample-size (power) calculations are a crucial step in designing such studies to ensure an adequate sample to detect a significant effect. Here we extend Rochon's method of sample-size estimation with a repeated binary response to the ordinal case. The proposed sample-size calculations are based on an analysis with generalized estimating equations (GEE) and inference with the Wald test. Simulation results demonstrate the merit of the proposed power calculations. Analysis of an arthritis clinical trial is used for illustration.

How big does my sample need to be? A primer on the murky world of sample size estimation

Background An explicit justification of sample size is now mandatory for most proposals submitted to funding bodies, ethics committees and, increasingly, for articles submitted for publication in journals. However, the process of sample size estimation is often confusing. Aim Here, we present a primer of sample size estimation in an attempt to demystify the process. Method First, we present a discussion of the parameters involved in power analysis and sample size estimation. These include the smallest worthwhile effect to be detected, the Types I and II error rates, and the variability in the outcome measure. Secondly, through a simplified, example ‘dialogue’, we illustrate the decision-making process involved in assigning appropriate parameter values to arrive at a ballpark figure for required sample size. We adopt a hypothetical, parallel-group, randomized trial design, though the general principles and concepts are transferable to other designs. The illustration is based on a traditional, power-analytic, null hypothesis-testing framework. In brief, we also address sample size estimation methods based on the required precision of the mean effect estimate. Conclusion Rigorous sample size planning is important. Researchers should be honest and explicit regarding the decisions made for each of the parameters involved in sample size estimation.

Continuous Covariates in Genetic Association Studies of Case-Parent Triads: Gene and Gene-Environment Interaction Effects, Population Stratification, and Power Analysis

We propose a multinomial logistic regression method which permits estimation and likelihood ratio tests for allele effects, their interactions with continuous covariates, and assessment of the degree of population stratification in genetic association studies of case-parent triads. Our approach overcomes the constraint imposed by the categorical nature of explanatory variables in the log-linear model. We also demonstrate that the multinomial logistic method can yield efficient inference in the presence of missing parental genotype data via the use of the Expectation-Maximization (EM) algorithm. We performed simulations to compare the multinomial logistic model with the case-pseudosibling conditional logistic model approach, both of which permit the incorporation of continuous covariates. Simulation results indicate that the multinomial logistic model and the conditional logistic model lead to similar estimates in large samples. A simulation-based method of sample size estimation is also used to show that the two models are approximately equivalent in sample size requirements. When parental genotype data are missing, either completely at random or dependent on covariates, the use of the EM algorithm gives multinomial logistic model greater power. Since the multinomial logistic model offers the possibility of assessing the degree of population stratification in the sample and can also provide efficient inference in the presence of missing parental genotypes, the proposed model has an important application in epidemiological family-based association studies.

The use of bootstrap methods for estimating sample size and analysing health‐related quality of life outcomes

Health-related quality of life (HRQoL) measures are increasingly used in trials as primary outcome measures. Investigators are now asking statisticians for advice on how to plan and analyse studies using such outcomes. HRQoL outcomes, like the SF-36, are usual measured on an ordinal scale, although most investigators assume that there exists an underlying continuous latent variable and that the actual measured outcomes (the ordered categories) reflect contiguous intervals along this continuum. The ordinal scaling of HRQoL measures means they tend to generate data that have discrete, bounded and skewed distributions. Thus, standard methods of analysis that assume Normality and constant variance may not be appropriate. For this reason, conventional statistical advice would suggest non-parametric methods be used to analyse HRQoL data. The bootstrap is one such computer intensive non-parametric method for estimating sample sizes and analysing data. We describe three methods of estimating sample sizes for two-group cross-sectional comparisons of HRQoL outcomes. We then compared the power of the three methods for a two-group cross-sectional study design using bootstrap simulation. The results showed that under the location shift alternative hypothesis, conventional methods of sample size estimation performed well, particularly Whitehead's method. Whitehead's method is recommended if the HRQoL outcome has a limited number of discrete values (<7) and/or the expected proportion of cases at either of the bounds is high. If a pilot data set is readily available then bootstrap simulation will provide a more accurate and reliable estimate, than conventional methods.Finally, we used the bootstrap for hypothesis testing and the estimation of standard errors and confidence intervals for parameters, in an example data set. We then compared and contrasted the bootstrap with standard methods of analysing HRQoL outcomes. In the data set studied, with the SF-36 outcome, the use of the bootstrap for estimating sample sizes and analysing HRQoL data produces results similar to conventional statistical methods. These results suggest that bootstrap methods are not more appropriate for analysing HRQoL outcome data than standard methods.

Power analyses for longitudinal trials and other clustered designs.

Existing methods for power and sample size estimation for longitudinal and other clustered study designs have limited applications. In this paper, we review and extend existing approaches to improve these limitations. In particular, we focus on power analysis for the two most popular approaches for clustered data analysis, the generalized estimating equations and the linear mixed-effects models. By basing the derivation of the power function on the asymptotic distribution of the model estimates, the proposed approach provides estimates of power that are consistent with the methods of inference for data analysis. The proposed methodology is illustrated with numerous examples that are motivated by real study designs.

Sample size estimation in research with dependent measures and dichotomous outcomes.

I reviewed sample estimation methods for research designs involving nonindependent data and a dichotomous response variable to examine the importance of proper sample size estimation and the need to align methods of sample size estimation with planned methods of statistical analysis. Examples and references to published literature are provided in this article. When the method of sample size estimation is not in concert with the method of planned analysis, poor estimates may result. The effects of multiple measures over time also need to be considered. Proper sample size estimation is often overlooked. Alignment of the sample size estimation method with the planned analysis method, especially in studies involving nonindependent data, will produce appropriate estimates.

Design and analysis of group-randomized trials: a review of recent practices.

We reviewed group-randomized trials (GRTs) published in the American Journal of Public Health and Preventive Medicine from 1998 through 2002 and estimated the proportion of GRTs that employ appropriate methods for design and analysis. Of 60 articles, 9 (15.0%) reported evidence of using appropriate methods for sample size estimation. Of 59 articles in the analytic review, 27 (45.8%) reported at least 1 inappropriate analysis and 12 (20.3%) reported only inappropriate analyses. Nineteen (32.2%) reported analyses at an individual or subgroup level, ignoring group, or included group as a fixed effect. Hence increased vigilance is needed to ensure that appropriate methods for GRTs are employed and that results based on inappropriate methods are not published.

Sample size and power estimation for studies with health related quality of life outcomes: a comparison of four methods using the SF-36

We describe and compare four different methods for estimating sample size and power, when the primary outcome of the study is a Health Related Quality of Life (HRQoL) measure. These methods are: 1. assuming a Normal distribution and comparing two means; 2. using a non-parametric method; 3. Whitehead's method based on the proportional odds model; 4. the bootstrap. We illustrate the various methods, using data from the SF-36. For simplicity this paper deals with studies designed to compare the effectiveness (or superiority) of a new treatment compared to a standard treatment at a single point in time. The results show that if the HRQoL outcome has a limited number of discrete values (< 7) and/or the expected proportion of cases at the boundaries is high (scoring 0 or 100), then we would recommend using Whitehead's method (Method 3). Alternatively, if the HRQoL outcome has a large number of distinct values and the proportion at the boundaries is low, then we would recommend using Method 1. If a pilot or historical dataset is readily available (to estimate the shape of the distribution) then bootstrap simulation (Method 4) based on this data will provide a more accurate and reliable sample size estimate than conventional methods (Methods 1, 2, or 3). In the absence of a reliable pilot set, bootstrapping is not appropriate and conventional methods of sample size estimation or simulation will need to be used. Fortunately, with the increasing use of HRQoL outcomes in research, historical datasets are becoming more readily available. Strictly speaking, our results and conclusions only apply to the SF-36 outcome measure. Further empirical work is required to see whether these results hold true for other HRQoL outcomes. However, the SF-36 has many features in common with other HRQoL outcomes: multi-dimensional, ordinal or discrete response categories with upper and lower bounds, and skewed distributions, so therefore, we believe these results and conclusions using the SF-36 will be appropriate for other HRQoL measures.

Statistical methods in epidemiology. II: A commonsense approach to sample size estimation

Purpose: It has been argued, by many, that mathematical formulae for estimating sample size are unnecessarily complex, so much so that researchers may be reluctant to seek statistical advice.Method: This paper reviews methods of sample size estimation arguing that two formulae (one based on comparison of proportion of 'successes', the other based on comparison of means of normally distributed data) suffice for many situations. This paper argues the case by taking examples drawn mainly from clinical trials research. However, the methods outlined can also be used in epidemiology specifically in both case-control and cohort studies with no loss of information.Results: For the situations outlined, worked examples are provided.Conclusions: Sample size estimation need not necessarily be a complex process. Simple techniques exist which enable the clinician and the statistician to work together. Continued dialogue between both parties is required so that good ideas do not go to waste.

How many subjects? Statistical power analysis in research

It is most satisfying to find a statistical book that is informative and mathematically rigorous, yet very pleasant reading in a manageable format. How Many Subjects?, by Helena Chmura Kraemer and Sue Thiemann, is just such a little gem. This stylishly slim volume (120 pages, of which 20 are chapter dividers or blanks) presents the authors' method of sample size estimation for a wide variety of statistical procedures, along with much useful instruction and many insights into general experimental design. Research investigators are increasingly being called upon to justify the number of observations required for their work (cf. the NIH Guide for the Care and Use of LaboratoryAnimals, Appendix D). Inaddition to obviating unnecessary distress to subjects, careful planning of experiments promotes economy and plain good science. Of the many sources of information on statistically appropriate sample size, many are task-specific, being limited to a single or small number of statistical procedures. Some are exhaustive and exhausting because of the mass and diversity of material. A regrettable few-unfortunately attractive to the beginner because of their simplicity-handle only Type I (0:) error and ignore the very important power side (Type II or I (3) of the sample size issue. Kraemer and Thiemann's book begins with a brief but lucid overview of the concepts of statistical hypothesis testing, critical effect size, and power analysis in the overall context of research study planning. A very elegant feature of the book is the authors' innovation of a single method of power calculation that is appropriate for a wide variety of different tests. Not only is this aesthetically