Conducting sensitivity analyses to identify and buffer power vulnerabilities in studies examining substance use over time

Abstract

IntroductionA priori power analysis is increasingly being recognized as a useful tool for designing efficient research studies that improve the probability of robust and publishable results. However, power analyses for many empirical designs in the addiction sciences require consideration of numerous parameters. Identifying appropriate parameter estimates is challenging due to multiple sources of uncertainty, which can limit power analyses' utility. MethodWe demonstrate a sensitivity analysis approach for systematically investigating the impact of various model parameters on power. We illustrate this approach using three design aspects of importance for substance use researchers conducting longitudinal studies base rates, individual differences (i.e., random slopes), and correlated predictors (e.g., co-use) and examine how sensitivity analyses can illuminate strategies for controlling power vulnerabilities in such parameters. ResultsEven large numbers of participants and/or repeated assessments can be insufficient to observe associations when substance use base rates are too low or too high. Large individual differences can adversely affect power, even with increased assessments. Collinear predictors are rarely detrimental unless the correlation is high. ConclusionsIncreasing participants is usually more effective at buffering power than increasing assessments. Research designs can often enhance power by assessing participants twice as frequently as substance use occurs. Heterogeneity should be carefully estimated or empirically controlled, whereas collinearity infrequently impacts power significantly. Sensitivity analyses can identify regions of model parameter spaces that are vulnerable to bad guesses or sampling variability. These insights can be used to design robust studies that make optimal use of limited resources.