Statistical power estimation in non-invasive brain stimulation studies and its clinical implications: An exploratory study of the meta-analyses.

Abstract

Non-invasive brain stimulation (NIBS) techniques have emerged as a promising tool for understanding and treating psychiatric disorders, necessitating a caution in terms of interpreting research results. This study aimed at systematically evaluating a representative sample of research conducted using NIBS interventions in neuro-psychiatric conditions, and assessing the power these studies achieved, given their sample sizes. A database search was conducted with defined keyword combinations. Using reported summary effects of the meta-analyses as estimate of the true effects, we calculated achieved power of each individual study to detect the effect indicated by the corresponding meta-analysis. Findings suggest that mean and median powers in the field of NIBS were 0.50, with a mode at 0.83 (range 0.05-1.00). When analysed separately, the median powers were 0.27 for tDCS, 0.70 for TMS and 0.97 for ECT. These studies had a mean total sample size of 22.2 ± 24.9 subjects and the median reported effect size across all studies was 0.61. According to our findings, studies conducted in NIBS miss around 50% of true positive results. Further, it appears that most of the researchers in this field chase statistical significance with small sample sizes, thus compromising the quality of their conclusions.