Abstract
Poor response to treatment is a defining characteristic of reading disorder. In the present systematic review and meta-analysis, we found that the overall average effect size for treatment efficacy was modest, with a mean standardized difference of 0.38. Small true effects, combined with the difficulty to recruit large samples, seriously challenge researchers planning to test treatment efficacy in dyslexia and potentially in other learning disorders. Nonetheless, most published studies claim effectiveness, generally based on liberal use of multiple testing. This inflates the risk that most statistically significant results are associated with overestimated effect sizes. To enhance power, we propose the strategic use of repeated measurements with mixed-effects modelling. This novel approach would enable us to estimate both individual parameters and population-level effects more reliably. We suggest assessing a reading outcome not once, but three times, at pre-treatment and three times at post-treatment. Such design would require only modest additional efforts compared to current practices. Based on this, we performed ad hoc a priori design analyses via simulation studies. Results showed that using the novel design may allow one to reach adequate power even with low sample sizes of 30–40 participants (i.e., 15–20 participants per group) for a typical effect size of d = 0.38. Nonetheless, more conservative assumptions are warranted for various reasons, including a high risk of publication bias in the extant literature. Our considerations can be extended to intervention studies of other types of neurodevelopmental disorders.
Highlights
The issue of low power when assessing treatment efficacy in dyslexiaPoor response to treatment is a defining feature of specific learning disorders (American Psychiatric Association [APA], 2013)
We aimed to show how the design analysis for treatment efficacy in dyslexia is affected using a more simplified inferential procedure based on the Bayes factor (BF)
Testing small samples is often unavoidable in studies on neurodevelopmental disorders, including randomized controlled trials assessing treatment efficacy in learning disorders
Summary
The issue of low power when assessing treatment efficacy in dyslexiaPoor response to treatment is a defining feature of specific learning disorders (American Psychiatric Association [APA], 2013). The estimated effect size (i.e., treatment-control standardized difference at post-test, corrected for the pre-test scores) for the most frequently used treatment approach as well as the only one for which a statistically significant meta-analytic estimate emerged, phonics instruction, was Hedge’s g = 0.32. This effect size can be considered relatively small (Cohen, 1988). Large effect sizes may be due to noisy estimates (e.g., large standard error due to small sample size, unreliable reading measures) or inappropriate diagnostic procedures (e.g., children’s low reading performance being only transient and not due to dyslexia)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
