Abstract
BackgroundThe lack of available measures that can reliably characterize early developmental skills in children with neurogenetic syndromes (NGS) poses a significant challenge for research on early development in these populations. Although syndrome-specific measures may sometimes be necessary, a more cost- and time-efficient solution would be to identify existing measures that are appropriate for use in special populations or optimize existing measures to be used in these groups. Reliability is an important metric of psychometric rigor to consider when auditing and optimizing assessment tools for NGS. In this study, we use Generalizability Theory, an extension of classical test theory, as a novel approach for more comprehensively characterizing the reliability of existing measures and making decisions about their use in the field of NGS research.MethodsWe conducted generalizability analyses on a popular early social communication screener, the Communication and Symbolic Behavior Scales—Infant-Toddler Checklist (CSBS-ITC), collected on 172 children (41 Angelman syndrome, 30 Prader-Willi syndrome, 42 Williams syndrome, 59 low-risk controls).ResultsOverall, the CSBS-ITC demonstrated at least adequate reliability in the NGS groups included in this study, particularly for the Prader-Willi and Williams syndrome groups. However, the sources of systematic error variance in the CSBS-ITC varied greatly between the low-risk control and NGS groups. Moreover, as unassessed in previous research, the CSBS-ITC demonstrated substantial differences in variance sources among the NGS groups. Reliability of CSBS-ITC scores was highest when averaging across all measurement points for a given child and was generally similar or better in the NGS groups compared to the low-risk control group.ConclusionsOur findings suggest that the CSBS-ITC communicates different information about the reliability of stability versus change, in low-risk control and NGS samples, respectively, and that psychometric approaches like Generalizability Theory can provide more complete information about the reliability of existing measures and inform decisions about how measures are used in research on early development in NGS.
Highlights
The lack of available measures that can reliably characterize early developmental skills in children with neurogenetic syndromes (NGS) poses a significant challenge for research on early development in these populations
While we expect that most children with a NGS will demonstrate atypical social communication development, the three NGS groups included in this study have unique phenotypes that will allow us to examine the reliability of the CSBS-ITC across a range of phenotypic profiles that vary in the severity and variability of intellectual disability and language skills
Age accounted for much lower proportions of variance in the NGS groups compared to the Low-risk control (LRC) group, and while age still accounted for the highest proportion of variance within the Prader-Willi syndrome (PWS) (27%) and Williams syndrome (WS) (28%) groups, it accounted for very little variance in the Angelman syndrome (AS) group (2%)
Summary
The lack of available measures that can reliably characterize early developmental skills in children with neurogenetic syndromes (NGS) poses a significant challenge for research on early development in these populations. Measures with low reliability are more likely to either miss effects that are truly present (i.e., type II errors) or suggest the presence of statistically significant effects that are due to error (i.e., type I errors). These risks are hazardous in NGS research, which often deals with very small, highly heterogeneous samples, and whose findings are often given substantial weight in determining the effectiveness of treatment protocols, especially in clinical trials [1]. Can a measure be used reliably in its current form, does it require optimization for use in NGS, or is wholesale replacement by an instrument designed to measure developmental skills in NGS justified? This critical evaluation of the reliability of existing measures in NGS is one aspect of this comprehensive evaluation that will provide better understanding of the strengths and limitations of common tools in the field
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.