Increasing dependability of caregiver implementation fidelity estimates in early intervention: A generalizability and decision study.
There is an increasing need to measure caregiver implementation of strategies from Naturalistic Developmental Behavioral Interventions (NDBIs) as a possible key mediator of outcomes in a child. The NDBI Fidelity (NDBI-Fi) rating scheme is a macro-code to estimate the implementation of core strategies. Yet, there is a need to understand the dependability of this measure to ensure intervention study findings are generalizable to everyday interactions and comparable across studies. We addressed this by evaluating the dependability or consistency of NDBI-Fi scores for 20 caregivers, averaged across observations of two occasions of two routines that were each scored by two raters. Our findings indicated that a single score (i.e. from a single occasion, single routine, and single rater) from the measure has low dependability (g = 0.43). When scores were averaged across two observations each of two routines scored by two raters (i.e. eight scores total), the score was more dependable (g = 0.77). The majority (81.6%) of absolute error variance was attributable to occasions of observation and its interaction with other facets (routine or rater). Therefore, we recommend the NDBI-Fi be applied to more than one observation of more than one routine to strengthen confidence that scores are generalizable to everyday parent-child interactions.Lay abstractOutcomes from caregiver-mediated interventions typically include measuring the caregiver's use of key techniques. The Naturalistic Developmental Behavioral Intervention-Fidelity (NDBI-Fi) tool is a valid measurement strategy for estimating caregiver use. In this study, we sought to understand how to improve data collection from natural observations of caregivers with their children to ensure the scores are representative of how the caregiver and child typically interact. We observed 20 caregiver-child pairs via telehealth in snack and play routines over two different days. Each video was rated using the NDBI-Fi by two observers. We learned that increasing the number of observations may be the best way to improve the dependability of scores from natural caregiver-child observations. This study adds to recent research seeking to understand how to best measure caregiver strategy use. These findings may guide future researchers and clinicians to consider increasing the number of observations used to evaluate caregiver use of intervention techniques in research studies or clinical practice.
- 10.1044/2024_jslhr-24-00195
- Jan 2, 2025
- Journal of speech, language, and hearing research : JSLHR
84
- 10.1007/s10803-017-3268-0
- Aug 31, 2017
- Journal of Autism and Developmental Disorders
416
- 10.1037/bul0000215
- Jan 1, 2020
- Psychological Bulletin
72
- 10.1044/2020_jslhr-19-00167
- May 8, 2020
- Journal of Speech, Language, and Hearing Research
2258
- 10.1542/peds.2009-0958
- Jan 1, 2010
- Pediatrics
1098
- 10.1007/s10803-015-2407-8
- Jan 1, 2015
- Journal of Autism and Developmental Disorders
71
- 10.1177/1362361320944011
- Jul 30, 2020
- Autism
309
- 10.1037/0003-066x.44.6.922
- Jan 1, 1989
- American Psychologist
32
- 10.1007/s10803-019-04342-0
- Jan 10, 2020
- Journal of autism and developmental disorders
13
- 10.1177/13623613211015003
- May 24, 2021
- Autism
- Research Article
- 10.1177/13623613251328463
- Apr 10, 2025
- Autism : the international journal of research and practice
Naturalistic developmental behavioral interventions target developmentally appropriate skills in young children with autism spectrum disorders using behavioral techniques in naturalistic interactions. Naturalistic developmental behavioral interventions demonstrate strong empirical support and frequently utilize caregiver training of intervention strategies. However, our understanding of the mechanisms of change linked to naturalistic developmental behavioral interventions remains limited. Based on retrospective, secondary data analyses from three previously conducted randomized controlled trials of caregiver-mediated naturalistic developmental behavioral interventions, this study examined the direct effect of intervention on caregiver naturalistic developmental behavioral intervention strategy use and how caregiver changes mediate intervention effects on children's social communication. A total of 419 videos from 229 dyads consisting of autistic children (M = 32.5 months/ages 1-5 years) and caregivers were included in analyses using the masked ratings of the measure of naturalistic developmental behavioral intervention strategy implementation-caregiver changes (MONSI-CC) and brief observation of social communication change (BOSCC-SC). Using longitudinal mediation analyses, we examined the mediation effect of caregiver behavioral changes on children's outcomes. Results demonstrated a significant intervention effect on changes of caregiver naturalistic developmental behavioral intervention strategy use (MONSI-CC) and of changeds of caregiver strategy use (MONSI-CC) on child social communication change (BOSCC-SC) scores, but no overall total effect of intervention on changes of child social communication (BOSCC-SC). This study demonstrates the mediating role of caregiver behavior on the child intervention response, a first step toward better understanding underlying mechanisms in naturalistic developmental behavioral interventions.Lay AbstractNaturalistic developmental behavioral interventions use behavioral techniques in naturalistic interactions and settings. Naturalistic developmental behavioral interventions also teach caregivers to use strategies to enhance social communication in their children with autism spectrum disorder. Naturalistic developmental behavioral interventions are effective in supporting young autistic children but how naturalistic developmental behavioral interventions work ("underlying mechanisms") is not known yet. Therefore, by applying new instruments to videos already collected from the three previously conducted randomized controlled trials of caregiver-mediated naturalistic developmental behavioral interventions, we examined how caregivers change in their strategy use over the course of intervention and how that relates to changes in social communication in their autistic children. A total of 419 videos from 229 dyads consisting of autistic children (M = 32.5 months/ages 1-5 years) and caregivers were included in analyses using two measures, one for the caregiver behavioral changes (the measure of naturalistic developmental behavioral intervention strategy implementation-caregiver changes [MONSI-CC]), and another for the children's social communication (the brief observation of social communication change [BOSCC-SC]). We found that naturalistic developmental behavioral interventions were effective in leading to increased mastery of caregiver naturalistic developmental behavioral intervention strategy use, which then affected the changes in child's social communication skills. However, we found no direct effect of treatment for child social communication skills. This study highlights the mediating role of caregiver behavior on the child intervention response, supporting the importance of involving caregivers as partners in the intervention for young autistic children.
- Research Article
41
- 10.1136/bmj-2023-076733
- Nov 14, 2023
- BMJ
ObjectiveTo summarize the breadth and quality of evidence supporting commonly recommended early childhood autism interventions and their estimated effects on developmental outcomes.DesignUpdated systematic review and meta-analysis (autism intervention meta-analysis; Project...
- Supplementary Content
1
- 10.1136/bmj-2023-076773
- Jan 12, 2024
- BMJ
ObjectiveTo summarize the breadth and quality of evidence supporting commonly recommended early childhood autism interventions and their estimated effects on developmental outcomes.DesignUpdated systematic review and meta-analysis (autism intervention meta-analysis; Project...
- Research Article
71
- 10.1177/1362361320944011
- Jul 30, 2020
- Autism
Naturalistic developmental behavioral interventions for young children with autism spectrum disorder share key elements. However, the extent of similarity between programs within this class of evidence-based interventions is unknown. There is also currently no tool that can be used to measure the implementation of their common elements. This article presents a multi-stage process which began with defining all intervention elements of naturalistic developmental behavioral interventions. Next, intervention experts identified the common elements of naturalistic developmental behavioral interventions using a survey. An observational rating scheme of those common elements, the eight-item NDBI-Fi, was developed. We evaluated the quality of the NDBI-Fi using videos from completed trials of caregiver-implemented naturalistic developmental behavioral interventions. Results showed that the NDBI-Fi measure has promise; it was sensitive to change, related to other similar measures, and demonstrated adequate agreement between raters. This unique measure has the potential to advance intervention science in autism spectrum disorder by providing a tool to measure the implementation of common elements across naturalistic developmental behavioral intervention models. Given that naturalistic developmental behavioral interventions have numerous shared strategies, this may ease clinicians' uncertainty about choosing the "right" intervention package. It also suggests that there may not be a need for extensive training in more than one naturalistic developmental behavioral intervention. Future research should determine whether these common elements are part of other treatment approaches to better understand the quality of services children and families receive as part of usual care.
- Research Article
24
- 10.1177/13623613221121427
- Sep 2, 2022
- Autism
Naturalistic Developmental Behavioral Interventions (NDBIs) are a group of early interventions that use a variety of strategies from applied behavioral and developmental sciences. Although Naturalistic Developmental Behavioral Interventions have been demonstrated effective, Naturalistic Developmental Behavioral Interventions are not implemented on a wide scale within early intervention programs for children on the autism spectrum. Potential reasons likely stem from differing theoretical orientations of developmental and behavioral sciences and practitioners' lack training, knowledge, and support for implementing Naturalistic Developmental Behavioral Interventions. In support of efforts to promote wide-scale implementation of Naturalistic Developmental Behavioral Interventions, we (1) clarify their common features, (2) discuss possible misconceptions, and (3) offer reasons why Naturalistic Developmental Behavioral Interventions should be widely implemented. We also provide recommendations to the autism service community, intervention developers, and researchers.
- Research Article
48
- 10.1111/dmcn.15258
- May 18, 2022
- Developmental medicine and child neurology
To identify which interventions are supported by evidence and the quality of that evidence in very young children with or at high likelihood for autism spectrum disorder (ASD) to improve child outcomes. We conducted an overview of reviews to synthesize early intervention literature for very young children with or at high likelihood for ASD. Cochrane guidance on how to perform overviews of reviews was followed. Comprehensive searches of databases were conducted for systematic reviews and meta-analyses between January 2009 and December 2020. Review data were extracted and summarized and methodological quality was assessed. Primary randomized controlled trial evidence was summarized and risk of bias assessed. This overview of reviews was not registered. From 762 records, 78 full texts were reviewed and seven systematic reviews and meta-analyses with 63 unique studies were identified. Several interventional approaches (naturalistic developmental behavioral intervention, and developmental and behavioral interventions) improved child developmental outcomes. Heterogeneity in design, intervention and control group, dose, delivery agent, and measurement approach was noted. Inconsistent methodological quality and potential biases were identified. While many early interventional approaches have an impact on child outcomes, study heterogeneity and quality had an impact on our ability to draw firm conclusions regarding which treatments are most effective. Advances in trial methodology and design, and increasing attention to mitigating measurement bias, will advance the quality of the ASD early intervention evidence base. Naturalistic developmental behavioral interventions, as well as developmental and behavioral interventions, improve child outcomes in autism spectrum disorder (ASD). If only randomized controlled trials are considered, guidelines for early intensive behavioral intervention in younger children should be revisited. The greatest intervention impacts were on proximal, intervention-specific outcomes. Inadequacies in the quality of the early ASD intervention evidence base were observed.
- Book Chapter
2
- 10.1016/bs.irrdd.2020.09.003
- Jan 1, 2020
Augmented naturalistic developmental behavioral intervention for toddlers with autism spectrum disorder: A community pilot study
- Research Article
2
- 10.1007/s10803-023-06203-3
- Dec 11, 2023
- Journal of autism and developmental disorders
An expanding evidence base has advocated for delivery of naturalistic developmental behavioral interventions (NDBIs) within community systems, thus extending the reach of these practices to young autistic children. The current study examined provider-reported use of NBDIs within a Part C Early Intervention (EI) system and the extent to which provider background, attitudes, and perceived organizational support predicted NDBI use. Results from 100 EI providers representing multiple disciplines indicated reported use of NDBI strategies within their practice despite inconsistent reported competency with manualized NDBI programs. Although NDBI strategy use was not predicted by provider experiences or perceived organizational support, provider openness to new interventions predicted the reported use of NDBI strategies. Future directions include mixed methods data collection across and within EI systems to better understand NDBI use and ultimately facilitate NDBI implementation.
- Research Article
5
- 10.1044/2022_ajslp-22-00190
- Jan 11, 2023
- American Journal of Speech-Language Pathology
Naturalistic developmental behavioral interventions (NDBIs) have demonstrated initial promise in facilitating social communication development for autistic toddlers, but their highly structured protocols may be a barrier toward their use by early intervention (EI) providers who must individualize intervention according to family-centered principles. This study aimed to characterize the extent to which EI speech-language pathologists (SLPs) use NDBI strategies, and the range of skills and behaviors addressed during their EI sessions, to contextualize the role of NDBIs within the scope of needs of families with autistic children in EI. This observational study included 25 families with an autistic toddler and their EI SLP. One home-based session was recorded for each family, and an observational measure was used to describe SLPs' NDBI strategy use. Qualitative content analyses were also used to characterize the strategies SLPs recommended to families, and the child skills and behaviors they discussed. SLPs did not implement NDBI strategies with high quality, but they implemented developmental NDBI strategies with significantly higher quality than behavioral NDBI strategies. SLPs discussed many strategies and skills across disciplines within the session. SLPs may require further training to implement NDBI strategies, but given the breadth and depth of skills addressed during sessions, researchers should investigate and report on the impact of NDBIs on a wider range of communication skills and developmental domains. This will facilitate clinical decision making and make these interventions better aligned with family-centered EI principles. https://doi.org/10.23641/asha.21834480.
- Research Article
1
- 10.1177/13623613241273081
- Oct 9, 2024
- Autism : the international journal of research and practice
Many families seek access to evidence-based therapy to support their child's learning. Naturalistic developmental behavioral intervention is a set of practices that use a child's natural motivation and interest to teach skills in everyday routines. Many families find naturalistic developmental behavioral interventions appealing and they have been proven to be effective. However, families may not be able to enroll in naturalistic developmental behavioral intervention-based programs due to the limited availability of trained service providers. Telehealth is the use of technology to engage with care providers, including doctors and therapists. Telehealth is an effective tool for improving access to services, especially for people in rural and underserved communities. Telehealth offers a way for providers to connect with families but it does not address the low numbers of trained providers. In this article, we share a statewide model developed to increase access to naturalistic developmental behavioral interventions for families while increasing training opportunities for early intervention providers. Through this model, expert consultants worked with caregivers and providers via telehealth for a brief series of visits. During these visits, consultants taught caregivers and providers strategies based in naturalistic developmental behavioral interventions. Feedback from caregivers and providers, along with improvement in child skills, show that this model was effective and acceptable.
- Supplementary Content
- 10.7759/cureus.92055
- Sep 1, 2025
- Cureus
Autism Spectrum Disorder (ASD) affects multiple developmental domains, and early intervention is critical for optimizing long-term outcomes. Early intensive behavioral and developmental interventions (EIBIs/EDIs), including Applied Behavior Analysis (ABA), the Early Start Denver Model (ESDM), and Pivotal Response Training (PRT), have emerged as key evidence-based strategies. This narrative review aims to evaluate the effectiveness of these interventions on cognitive, language, adaptive, and social outcomes in children under seven years of age, drawing from both primary studies and systematic reviews.Findings from meta-analyses and primary studies indicate that EIBIs and naturalistic developmental behavioral interventions (NDBIs) are associated with significant improvements in IQ (gains of 9-15 points) and language development. However, effects on core autism symptoms are more variable. Parent-mediated and lower-intensity intervention models show promise in maintaining effectiveness while improving accessibility. Despite these encouraging outcomes, variability in study designs, outcome measures, and individual responses highlights the need for more personalized approaches.The key gaps include limited long-term follow-up, inconsistent symptom reduction, and disparities in access. Future research should prioritize precision medicine frameworks, adaptive intervention models, and culturally responsive implementation strategies to enhance equity and maximize developmental potential for children with ASD.
- Research Article
25
- 10.1007/s10803-021-04882-4
- Jan 23, 2021
- Journal of Autism and Developmental Disorders
Naturalistic developmental behavioral interventions (NDBIs) are evidence-based interventions for young children with autism spectrum disorder. There has been growing interest in implementing manualized NDBIs within the early intervention (EI) system without a clear understanding of how these programs and the broader strategies encompassed within them are already used by EI providers. This study examined the use of manualized NDBI programs and broader NDBI strategies within an EI system and factors that impacted their use. Eighty-eight EI providers completed a measure of NDBI program and strategy use. Thirty-three providers participated in a supplemental focus group or interview. Overall, providers described using broader NDBI strategies and the need to adapt manualized NDBI programs. Provider-, intervention-, and organization-level factors impacted their use of NDBI programs and strategies.
- Research Article
6
- 10.1007/s10803-024-06382-7
- Jun 7, 2024
- Journal of Autism and Developmental Disorders
Both naturalistic developmental behavioral interventions (NDBIs) and augmentative and alternative communication (AAC) have been shown to support the language development of children with a diagnosis of autism spectrum disorder and minimal speech. However, little research has addressed the impact of incorporating AAC systems within NDBIs. This systematic review was conducted to assess the relative impact of NDBI procedures with and without AAC on the language development of children on the autism spectrum with minimal speech. Relevant studies were located through systematic database searching, targeted review of relevant journals, and ancestral search of references from identified and associated papers. Relevant study characteristics were coded for all included studies, as well as determining certainty of evidence and calculating effect sizes for language variables. All procedures followed the systematic review guidelines set by the Cochrane Collaboration. A total of 29 relevant studies were included within this review, covering both single-case and group design research. Three studies were identified that directly compared NDBI and AAC interventions. NDBIs had a strong impact on language across study types (i.e., with and without AAC), though both aggregate and comparative effect sizes were notably larger when AAC was included within NDBI procedures, as compared to NDBIs without AAC. Results suggest that combining AAC with NDBI procedures may lead to better language outcomes than NDBIs alone for children on the autism spectrum with minimal speech.
- Research Article
2
- 10.1177/13623613241227516
- Feb 6, 2024
- Autism : the international journal of research and practice
Naturalistic Developmental Behavioral Interventions have been described as culturally responsive and family-friendly interventions, with research demonstrating improvements in children's development following the receipt of these interventions. Given the child-directed nature of Naturalistic Developmental Behavioral Interventions and the intervention's integration within families' daily routines, many studies have examined the impact of Naturalistic Developmental Behavioral Interventions on family and family member quality of life. We conducted a systematic review and meta-analysis to explore the relationship between Naturalistic Developmental Behavioral Interventions and family quality of life. Results suggest that the provision of a Naturalistic Developmental Behavioral Intervention neither improved nor worsened family or family member quality of life. For those involved in delivering Naturalistic Developmental Behavioral Intervention services, there is an immediate need to convey to families that children's improvements will likely not translate into improvements in family quality of life.
- Research Article
- 10.1186/s12887-025-05587-8
- Apr 1, 2025
- BMC Pediatrics
BackgroundParticipation in Naturalistic Developmental Behavioral Interventions (NDBI) is associated with significant improvements in functioning for toddlers with, and showing early signs of, autism spectrum disorder. The Part C Early Intervention (EI) system, which is publicly funded and available in all U.S. states, offers an optimal infrastructure through which toddlers can receive NDBIs. This study seeks to assess the effectiveness and fit of one NDBI, Caregiver Implemented Reciprocal Imitation Teaching (CI-RIT), within the Part C EI system.MethodsThis hybrid type 1 effectiveness/implementation trial uses a multi-site randomized control design to simultaneously test effectiveness and collect implementation data on CI-RIT in the Part C EI system across four states: Illinois, Massachusetts, Michigan and Washington. Participants include EI providers (target n = 160) who are randomized to either CI-RIT or treatment as usual (TAU), and child/caregiver dyads on their caseloads (target n = 440). Primary effectiveness outcomes focus on (1) child social communication, joint attention, motor imitation; and (2) caregiver responsivity, implementation fidelity of RIT, and self-efficacy, which are all measured at baseline and then 4-months and 9-months after baseline. Implementation outcomes include CI-RIT modifications, treatment acceptability, fidelity of CI-RIT coaching, and RIT session completion.DiscussionThis study represents a critical effort to transport an evidence-based NDBI, CI-RIT, into a national service delivery setting, the Part C EI system. The large, multi-site nature of the trial provides the opportunity to address critical questions about training and intervention effectiveness, which will, in turn, optimize and support CI-RIT implementation at scale.Trial registrationThe trial protocol is registered at ClinicalTrials.gov (NCT05114538; Registration date: 10/28/2021).
- Research Article
- 10.1177/13623613251365979
- Nov 1, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251379920
- Oct 30, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251384436
- Oct 30, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251379945
- Oct 30, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251386983
- Oct 30, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251383343
- Oct 28, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251385029
- Oct 28, 2025
- Autism : the international journal of research and practice
- Research Article
- 10.1177/13623613251380340
- Oct 24, 2025
- Autism : the international journal of research and practice
- Front Matter
- 10.1177/13623613251389295
- Oct 24, 2025
- Autism : the international journal of research and practice
- Discussion
- 10.1177/13623613251383347
- Oct 22, 2025
- Autism : the international journal of research and practice
- Ask R Discovery
- Chat PDF
AI summaries and top papers from 250M+ research sources.