Social Validity Assessment Research Articles

Teaching students with autism to solve mathematical story problems: A replication and extension

AbstractSolving mathematical story problems has proven to be a challenge for primary school students with and without developmental disabilities. The present study replicates a behavior analytic study (Neef et al.) by teaching three autistic Chinese students in inclusive education classes to solve addition/subtraction story problems by acquiring an overt precurrent behavior chain. The intervention was effective for all participants by employing a multiple baseline design across behaviors. Additionally, the study extends Neef et al. by conducting maintenance, generalization, and social validity assessments, and expands the teaching format to distance learning (telehealth). The results and limitations are discussed based on Skinner's conceptualization of problem‐solving and its clinical application in teaching complex mathematical skills to students with developmental disabilities.

Cultural and Linguistic Adaptation of Parent-Implemented Language Interventions for At-Risk Children: A Systematic Review

ABSTRACT Children from low socioeconomic backgrounds are at an increased risk for delayed language and literacy development. In recognizing the role parents play in children’s language and literacy development, organizations throughout the world are identifying and implementing parent-focused interventions early in children’s lives. Impacting the success of parent-implemented language interventions, however, is the fit with recipient families’ culture. Although cultural and linguistic adaptation is gaining more emphasis in the education field, there is little focus on cultural and linguistic adaptation of parent-implemented language interventions. Twenty-four studies were evaluated for quality and effectiveness and were examined to identify how researchers adapted interventions culturally and linguistically. We further evaluated how researchers addressed social validity in these interventions. Interventions were of medium to high quality and were reported to effectively enhance children’s language and literacy outcomes. Overall, the studies supported the sociolinguistic pride of participants’ culture by using community metaphors and concepts and supported the use of participants’ home language. Twelve studies included an assessment of social validity, primarily in the aspects of intervention procedures and content, and almost exclusively at the end of the intervention. This review emphasizes the importance of adapting parent-implemented language interventions with cultural and linguistic practices of local populations to enhance children’s language outcomes.

The effectiveness of an executive function program for individuals diagnosed with autism spectrum disorder

AbstractAutistics/individuals with autism spectrum disorder (ASD) commonly display deficits in executive functioning. These deficits negatively impact social success and the autistic's/individual with ASD's ability to form meaningful friendships. Yet, there is limited research provided for the effective teaching of executive functioning skills to autistics/individuals with ASD. The current study provides an evidence‐based behavior analytic approach for developing executive functioning skills for autistics/individuals diagnosed with ASD. The study also provides the preliminary steps to creating a curriculum that is easy to implement and uses minimal resources. Finally, the study extends current literature on the Teaching Interaction Procedure (TIP) and evaluates the effectiveness of the TIP for developing executive functioning skills for older children diagnosed with ASD. The executive function program resulted in acquisition of all six targeted executive functioning skills for all the participants. While generalization and maintenance data were slightly variable across participants, all three participants displayed improvement in executive functioning skills following the implementation of the executive function program and a social validity assessment indicated that the participants were satisfied with the skills taught, the treatment, and the outcomes.

Improving undergraduate students' email etiquette with computer-based instruction.

Email etiquette is an important skill, especially in professional settings. Research demonstrates that undergraduate students follow email rules when given written instructions plus an example. The purpose of the present study was to evaluate the effects of computer-based email instruction on undergraduate students' email etiquette and to assess the social validity of that instruction. The email instruction package was delivered through an online learning management system and consisted of written instructions plus a model, a quiz, and quiz feedback. We measured email etiquette by scoring emails from course assignments using a checklist. The instruction package produced a replicated increase in mean email checklist scores across two sections of introductory psychology, and checklist scores were elevated when participants sent emails in a novel context. The results of the social-validity assessments suggest that the goals were important, the procedures were acceptable, and there were meaningful improvements in email etiquette for some participants.

Assessment of Adverse Events, Side Effects, and Social Validity in Evidence-Based Behavioral Interventions for Autistic Students

Research has demonstrated positive impacts of behavioral interventions on various educational outcomes for autistic youth, and implementation of these interventions in education settings has been widely advocated. However, recent studies have identified methodological shortcomings in the behavioral intervention evidence base, including lack of reporting on side effects and social validity. This review including 98 studies identified as evidence-based practices by the National Clearinghouse on Autism Evidence and Practice further highlights the lack of evaluation of side effects and social validity in behavioral intervention research. Suggestions are given regarding assessment of side effects, embedding social validity into intervention, and practical takeaways for educators. Future research and practice should prioritize addressing potential side effects and advancing ethical implementation of evidence-based behavioral interventions.

Using general‐case procedures outside of autism intervention: A systematic review

AbstractRecently, there has been a call for the application of behavior analytic technology to more diverse domains. Although general‐case procedures are most often applied in autism service delivery, they can be applied to more diverse domains ranging from organizational behavior management to public health initiatives. This literature review, therefore, provides an overview of technological descriptions of general‐case procedures and their applications in areas other than autism intervention. Evaluated variables include the population, independent and dependent variables, generalization measures, and social validity assessments. The results suggest that most researchers are not using a technological description of general‐case procedures nor are they assessing for generalization. Recommendations for future researchers and clinicians include better descriptions of general‐case procedures, evaluation with different populations, assessment of acceptability and more comprehensive evaluations of generalization.

Using the Preschool Life Skills Program to Support Skill Development for Children with Trauma Histories.

The Preschool Life Skills (PLS) program has a wealth of evidence demonstrating efficacy in remediating important social and learning-to-learn skill deficits in at-risk preschoolers. Those same skill deficits also are common in older children in foster or residential care, most of whom have experienced some sort of trauma or other adverse childhood events. This study sought to evaluate individualized PLS curricula for two boys with substantial trauma histories and demonstrate how the PLS program could be delivered within a trauma-informed framework. We delivered the program and evaluated skill acquisition in a one-to-one setting in a UK-based clinic, and asked caregivers to assess skills at home. Results showed that both boys acquired skills targeted in their individual curriculum, but maintenance was sometimes inconsistent. Social validity assessments suggested that both boys enjoyed the training but were less definitive about its overall benefits. Caregivers rated the program highly and reported skill improvements at home. We discuss the implications of these findings in terms of adapting the PLS program to children with trauma histories.

Considering Social Validity in Special Education Research

This article is part of a special LDRP research–to–practice series introducing key concepts to enable special education practitioners and other nonresearchers to be more informed research consumers. In the article, we explore how social validity is assessed in special education research and how to interpret social validity assessments. Rather than focusing on measuring intervention effects, social validity involves assessing the social importance of the goals, procedures, and outcomes of interventions and programs. We define social validity, provide questions to consider when examining assessments of social validity in research papers, review approaches commonly used to assess social validity with examples from the research literature, and make recommendations for reconciling findings of positive intervention effects on targeted outcomes but absent or negative findings related to social validity in a study. Our take–home message is that considering social validity assessments helps research consumers interpret study findings and informs how to apply findings in practice.

A Systematic Review of Social Validation Procedures in Intervention Research With Transition-Age Autistic Youth

In this secondary analysis of a previously conducted systematic review, we analyze social validity assessments in intervention research for transition-age autistic youth. Social validity is concerned with the acceptability of the intervention goals, the acceptability and feasibility of the intervention procedures, and the perceived importance of the intervention outcomes. We found that although just over half of intervention studies assessed some aspect of social validity, only 43% of those studies examined all three dimensions. There were several shortcomings of the social validation procedures, including a lack of psychometric validation for quantitatively scored questionnaires and a failure to describe qualitative procedures for analyzing open-ended questions. These shortcomings likely explain why interpretations of social validity assessments were nearly universally positive.

Implementation of Check-In/Check-Out to Improve Classroom Behavior of At-Risk Elementary School Students.

The multi-tiered school-wide positive behavioral interventions and supports offers a comprehensive model for the prevention of behavioral and academic problems in schools. This study evaluated Check-in/Check-out (CICO), a Tier 2 intervention, with three elementary school students from a high-need population, whose problem behavior was hypothesized to be maintained by teacher attention. The study employed a concurrent multiple baseline design across participants, a single case experimental design to examine the effects of CICO on student academic engagement and problem behavior during instruction. Results indicated that implementation of CICO with fidelity by the CICO coordinator, classroom teachers, and parents lead to increased academic engagement and reduced problem behavior in all three students. Tau-U Effect sizes were medium to large for academic engagement and small to medium for problem behavior across students. Data on two students indicate that systematically fading the number of times teachers utilize the daily report card has the potential for promoting maintenance effects. For one student, fading of the coordinator was successful. Social validity assessment indicated high levels of satisfaction with the CICO intervention by the participating students, teachers, and one parent.

Grandparent-Implemented Interventions to Reduce Challenging Behavior of an Adult With Autism: A Pilot Telehealth Case Study

Adults with autism spectrum disorder (ASD) often engage in challenging behavior that requires individualized intervention to remediate. However, access to high-quality services is limited for adults, leaving this population without necessary behavioral healthcare. This study addressed the needs of a 22-year-old man with ASD and intellectual disability (ID) who lived with his grandmother and engaged in tangibly maintained challenging behavior. The interventionist coached his grandmother via telehealth in the implementation of two behavior analytic interventions: signaled multiple schedule arrangement and a modified contingency contract. The effects of the interventions were evaluated using an alternating treatments design with baseline. While both interventions were effective at decreasing challenging behavior when compared to baseline, the signaled multiple schedule arrangement resulted in zero instances of challenging behavior. Both the participant and the grandparent completed post-intervention social validity assessments and reported overall positive satisfaction with the interventions. Practice recommendations are presented.

Multiple schedules with response interruption and redirection to reduce stereotypy in children with autism spectrum disorder: Generalization and social validity

Although response interruption and redirection (RIRD) and multiple schedule arrangements have sometimes been shown to be effective in reducing stereotypy for individuals with autism spectrum disorder (ASD), important questions remain about optimal methods for facilitating generalization and social validity. In the current study, we sought to extend the literature on stereotypy treatment and MSAs in several ways. First, we programmed for generalization by conducting sessions with two exemplars of two different categories of activities (i.e., school work, toy play) and assessing generalization with novel exemplars and activities and during the participant's typical instructional program. Second, we used RIRD instructions that were contextual models of appropriate behavior relevant to each activity. Third, during the free access component of the multiple schedule, we used a child's play tent as a first step toward teaching participants to engage in stereotypy during private free time. For all three participants with ASD, stereotypy decreased quickly and substantially, with reductions generalizing to novel activities and contexts. An extensive social validity assessment with board-certified behavior analysts indicated high social acceptability of the goals, procedures, and outcomes.

Social validity of telepractice in early intervention: Effectiveness of family‐centered practices

AbstractObjectiveThe present study evaluates the impact of family‐centered practices on families' ratings of social validity of telepractice in 659 Spanish families in early childhood intervention (ECI) services.BackgroundThe integration of telepractice in ECI services has been accelerated by the social and health situation caused by the COVID‐19 pandemic. Until then, telepractice was a well‐known but scarcely used resource in early intervention. Social validity assessments are critical for telepractice because services are focused on supporting primary caregivers. Social validity, therefore, refers to whether the intervention approach and outcomes match the needs of children and families, allowing for improved planning, implementation, and evaluation of early intervention programs.MethodA survey‐based cross‐sectional study on families' ratings of social validity was carried out. A structural equations model was performed in order to assess the influence of families' ratings of the social validity of telepractice. The model analyzed a mediation through latent variables: family‐centered practices (X), fit of services to family needs (M), and social validity of telepractice (Y).ResultsOur study shows an overall good perception of social validity of telepractice by families. Focusing on family needs and priorities and professionals encouraging families to have an active role completely mediates the relationship between family‐centered practices and family social validity scores. No differences in telepractice modalities were found.Conclusions and ImplicationsA family‐centered approach to telepractice in ECI is needed, not only for greater social validity, but also for improved child and family outcomes, especially when services support real family needs and priorities.

Effects of Feedback Timing on Accuracy of Functional Analysis Implementation

ABSTRACT The current study evaluated the effects of the temporal placement of positive and corrective feedback on the accuracy of implementing idiosyncratic functional analysis (FA) procedures with three adult learners. The three feedback conditions included 1) delivering all feedback immediately after performance, 2) all feedback immediately before the next performance, and 3) corrective feedback before performance/positive feedback after performance. The conditions were counterbalanced across three idiosyncratic FA procedures via an adapted alternating treatment design embedded in a multiple baseline design across participants. For one out of three participants, corrective feedback before/positive feedback after and all feedback before resulted in greater increases in accuracy than all feedback after. For the remaining two participants, all three feedback conditions resulted in accuracy increasing to the mastery criterion. All participants selected all feedback before as their preferred condition in a social validity assessment; practical implications and directions for future research are discussed.

Social Validity in Behavioral Research: A Selective Review.

Through the application of behavioral principles, behavior analysts seek to produce socially meaningful behavior change, defined as alterations in behavior that yield important outcomes immediately beneficial for the direct consumers of interventions and key stakeholders. Behavioral practitioners and researchers often engage in assessment and reporting of the meaningfulness of behavior change using social validity assessments. These assessments ensure that target behaviors are appropriately selected, intervention procedures are acceptable, and satisfactory outcomes are produced. The purpose of this review is to identify the current state of social validity within behavioral literature. We reviewed eight peer-reviewed journals between 2010 and 2020. We found that 47% of the intervention studies reviewed included a social validity assessment. Social validity assessment across journals has increased over time, with a significant rise from 2019 to 2020. Implications of these findings and suggestions for future work are discussed.

The fundamental role of social validity in behavioral consultation in school settings

AbstractSocial validity refers to the extent to which applied behavior analytic (ABA) treatment goals, methods, and/or outcomes are acceptable and meaningful to stakeholders. The aim of this conceptual paper is to suggest how social validity may be promoted during behavioral consultation in school systems and how the assessment of various aspects of social validity contributes to this throughout the consultation process. Literature pertaining to the concept of social validity, rapport‐building, collaboration with stakeholders in behavioral school consultation, and the assessment of social validity are reviewed and synthesized. Social validity may be promoted in several domains: communication with consultees, collaborative selection of treatment methods and student objectives, consultee training, and measurement of outcomes meaningful to stakeholders. Social validity assessment can occur in all of these domains and is ideally a continuous process that informs all phases of consultation. There is a particular need for direct measurement of the social significance of behavioral consultation services. Social validity is integral to all aspects of the behavioral school consultation process, and it may be facilitated by incorporating valid social validity assessments into every stage of the consultation process.

Augmented Reality in Physics Education: Students with Intellectual Disabilities Inquire the Structure of Matter

Abstract Immersive technologies support educational activities and provide motivating contexts which are increasingly implemented in special education settings. Augmented Reality (AR) seems to improve the level of engagement in teaching and learning processes for all students, including students with Intellectual Disabilities (ID). However, there is a lack of research that investigates AR learning environments where students with ID can be involved in inquiry-based activities and acquire academic content linked to real situations. The purpose of this study was to implement a single-subject design and evaluate the effects of an AR system on students’ performance on the microscopic level of the structure of matter and especially the phase-states of water. A functional relationship was found between students’ correct responses during probe sessions and the AR inquiry-based intervention. In addition, a social validity assessment indicated that the AR glasses helped students with ID to acquire physics concepts, as well as inquiry skills in a vivid experience. The students also reported satisfaction from using the AR glasses. Suggestions for future research include the design of AR-based interventions for other science concepts for students with ID as well as other special educational needs.

Social-Emotional and Behavioral Support for First- and Second-Grade Black Learners at Risk for Emotional and Behavioral Problems

There is a need to provide evidence-based social, emotional, and behavioral interventions for learners at risk for emotional and behavioral problems. Evidence-based interventions for social and emotional success are especially necessary for Black learners with emotional or behavioral problems. In the current study, we used a multiple-baseline design time-lagged across groups (i.e., classrooms) to examine the effects of a culturally adapted version of the Strong Start curriculum plus Check-In/Check-Out on externalizing problem behaviors of 18 first-grade and second-grade Black learners. Results indicate a functional relation between the intervention package and externalizing problem behaviors. Additionally, post-intervention social validity assessment indicates that participating teachers viewed the intervention package as feasible and participating learners reported a positive experience.

Parent-Implemented Interventions via Telepractice in Autism Research: A Review of Social Validity Assessments.

Parent-implemented autism interventions are considered empirically validated interventions and the use of telepractice in these interventions is reported as effective. However, little is known about the social validity assessments and outcomes of these interventions. The purpose of this review is to explore the current practices of conducting social validity assessment and reporting its outcomes within parent-implemented telepractice autism interventions. The 11 reviewed studies included caregivers as participants, telepractice intervention focusing on social communication outcomes of young autistic children, and were published in a peer-reviewed journal within the past five years. The researchers in the reviewed articles reported positive outcomes for parent-implemented telepractice autism interventions. Notably, however, information about social validity assessments of the interventions was limited. Researchers are encouraged to evaluate the social validity of interventions using multiple data sources and methods, and report on their findings as they relate to other types of data.

Evaluating the Acceptability and Social Validity of a Caregiver-Led Technology-Based Menstrual Hygiene Management Intervention for Youth on the Autism Spectrum.

ObjectivesMenstrual hygiene management (MHM) is a critical area of daily living skills for neurodiverse menstruators in which few evidence-based interventions exist. This study evaluated the acceptability and social validity of a caregiver-mediated intervention for youth on the autism spectrum utilizing behavior skills training, animated video modeling, task analysis, and telehealth services to improve changing a menstrual pad.MethodsThe consultant recruited three youth and caregiver dyads to test the intervention and collect acceptability and social validity assessments. Materials varied based on the phase of the study; however, a typical trial required data collection sheets, youth’s underwear, a menstrual pad, and, during intervention, the video model and hardware technology. A multiple baseline design across participants and reversal design were used to demonstrate experimental effects. Due to the sensitive nature of the study, independence and accuracy data were collected by caregivers and analyzed later by the consultant through visual analysis.ResultsOverall, youth participants improved their percentage of independent and accurate steps of changing a menstrual pad. Furthermore, caregivers reported moderate to high levels of acceptability of the telehealth service delivery and animated video modeling, as well as the overall MHM training program measured by the Technology Acceptability Model-Fast Form (FF-TAM) and the Behavior Intervention Rating Scale (BIRS), respectively. Youth reported high levels of acceptability of the MHM training program as measured by the Child Intervention Rating Profile (CIRP).ConclusionsResults suggest that a technology-based, caregiver-mediated MHM intervention for neurodiverse youth would be considered acceptable and socially valid by caregiver and youth participants.Supplementary InformationThe online version contains supplementary material available at 10.1007/s41252-022-00261-x.