Social Validity Measures Research Articles

Comparing Two Data Collection Procedures to Teach Academic Targets within a Group Instructional Format

ABSTRACT This study aimed to extend previous literature comparing continuous methods of data collection to estimation data, but this time implementing the data collection procedures within a group discrete trial teaching format with three individuals diagnosed with autism spectrum disorder. Group discrete trial teaching was conducted in a classroom setting using relevant academic targets for each child. The data collection procedures were examined in terms of accuracy of data collection, efficiency of teaching (i.e. number of trials delivered per session), and rate of child acquisition of targets. Results of the adapted alternating treatment design replicated across sets and multiple targets indicated that estimation data was found to be a valid data collection method in identifying mastery, with only two instances where the mastery probe did not confirm mastery. Trial-by-trial data indicated mastery four times in which a probe for mastery did not confirm mastery of targets. Additionally, it was found that instructors conducted more teaching trials within the estimation data condition across all learners and training sets and social validity measures indicated overall favorable responses toward the estimation data collection procedure from both instructors.

Assessing physiological arousal and emotional valence during behavioral intervention for pediatric feeding difficulties: A pilot study

AbstractThis study's purpose is to introduce a new measurement system that objectively assesses the social validity of behavioral feeding intervention from the child's perspective via the concurrent measurement of two dimensions of emotion. To date, the primary measures of social validity for behavioral feeding intervention are caregiver treatment acceptability and satisfaction surveys. This is the first study to objectively measure two dimensions of emotion, physiological arousal and emotional valence, while children received behavioral intervention for feeding difficulties. Data collectors used a new open‐source data collection software, cometrics, developed to synchronize and record physiological and observational data. Physiological data was collected using a wearable biosensor and observers recorded an index of child happiness, unhappiness, or neither using definitions by Phipps et al. (2022) for six children with a pediatric feeding disorder. Two out of six children needed programmed habituation to the device before data collection. All children assented to wearing the device during data collection. Recorded indices mapped to separable physiological states using electrodermal activity and its constituent components, skin conductance responses and skin conductance level, in addition to skin temperature. Findings demonstrated the feasibility of measuring two dimensions of child emotions during behavioral feeding intervention and revealed that children's physiological responses were distinctly different during intervals scored as happy, unhappy, or neither. This new data collection system has implications for future research on the child's emotional experience of behavioral feeding treatment and increases the possibilities for improving clinical practice.

Facilitators and barriers to the implementation of the picture exchange communication system (PECS): a systematic review

Background and aimsAlthough there are several published systematic reviews and meta-analyses of the effectiveness of the Picture Exchange Communication System (PECS) protocol, no previous review has focused on factors affecting its implementation. The present review attempted to identify potential barriers and facilitators in implementing PECS, as well as examining authors’ and stakeholders’ views and experiences in terms of implementation.Methods and proceduresA total of 49 studies were identified through a systematic review search process and were included in the data extraction phase. A Content Analysis of previous researchers’ comments was also carried out.Outcome and resultsDuring the data extraction and synthesis phase, it became evident that important features of PECS interventions are not systematically reported in published literature. Thus, our primary research question remained partially unanswered. A Content Analysis of previous researchers’ comments revealed four potential implementation factors, including Contextual & Environmental Factors, PECS Training/Consultation, Supervision and Implementation Fidelity and Adherence to the Protocol.Conclusions and implicationsThe identification of potential barriers or facilitators affecting the implementation of PECS is not currently possible. The present review’s identified implementation factors were extracted from qualitative analysis of previous researchers’ anecdotal remarks or from the social validity measures. Implications in terms of previous PECS literature findings are discussed.Systematic review registration: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018099767.

The Effects of an Instructional Package on Reading Skills for Multilingual Learners With Extensive Support Needs

In the current study, investigators examined the effects of an instructional reading package consisting of shared reading and systematic instruction (e.g., task analysis, system of least prompts, visual supports, and native language support) in a small group setting on percentage of correct consonant-vowel-consonant (CVC) words for three elementary participants with extensive support needs who also were multilingual learners. A multiple probe design across word sets with replication across participants resulted in a functional relation between the percentage of correct CVC words acquired and the instructional package. In addition, there were positive changes in exploratory (secondary) dependent variables that included components of reading (i.e., phonemic awareness, fluency, vocabulary, comprehension, and generalization) as demonstrated through pre-posttest assessments. Social validity measures for parents, students, and paraprofessionals also revealed positive pre-post changes. Limitations and future research are discussed.

Using the disequilibrium theory in behavior change projects on homework and social media usage

AbstractThe disequilibrium theory suggests restricting a behavior below baseline levels will induce response deficit and make that behavior a more impactful reinforcer. This reinforcement principle was incorporated into a behavior change project for eight students, where the instrumental behavior was homework, and the contingent behavior was social media (SM) access. Students self‐selected their level of SM access deficit and completed both a baseline and treatment phase during the first 8 weeks of an undergraduate learning and behavior course. Most students increased daily homework rates during treatment relative to baseline, although the average increase was not statistically significant. Daily SM rates were significantly decreased during treatment relative to baseline, which was evidence of response deficit. Students rated the behavior change project high on most social validity measures. These results indicate that behavior change projects based on the disequilibrium theory are a viable way to induce changes in socially significant behaviors.

Using Video Modeling to Teach Neurotypical Adolescents to Interact Socially with Peers with ASD.

Research has shown video modeling to be effective for training adult service providers to administer evidence-based practices to children with autism. This study examined the effects of video modeling training (VMT) on neurotypical adolescents' performance of peer mediated social interaction (PMSI), a 10-step procedure of simplified behavioral practices, during roleplay with an adult actor. A multiple probe design across participants evaluated the effects of VMT on delivery of PMSI by five neurotypical adolescents. All participants demonstrated immediate increases and generalized delivery of PMSI to four adolescents with autism following VMT. Social interaction for two additional youths with autism also improved when evaluated within a peer mediated setting, as a measure of social validity, before and after VMT.

A career education program for early childhood youth: Development and initial evaluation

AbstractFostering occupational knowledge during early childhood could help build a foundation for more sophisticated future knowledge of the world of work. Thus, we developed and initially evaluated the effectiveness of a career education program designed to promote occupational knowledge of kindergarten children. Participants were 133 Italian kindergarten children with a mean age of 4.62 years (SD = 0.47) who were randomly assigned to an experimental or control group. Results indicated that, compared to the control group, the experimental group experienced a significantly greater overall increase in occupational knowledge after the intervention. Results of social validity measures indicated that parents and teachers of the experimental group perceived the intervention very favorably in terms of its goals, acceptability and usability of the procedures, and outcomes. The study results provide initial support for the potential of the career education program to promote understanding of the world of work and occupations in early childhood.

Incorporating social validity into practice: Treatment progression across pediatric feeding skill domains

AbstractA small but growing body of research in pediatric feeding disorders asserts the importance of comprehensively measuring social significance of goals, procedures, and effects of intervention, and incorporating social validity into practice to inform treatment. This report sought to extend this literature by detailing procedures to measure and improve social validity during a clinical case of a 3.5‐year‐old during a home‐based intensive feeding program. A multiple baseline design demonstrated effectiveness of nonremoval and re‐presentation added to a treatment package. Repeated choice via direct child preference assessments informed demand fading and gradual progression across six feeding skill domains (medication, cup drinking, independence, texture, volume, variety) and arrangements of response effort (preference, skill) with layers of reinforcer parameters (quality, magnitude, rate, immediacy). Indices of happiness definitions were modified, and extinction bursts examined. Fostering a collaborative approach, caregivers provided detailed input on social validity measures pretreatment, at discharge, and long‐term follow‐up (6‐month, 1‐year), inclusive of both qualitative and quantitative responses, written and verbal communication, and permanent product data. Further implications for practitioners included detailing the process for caregiver training and generalization to family meals with siblings and community settings, and providing adaptable full‐text guidelines for free access/choice contexts.

Self-monitoring and public posting improve competitive youth cyclists' training performance.

This study investigated the effects of self-monitoring and public posting on the cycling performance of competitive youth cyclists. We measured three primary dependent variables: performance volume, intensity precision, and performance-duration deviation. In addition, we evaluated self-monitoring accuracy and social validity. The participants were three males aged 14-16 years. We used an ABAB design to evaluate an intervention package that consisted primarily of self-monitoring and public posting. Athletes self-monitored their performance after training using an online summative Google Form. The coach publicly posted performance-based rankings on the social media application WhatsApp. Results indicate that the intervention package positively improved all performance measures across all athletes. Social-validity measures were also favorable.

The generalized impacts of systematic instruction strategy training on the instructional staff of a vocational training center

BACKGROUND: Vocational training centers improve outcomes for people with intellectual and developmental disabilities (IDD), but employment rates remain low. Additionally, vocational direct support professionals (DSPs) often lack the training to implement effective instruction and the administrative procedures needed for change. OBJECTIVE: This study examines the effect of behavioral skills training (BST) on the maintained procedural fidelity of evidence-based practice (EBP) instruction in direct support providers with clients who have moderate to extensive support needs (M-ESN). METHOD: Three instructor-identified EBPs for teaching individuals with IDD were selected for professional development (PD): most-to-least prompting, time delay procedures, and system of least prompts. Three single case multiple probe across behaviors experiments evaluated the efficacy of the researcher-delivered BST instruction model on trainee accuracy as assessed by trained center administrators. RESULTS: A functional relation between the BST instruction and generalized DSP implementation was indicated, despite challenges and limitations related to COVID-19. Social validity measures found the goals, procedures, and outcomes were socially significant, and the intervention was practical and effective. CONCLUSION: Practices in vocational training should reflect the high standards for systematic instruction in the K-12 setting. EPB BST training with administration progress monitoring can improve DSPs’ instruction with clients, thus improving learning outcomes for adults with M-ESN.

Evaluating the Impact of PWS Smart-Start: A Behavior Analytic Caregiver Training Program for Prader-Willi Syndrome

PWS Smart-Start is a behavioral caregiver training program developed specifically for caregivers of children with Prader-Willi syndrome (PWS) aged 3 to 14. The purpose of the current study was to evaluate the acceptability and preliminary efficacy of the program. Thirty-four caregivers of children with PWS received the PWS Smart-Start training using online live video-coaching across a 10-week period. The impact of the training on a variety of variables including caregiver and child behavior, caregiver stress and burnout, and family functioning were evaluated using a quasi-experimental pretest posttest design. Results indicated statistically significant decreases across child behavior challenges and parental stress and burnout following the training. Statistically significant improvements were also seen in parenting practices, the quality of the parent–child relationship, and family quality of life. Results of social validity measures indicated caregiver satisfaction with the program. These preliminary results indicate that the PWS Smart-Start program has the potential to not only support children with PWS through behavioral challenges but also reduce caregiver stress and burnout and improve a variety of aspects of family life. Future research is needed to replicate and further validate these results and to extend the exploration of other behavior analytic interventions for this population.

Further evaluation of a hands‐on‐shoulder plus praise procedure to reduce toe walking

AbstractIdiopathic toe walking (ITW) refers to ambulation with a toe‐to‐toe gait in the absence of a known cause. Although it is exhibited by typically developing children, it is more common among children with autism spectrum disorder (ASD). Persistent ITW can cause several long‐term medical problems, including ankle deformities and balance and posture abnormalities. In the current study, we first conducted a series of no interaction or alone conditions to verify that ITW exhibited by four children with ASD persisted in the absence of social consequences. We then evaluated a hands‐on‐shoulder plus praise procedure in which gentle pressure is applied to participants' shoulders to prompt their heels to contact the ground, along with praise for appropriate steps in order to reduce ITW. Results suggest that the procedure was effective with all participants, and measures of social validity indicate that caregivers approved of the procedure. We discuss the results in terms of the putative mechanisms responsible for the effects of the procedure as well as the conditions under which the procedure might be used.

Staying on Task: Use of Self-Monitoring and Peer Coaching to Support Inclusive Postsecondary Education

ABSTRACT Although applications (apps) for technology-based self-monitoring have received little attention in the literature for college-level students with intellectual disability (ID) in inclusive postsecondary education (IPSE) programs at institutions of higher education, novel tools regularly arrive and are applied within current support structures. I-Connect, a smartphone or web-based self-monitoring tool, has the potential to be an important instrument for promoting independence and access to inclusive academic and workplace environments. The current study used a multi-treatment A-B-A-B-C withdrawal single-case design to determine the effects of peer coaching and a smartphone-based self-monitoring app on the on-task behavior of a college student with autism and intellectual disability. Measurement included the impact of the I-Connect app and peer tutoring in combination with the app and the social validity of the intervention. Findings demonstrated a functional relation for the use of I-Connect with some limitations. The data showed promise for the addition of peer support and were confirmed by a social validity measure. The combination of technology-based self-monitoring supports with the peer supports typically available in inclusive postsecondary education (IPSE) programs can potentially increase key self-determination outcomes. The authors discuss further implications and suggestions for confirming these findings in subsequent inquiries.

Enhanced milieu teaching with phonological emphasis: A pilot, telepractice study for parent training in early intervention

Purpose This pilot study evaluated enhanced milieu teaching with phonological emphasis (EMT + PE) parent training intervention delivered through telepractice and its impact on parent implementation, child speech outcomes, and child language outcomes. Method A multiple baseline design across behaviours was used to assess response to parent training and child outcomes. The intervention was delivered to four parents and their young children with repaired cleft palate. Speech production, receptive language, and expressive language were assessed prior to intervention. EMT + PE strategies including matched turns, modelling and expansions, and prompting/speech recasting were taught to parents using the teach-model-coach-review model. Social validity questionnaires were administered pre- and post-intervention to assess parents’ rating of competence and confidence for managing their child’s early development. Reliability and procedural fidelity were conducted. Result All parents increased their use of modelling and expansions, and prompting and speech recasting in response to training, as evidenced by large effect sizes. Children also responded positively to these strategies by increasing percentage of consonants correct (PCC) and target vocabulary, although PCC gains were less than observed in a prior hybrid telepractice and face-to-face intervention. Social validity measures indicated parents increased their confidence to provide intervention for their children. Conclusion This pilot study indicated that effective parent training in EMT + PE strategies can be delivered through telepractice.

Using Behavioral Skills Training to Teach Functional Assessment Interviewing, Cultural Responsiveness, and Empathic and Compassionate Care to Students of Applied Behavior Analysis.

Within the field of applied behavior analysis, there is a recognized need for increased training for practitioners on cultural responsiveness, as well as to improve behavior analysts' demonstration of compassion and empathy towards the families with whom they work. The present study used behavioral skills training via telehealth to teach three skillsets-functional assessment interviewing, empathic and compassionate care, and cultural responsiveness. Participants were seven graduate students who had no previous coursework in behavioral assessment and whose caseload mainly included clients who did not share the participant's cultural, ethnic, or religious backgrounds. The results showed that behavioral skills training was effective in improving performance across all three skillsets. In addition, high levels of responding maintained following the completion of the training for the majority of the participants. Several levels of social validity measures support the utility and impact of this training. The findings have implications for training practitioners on these vital skills.

A comparison of fixed momentary differential reinforcement of other behavior to variable momentary differential reinforcement of other behavior to reduce challenging behavior

AbstractDifferential reinforcement of other behavior (DRO) is commonly used to reduce behavioral excesses. Momentary DRO schedules involve delivery of reinforcement contingent upon the absence of the target behavior at a given moment. Two variations of momentary DRO exist: fixed‐momentary (FM) DRO and variable‐momentary (VM) DRO. In the current study, we directly compared FM‐DRO and VM‐DRO schedules to reduce challenging behavior maintained by automatic reinforcement exhibited by four children with autism spectrum disorder. The results suggest that both the DRO schedules were equally effective to reduce challenging behavior. A social validity measure showed that most caregivers rated the VM‐DRO as a preferred schedule and noted the potential for FM‐DRO schedule to become more discriminable over time, which could reduce its effectiveness. However, most caregivers also commented that the FM‐DRO schedule was easier to implement.

Remote First-Language Assessment: Feasibility Study With Vietnamese Bilingual Children and Their Caregivers.

There is a shortage of bilingual speech-language pathologists (SLPs) in the United States. For Vietnamese, less than 1% of SLPs speak the language compared with a Vietnamese American population of > 2.1 million. This study examines the feasibility and social validity of remote child language assessment with the help of a caregiver to address the need for first language assessments among Vietnamese-speaking children. Twenty-one dyads of caregivers and typically developing children (aged 3-6 years) completed two assessment sessions in their first language, Vietnamese, using Zoom videoconferencing. Sessions were counterbalanced between two conditions in which either the clinician or the caregiver was the task administrator. Children's language samples were elicited using narrative tasks. Social validity was also assessed through caregiver and child questionnaires at the end of each session. There were no significant differences between conditions on language sample measures nor the measures of social validity. Both caregivers and their children felt positively about the sessions. The caregivers' feelings were related to their perception of children's feelings about the sessions. Children's feelings were related to their Vietnamese language proficiency, caregiver-reported language ability, and whether they were born outside of the United States. Findings build the evidence base for telepractice as an effective and socially valid service delivery model for bilingual children in the United States. This study supports the potential for caregivers as task administrators in a telepractice setting, making assessment in a child's first language more feasible and accessible. Future investigation is needed to extend results to bilingual populations with disorders.

Social and Emotional Intervention Research as Justice: A Case for Accountability

The field of special education is recognized for conducting research designed to improve the quality of life for the children and families we serve. However, our field has been criticized for empirical approaches that are inconsistent with the values and beliefs we articulate as central to our scientific practice. As such, a shift in our research endeavors is warranted. Given the specific calls within early childhood special education for research programs that consider the marginalized experiences of children and families, we posit five guidelines to reframe social and emotional intervention research in early childhood special education as justice-oriented. These guidelines are (a) understand the historical and immediate context of research participants; (b) incorporate best practices in social and emotional assessment to reduce bias; (c) center children’s and families’ voices in social validity measures; (d) address confounding variables to fidelity of implementation; and (e) examine diversity in research team composition.

An updated review of the customized employment literature

BACKGROUND: There is clear federal priority to develop effective strategies to mitigate the continued disparity in employment opportunities and outcomes for people with significant disabilities. At the same time, there are calls for rehabilitation practitioners to understand, utilize, and implement evidence-based practices (EBP) with fidelity to improve rehabilitation outcomes, such as competitive integrated employment. Customized employment (CE) emerged as a promising practice that promotes improved employment outcomes. OBJECTIVE: The purpose of this manuscript is to describe published literature on CE and to make recommendations on the best methods for establishing CE as an evidence-based practice. METHOD: We included 10 articles classified as “data-based” articles from a review of the literature on CE conducted by Riesen et al. We also conducted an electronic search of articles on CE published between 2015 and 2021. RESULTS: This review indicates that CE research is predominantly descriptive. Based on the results of the review, we categorized CE research into three types: model description and project evaluation, perceptions of CE, and essential structure and competency. The descriptive studies suggest that CE produces quality employment outcomes for people with disabilities. CONCLUSION: While research on CE has expanded, future research on CE should use more robust indicators including (a) quality descriptions of the participants, (b) quality descriptions of settings and conditions, (c) descriptions of the independent variables, (d) descriptions of fidelity to procedures, and (e) quality descriptions of social validity measures. Researchers should also focus on using correlational research to establish customized employment as an evidence-based practice.

Teaching Compassion Skills to Students of Behavior Analysis: A Preliminary Investigation.

Socially valid practices are at the heart of applied behavior analysis and can influence how interventions are experienced by families. However, the training of practitioners of applied behavior analysis is primarily focused on the implementation of technical procedures with little focus on therapeutic approaches. Empathy and therapeutic rapport have been associated with improved outcomes in allied professions (Beach et al., Journal of the American Board of Family Practice, 15(1), 25–38, 2006; Hojat et al., Academic Medicine, 86(3), 359, 2011; Horst et al., Journal of Child & Family Nursing, 3, 5–14, 2000), but have been minimally studied within the field of behavior analysis. In the present study, several sources were utilized to identify and define empathic and compassionate care skills. These skills were divided into three skill areas (i.e., basic interviewing skills, interest in the family, joining with the family) and taught to ABA master’s students using behavioral skills training via a telehealth platform. All four participants significantly improved their engagement in compassionate care skills following training and maintained these skills in follow-up probes and with a different experimenter. Several post-study measures of outcome were taken, including social validity measures from participants, ratings of compassion from consumer and professional experts, as well as comparison measures on the Jefferson Scale of Physician Empathy. Participant reports of social validity were high, as were consumer and professional ratings of compassionate behaviors. Improvements on the Jefferson Scale of Physician Empathy were also observed. Implications for training practitioners and for expanding the focus on compassionate care skill development within the field are explored.