Single-case Experimental Design Data Research Articles

Evaluating the impact of movement tracking feedback on engagement with home exercise programmes of children with cerebral palsy using a new therapy app: a protocol for a mixed-methods single-case experimental design with alternating treatments

IntroductionChildren with cerebral palsy (CP) are prescribed home exercise programmes (HEPs) to increase the frequency of movement practice, yet adherence to HEPs can be low. This paper outlines the protocol...

How to assess and take into account trend in single-case experimental design data

ABSTRACT One of the data features that are expected to be assessed when analyzing single-case experimental designs (SCED) data is trend. The current text deals with four different questions that applied researchers can ask themselves when assessing trend and especially when dealing with improving baseline trend: (a) What options exist for assessing the presence of trend?; (b) Once assessed, what criterion can be followed for deciding whether it is necessary to control for baseline trend?; (c) What strategy can be followed for controlling for baseline trend?; and (d) How to proceed in case there is baseline trend only in some A-B comparisons? Several options are reviewed for each of these questions in the context of real data, and tentative recommendations are provided. A new user-friendly website is developed to implement the options for fitting a trend line and a criterion for selecting a specific technique for that purpose. Trend-related and more general data analytical recommendations are provided for applied researchers. Trial registration: ClinicalTrials.gov identifier: NCT04560777

OSCAR (Osteopathic Single CAse Research) – Assessing the effect of standard and biopsychosocial osteopathic management for patients with non-specific low back pain: Protocol for a Single Case Experimental Design (SCED)

BackgroundOsteopathy has been shown to be effective in the management of chronic low back pain (LBP). Guidelines recommend biopsychosocial care for chronic, complex musculoskeletal conditions, including non-specific LBP. ObjectivesThis study has four aims: 1/to assess how patients with LBP improve after osteopathic treatment, both before and after an osteopath has completed a Biopsychosocial Pain Management course; 2/to assess if it is feasible and acceptable for osteopaths to receive weekly SCED data and use it to guide patient management; 3/to assess if it is acceptable for patients to submit daily data and discuss weekly summary with their osteopaths; and 4/to test feasibility for researchers in collecting, managing and storing large quantities of individual patient data. MethodsA multiple baseline single case experimental design trial with up to 10 UK osteopaths with more than 15 years in practice and 60 patients will assess how change occurs as a result of osteopathic treatment for patients with non-specific LBP of more than 12 weeks’ duration. Statistical analysis will assess the degree and rate of change between baseline, intervention and follow-up periods, and whether differences in effect are observed after the osteopaths have completed the biopsychosocial patient management training course. Primary outcomes will be the Numeric Pain Rating and Patient Specific Function Scales, measured daily at baseline and for 6 weeks during the intervention stage, and weekly or fortnightly during a 12-week follow-up period. EthicsThis research was approved by the University College of Osteopathy Research Ethics Committee.

Modeling multiple dependent variables in meta-analysis of single-case experimental design using multilevel modeling.

Although meta-analyses of single-case experimental design (SCED) often include multiple types of dependent variables (DVs), multiple DVs are rarely considered within models in the analysis. Baek et al. (Journal of Experimental Education, 90(4), 934-961, 2022) identified several statistical issues that arise when researchers fail to model multiple DVs in meta-analyses of SCED data. However, the degree to which non-modeling of multiple DVs impacts the results of the meta-analysis of SCED has not been fully examined. In this simulation study, we have systematically investigated the impact of non-modeling of multiple DVs when analyzing meta SCED data using multilevel modeling. The result demonstrates that modeling multiple DVs has advantages over the non-modeling option for meta-analysis of SCED. Modeling multiple DVs enables the determination of precise effects from different DVs in addition to the unbiased and accurate average effect and accurate estimates and inferences for the error variances at the study level as well as the observation level. The current study also reveals potential factors (i.e., the number of DVs, degree of heterogeneity in the level-1 error variances and autocorrelation, and presence of the moderator effect) that impact the precision and accuracy of the variance parameters.

Interpreting treatment effect size from single case experimental design data: a preliminary analysis of differential effects of treatments designed to increase or decrease behaviour.

Estimates of treatment effect size from single case experimental design (SCED) data may be impacted by the direction for treatment effects (i.e. ascending or descending slope for the dependent variable). Estimating effect sizes for treatments designed to decrease behaviour are potentially more restricted because the intended direction for treatment is zero (i.e. an absolute basal). Conversely, effect sizes for interventions that increase behaviour are less restricted due to a relatively unconstrained ceiling from a pure measurement standpoint (i.e. no absolute ceiling). That is, treatments that increase behaviour have a broader range of possible effect size values as the ceiling is only limited by demand characteristics and the learners' skills and motivation to exhibit the behaviour. The current study represents a preliminary analysis of the mean and range of SCED effect sizes for treatments designed to either increase or decrease target behaviour. A within-case Cohen's d measure that was developed for SCED data was used to estimate treatment effect sizes. Results indicated that the mean and range of effect size values for treatments that increased behaviour were significantly greater compared with treatments that decreased behaviour. Results are discussed in terms of developing standards, or best practices, specific to interpreting effect size values and meeting quality control requirements for inclusion of the data set in future SCED meta-analytic studies estimating treatment effect size. Specifically, preliminary results suggest that benchmarks for low, medium and high SCED effect size values need to be developed separately for treatments that increase or decrease levels of the dependent variable.

The Effects of a Virtual Reality-Based Training Program for Adolescents With Disruptive Behavior Problems on Cognitive Distortions and Treatment Motivation: Protocol for a Multiple Baseline Single-Case Experimental Design.

BackgroundSerious disruptive behavior among adolescents is a prevalent and often persistent problem. This highlights the importance of adequate and effective treatment to help adolescents with disruptive behavior problems react less hostile and aggressive. In order to create a treatment environment in which behavioral change can be enhanced, treatment motivation plays an essential role. Regarding treatment itself, a focus on challenging self-serving cognitive distortions in order to achieve behavioral change is important. Street Temptations (ST) is a new training program that was developed to address both treatment motivation and cognitive distortions in adolescents with disruptive behavior problems. One of the innovative aspects of ST is the use of virtual reality (VR) techniques to provide adolescents during treatment with visually presented daily social scenarios to activate emotional engagement and dysfunctional cognitions. By using the VR scenarios as an integral starting point of ST’s sessions and transferring the power of the VR experience into playful and dynamic exercises to practice social perspective–taking, adolescents are encouraged to reflect on both their own behavior and that of others. This focus on reflection is grounded in ST’s main treatment mechanism to influence treatment motivation and cognitive distortions, namely, mentalizing (ie, reflective functioning).ObjectiveThe aim of this study is to describe the research protocol to evaluate the effects of ST on treatment motivation and cognitive distortions. We take a closer look at the use of ST and the methodology used, namely, the repeated single-case experimental design (SCED).MethodsThe effects of ST are studied through a multiple baseline SCED, using both quantitative and qualitative data. In total, 18 adolescents from secure residential youth care facilities and secondary special education schools are randomly assigned to 1 of the 3 different baseline conditions. Throughout the baseline phase (1, 2, or 3 weeks), intervention phase (4 weeks), and follow-up phase (1, 2, or 3 weeks), daily measurements on treatment motivation and cognitive distortions are conducted. Secondary study parameters are assessed before baseline, after intervention, and after follow-up. Qualitative data are collected after intervention, as well as at 3 months and 6 months after the intervention.ResultsData collection for this study started in November 2021 and is planned to be completed by August 2023. The results will be published in peer-reviewed journals and presented at national and international conferences.ConclusionsST aims to improve the disruptive behavior problems of adolescents. This study will be the first to gain insights into the effectiveness of ST. The strengths of this study include its thorough and individually focused design (SCED), the focus on a residential as well as a secondary special education setting, and the ecological validity. The implications for practice are discussed.Trial RegistrationCentral Committee on Research Involving Human Subjects NL75545.029.20. Netherlands Trial Register NL9639; https://www.trialregister.nl/trial/9639International Registered Report Identifier (IRRID)PRR1-10.2196/33555

Introduction to the Special Section: Translating Advanced Quantitative Techniques for Single-Case Experimental Design Data.

The articles in this special section offer strategies to single-case experimental design (SCED) researchers to interpret their outcomes, communicate their results, and compare the results using common, quantitative results. Advancing quantitative methods applied to SCED data will facilitate communication with scientists and other professionals that do not typically interpret graphed data of the dependent variable. Horner and Ferron aptly note that innovative statistical procedures are improving the precision and credibility of SCED research as disseminate our findings to an increasingly diverse audience. This special section promotes the translation of these quantitative methods to encourage their adoption in research using single case experimental designs.

Structured visual analysis of single-case experimental design data: Developments and technological advancements.

Visual analysis is the primary method used to interpret single-case experimental design (SCED) data in applied behavior analysis. Research shows that agreement between visual analysts can be suboptimal at times. To address the inconsistent interpretations of SCED data, recent structured visual-analysis technological advancements have been developed. To assess the extent to which structured visual analysis is used to guide or supplement applied behavior analysts' interpretation of SCED graphs, a systematic review between the years of 2015 to 2020 in the Journal of Applied Behavior Analysis was conducted. Findings showed that despite recent efforts to develop structured visual-analysis tools and criteria, these methods are rarely used to analyze SCED data. An overview of structured visual-analysis tools is shared, their utility is delineated, common characteristics are brought to light, and future directions for both research and their clinical use are highlighted.

Meta-analyses and effect sizes in applied behavior analysis: A review and discussion.

For more than four decades, researchers have used meta-analyses to synthesize data from multiple experimental studies often to draw conclusions that are not supported by individual studies. More recently, single-case experimental design (SCED) researchers have adopted meta-analysis techniques to answer research questions with data gleaned from SCED experiments. Meta-analyses enable researchers to answer questions regarding intervention efficacy, generality, and condition boundaries. Here we discuss meta-analysis techniques, the rationale for their adaptation with SCED studies, and current indices used to quantify the effect of SCED data in applied behavior analysis.

Characteristics of Moderators in Meta-Analyses of Single-Case Experimental Design Studies.

Hierarchical linear modeling (HLM) has been recommended as a meta-analytic technique for the quantitative synthesis of single-case experimental design (SCED) studies. The HLM approach is flexible and can model a variety of different SCED data complexities, such as intervention heterogeneity. A major advantage of using HLM is that participant and-or study characteristics can be incorporated in the model in an attempt to explain intervention heterogeneity. The inclusion of moderators in the context of meta-analysis of SCED studies did not yet receive attention and is in need of methodological research. Prior to extending methodological work validating the hierarchical linear model including moderators at the different levels, an overview of characteristics of moderators typically encountered in the field is needed. This will inform design conditions to be embedded in future methodological studies and ensure that these conditions are realistic and representative for the field of SCED meta-analyses. This study presents the results of systematic review of SCED meta-analyses, with the particular focus on moderator characteristic. The initial search yielded a total of 910 articles and book chapters. After excluding duplicate studies and non peer-reviewed studies, 658 unique peer-reviewed studies were maintained and screened by two independent researchers. Sixty articles met the inclusion criteria and were eligible for data retrieval. The results of the analysis of moderator characteristics retrieved from these 60 meta-analyses are presented. The first part of the results section contains an overview of moderator characteristics per moderator level (within-participant level, participant level, and study level), including the types of moderators, the ratio of the number of moderators relative to the number of units at that level, the measurement scale, and the degree of missing data. The second part of the results section focuses on the metric used to quantify moderator effectiveness and the analysis approach. Based on the results of the systematic review, recommendations are given for conditions to be included in future methodological work.

Accurate models vs. accurate estimates: A simulation study of Bayesian single-case experimental designs

Although statistical practices to evaluate intervention effects in single-case experimental design (SCEDs) have gained prominence in recent times, models are yet to incorporate and investigate all their analytic complexities. Most of these statistical models incorporate slopes and autocorrelations, both of which contribute to trend in the data. The question that arises is whether in SCED data that show trend, there is indeterminacy between estimating slope and autocorrelation, because both contribute to trend, and the data have a limited number of observations. Using Monte Carlo simulation, we compared the performance of four Bayesian change-point models: (a) intercepts only (IO), (b) slopes but no autocorrelations (SI), (c) autocorrelations but no slopes (NS), and (d) both autocorrelations and slopes (SA). Weakly informative priors were used to remain agnostic about the parameters. Coverage rates showed that for the SA model, either the slope effect size or the autocorrelation credible interval almost always erroneously contained 0, and the type II errors were prohibitively large. Considering the 0-coverage and coverage rates of slope effect size, intercept effect size, mean relative bias, and second-phase intercept relative bias, the SI model outperformed all other models. Therefore, it is recommended that researchers favor the SI model over the other three models. Research studies that develop slope effect sizes for SCEDs should consider the performance of the statistic by taking into account coverage and 0-coverage rates. These helped uncover patterns that were not realized in other simulation studies. We underline the need for investigating the use of informative priors in SCEDs.

A Bayesian Rate Ratio Effect Size to Quantify Intervention Effects for Count Data in Single Case Experimental Research

Single case experimental design (SCED) is an indispensable methodology when evaluating intervention efficacy. Despite long-standing success with using visual analyses to evaluate SCED data, this method has limited utility for conducting meta-analyses. This is critical because meta-analyses should drive practice and policy in behavioral disorders more than evidence derived from individual SCEDs. Even when analyzing data from individual studies, there is merit to using multiple analytic methods since statistical analyses in SCED can be challenging given small sample sizes and autocorrelated data. These complexities are exacerbated when using count data, which are common in SCEDs. Bayesian methods can be used to develop new statistical procedures that may address these challenges. The purpose of the present study was to formulate a within-subject Bayesian rate ratio effect size (BRR) for autocorrelated count data that would obviate the need for small sample corrections. This effect size is the first step toward building a between-subject rate ratio that can be used for meta-analyses. We illustrate this within-subject effect size using real data for an ABAB design and provide codes for practitioners who may want to compute BRR.

Estimating effect size with respect to variance in baseline to treatment phases of single-case experimental designs: A Bayesian simulation study

The current study examined the relation between the ratio of baseline to treatment sessions and how differences in this ratio can influence estimation of treatment effect size from temporally adjacent baseline and treatment phases of any single-case experimental design (SCED). The current study describes how Bayesian statistical analyses can be used to aggregate treatment outcomes across subjects to meta-analyze SCED data. One-third of all A versus B comparisons (based upon simulated average values) did have a 10% or more bias, with the vast majority of the bias being substantially fewer data points in baseline compared to treatment sessions. SCEDs require relatively steady state responding; thus researchers may run relatively more B sessions compared to A sessions in the course of visually inspecting graphically depicted data. When the standard deviation for the number of A sessions was approximately twice as large or more than the B phase standard deviation, the degree of AB sessions ratio bias decreased substantially. SCED practitioners can use results of the current study to determine the potential benefits of running additional baseline or treatment sessions.

Using linear mixed effects models: A single-case experimental design meta-analysis of functional communication training

In this study, we extended the meta-analytic literature on single-case experimental design (SCED) studies on functional communication training (FCT) using a linear mixed effects model (LMEM). First, we evaluated the methodological adequacy of each study using the What Works Clearinghouse (WWC) Standards. Second, we calculated a standardized rate difference (SRD) between the rates of alternative communicative responses and problem behavior. Third, we estimated a LMEM using SRD as the dependent variable to examine the effect of FCT across participants with different disabilities. Finally, we estimated a LMEM including only studies in which participants had a diagnosis of Autism Spectrum Disorder (ASD), without standardizing the rate difference. Results indicate that FCT was effective in increasing the SRD as suggested by the greater number of alternative communicative responses compared to the number of problem behaviors over time. Results also suggest that FCT is an effective intervention for participants with ASD and that it might not be necessary to standardize scores when using LMEMs to analyze data from participants with the same disability. We discuss the utility of analyzing SCED data using LMEMs and make recommendations for an SCED data warehouse protocol.

Tau-U: A Quantitative Approach for Analysis of Single-Case Experimental Data in Aphasia.

Tau-U is a quantitative approach for analyzing single-case experimental design (SCED) data. It combines nonoverlap between phases with intervention phase trend and can correct for a baseline trend (Parker, Vannest, & Davis, 2011). We demonstrate the utility of Tau-U by comparing it with the standardized mean difference approach (Busk & Serlin, 1992) that is widely reported within the aphasia SCED literature. Repeated writing measures from 3 participants with chronic aphasia who received computer-based writing treatment are analyzed visually and quantitatively using both Tau-U and the standardized mean difference approach. Visual analysis alone was insufficient for determining an effect between the intervention and writing improvement. The standardized mean difference yielded effect sizes ranging from 4.18 to 26.72 for trained items and 1.25 to 3.20 for untrained items. Tau-U yielded significant (p < .05) effect sizes for 2 of 3 participants for trained probes and 1 of 3 participants for untrained probes. A baseline trend correction was applied to data from 2 of 3 participants. Tau-U has the unique advantage of allowing for the correction of an undesirable baseline trend. Although further study is needed, Tau-U shows promise as a quantitative approach to augment visual analysis of SCED data in aphasia.

Approaches for Specifying the Level-1 Error Structure When Synthesizing Single-Case Data

ABSTRACTMultilevel modeling has been utilized for combining single-case experimental design (SCED) data assuming simple level-1 error structures. The purpose of this study is to compare various multilevel analysis approaches for handling potential complexity in the level-1 error structure within SCED data, including approaches assuming simple and complex error structures (heterogeneous, autocorrelation, and both) and those using fit indices to select between alternative error structures. A Monte Carlo study was conducted to empirically validate the suggested multilevel modeling approaches. Results indicate that each approach leads to fixed effect estimates with little to no bias and that inferences for fixed effects were frequently accurate, particularly when a simple homogeneous level-1 error structure or a first-order autoregressive structure was assumed and the inferences were based on the Kenward-Roger method. Practical implications and recommendations are discussed.

Development of an in-vivo metric to aid visual inspection of single-case design data: Do we need to run more sessions?

Comparing visual inspection results of graphed data reveals inconsistencies in the interpretation of the same graph among single-case experimental design (SCED) researchers and practitioners. Although several investigators have disseminated structured criteria and visual inspection aids or strategies, inconsistencies in interpreting graphed data continue to exist even for individuals considered to be experts at interpreting SCED graphs. We propose a fail safe k metric that can be used in conjunction with visual inspection, and it can be used in-vivo after each additional data point is collected within a phase to determine the optimal point in time to shift between phases (e.g., from baseline to treatment). Preliminary proof of concept data are presented to demonstrate the potential utility of the fail safe k metric with a sample of previously published SCED graphs examining the effects on noncontingent reinforcement on occurrences of problem behavior. Results showed that if the value of fail safe k is equal to or less than the number of sessions in the current phase, then the data path may not be stable and more sessions should be run before changing phases. We discuss the results in terms of using the fail safe k as an additional aid for visual inspection of SCED data.

Single-case experimental designs: Reflections on conduct and analysis

In this editorial discussion we reflect on the issues addressed by, and arising from, the papers in this special issue on Single-Case Experimental Design (SCED) study methodology. We identify areas of consensus and disagreement regarding the conduct and analysis of SCED studies. Despite the long history of application of SCEDs in studies of interventions in clinical and educational settings, the field is still developing. There is an emerging consensus on methodological quality criteria for many aspects of SCEDs, but disagreement on what are the most appropriate methods of SCED data analysis. Our aim is to stimulate this ongoing debate and highlight issues requiring further attention from applied researchers and methodologists. In addition we offer tentative criteria to support decision-making in relation to the selection of analytical techniques in SCED studies. Finally, we stress that large-scale interdisciplinary collaborations, such as the current Special Issue, are necessary if SCEDs are going to play a significant role in the development of the evidence base for clinical practice.

Single case experimental designs: Introduction to a special issue of Neuropsychological Rehabilitation

This paper introduces the Special Issue of Neuropsychological Rehabilitation on Single Case Experimental Design (SCED) methodology. SCED studies have a long history of use in evaluating behavioural and psychological interventions, but in recent years there has been a resurgence of interest in SCED methodology, driven in part by the development of standards for conducting and reporting SCED studies. Although there is consensus on some aspects of SCED methodology, the question of how SCED data should be analysed remains unresolved. This Special Issues includes two papers discussing aspects of conducting SCED studies, five papers illustrating use of SCED methodology in clinical practice, and nine papers that present different methods of SCED data analysis. A final Discussion paper summarises points of agreement, highlights areas where further clarity is needed, and ends with a set of resources that will assist researchers conduct and analyse SCED studies.