Average Weighted Effect Size Research Articles

Meta-Analyses and Narrative Review of Home-Based Interventions to Improve Literacy and Mathematics Outcomes for Children Between the Ages of 3 and 5 Years Old

The purpose of these meta-analyses was to examine the effectiveness of home-based interventions aimed at improving literacy and mathematics outcomes for preschool-age children (mean age = 4.29 years; range = 3.07–5.32 years) and to develop an understanding of what home-based interventions work in different contexts. A total of 32 studies met the inclusion criteria for these meta-analyses; 30 studies included sufficient data for inclusion in the meta-analyses, and two studies did not contain sufficient quantitative data and instead were summarized in a narrative review. The average weighted effect size for interventions with literacy (d = 0.10; CI = [−0.17, 0.38]; n = 27) and mathematical outcomes (d = 0.18; CI = [−1.62, 1.99]; n = 8) were minimal. Hence, these meta-analyses showed that home-based interventions had minimal effect on literacy and mathematical outcomes for preschoolers. There were more home-based interventions with literacy (N = 28) than mathematical outcomes (N = 10). The heterogeneity showed no variability, indicating that all intervention impacted on children’s outcomes to similar effect. Overall, many interventions were relatively light touch (i.e., time spent engaging in parent training), and the engagement requirement of the parent in some studies was minimal (e.g., reading a short text message). More in-depth research into the components of interventions (e.g., focus, training approaches) and evaluation of interventions before they are implemented is essential for ensuring that early interventions will be effective and lead to the development of the intended skills.

Toward a Model of Statistical Learning and Reading: Evidence From a Meta-Analysis

A compelling demonstration of implicit learning is the human ability to unconsciously detect and internalize statistical patterns of complex environmental input. This ability, called statistical learning, has been investigated in people with dyslexia using various tasks in different orthographies. However, conclusions regarding impaired or intact statistical learning in dyslexia remain mixed. This study conducted a systematic literature search of published and unpublished studies that compared statistical learning between people with and without dyslexia using different learning paradigms in different orthographies. We identified 49 papers consisting of 59 empirical studies, representing the data from 1,259 participants with dyslexia and 1,459 typically developing controls. The results showed that, on average, individuals with dyslexia performed worse in statistical learning than age-matched controls, regardless of the learning paradigm or orthography (average weighted effect size d = 0.47, 95% confidence interval [0.36, 0.59], p < .001). Meta-regression analyses further revealed that the heterogeneity of effect sizes between studies was significantly explained by one reader characteristic (i.e., verbal IQ) but no task characteristics (i.e., task paradigm, task modality, and stimulus type). These findings suggest domain-general statistical learning weakness in dyslexia across languages, and support the need for a new theoretical model of statistical learning and reading, that is, the SLR model, which elucidates how reader and task characteristics are regulated by a multicomponent memory system when establishing statistically optimal representations for deep learning and reading.

A Meta-Analysis of Single-Case Research on Word-Problem Instruction for Students With Learning Disabilities

In this meta-analysis of single-case designs, the researchers investigated the effects of word-problem instruction for students with learning disabilities in Grades 1 through 12, as reported in 20 peer-reviewed articles published between 1975 and 2020. A two-level multilevel model with a robust variance estimation method was implemented to calculate the effect sizes. A significant overall effect of word-problem instruction for students with learning disabilities was found, and the results indicate that studies meeting all three quality indicators of implementation fidelity (i.e., adherence, dosage, and regularity) showed greater effects. The average weighted effect sizes of word-problem instruction varied depending on the content and practice standards of the Common Core State Standards for Mathematics. Limitations and future research directions are also discussed.

The association between therapeutic alliance and outcome in internet-based psychological interventions: A meta-analysis

BackgroundInternet-based psychological interventions have proven effective and yield advantages that make them a viable alternative to face-to-face therapy in many fields. Yet, the role of therapeutic alliance in technologically mediated interventions has been discussed critically. The aim of this meta-analysis is to summarize the association between therapeutic alliance and outcome in therapist-assisted online interventions. MethodsA systematic search of the databases PubMed, Web of Science, PsycINFO and PubPsych was conducted for articles published before February 2020 that assess the association between therapeutic alliance and outcome in internet-based interventions involving remote therapist contact. Results were systematically screened and information on the alliance-outcome-association was extracted. A multilevel meta-analysis was conducted. ResultsOverall, 51 effect sizes were extracted from 20 included studies. The average weighted effect size is r = 0.203 (p < .0001). The correlation was larger when alliance was measured near the end of an intervention. There was no impact of therapist contact frequency or mode and availability of self-help content on the effect size. ConclusionsTherapeutic alliance and outcome are significantly correlated in internet-based therapy. This points to the relevance of a stable alliance in internet-based interventions and suggests that fostering alliance might be beneficial to treatment success.

Are Women More Likely than Men Are to Care Excessively about Maintaining Positive Social Relationships? A Meta-Analytic Review of the Gender Difference in Sociotropy

Sociotropy is defined as the tendency to overemphasize maintaining positive social relationships (Beck 1983). Although the stereotype that women care more about interpersonal relationships than men do is well-documented (Cross and Madson 1997), the literature provides mixed support as to whether women are more sociotropic than men are. This is important to establish because sociotropy consistently correlates positively with depression (Robins et al. 1994) and thus a gender difference in sociotropy could contribute to the well-documented gender difference in depression (Girgus and Nolen-Hoeksema 2006). The present meta-analysis asks whether the gender difference in sociotropy exists, and if so, at what magnitude, by aggregating 108 independent effect sizes from 90 papers (n = 30,372 participants). The average weighted effect size of the gender difference was d = .34, with women scoring higher than men on sociotropy. Culture was a significant moderator: The gender difference in sociotropy was significantly smaller in research from collectivist countries, where interpersonal harmony and cooperation are emphasized for both genders, than in research from individualistic countries, where men are supposed to be independent and agentic and women are supposed to be communal and concerned with relationships. Further research is needed to explore the development of this gender difference and its relationship to the gender difference in depression.

Technology integration in postsecondary education: A summary of findings from a set of related meta-analyses

Although the overall research literature on the application of educational technologies to classroom instruction tends to favor their use over their non-use, these results vary considerably depending on what kind of technology is used, who it is used with and, more importantly, under what circumstances and for what instructional purposes it is used. Relatively recent, but well-developed and powerful methodology of systematic reviews, particularly quantitative syntheses (also known as meta-analyses) is especially suitable for addressing questions of that type by systematically summarizing research evidence in given areas of interest in social sciences.This meta-analysis summarizes data from 674 independent primary studies that compared higher degrees of technology use in the experimental condition with less technology in the control condition, in terms of their effects on student learning outcomes in postsecondary education. The result was an overall average weighted effect size of = 0.27 (k = 879, p &lt; .01), indicating low but significant positive effect of technology integration on learning. The follow-up analyses revealed the influence of educational technology used for cognitive support and blended learning instructional settings designed interaction treatments, and technology integration in teacher training, especially when student-centered pedagogical frameworks are used. These findings are of potentially high interest and applied value for educational practitioners, including teachers and school administrators, as well as for instructional designers and developers of educational software.

Protocol for a systematic review: Interventions to improve mathematics achievement in primary school-aged children: a systematic review.

Protocol for a systematic review: Interventions to improve mathematics achievement in primary school-aged children: a systematic review.

Visual artificial grammar learning in dyslexia: A meta-analysis

BackgroundLiteracy impairments in dyslexia have been hypothesized to be (partly) due to an implicit learning deficit. However, studies of implicit visual artificial grammar learning (AGL) have often yielded null results. AimsThe aim of this study is to weigh the evidence collected thus far by performing a meta-analysis of studies on implicit visual AGL in dyslexia. Methods and proceduresThirteen studies were selected through a systematic literature search, representing data from 255 participants with dyslexia and 292 control participants (mean age range: 8.5–36.8 years old). ResultsIf the 13 selected studies constitute a random sample, individuals with dyslexia perform worse on average than non-dyslexic individuals (average weighted effect size=0.46, 95% CI [0.14 … 0.77], p=0.008), with a larger effect in children than in adults (p=0.041; average weighted effect sizes 0.71 [sig.] versus 0.16 [non-sig.]). However, the presence of a publication bias indicates the existence of missing studies that may well null the effect. Conclusions and implicationsWhile the studies under investigation demonstrate that implicit visual AGL is impaired in dyslexia (more so in children than in adults, if in adults at all), the detected publication bias suggests that the effect might in fact be zero.

The Role of the Therapeutic Relationship in Psychopharmacological Treatment Outcomes: A Meta-analytic Review.

Patient nonadherence to psychopharmacological treatment is a significant barrier to effective treatment. The therapeutic relationship is known to be a critical component of effective psychological treatment, but it has received limited study. A meta-analysis was conducted to examine the role of the therapeutic relationship in the delivery of effective psychopharmacological treatment. PubMed, PsycINFO, CINAHL, Google Scholar, Ingenta, and the Web of Science-Science Citation Index were searched, including reference lists of found articles. Meta-analytic methods were used to examine the association between the physician-patient therapeutic relationship and outcomes in psychopharmacological treatment. Eight independent studies of psychopharmacological treatment reported in nine articles met the inclusion criterion (1,065 participants) of being an empirically based study in which measures of the therapeutic relationship were administered and psychiatric treatment outcomes were assessed. The overall average weighted effect size for the association between the therapeutic relationship and treatment outcomes was z=.30 (95% confidence interval=.20-.39), demonstrating a statistically significant, moderate effect. Findings indicate that a positive therapeutic relationship or alliance between the physician and the psychiatric patient is associated with patient improvement over the course of psychopharmacological treatment. Results suggest that more attention should be paid to psychiatrist communication skills that may enhance the therapeutic alliance in psychopharmacological treatment.

Comparing Students With and Without Reading Difficulties on Reading Comprehension Assessments: A Meta-Analysis

Researchers have increasingly investigated sources of variance in reading comprehension test scores, particularly with students with reading difficulties (RD). The purpose of this meta-analysis was to determine if the achievement gap between students with RD and typically developing (TD) students varies as a function of different reading comprehension response formats (e.g., multiple choice, cloze). A systematic literature review identified 82 eligible studies. All studies administered reading comprehension assessments to students with RD and TD students in Grades K–12. Hedge’s g standardized mean difference effect sizes were calculated, and random effects robust variance estimation techniques were used to aggregate average weighted effect sizes for each response format. Results indicated that the achievement gap between students with RD and TD students was larger for some response formats (e.g., picture selection ES g = −1.80) than others (e.g., retell ES g = −0.60). Moreover, for multiple-choice, cloze, and open-ended question response formats, single-predictor metaregression models explored potential moderators of heterogeneity in effect sizes. No clear patterns, however, emerged in regard to moderators of heterogeneity in effect sizes across response formats. Findings suggest that the use of different response formats may lead to variability in the achievement gap between students with RD and TD students.

Does Use of Text-to-Speech and Related Read-Aloud Tools Improve Reading Comprehension for Students With Reading Disabilities? A Meta-Analysis

Text-to-speech and related read-aloud tools are being widely implemented in an attempt to assist students’ reading comprehension skills. Read-aloud software, including text-to-speech, is used to translate written text into spoken text, enabling one to listen to written text while reading along. It is not clear how effective text-to-speech is at improving reading comprehension. This study addresses this gap in the research by conducting a meta-analysis on the effects of text-to-speech technology and related read-aloud tools on reading comprehension for students with reading difficulties. Random effects models yielded an average weighted effect size of ( = .35, with a 95% confidence interval of .14 to .56, p < .01). Moderator effects of study design were found to explain some of the variance. Taken together, this suggests that text-to-speech technologies may assist students with reading comprehension. However, more studies are needed to further explore the moderating variables of text-to-speech and read-aloud tools’ effectiveness for improving reading comprehension. Implications and recommendations for future research are discussed.

Writing Characteristics of Students With Learning Disabilities and Typically Achieving Peers

There is a general consensus that writing is a challenging task for students with learning disabilities (LD). To identify more precisely the extent and depth of the challenges that these students experience with writing, the authors conducted a meta-analysis comparing the writing performance of students with LD to their typically achieving peers. From 53 studies that yielded 138 effect sizes, the authors calculated average weighted effect sizes, showing that students with LD obtained lower scores than their peers on the following writing outcomes: writing quality (–1.06); organization (–1.04); vocabulary (–0.89); sentence fluency (–0.81); conventions of spelling, grammar, and handwriting (–1.14); genre elements (–0.82); output (–0.87); and motivation (–0.42). Implications for research and practice are provided based on these findings.

Rapid response to eating disorder treatment: A systematic review and meta-analysis.

This review aimed to (a) examine the effects of rapid response on behavioral, cognitive, and weight-gain outcomes across the eating disorders, (b) determine whether diagnosis, treatment modality, the type of rapid response (changes in disordered eating cognitions or behaviors), or the type of behavioral outcome moderated this effect, and (c) identify factors that predict a rapid response. Thirty-four articles met inclusion criteria from six databases. End of treatment and follow-up outcomes were divided into three categories: Behavioral (binge eating/purging), cognitive (EDE global scores), and weight gain. Average weighted effect sizes(r) were calculated. Rapid response strongly predicted better end of treatment and follow-up cognitive and behavioral outcomes. Moderator analyses showed that the effect size for rapid response on behavioral outcomes was larger when studies included both binge eating and purging (as opposed to just binge eating) as a behavioral outcome. Diagnosis, treatment modality, and the type of rapid response experienced did not moderate the relationship between early response and outcome. The evidence for weight gain was mixed. None of the baseline variables analyzed (eating disorder psychopathology, demographics, BMI, and depression scores) predicted a rapid response. As there is a solid evidence base supporting the prognostic importance of rapid response, the focus should shift toward identifying the within-treatment mechanisms that predict a rapid response so that the effectiveness of eating disorder treatment can be improved. There is a need for future research to use theories of eating disorders as a guide to assess within-treatment predictors of rapid response. © 2016 Wiley Periodicals, Inc. Int J Eat Disord 2016; 49:905-919.

Writing Characteristics of Students with Attention Deficit Hyperactive Disorder: A Meta‐Analysis

Students with Attention Deficit Hyperactivity Disorders (ADHD) frequently experience significant difficulty mastering basic academic skills. This meta‐analysis focuses on one specific potential area of learning difficulties for these students: namely, writing. To identify the extent and depth of the potential writing challenges faced by students with ADHD, we conducted a meta‐analysis comparing the writing performance of grade 1 to 12 students with ADHD to their normally achieving peers. We located 44 papers, yielding 45 studies with 87 effect sizes. The average weighted effect sizes showed that students with ADHD obtained lower scores than their normally achieving peers for writing quality (–0.78), output (–0.64), number of genre elements (–0.69), vocabulary (–0.76), spelling (–0.80), and handwriting (–0.62). Contrary to expectations, moderator analyses found that neither study quality nor the source from which ADHD students were drawn (i.e., school/community vs. clinic/hospital) accounted for variability above sampling error alone.

Formative Assessment and Writing

AbstractTo determine whether formative writing assessments that are directly tied to everyday classroom teaching and learning enhance students’ writing performance, we conducted a meta-analysis of true and quasi-experiments conducted with students in grades 1 to 8. We found that feedback to students about writing from adults, peers, self, and computers statistically enhanced writing quality, yielding average weighted effect sizes of 0.87, 0.58, 0.62, and 0.38, respectively. We did not find, however, that teachers’ monitoring of students’ writing progress or implementation of the 6 + 1 Trait Writing model meaningfully enhanced students’ writing. The findings from this meta-analysis provide support for the use of formative writing assessments that provide feedback directly to students as part of everyday teaching and learning. We argue that such assessments should be used more frequently by teachers, and that they should play a stronger role in the Next-Generation Assessment Systems being developed by Smart...

A meta-analysis and meta-regression of serial reaction time task performance in Parkinson's disease.

This article reports findings of a meta-analysis and meta-regression summarizing research on implicit sequence learning in individuals with Parkinson's disease (PD), as measured by the Serial Reaction Time (SRT) task. Following a systematic search of the literature, we analyzed a total of 27 studies, representing data from 505 participants with PD and 460 neurologically intact control participants. Overall, the meta-analysis indicated significantly (p < .001) worse sequence learning by the PD group than the control group. The average weighted effect size was found to be .531 (95% CI [.332, .470]), which is a medium effect size. However, moderate to high levels of heterogeneity (differences) were found between study effect sizes (I(2) = 58%). Meta-regression analysis suggested that presentation of the SRT task under dual task conditions coupled with PD severity or characteristics of the sequence might affect study effect sizes. The meta-analysis provides clear support that learning in procedural memory (procedural learning), which underlies implicit sequence learning in the SRT task, is impaired in PD.

Effects of CALL Versus Traditional L1 Glosses on L2 Reading Comprehension

Meta-analytic research of experiments conducted on the effects of native-language (L1) glosses on second-language (L2) reading comprehension have revealed a significant difference between groups of studies with traditional and computer-assisted L1 glosses. This means that learners provided with L1 glosses comprehend significantly more text--through the medium of a computer--than learners with traditional, paper-based L1 glossing aids. The average weighted effect size of studies with computer-assisted L1 glosses is large (g = 1.09) and differs significantly (p &lt; .001) when compared to studies with traditional L1 glosses (g = .39). This article explores factors explaining how and when computer-assisted L1 glosses can be effective in L2 reading comprehension.

Recasts in the L2 classroom: A meta-analytic review

In order to examine what we know and understand about the effectiveness of recasts in the second language classroom, this paper presents the results of a meta-analytic review of the current research on recasts. Initial analysis showed an average weighted effect size of 0.38. A Q-statistic test revealed that the studies contain heterogeneous data, where significant difference occurred in the type of treatment given to the participants, the type of grammatical structure studied, and in the difference in the second language being studied.

The Process Writing Approach: A Meta-analysis

ABSTRACT The process approach to writing instruction is one of the most popular methods for teaching writing. The authors conducted meta-analysis of 29 experimental and quasi-experimental studies conducted with students in Grades 1–12 to examine if process writing instruction improves the quality of students’ writing and motivation to write. For students in general education classes, process writing instruction resulted in a statistically significant, but relatively modest improvement in the overall quality of writing (average weighted effect size [ES] = 0.34). Variation in ES was not related to grade, reliability of the writing quality measure, professional development, genre assessed, or quality of study. The process writing approach neither resulted in a statistically significant improvement in students’ motivation nor enhanced the quality of struggling writers’ compositions.

Use of the Wechsler Adult Intelligence Scale Digit Span subtest for malingering detection: A meta-analytic review

Twenty-four studies utilizing the Wechsler Adult Intelligence Scale (WAIS) Digit Span subtest—either the Reliable Digit Span (RDS) or Age-Corrected Scaled Score (DS-ACSS) variant—for malingering detection were meta-analytically reviewed to evaluate their effectiveness in detecting malingered neurocognitive dysfunction. RDS and DS-ACSS effectively discriminated between honest responders and dissimulators, with average weighted effect sizes of 1.34 and 1.08, respectively. No significant differences were found between RDS and DS-ACSS. Similarly, no differences were found between the Digit Span subtest from the WAIS or Wechsler Memory Scale (WMS). Strong specificity and moderate sensitivity were observed, and optimal cutting scores are recommended.