The effect of linguistic comprehension instruction on generalized language and reading comprehension skills: A systematic review.

Abstract

The linguistic comprehension programs included in this review display a small positive immediate effect on generalized outcomes of linguistic comprehension. The effect of the programs on generalized measures of reading comprehension is negligible. Few studies report follow-up assessment of their participants. Children who begin school with proficient language skills are more likely to develop adequate reading comprehension abilities and achieve academic success than children who struggle with poor language skills in their early years. Individual language difficulties, environmental factors related to socioeconomic status (SES), and having the educational language as a second language are all considered risk factors for language and literacy failure. What is the aim of this review? This Campbell systematic review examines the effects of linguistic comprehension instruction on generalized measures of language and reading comprehension skills. The review summarizes evidence from 43 studies, including samples of both preschool and school-aged participants. This review included studies that evaluate the effects of linguistic comprehension interventions on generalized language and reading outcomes. A total of 43 studies were identified and included in the final analysis. The studies span the period 1992–2017. Randomized controlled trials (RCTs) and quasi-experiments (QEs) with a control group and a pre–post design were included in the review. The effect of linguistic comprehension instruction on generalized outcomes of linguistic comprehension skills is small in studies of both the overall immediate and follow-up effects. Analysis of differential language outcomes shows small effects on vocabulary and grammatical knowledge and moderate effects on narrative and listening comprehension. Linguistic comprehension instruction has no immediate effects on generalized outcomes of reading comprehension. Only a few studies have reported follow-up effects on reading comprehension skills, with divergent findings. Linguistic comprehension instruction has the potential to increase children's general linguistic comprehension skills. However, there is variability in effects related to the type of outcome measure that is used to examine the effect of such instruction on linguistic comprehension skills. One of the overall aims of linguistic comprehension intervention programs is to accelerate children's vocabulary development. Our results indicated that the type of intervention program included in this review might be insufficient to accelerate children's vocabulary development and, thus, to close the vocabulary gap among children. Further, the absence of an immediate effect of intervention programs on reading comprehension outcomes indicates that linguistic comprehension instruction through the type of intervention program examined in this study does not transfer beyond what is learned to general types of text. Despite clear indications from longitudinal studies that linguistic comprehension plays a vital role in the development of reading comprehension, only a few intervention studies have produced immediate and follow-up effects on generalized outcomes of reading comprehension. This indicates that preventing and remediating reading comprehension difficulties likely requires long-term educational efforts. Finally, it is likely that other outcome measures that are more closely aligned with the targeted intervention (use of targeted instructed words in the texts) would yield a different pattern of results. However, such tests were not included in this review. The review authors searched for studies up to October 2018. Well-developed vocabulary and language comprehension skills are not only critical in themselves but also fundamental to the development of adequate reading comprehension abilities and achieving academic success. Children with poor language skills, children from low socioeconomic areas, and second-language learners are at risk for subsequent reading comprehension problems. Reading comprehension difficulties are relatively common in school-aged children, and intervention programs have been designed to support children's linguistic comprehension skills. The primary objective of this review was to examine the extent to which linguistic comprehension instruction in educational settings is effective when measured by generalized outcomes of linguistic comprehension and reading comprehension. Specific electronic searches for literature dating back to 1986 were conducted in the following databases: Eric (Ovid), PsycINFO (Ovid), ISI Web of Science, Proquest Digital Dissertations, Linguistic and Language Behavior Abstracts (LLBA), Scopus Science Direct, Open Grey and Bielefeld Academic Search Engine (BASE). The search was limited to publications reported in English. The literature search also utilized citations, Google Scholar, prior meta-analyses and key journals. In addition, authors in the field were contacted for unpublished or in-press manuscripts. The review included RCTs and QEs with a pretest–posttest-controlled design. It was imperative that the intervention programs were conducted in a preschool or later educational setting, up to the end of secondary school. Intervention programs implemented by parents or other persons in the child's home environment were not included in the review. Further, the sample of participants included both monolingual and second-language learners, unselected typically achieving children, children with language delay/weaknesses, or children from low socioeconomic backgrounds. Samples of children with a special diagnosis, like autism, or other physical, mental, or sensory disabilities were not eligible for inclusion in the review. Moreover, studies had to report generalized outcomes of language and reading comprehension to be included in the review. Studies that only reported proximal outcomes designed by researchers to measure the direct effect of trained words were not included. Two electronic searches were conducted for this review. The first search was conducted in October 2016, followed by the same electronic search strategy in October 2018; 4,991 references for the original and 1,776 references for the follow-up search were identified and screened for eligibility. Among these, 871 references for the original and 175 references for the follow-up search were included for a full-text screening procedure. Analyses were conducted using the Comprehensive meta-analysis program by Borenstein, Hedges, Higgins, and Rothstein (2014). Overall, 43 references met the inclusion criteria and were included in the review. Linguistic comprehension instruction showed small effects on generalized measures of vocabulary and grammar in favor of the treatment groups. Further, the effect of linguistic comprehension instruction on narrative and listening comprehension skills showed positive moderate effects in favor of the treatment groups. However, there was no clear evidence of effect of linguistic comprehension instruction on general reading comprehension outcomes from the type of trials included in this review. The evidence indicated that the type of intervention program included in this review has the potential to increase children's general linguistic comprehension skills. However, these programs are probably not sufficiently effective to accelerate children's vocabulary knowledge and close the vocabulary gap among children. Programs with longer time frames and follow-up assessments than what was included in this review must be developed in the future. Simultaneously, more information from RCTs is needed to ensure that no systematic differences between intervention groups affect the outcome. The ability to understand and express language in both its oral and written forms is a crucial aspect of human development. Language is vital to be able to communicate with others and is closely linked to both social and emotional functioning. Children with poor language skills may experience more problems related to social, emotional, and behavioral aspects relative to their peers (Norbury et al., 2016). Researchers indicate less engagement in conversational interactions, poorer discourse skills, and more communication misunderstandings among children with poor language skills as compared to their typical peers (Durkin & Conti-Ramsden, 2007, 2010). Children with poor language skills are also considered to be at risk for poor academic achievement. Proficient language skills are fundamental to all higher-level cognitive activities and set the stage for reading development and academic success (McNamara & Magliano, 2009). Even though most children develop language naturally at a rapid pace, poor language skills in early childhood are not uncommon. An epidemiological study by Norbury et al. (2016) estimated the prevalence of language disorders of unknown origin to be approximately two children in every first-year classroom (7.58%). Both genetic risk factors within the child (Puglisi, Hulme, Hamilton, & Snowling, 2017; Stromswold, 2001) combined with environmental factors related to the amount and quality of language exposure (Hoff, 2003) are likely to explain the large variations between children and why some children will be at a greater risk for developing poor language skills than others. In addition, substantial portions of children entering school across countries come from families in which a language other than the educational language is practiced. Even though researchers have indicated cognitive advantages of growing up multilingual, like executive control, (Bialystok & Viswanathan, 2009) benefits related to linguistic processing is not reported. In a meta-analysis comparing first- and second-language learners, Melby-Lervåg and Lervåg (2014) found that second-language learners displayed a large deficit in language comprehension (d = −1.12 in favor of first-language learners). Thus, as a group, second-language learners display poorer linguistic comprehension skills in the second language than their monolingual peers. Their challenges are particularly related to vocabulary acquisition in the second language, and there appears to be limited transfer from the first language to the second language (Melby-Lervåg & Lervåg, 2011; Snow & Kim, 2006). In school-aged children, poor language skills may manifest themselves as reading comprehension problems. This becomes a problem when children reach fourth grade and are expected to begin reading to learn. In general, difficulties with reading comprehension are prevalent among students across countries. In the United States, 32% of students in the fourth grade and 24% of students in the eighth grade performed below the basic level on the National Assessment of Educational Progress (NAEP) reading test in 2017 (National Center for Educational Statistics [NCES], 2018). The proportion of children reading below the basic level is reported to be higher among children from families with low SES and from minority race/ethnicity groups, such as black and Hispanic children. In 2017, White fourth-grade students outperformed their Black peers with 26 scaled scores, and Hispanic students with 23 scaled scores (NCES, 2018). Recent assessments on NAEP also showed that the average reading score for second-language learners in eighth grade was 43 scaled scores lower than the average score for peers who are not second-language learners (NCES, 2018). The situation of low-level reading skills among students is similar in North America and several European countries (Organization for Economic Co-operation and Development [OECD], 2010a, 2010b). As children who lack a strong foundation of linguistic and reading comprehension skills are more likely to experience academic difficulties and drop-out from school, developing effective instructional practices is of the utmost importance to the field of education. The aim of this review was to improve our understanding of intervention studies targeting two core constructs: linguistic comprehension and reading comprehension. Linguistic comprehension is defined as the process by which lexical (i.e., word) information, sentences, and discourses are interpreted (Gough & Tunmer, 1986). It refers to the ability to understand oral language, often assessed by tests of vocabulary or listening comprehension (Bornstein, Hahn, Putnick, & Suwalsky, 2014; Foorman, Herrera, Petscher, Mitchell, & Truckenmiller, 2015; Klem et al., 2015; Melby-Lervåg & Lervåg, 2014). Vocabulary is a core component of linguistic comprehension. Vocabulary has typically been divided into either expressive and receptive vocabulary or depth and breadth vocabulary (Ouellette, 2006). However, several more recent studies, using latent variables, have shown that these are highly related constructs that are difficult to differentiate (e.g., Bornstein et al., 2014; Lervåg, Hulme, & Melby-Lervåg, 2017). Although vocabulary is a core component in linguistic comprehension, skills such as syntax (the ability to understand and formulate sentences) and morphology (how words are formed), which build directly on vocabulary knowledge, are also often considered to be part of a broader linguistic comprehension construct (e.g., Klem et al., 2015). Reading comprehension can be defined as the active extraction and construction of meaning from all kinds of texts (Snow, 2001). Linguistic comprehension is commonly understood as an important factor that underpins the development of reading comprehension beyond word-level reading (Gough & Tunmer, 1986). In later grades, when decoding skills are fully mastered and the contribution of decoding skills to reading comprehension has lessened, linguistic comprehension and reading comprehension are almost isomorphic constructs (Lervåg et al., 2017; Muter, Hulme, Snowling, & Stevenson, 2004; Storch & Whitehurst, 2002). The primary aim of this review was to provide an overview of studies on interventions targeting linguistic comprehension and their effects on measures of generalized linguistic comprehension skills. Because linguistic comprehension skills are understood to be a prerequisite for subsequent reading comprehension skills, the second aim of this review was to examine possible transfer effects from instruction to generalized reading comprehension outcomes. We know from earlier trials and prior reviews that children learn words that they have been directly instructed in. However, the evidence to which educational intervention programs can produce effects on generalized language and reading comprehension tests that are not targeted for specific intervention (distal effects) has been unclear. Moreover, it must be noted that the terms generalized linguistic comprehension and reading comprehension outcomes refer to tests that are not targeted for the specific intervention. This implies that it is the distal treatment effects that are of interest and that the outcomes are not inherent to treatment (e.g., standardized tests; see Cheung & Slavin, 2016). By strengthening our knowledge of this subject, we can potentially obtain insights into how related deficits can be ameliorated. This information is critical in making policy decisions regarding whether such programs are suitable for implementation in early childhood education and later schooling. In addition, reviewing intervention studies may also provide a more refined understanding of the underlying causal mechanisms through which interventions are effective. This aspect is vital for providing a sound theoretical foundation for constructing better and more targeted intervention programs. In general, intervention research that targets linguistic comprehension instruction typically targets groups of children who are at risk for language and reading comprehension difficulties. Samples of participants may represent classroom students from low socioeconomic areas, children with poor language skills who are selected based on screening tests for language proficiency, and samples with participants who have their educational language as a second language. The content of the intervention programs reviewed in this paper involves instruction in linguistic comprehension skills (e.g., vocabulary, grammar, and narrative skills). The intervention programs of interest aim, at an overall level, to provide children or students with rich exposures to language learning situations. The overall aim was to obtain the effects of linguistic instruction on generalized outcomes of linguistic comprehension and reading comprehension. Vocabulary instruction is the main building block in the design of linguistic comprehension instruction programs. Vocabulary knowledge serves as a proxy for the development of spoken language skills and plays a crucial role in the understanding of texts (Anderson & Freebody, 1981; Graves, 1986). Therefore, the focus on vocabulary instruction is highly valued in experimental studies among both preschool and school-aged children. Recognized theorists have identified principles for effective word teaching that are typically included as instructional features in the type of studies examined in this review: the instruction must provide definitional and contextual information and repeated exposures as well as facilitate active processing (e.g., McKeown & Beck, 2014; Stahl & Fairbanks, 1986). However, learning a number of word meanings does not necessarily provide children with the competence to acquire the knowledge of new words independently. Therefore, intervention studies typically provide direct word instruction as an embedded feature within a broader comprehensive program. Then, the aim is to obtain effects on generalized outcomes of linguistic comprehension and reading comprehension (both of which are the focus of this review). A commonly employed strategy to teach children words in intervention studies has been to provide children with direct instruction in word meanings through storybook reading. This direct vocabulary instruction has been practiced in various ways. One instructional approach is to provide children with brief explanations of word meanings during reading. This embedded vocabulary instruction targets the breadth of their vocabulary knowledge and has the benefit of being time-efficient, as it allows for the instruction of numerous word meanings during a training session (Coyne, McCoach, Loftus, Zipoli, & Kapp, 2009). Another instructional approach is to provide children with a rich instruction of words following storybook reading (Beck & McKeown, 2007). This includes providing multiple explanations and examples related to multiple contexts and letting children actively engage in the explanations and discussions of word meanings. This technique is expected to foster a child's depth of vocabulary knowledge, in contrast to increasing the number of word meanings that a child knows (Beck & McKeown, 2007; Coyne et al., 2009). In addition to the quantity of instruction and direct word instruction, experimental programs are typically designed based on principals for instructional quality of interactions and extended talk during activities (Snow, Tabors, & Dickinson, 2001). Selected topics and instruction of words are typically used as a gateway for discussions (Weizman & Snow, 2001). Shared book reading is a common recommended activity to support young children's language skills (Dickinson & Tabors, 2001), and a commonly used activity in experimental studies. Shared book reading implemented using the methodology known as dialogic reading (Whitehurst et al., 1994) provides an opportunity for the instruction of words that are presented in text along with opportunities to focus the instruction on active listening skills and building narrative competencies. Similarly, intervention programs for school-aged children typically value vocabulary instruction that contains explicit explanations within the context of both discussions and text reading (Lawrence, Crosson, Paré-Blagoev, & Snow, 2015) and utilizes instructional features to increase students’ word consciousness by building morphological awareness (Brinchmann, Hjetland, & Lyster, 2015; Lesaux, Kieffer, Kelley, & Harris, 2014). In closing, even though the principals of vocabulary instruction and principals for instruction are often aligned across studies, there may be differences among the studies in terms of which specific activities they focus on. Table 1 lists a few core activities that are often included, to varying degrees, in such trials. However, when examining program content across trials, it becomes evident that although it would be interesting to examine studies according to dimensions of instructional features or activities, it is not straightforward to separate studies into different types of instruction. In many of the intervention programs, particularly for preschool children, the teacher plays an important role in facilitating discussions during the intervention period. By providing definitions and examples, asking open-ended questions, asking for clarifications, and engaging the children in active talk, children are encouraged to utilize active listening skills and express themselves. As evident from many intervention programs, participating teachers are instructed in strategies for facilitating high-level discussions prior to the start of the intervention. The type of intervention programs that are included in this review provide children with language instruction sessions that are conducted in educational settings. This implies that the instruction programs can be considered as supplemental instruction, as the control group follows regular practice in preschools or school settings. However, studies may vary according to intensity (e.g., hours or days per weeks) and length of instruction (e.g., number of weeks). At least three important theoretical perspectives set the stage for this review and are important in the discussion about how the intervention might work: The development of linguistic comprehension; the relationship between linguistic comprehension and reading comprehension; and the mechanism of why linguistic comprehension instruction should lead to transfer effects on outcomes of generalized language and reading comprehension skills. The first perspective to be addressed is that linguistic comprehension appears to develop with a high degree of interdependence. Several cross-sectional and longitudinal studies using observed variables have indicated that expressive and receptive vocabulary, grammar and syntax, and verbal memory are related skills that reflect a common factor (Colledge, et al., 2002; Johnson et al., 1999; MacDonald & Christiansen, 2002; Pickering & Garrod, 2013). This hypothesis has gained more conclusive support in large-scale longitudinal studies that employ latent variables that correct for measurement errors: Bornstein et al. (2014) found a unitary core language construct from early childhood to adolescence. In addition, Klem et al. (2015) found a unidimensional latent language factor (defined by sentence repetition, vocabulary knowledge, and grammatical skills) in a longitudinal study of children aged 4–6 years. Further, recent studies that have included listening comprehension tests have also made arguments for a single language construct, in which different language assessment tools share a common variance. Justice et al. (2017) examined the development of language constructs in preschool through third-grade children and reported that the latent variables “oral language” (indicated by receptive and expressive vocabulary and syntax) and “listening comprehension” (indicated by tests assessing the ability to comprehend narrative and expository passages as well as inferential skills) appeared to assess the same underlying construct. Similarly, a study by Lervåg et al. (2017) found that a latent language factor defined by vocabulary, grammar, verbal working memory, and inference skills was a clear predictor of the variation in “listening comprehension” measured by oral comprehension tests (explaining 95% of the variation in listening comprehension). Overall, these findings suggest that different language outcomes share a lot of common variance and that language skills, across domains (e.g., vocabulary and grammar) and modalities (expressive and receptive), are supportive of each other in development. A second important issue is the robust longitudinal stability within the linguistic comprehension domain. A stable rank order of children's vocabulary knowledge is preserved during both preschool and later school years (Melby-Lervåg and Hulme, 2012; Storch & Whitehurst, 2002). The studies by Bornstein et al. (2014) and Klem et al. (2015) also indicate that the unitary core construct is highly stable over time. All these studies suggest that although all children's linguistic comprehension skills improve over time, the rank order between children is more or less preserved. This implies that altering children's language levels relative to other children is a complex and challenging endeavor. Nonetheless, as Bornstein et al. (2014) note, stability does not imply that it is impossible to change language skills through intervention. Thus, this review sheds light on important theoretical issues related to the nature of language learning, such as to what extent we—despite the high stability of linguistic comprehension and reading comprehension—can alter these skills and whether skills transfer from specific tasks integrated in the intervention to more generalized tasks in standardized tests. The second theoretical issue involves the relationship between our primary outcomes of linguistic comprehension and reading comprehension. How could improvement in linguistic comprehension transfer to reading comprehension? A close relationship between linguistic comprehension skills and the development of reading comprehension has been demonstrated in several longitudinal studies (Foorman et al. 2015; Lervåg et al., 2017; Torppa et al., 2016). Linguistic comprehension is a well-known precursor to reading comprehension success, and it develops long before formal reading instruction begins (Hjetland et al., 2018; Snow, Burns, & Griffin, 1998). These studies align with the Simple View of Reading, which is a well-established theoretical model of reading comprehension (Gough & Tunmer, 1986). This model presents reading comprehension as the product of decoding and linguistic comprehension skills and is formalized as the equation “Decoding × Linguistic comprehension = Reading Comprehension.” In this model, linguistic comprehension is an important underpinning in the development of reading comprehension beyond word-level reading (Gough & Tunmer, 1986). While decoding is an important predictor of reading skills in the early reading phase, linguistic comprehension is understood as an essential predictor for the further development of reading comprehension (Hoover & Gough, 1990; Muter et al., 2004; Storch & Whitehurst, 2002). Studies on both second-language learners and monolingual children with language delays have shown that the challenges they experience related to the understanding of texts are not characterized by a lack of decoding skills (Bowyer-Crane et al., 2008; Spencer, Quinn, & Wagner, 2014). This indicates the importance of fostering linguistic comprehension skills to ensure proficient reading comprehension development. However, notably, at an older age (when linguistic comprehension explains the majority of variation in reading comprehension), reading comprehension has also proven to be a highly stable construct (Lervåg & Aukrust, 2010). Theories on the nature of how and to what extent we can transfer what we learn are an important aspect of this review (see Bransford & Schwartz, 1999; Carraher & Schliemann, 2002). In this regard, two issues are at play: (a) the transfer of effects from criterion measures that contain the specific words that are used in the intervention to standardized tests of linguistic comprehension, and (b) the transfer of effects on linguistic comprehension to reading comprehension. Numerous studies indicate that children can easily be taught the meaning of novel words with which they are presented in an intervention (Elleman, Lindo, Morphy, & Compton, 2009). This phenomenon is often referred to as “near transfer.” However, in an intervention program, a child is typically presented with 3–6 novel words per week (Elleman et al., 2009). This amount is hardly sufficient to close the gap with children who have superior linguistic comprehension or the gap that exists between first- and second-language learners because the comparison children also continuously develop their language skills. For example, among studies that provided direct vocabulary instruction that was either embedded in story book reading or as a separate component, it is important to note that there have been no intervention studies that have taught over 150 words or that have lasted over 104 hours (at least up until 2009; Elleman et al., 2009). Thus, for the studies that do show positive effects on generalized measures (e.g., Bowyer-Crane et al., 2008; Clarke, Snowling, Truelove, & Hulme, 2010; Fricke, Bowyer-Crane, Haley, Hulme & Snowling, 2013), it is not likely that instructing specific definitions of words is the causal factor that underpins this improvement. It is most likely that there are other factors in the instruction that led to the gains on standardized measures. Language interventions must teach children skills that are transferrable so that they can use them for general language development. These strategies can then be used when they encounter new words and unfamiliar sentences and not merely for the specific words taught in the intervention. As Taatgen (2013) stated, “Transfer in education is not necessarily based on content and semantics but also on the