Frequency in Incidental Vocabulary Acquisition Research: An Undefined Concept and Some Consequences

Abstract

The most commonly examined variable in the growing body of literature on incidental second language (L2) vocabulary acquisition is frequency of exposure to target words. Earlier studies even ventured claims for minimum or optimal frequency of exposures needed in order for learners to acquire target words. More recent research addresses more nuanced questions with frequency still figuring prominently among the factors examined. In this article we show that the notion of frequency is less than straightforward, that it has unintentionally been put to use differently by different researchers, and that this difference across studies matters in several ways. We also elucidate specifically why it matters and suggest ways of bringing some coherence to the divergent practices within the research community. Implicit in any research that takes frequency as a variable are notions of identity across multiple tokens, that is, notions of what constitutes the same word encountered more than once. For any given text, the frequency of a word is the number of its tokens occurring in the text. Tokens, however, are not in and of themselves but are tokens by virtue of what they are tokens of, that is, by the type they betoken. We want to suggest that it is in this construed relation of token to type where differences can arise in research practice and that these differences matter. Say, for example, that I encounter the string create and the string creation each once on a page of running text. Have I encountered two tokens of the same single type, of the word create, cast broadly to include its derivational variants? Or have I encountered one token each of two different words, two different types, one the lemma create and the other the lemma creation? This depends on how abstractly or broadly I construe the notion of word as a type. Again, we will show that researchers diverge in exactly these ways of construing word type and, moreover, that such divergence is left implicit. In addition to yielding incommensurable acquisition research results, such implicit differences also make it impossible to address a basic question about the learners' perspective, one that has so far been left unasked. Do learners consider encounters with tokens to be repeated exposures to the same word or disparate exposures to different, unrelated words? While some researchers counting frequency of exposure may consider derivational variations within a word family to be tokens of the same word, tokens of one type, it is not clear whether learners do; whether, for example, they have the morphological wherewithal to detect these variants as repeated exposures to the same word. For learners who do not, we should ask what that says about calculating frequency in ways incommensurate with their perceptions. For learners who do, we should ask what is the evidence that tells us they do. In this article we show how practices in the operationalization of repeated tokens of target words vary across studies reported in the literature, and we trace some of the consequences of these divergent practices for evaluating the significance of this growing body of literature. We then recommend a way of operationalizing the counting of repeated tokens in the hope of standardizing how the concept of frequency is defined in this research community. Incidental vocabulary acquisition from reading refers to words being acquired while learners are reading for purposes other than learning vocabulary. Hulstijn (2001) defines incidental vocabulary acquisition in terms of experimental research design, suggesting it occurs when research subjects are asked to read a text without being made aware they will subsequently be assessed on vocabulary knowledge. Any words acquired during the reading are assumed to have been acquired incidentally. A small sampling of past studies researching incidental vocabulary acquisition from reading show they were based on different text genres including sentence-long contexts (Chen & Truscott, 2010; Heidari-Shahreza & Tavakoli, 2012; Webb, 2007, 2008), specially constructed paragraph readings (Negari & Rouhi, 2012; Rott, 1999), dramatic plays (Dupuy & Krashen, 1993), short stories (Day, Omura, & Hiramatsu, 1991; Hulstijn, Hollander, & Greidanus, 1996; Tekmen & Daloğlu, 2006; Zahar, Cobb, & Spada, 2001), expository texts (Hulstijn, 1992), comic books (Horst & Meara, 1999), academic texts (Vidal, 2011), graded readers (Brown, Waring, & Donkaewbua, 2008; Horst, Cobb, & Meara, 1998; Pigada & Schmitt, 2006; Waring & Takaki, 2003), and novels (Kweon & Kim, 2008; Lehmann, 2007; Pellicer-Sánchez & Schmitt, 2010; Pitts, White & Krashen, 1989). The results of these studies suggest varying degrees of correlation between frequency of occurrence of target words in the reading texts and their acquisition by subjects. Findings generally indicate modest amounts of vocabulary growth with estimates of the number of exposures to target words needed to acquire them ranging from 6 to more than 20 exposures, depending on the study (Waring & Nation, 2004). Target word frequency of exposure has been the main variable investigated. A primary motivation for this article, however, is the near-total absence from this growing literature of any descriptions of how frequency was calculated; that is, how researchers counted the tokens of target words. Only 2 out of 2511 Some articles report more than one study. studies cited above give any indication of how target word frequencies were calculated (Horst & Meara, 1999; Kweon & Kim, 2008). Attempts were made through e-mail to contact the authors of the remaining 20 articles cited above to seek additional information regarding how frequency was operationalized for their studies. We received responses from authors of 13 of the 20 articles. For three articles authors said they counted only the exact word form (M. A. Heidari-Shahreza, personal communication, June 3, 2013; M. Rouhi, personal communication, June 1, 2013; S. Rott, personal communication, July 7, 2013); for four articles the authors also counted inflected forms (J. Truscott, personal communication, May 31, 2013; S. Krashen, personal communication, July 8, 2013; S. Webb, July 13, 2013); for two articles authors counted both inflected and derived forms (T. Cobb, personal communication, June 12, 2013; A. Pellicer-Sánchez, personal communication, June 31, 2013); the data could not be located for three articles (J. Hulstijn, personal communication, January 12, 2013; N. Schmitt, personal communication, June 11, 2013); and finally one author said frequency was operationalized differently depending on the target word (M. Horst, personal communication, July 9, 2013). The responses to the e-mail inquiries indicate that incidental vocabulary acquisition researchers have differentially operationalized repeated tokens of target words. The decision for researchers to differentially count the tokens of target words may be indicative of their stance on L2 readers being more or less equipped at associating related forms of a target word as a single lemma. When Nagy and Anderson (1984) estimated the number of lexical items acquired by native speakers during a school year, they found it helpful to create word categories based on how dictionaries have traditionally treated words. Inflectional variants such as crash, crashing, and crashed have been treated as variants of the same word in many dictionaries under the headword crash. From an instructional point of view, an L2 learner that has mastered the basics of English verb inflection should not have problems associating the three forms. The same holds for plural inflection of countable nouns, such as daughter and daughters. On the other hand, dictionaries may or may not take additional types of relatedness into consideration. Some dictionaries list or cross-reference words with semantically transparent derivatives, associating for example fear and fearful. From the researchers' point of view, the ability to associate inflected and derived variants and vice versa is apparent; however, when considering the point of view of L2 readers, researchers should first ask what work must be done by an L2 reader in order to associate one form of a word (e.g., create) with a related form of that word (e.g., creation). For the L2 reader, is this one or two words? Brown (2013), for example, has drawn attention to the fact that the use of word family as a unit of counting frequency could be problematic in “that the application of word-building knowledge may have a limited role [for learners] and instead the frequency of individual word forms themselves may determine whether [words] are problematic … or not” (p. 1050). Moreover, Ward and Chuenjundaeng (2009) found L2 learners had difficulties associating related forms of words, especially words with classical or latinate etymologies. Previous incidental vocabulary acquisition studies may have adopted unclear criteria for relatedness when identifying recurrent tokens of the same target word. Lack of explicitness or clarity here, however, makes it impossible to examine, for example, the relation of morphological processes to acquisition that could potentially account for certain anomalies in the results found in previous studies. Therefore, we believe it is worth examining whether the notion of word identity across tokens has been operationalized consistently within and across studies in the literature. Since the core notion is left implicit in this literature, we must approach the question indirectly. Thus, we adopt three plausible and widely acknowledged degrees of word form variation and re-analyze data from published studies. From strict to permissive, the three units of word form variation among target word tokens are: (1) exact word form (thus create and created are not tokens of the same word), (2) lemma (thus create and created are tokens of the same word), (3) word family (thus create, created, and creativity are variants of the same word). From the target texts used in the studies, we generate three frequency counts, one for each of these construals of word form variation, and compare these counts with those reported in the published studies. Through this post hoc analysis we uncover a lack of consistency both within and among studies in how word frequency is calculated and trace some potential consequences of this inconsistency. Two criteria were used to identify studies to be included in our analysis. First, only published studies were included. Second, only studies reporting the frequency and/or acquisition rates for individual target words were included. It was only for these studies that we could determine both what construal of word type was used in calculating frequencies and the acquisition results for those words. Horst et al. (1998), for example, report individual target word data. A frequency of 12, for example, is reported for the target word furmity with 16 out of 34 participants having acquired it. Horst et al.'s (1998) study allows us to search the target text and confirm whether the 12 occurrences of furmity include inflected and derived variations of this target word. In contrast, while Kweon and Kim (2008) report that participants showed a 24.83% acquisition of the nouns appearing in a 7–19 frequency band, from their data it cannot be determined exactly which words were acquired and how many participants acquired them. Nor can we examine, as we can with the Horst et al. (1998) study, whether they allow for inflectional or derivational variation among tokens of the same word. A total of 6 out of 25 studies were found to meet our selection criteria. Although this leaves an admittedly small data set for further analysis, this fact in itself makes one of our main points: the inclusion of explicit data on individual word frequencies and acquisition rates in published research is regrettably rare. In recommending that researchers report individual target word data we have not overlooked the fact that publishers may limit the space provided to authors. Upon examination of 22 published articles we found that space did not seem to be the reason for not including individual acquisition results, because 14 of the articles provide the target words assessed but not the number of participants having acquired those target words. Furthermore, with the addition of online accessible supplementary materials, researchers can begin to make this type of data available to readers.22 The 22 articles reviewed are listed in an online supplement to this article. To examine whether practices of frequency counting vary across studies and what consequences follow if they do, we turn to the six studies that did supply the relevant data. Horst et al. (1998) report the frequency and acquisition rates for 45 target words that appear in the 21,232-token graded reader Mayor of Casterbridge (Jones, 1979). Zahar et al. (2001) and a replication of Zahar et al.'s (2001) study by Tekmen and Daloğlu (2006) report the frequency and acquisition rates for the same 30 target words that appear in the 2,383-token short story “The Golden Fleece.” Brown et al. (2008) report the frequency of 84 target words appearing in the 5,522-token novel One-Way Ticket (Bassett, 1991b), 5,765-token novel The Witches of Pendle (Akinyemi, 1994), and 5,415-token novel The Elephant Man (Vicary, 1989). Pellicer-Sánchez and Schmitt (2010) report the frequency of 34 target words appearing in the approximately 65,000-token novel Things Fall Apart (Achebe, 2001). Waring and Takaki (2003) report the frequency of 25 target words appearing in the 5,872-token novel A Little Princess (Bassett, 1991a). First, the tokens of the target words from target texts used in the published studies were counted and compared to cited frequencies. Three units of counting were used to generate three frequency counts for each target word: (1) exact word form, (2) lemma, and (3) word family. One of the authors and a research assistant independently counted and compared results for the three frequency counts of each target word to illustrate how differences in researchers' implicit definitions of target word repetition will generate different frequencies for the same target word. Next, we planned to use the individual target word acquisition results from the six studies to show how the use of frequency bands in reporting acquisition results can be misleading. Unfortunately, the acquisition results of target words from “The Golden Fleece” reported in Zahar et al. (2001) and Tekmen and Daloğlu (2006) refer only to participant posttest results (T. Cobb, personal communication, December 18, 2011; A. Daloğlu, personal communication, December 19, 2011) and the remaining three studies do not report acquisition results for individual target words. Therefore, only the individual target word acquisition results from Horst et al. (1998) could be used to highlight the possible disadvantages of using frequency bands to report target word acquisition results. A detailed analysis of the six studies that supplied the specific word frequency and acquisition data, using different construals of word type when counting target word tokens, yielded different frequency counts for target words. The tokens of the nine target words from “The Golden Fleece” that shift in frequency were examined to determine what degree of form variation was used in calculating target word frequency in the previous studies (see Table 1). It appears that inconsistent standards for counting were used by Zahar et al. (2001) and by Tekmen and Daloğlu (2006) in determining frequency for target words. For the words breath and fight only the exact word forms were counted. Yet, the frequency counts the authors cite for six words (centaur, crash, fearful, helmet, oracle, rush) are of lemmas. Furthermore, word family was used to count tokens for the target word ruler. Therefore, no consistent pattern of acceptance in variation of target word tokens could be determined. No clear pattern of accepted variation in word tokens could be found for 39 out of the 45 target words selected from Mayor of Casterbridge used by Horst et al. (1998) (see Table 2). Cited frequencies for five of the words (root, flame, swear, treat, wheat) could be arrived at only by counting tokens showing no inflectional or derivational variation, that is, tokens identical in word form. However, frequencies given for another 10 words (image, weary, cheek, willing, inquire, folk, lean, sorrow, burst, grain) were consistent only with counting tokens of lemmas, and the counts given for yet another 10 words (harvest, confuse, grave, affair, attempt, suffer, magistrate, maid, whisper, hay) could be arrived at only by counting tokens of word families. Furthermore, our counting found four words (stare, lean, inquire, confuse) with zero tokens appearing at the level of exact word form; this indicates that the form of the target word that appeared on the assessment and the form that appears in the target text were not the same. Two target words (lean, swear), both irregular verbs, have cited frequencies that were calculated using two different criteria. Tokens of leant were included for the frequency of the word lean but tokens of swore were not included for the frequency of the word swear. Lastly, we found eight words (ma'am, trade, skimmity, stare, dusk, relative, fellow, effect) having the same frequency for all three units of counting (exact word form, lemma, word family) but differing from the frequencies cited; no apparent reason attributable to variation in target word form could be determined. The tokens of the 35 out of 84 target words from the texts used by Brown et al. (2008) that vary in frequency were examined to determine what degree of form variation was used in calculating target word frequency (see Tables 3-5). From the alternative word forms cited, it appears Brown et al. counted word families to calculate frequency. Specifically, the counts given for five words (loud/ly, police, slowly, kind/ly, true/truth) could be arrived at only by counting tokens of word families. Still, some forms of target words that occurred in the text and were counted by the authors were not cited. For example, cited frequencies for 18 words (bag, bed, book/s, diamond, eye, friend, hole, holiday, horse, lady/ies, leg, nurse, picture, story/ies, tall/er, village, voice, woman/men) could be arrived at only by counting tokens of lemmas. The authors counted inflected forms for all 18 words but the alternative forms were indicated for only five of these words. Furthermore, the authors indicated alternative forms for three target words (day/s, door/s, quick/ly) whose inflected or derived forms did not occur in the text; the cited frequency counts for two of these words still matched ours. We also found three words with derived forms occurring in the texts whose frequency counts did not match those cited; the cited frequency for happy differed because the tokens of happily were not counted by the authors. We could not determine why our counts differed from those cited for the other two words (noise/y, shop/keeper). Our counting also found three words (chair, eye, leg) with zero tokens appearing at the level of same word form; as with other texts analyzed, this indicates that the form of the target word that appeared on the assessment and the form that appears in the target text were not the same. Lastly, we found nine words (cab, chair, difficult, drink, hat, judge, prison, quick/ly, warm) having the same frequency for all three units of counting but differing from the frequencies cited; no apparent reason attributable to variation in target word form could be determined. The tokens of 13 out of 34 target words from the texts used by Pellicer-Sánchez and Schmitt (2010) that vary in frequency were examined to determine what degree of form variation was used in calculating target word frequency (see Table 6). Frequencies cited for two words (ogwu, ozo) were consistent only with counting tokens of lemmas. Furthermore, our counting found three words with zero tokens appearing in the novel; this appears to be due to typographical errors (indichie for ndichie; uti for uri; ekwenzu for ekwensu). Lastly, we found seven words (egwuwu, foo-foo, ilo, iyi-uwa, nneka, obi, uli) having the same frequency for all three units of counting, but differing from the frequencies cited; no apparent reason attributable to variation in target word form could be determined. Although Waring and Takaki (2003) indicate lemmas were counted for some target words (e.g., year/s, house/s), no clear pattern of accepted variation in word tokens could be found for 12 out of the 25 target words selected from A Little Princess that yield a different frequency for the three construals of word type (see Table 7). Cited frequencies for one of the words (late) could be arrived at only by counting the exact word form. However, frequencies given for another six words (year/s, week, night, mine, house/s, face) were consistent only with counting tokens of lemmas. As with Brown et al. (2008), Waring and Takaki (2003) did not indicate alternative forms of tokens that were counted even when it appears they used alternative forms. The frequency for all three units of counting for two words (rich, beautiful) did not match those cited; it cannot be determined if variation in target word form was the cause for this difference in cited frequency. Lastly, we found three words (money, good, dead) having the same frequency for all three units of counting but differing from the frequencies cited; no apparent reason attributable to variation in target word form could be determined. It is an accepted practice in incidental vocabulary acquisition research to report the percentage of participants acquiring target words within frequency bands, which then may be used to recommend a threshold exposure number to encourage vocabulary acquisition. Frequency bands (based on Pellicer-Sánchez & Schmitt, 2010) and aggregated percentages for relative gain scores of target words from Horst et al. (1998) are reported in Table 8. The differences in results illustrate how dissimilar definitions of word repetition not only produce a difference in reported target word frequency but also affect the acquisition results when reported in frequency bands. This in turn may inadvertently trigger researchers to recommend different thresholds of exposure to target words. For example, a researcher using the text Mayor of Casterbridge who counts only repetitions of the exact form of target words would find 10 target words appearing in the 5–8 frequency band, whereas a researcher that allowed for inflectional variation in word form would find 16 target words, and a researcher that allowed for derivational variation in word form would find 24 such target words. As the degree of variation in form permitted shifts, so does the number of target words appearing in any given frequency band. Table 8 also reports results for the frequency of zero, indicating that the form of these target words in the assessment did not match the form of any of the occurrences of that word in the text. Based on the morphology of word variation, we recommended here an ontology that categorizes different ways of operationalizing target word form variation. The purpose is to make it possible to specify explicitly which degree of word form variation is being used in counting the frequency of recurrences of the same word in a text: same word form, same lemma, or same word family. It has been shown through post hoc analysis of individual target word data taken from published research that the criterion used in the counting of target word tokens in the studies examined was inconsistent. It has also been shown that the frequency of target words could change depending on the degree of variation permitted among tokens of the same word. Analysis of individual target word data showed cases where the form of the target word appearing in the assessment was not among the forms of the word used in the text. These cases may inadvertently be assessing more than just word acquisition, requiring, for example, the ability to recognize inflected or derived variations of a target word that the reader has never seen outside the assessment. To ensure that studies are investigating the same phenomenon, we recommend future studies operationalize repetition of target words by controlling for word form variation. Research methods should explicitly describe the level of variation permitted when reporting the frequencies of target words, for example by specifying whether tokens were all identical in form, were all inflectional variations of the same lemma, or inflectional and derivational variations of the same word family. Furthermore, individual target word data should be reported. Without access to individual target word data, a replication study could not compare results for individual target words. Publishing individual target word data allows for research into intricacies that are undetectable through aggregated mean scores. Furthermore, data from published studies could be used subsequently to investigate the effects of variables not originally considered by authors. Future research should further explore how variation in target word form may affect incidental acquisition of vocabulary. For example, in our analysis we found inconsistencies in counting tokens of irregular verbs. Lastly, when investigating the incidental acquisition of vocabulary through reading, we suggest that as a minimum researchers ensure that the form of the target word assessed is found within the text read by participants. It is worth exploring whether the amount of variation exhibited by target word tokens matters to L2 learners and whether this phenomenon exhibits any effect on the incidental acquisition of vocabulary through reading. Our analysis shows there is a need for attention to be placed on the issue of form variation across multiple tokens of target words that appear within texts that L2 learners read. However, the issue we are addressing is more than possible flawed methodological practices in previously published research. While it is important for researchers to ensure they are investigating the same phenomenon by including the operationalization of variables, they must also consider whether L2 readers perceive these variables in the same way as researchers have operationalized them. In other words, regardless of whether researchers agree on whether the three tokens create, creates, creation are one, two, or three words, it is imperative to determine how the L2 reader perceive these three tokens. In recent years incidental vocabulary acquisition as a research area has gained more attention as researchers have explored several variables in connection to frequency of exposure to target vocabulary. Unfortunately, what constitutes repeated exposure to target vocabulary has remained implicit and, in some cases, inconsistent. We have suggested a strategy for operationalizing repetition of target words that ensures the investigation of the same phenomenon across studies. In addition to providing explicit counting practices for target words, we hope future researchers will begin exploring what features of individual target words influence incidental acquisition from reading. This research is on the basis of the work supported and funded by Taiwan National Science Council Grant # NSC 100-2511-S-008-005-MY3. The authors thank Ms. Siou-Lan Wang for assistance with the counting of target words. Barry Lee Reynolds is an assistant professor in the Education Center for Humanities and Social Sciences at National Yang-Ming University, Taiwan. His research interests include incidental vocabulary acquisition, computer-assisted language learning, and second language writing instruction. His recent work examines what factors affect incidental vocabulary acquisition through reading. David Wible is a professor in the Graduate Institute of Learning and Instruction at National Central University, Taiwan. His theoretical research has been on comparative grammar, and his applied work has focused on how we learn words in a second language and on creating digital knowledge resources that can support this learning. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.