Learner Corpus Research Articles

From early to future learner corpus research

Abstract The aim of this article is to survey the field of learner corpus research from its origins to the present day and to provide some future perspectives. Key aspects of the field — learner corpus design and collection, learner corpus methodology, statistical analysis, research focus and links with related fields, in particular SLA, FLT and NLP — are compared in first-generation LCR, which extends from the late 1980s to 2000, and second-generation LCR, which covers the period from the early 2000s until today. The survey shows that the field has undergone major theoretical and methodological changes and considerably extended its range of applications. Future developments that are likely to gain ground are grouped into three categories: increased diversity, increased interdisciplinarity and increased automation.

Exploring the functions of lexical bundles in undergraduate opinion paragraphs for pedagogical use

Abstract Corpus-based studies of lexical bundles have opened new avenues for language teaching research. The fact that naturally occurring language consists of patterns of lexical repetition and multi-word units has given rise to the question of chunkiness in learner language. This study was designed to examine lexical bundles and their functions in a small, specialized learner corpus of opinion paragraphs written by Saudi English as a Foreign Language (EFL) students at the University of Jeddah. The study takes a frequency-driven approach to identify common lexical bundles. A learner corpus of 237 writing tasks produced by Saudi undergraduate students from 11 different sections is compiled and explored. The primary aim was to identify high-frequency five-word lexical bundles and explore their functions in the learner corpus, as well as investigate any distributional differences in bundle use across the various student sections. The findings revealed that learners utilized lexical bundles primarily to serve four key functions: expressing stance, supporting a point, introducing an item, and making recommendations. Notably, variations were observed in the distribution of these functional categories among the different student groups. The study concludes by outlining some pedagogical applications for educators and language practitioners, highlighting the value of learner corpus-informed approaches to enhancing learners’ awareness and mastery of lexical patterning in academic writing. By better understanding the role of formulaic language in learner production, instructors can tailor their teaching to more effectively support students’ linguistic development.

The Study on Sentence Difficulty of the Chinese Components

In recent years, reading comprehension has gradually become a proficiency indicator of interest in lexical and grammar. As sentences are the basic units of discourse structure, sentence difficulty is often applied to the study of text difficulty. Although there have been a number of studies on sentence difficulty, the lack of consistency in the indicators chosen or the discussion of specific grammatical issues have limited the research on sentence difficulty. Therefore, this study adopts a corpus-based approach, using a corpus as an objective and scientific data source. The study utilizes the Digital Platform for Chinese Grammar and the 8000 Chinese Words as important reference sources. Additionally, the CRIE 3.0 is employed to validate the texts and establish sentence difficulty indicators. However, due to the incomplete development of certain indicators in the "Chinese Grammar Digital Platform", the study refers to the Chinese Proficiency Grading Standards for International Chinese Language Education and Hanyu Shuiping Kaoshi to establish comprehensive sentence structure and sentence component difficulty indicators. Subsequently, the established difficulty indicators are validated by conducting comparative analyses using corpora as the basis. Native speaker corpora are used as benchmarks, while Mandarin learner corpora are used for comparison, and then validate objectively through the machine learning model. These validation aims to examine the validity and reliability of the selected indicators and establish a calculation method involving " level of grammar * point distribution ratio of grammar " to determine the difficulty indicators for Chinese sentences, Additionally, expert reliability is accessed to ensure the credibility of indicators.

Academic Word List and Cefr Levels: Profiling Academic Vocabulary in a Technical University Learner Corpus

Academic Word List and Cefr Levels: Profiling Academic Vocabulary in a Technical University Learner Corpus

The MuLeCo Project: A learner corpus of L1 German learners of Romance languages

The MuLeCo Project: A learner corpus of L1 German learners of Romance languages

From second language acquisition research to foreign language teaching through the prism of corpora

Corpus linguistics can be seen as a set of methods applicable to different branches of linguistics. This article deals with the application of corpus linguistics (including learner corpus research) to second language acquisition (SLA) research and foreign language teaching (FLT). It describes the existing links between these frameworks, arguing that the combinations usually work in pairs. By contrast, the concept of corpus-based theory-to-practice paths is introduced as a way of combining the three frameworks. More precisely, SLA research and FLT are brought together through the prism of corpora, which can provide data to test and possibly refine SLA hypotheses and then form the basis for pedagogical materials that can help solve some of the issues brought to light by the corpus analysis. Illustrations of such paths are given that centre around input-driven inductive learning, the natural order hypothesis and language transfer.

영어권 한국어 학습자의 보조사 ‘도’ 사용 발달 연구A Study of English-Speaking Korean Learners’ Acquisition of the Particle to “also”

ABSTRACT English-speaking learners of Korean frequently have difficulty acquiring the Korean particle “to,” which is translatable as also. This study conducts two analyses: an error analysis of English-speaking learners’ use of “to” and a comparison of the use of “to” by advanced learners and native speakers. First, our error analysis of the learner corpus finds that beginners frequently make errors in positioning “to” and in combining “to” with other particles; intermediate learners also have difficulty with positioning “to” correctly, as well as with using “to” to make parallel connections. Advanced learners still struggle with parallel connections, but their overall error rate is significantly less. Second, the study’s contrastive interlanguage analysis investigates whether advanced learners can use diverse polysemous meanings of the particle “to” and whether they can use complex grammatical patterns, including “to,” as proficiently as native speakers. Compared to native speakers, advanced learners use two meanings less frequently: an “extreme presentation” meaning, where “to” can be translated as even; and an “emphasis” meaning, which lacks a single-word English translation. Our analysis also found that advanced learners use two grammatical patterns less frequently than native speakers: complex combinations of particles including “to,” and adding the meaning of “to” to a predicate.

The Reliability of using ChatGPT in Rating EFL Writings

This paper explores the reliability of using ChatGPT in evaluating EFL writing by assessing its intra- and inter-rater reliability. Eighty-two compositions were randomly sampled from the Written English Corpus of Chinese Learners. These compositions were rated by three experienced raters with regard to ‘language’, ‘content’, and ‘organization’. The writing samples were also rated by ChatGPT twice over some time, and the average scores were calculated. Independent samples t-test was conducted to compare the average scores given by ChatGPT and human raters. Pearson correlation analyses were conducted between the two sets of overall scores given by ChatGPT to calculate the intra-rater reliability, as well as between average scores given by ChatGPT and human raters for inter-rater reliability. The results of comparative analysis shows that ChatGPT may be used for evaluating EFL essays, as the scores are similar to those provided by reliable human raters. However, the result of correlation analyses shows that the intra-rater reliability of ChatGPT is not h igh enough to be acceptable, r=0.575, p<0.01 and the strength of the inter-rater reliability is moderate as well, r=0.508, p<0.01. Besides, there is no significant relationship between their average scores on ‘organization’ of the writings, r=0.181, p>0.05. Thus, it can be concluded that ChatGPT is not a reliable tool to rate and score EFL writings using the prompt in this study. One of the possible reasons for the unreliability of ChatGPT as a rater of EFL writing seems to be related to scoring for the ‘organization’ of the essay. These findings imply that while ChatGPT has potential as an evaluative tool, its current limitations, particularly in assessing organization, must be addressed before it can be reliably used in educational settings.

Analysis of Syntactic and Lexical Complexity in a Learner Corpus

Analysis of Syntactic and Lexical Complexity in a Learner Corpus

Triangulating learner corpus and online experimental data: Evidence from gender agreement and relative clauses in L2 Greek

AbstractThe article introduces triangulation to converge evidence from corpus and experimental data, by means of two case studies in second language (L2) learners of Greek. The first case study investigates the acquisition of gender agreement, while the second probes the development of relative clauses. In both studies, findings from the corpus are tested against online experimentation using eye‐tracking and self‐paced reading tasks, a combination that is scarce in research implementing triangulation. The findings suggest that methodological convergence yields both congruous and incongruous evidence. However, both types of evidence contribute to a nuanced understanding of the linguistic phenomena under study, as well as the different perspectives from which they are approached. The article concludes that such an approach to triangulation can significantly contribute to enhancing the reliability and validity of the findings, provided that methodological convergence has been achieved at the design level of the study.

Seeking research funding in a peripheral context: A learner corpus genre study of grant proposal summaries

Although the grant proposal is a high-stakes genre for researchers, there are few analyses available for consultation by learners and most studies investigate only a limited number of successful proposals written by experienced academics. This study reports on a genre analysis of a learner corpus of grant proposal summaries (abstracts), written by researchers who operate at the periphery of academia. The proposals were written by exiled Syrian academics and submitted to the Council for At-Risk Academics (Cara) grant awarding body for research funding. A corpus of 102 proposal summaries was compiled consisting of 27 successful and 75 unsuccessful summaries, and a genre framework of three moves and ten steps was developed. Successful summaries were contrasted with unsuccessful summaries; this comparison reveals that unsuccessful summaries underuse the move Indicating the value of the research. Specifically, they tend to omit two steps: Importance (of the research) and Research Outcomes. All Cara summaries were also compared with Matzler's (2021) prototype; results show that both successful and unsuccessful summaries underuse the Methods step. These findings provide pointers to the genre functions likely to be most problematic for learners, and have immediate practical applications in pedagogic materials for proposal writing.

Usage-based approaches to assessing syntactic sophistication in second language writing: Interaction of genre and proficiency

Based on usage-based approaches to language learning, this study investigated how second language (L2) writers use verbs and argument structure constructions across different genres, while also exploring how this genre effect interacts with L2 writing proficiency. We analyzed five previously established usage-based indices using a learner corpus comprising argumentative and narrative essays elicited from 3196 college-level learners. Through a series of mixed-effects regression analyses, we identified a consistent genre effect, with argumentative essays featuring a more sophisticated use of verbs and constructions alongside a greater diversity in construction types. Additionally, a significant interaction emerged between genre and proficiency regarding construction frequency and the frequency of verb–construction combinations. These results indicate that while the previously observed genre effect extends to the L2 use of verbs and constructions, its manifestation varies depending on L2 proficiency. Overall, our study supports the usage-based learning of verbs and constructions in the context of L2 writing.

Crowdsourced Comparative Judgement for Evaluating Learner Texts: How Reliable are Judges Recruited from an Online Crowdsourcing Platform?

Abstract Recent studies of proficiency measurement and reporting practices in applied linguists have revealed widespread use of unsatisfactory practices such as the use of proxy measures of proficiency in place of explicit tests. Learner corpus research is one specific area affected by this problem: few learner corpora contain reliable, valid evaluations of text proficiency. This has led to calls for the development of new L2 writing proficiency measures for use in research contexts. Answering this call, a recent study by Paquot et al. (2022) generated assessments of learner corpus texts using a community-driven approach in which judges, recruited from the linguistic community, conducted assessments using comparative judgement. Although the approach generated reliable assessments, its practical use is limited because linguists are not always available to contribute to data collections. This paper, therefore, explores an alternative approach, in which judges are recruited through a crowdsourcing platform. We find that assessments generated in this way can reach near identical levels of reliability and concurrent validity to those produced by members of the linguistic community.

Causation in a Learner Corpus of Hungarian-English Summaries of Disease Descriptions

Causation in a Learner Corpus of Hungarian-English Summaries of Disease Descriptions

Prompt response variation in learner corpus research: Implications for data interpretation

While general first language corpora are composed of samples from various naturalistic sources (e.g., websites, books), language samples in most written learner corpora (LC) are texts produced in response to prompts. In this context, LC users need to develop a clear awareness of the affordances and limitations of specific prompts and how responses to said prompts may affect the investigation of their intended object(s) of study. Through an analysis of the presence/absence of specific Spanish verb tenses in texts written in response to two supposedly narrative prompts in a Spanish LC (COWS-L2H; Yamada et al., 2020), this article illustrates the impact of inter- and intra-prompt response variation on LC data interpretation. Based on this evidence, we caution against rapid assumptions about text content based solely on the superficial phrasing of LC writing prompts. Instead, we recommend that LC users perform in-depth quantitative and qualitative analyses of learners’ samples written in response to each prompt they aim to include in their study prior to running statistical models on those data.

Exemplification in student essay writing: A study of learner corpus of essay writing (LCEW)

AbstractIn academic writing, exemplification plays a crucial role in contextualizing complex ideational material through instances the readers can understand. In addition to illustrating ideas through concrete instances, the act of providing examples serves the purpose of helping the readers grasp the writer's intentions. However, strategically performing exemplification to elaborate the propositional material seems to be a challenge for many novice student writers. Although some studies have mentioned that students use significantly less frequently the exemplification resources in their writing, fewer studies have probed into EFL student writing to determine the problems they face in elaborating the ideas. Using the marker approach, which examines the discourse functions bottom‐up from markers to moves, the learner corpus of essay writing (LCEW) was analyzed for three major forms of exemplification: representation, argumentation, and analogy. The results indicate that the examples are strictly limited to certain patterns like specifying concepts through a subcategory and illustrating the arguments through everyday experiences. Moreover, many examples deviate from the usual patterns of exemplification causing confusion. These findings have pedagogic implications for academic writing courses in the EFL context.

An Investigation Into English Learners’ Speech Acts Using a Semi-Automatic Annotation Tool

This paper explores the speech acts most frequently performed by learners of English as a foreign language (EFL) through the Dialogue Annotation and Research Tool (DART), a semi-automatic annotation tool. The study used learner corpus research and an English as the first language (L1) speaker corpus as a reference to compare the speech acts performed by the EFL learners. The study involved 90 EFL learners of four nationalities. A spoken learner corpus was built with the dyadic interlanguage English conversation register. The most frequently performed speech acts were generally different for the English L1 speakers and the EFL learners. The speech acts labeled as correctSelf, referProcess, expressNonAwareness, and stateReason were in the top 10 for EFL learners, whereas they were not in the top 10 for the English L1 speakers. This difference was caused by three factors: the English proficiency of the EFL learners, the frequently used formulaic phrases, and the task requirements. Moreover, the corpus annotation has a problem with unrecognized speech acts, primarily due to the redundancies and fragments in the learner discourses. The findings of this study reveal the complementary relationship between grammatical and pragmatic performances.

A refined and concise model of indices for quantitatively measuring lexical richness of Chinese university students’ EFL writing

In the existing literature, scholars have proposed various indices to measure the lexical richness (LR) of English as a foreign language (EFL) writing. However, there are currently issues of redundant indices and inconsistent usage. Attempting to address the research question of which indices are the most sensitive and effective ones to distinguish between different grade levels of Chinese university students’ EFL writing, this study aims to put forward a refined and concise model of indices that can truthfully reflect LR in EFL writing. A total of 180 compositions were selected from a Chinese EFL learner corpus: <i>Spoken and written English corpus of Chinese learners</i>. Scores of 28 LR indices of these compositions were computed using the software <i>Lexical Complexity Analyzer</i>,<i> MATTR</i>,<i> </i>and<i> Coh-Metrix</i>. One-way ANOVA or Welch’s ANOVA, depending on the variable’s homogeneity of variances, was conducted for each index. Two criteria were applied to determine which index of a measure should be included in the refined model: whether the difference of an index is significant among different grade levels and the effect size of ANOVA. Based on the quantitative results of ANOVAs and qualitative human judgment based on literature, six indices of the six LR measures were included in the refined model: lexical density, lexical sophistication-I, verb sophistication-II, number of different words-expected sequence 50, corrected TTR, and squared verb variation-I. This refined model addresses the issues of redundancy and inconsistency in previous studies, providing a more accurate and efficient tool for assessing LR in EFL writing.

AI 기반 피드백의 교육적 활용 가능성 탐색

This study examines the potential of utilizing natural language processing technology to provide objective and efficient feedback in Korean language classrooms through an automated feedback system. To validate the effectiveness of AI-based automated feedback, the study compared corrections made by human annotators with those generated by the AI model, GPT4, using a corpus of beginner Korean language learners. The results indicated high concordance rates between human and GPT4 corrections, with Level 1 accuracy at 85.71% and Level 2 accuracy at 84.07%. Corrections primarily focused on grammatical corrections, averaging 61%, while lexical corrections comprised about 39%. Corrections across various parts of speech, including nouns, verbs, and adjectives, were performed at similar rates by both humans and GPT4. However, GPT4 showed limitations in appropriately correcting errors that required contextual understanding. Despite these limitations, the AI-based feedback system can be an effective supplementary tool in educational settings. Teachers can reduce their workload and focus more on content and structural feedback, while learners benefit from real-time, personalized feedback that encourages self-review and improves language proficiency. This study underscores the importance of AI-based educational tools and emphasizes the need for an integrated approach involving teacher collaboration to continually enhance the AI-based feedback system.

English Adjectives Used by Non-natives in ICNALE Spoken Dialogues

The purpose of this study is to examine English adjectives found in spoken language corpus of English learners in Asia. This study employed theories of adjectives from language typology perspective by Dixon (2010) and Frawley (1992). Descriptive-qualitative approach was applied, using ICNALE Spoken Dialogues as the data source. This study utilized the corpus analysis tools AntConc. The results identified the top 20 adjectives in the corpus, with Value, Human Propensity, and Difficulty each representing 20% of the findings. This study has implications for the significance of foreign language learning. Teachers can use adjectives from these categories to teach students, as they are basic vocabularies for English learners. Students are also expected to gain a better understanding of the functional aspect of adjectives, which could improve their language performance. The interest of this study lies in the high frequency of occurrence in the corpus. However, some limitations should also be considered.