Grammatical Research Articles

A Comparative Study on ERNIE Bot 4.0 Turbo and ChatGPT 4o’s Performance in Evaluating First-Year Undergraduate Persuasive Essays

This study compares the performance of two prominent AI language models, ERNIE Bot 4.0 Turbo and ChatGPT 4o, in evaluating first-year undergraduate persuasive essays within the social sciences domain. Drawing from the Louvain Corpus of Native English Essays, a comprehensive collection of academic writings by British and American university students, this study aims to examine the models’ capabilities in assessing the grammatical correctness, vocabulary usage, coherence, content depth, and writing style of the essays. This study adopts a structured evaluation framework based on IELTS writing criteria to assess the models’ performance. A 40 persuasive essays from the Louvain Corpus were evaluated by both AI models and compared with human raters’ evaluations to ensure validity. The findings reveal distinct differences in the assessment styles of the two models. ChatGPT 4o exhibits a more critical approach, pinpointing areas for improvement, such as lack of argument development, coherence issues, and grammatical errors. Conversely, ERNIE Bot 4.0 Turbo offers a more balanced assessment, acknowledging essays’ strengths and suggesting improvement areas. Notably, ERNIE Bot’s evaluation highlights potential biases in AI-based assessment systems, particularly in its unequal emphasis on viewpoints. This comparative examination offers valuable perspectives on the advantages and constraints of AI models in assessing scholarly compositions, underscoring the significance of amalgamating varied AI functionalities to establish more all-inclusive and efficient feedback systems for learners. By understanding these differences, researchers and educators can better utilize AI-assisted essay evaluation systems to enhance student learning experiences.

Developing explicit and implicit L2 knowledge with Duolingo’s input-based features: Case studies of L2 Spanish learning

In this single-case study with a quasi-experimental design, we set out to examine the role of two input-based features of the language learning app Duolingo on learners’ development of implicit and explicit second language (L2) knowledge. We investigated the effect of Duolingo’s Stories feature and Podcast feature on two Spanish-L2 learners’ performance on a battery of implicit and explicit L2 knowledge tests. Once a week, over a four-week period using these input-based features, both participants repeatedly performed oral narrative tasks, timed and untimed grammaticality judgment tasks, and metalinguistic knowledge tests. The participant assigned to the Podcast condition showed meager gains in productive implicit knowledge (measured by oral narrative tasks) but decreased in receptive explicit knowledge (measured by untimed grammaticality judgment tasks). The participant assigned to the Stories condition showed a large increase in receptive implicit knowledge of Spanish (measured by timed grammaticality judgment tasks). Neither participant showed meaningful gains in the measure of productive explicit knowledge (measured by metalinguistic knowledge tasks). Although there was a slight increase in receptive explicit knowledge for the participant using Stories, the data from both participants discount the idea that app-based mobile-assisted language learning (MALL) prioritizes receptive explicit L2 knowledge, at least when considering these novel input-based features of Duolingo. These findings add nuance to previous research showing that the primary outcome of MALL app use is receptive explicit L2 knowledge.

Exploring the flexibility of word position encoding in Chinese reading: the role of transposition effects

ABSTRACT Previous research in the alphabetic writing system has demonstrated that transposition distance (adjacent vs. non-adjacent) modulates word position encoding during sentence reading. To examine the generality of this pattern within a more holistic model of sentence processing, we investigated this effect in the logographic Chinese writing system. We manipulated the number of words intervening between the transposed words, creating four conditions: none, one, two, and three intervening words. Participants performed a rapid grammaticality judgment task. Results showed longer response latencies and higher error rates when words were transposed adjacent to each other compared to non-adjacent transpositions. Furthermore, more errors and longer response times were observed when one word intervened between the transposed words compared to two or three intervening words. However, no significant differences emerged between the two- and three-word interval conditions. These findings suggest that word position encoding exhibits graded, flexible tolerance to transposition distance, constrained by proximity.

Thematic control and criteria-based assessment of foreign language writing skills using artificial intelligence technologies

Importance. Currently, there is a tendency that when teaching a foreign language, teachers are increasingly resorting to the use of artificial intelligence technologies to plan training sessions, generate educational content, as well as to conduct automated testing of formed communication skills. The monitoring of academic achievements and assessment is one of the key components of the organization of the educational process. Traditional methods of control and assessment require significant time and labor-intensive costs from the teacher, while artificial intelligence technologies make it possible to simplify and automate routine tasks: check tests and written papers, analyze them and identify mistakes, provide feedback. Thanks to the integration of natural language processing (NLP) technologies into chatbots and adaptive intelligent learning systems, it becomes possible on a daily basis to check texts created by students, evaluate them from the point of view of grammatical and lexical correctness, as well as identify stylistic and factual errors in real time. The purpose of this work is to test the applicability of artificial intelligence technologies for thematic control and criteria-based assessment of educational achievements using the example of productive and reproductive writing.Research Methods. In carrying out this study, the following groups of methods were used: theoretical methods aimed at familiarizing with scientific and methodological literature on the topic of research, analysis and classification of theoretical and methodological material for conducting thematic control and criterion assessment in foreign language classes, as well as empirical methods that allowed modeling pedagogical control and assessment processes using artificial intelligence technologies, observation, analysis and description of the results obtained.Definition of Concepts. The main concepts used in the study are “control of educational results” and its variety “thematic control”, “criterion assessment”.Results and Discussion. In the course of the study, various types of monitoring educational results using artificial intelligence technologies were considered: preliminary, current, intermediate, thematic, final. The choice of thematic control is due to the opportunity to trace its applicability within the framework of a single lesson in a foreign language and identify the main difficulties in using this form of control. The following criteria for evaluating writing skills are used to evaluate learning outcomes using artificial intelligence technologies: a) the structure of the written text; b) compliance with the main topic; c) coherence; d) relevance; e) grammatical correctness; f) lexical correctness; g) ethics of writing and stylistic correctness.Conclusion. Artificial intelligence technologies at the present stage have a high degree of adaptability and include a wide range of software and hardware solutions that allow for such important pedagogical procedures as monitoring educational achievements and evaluation in accordance with user-defined evaluation criteria. The obtained results are proposed to be used in research devoted to the study of modern methods of monitoring educational achievements in the methodology of teaching foreign languages using artificial intelligence technologies.

Far transfer of retrieval-practice benefits: rule-based learning as the underlying mechanism.

Benefits of self-testing for learning have been consistently shown for simple materials such as word lists learned by rote memorization. Considerably less evidence for such benefits exists for complex, more educationally relevant materials and its application to new situations. The present study explores the mechanisms underlying this transfer. To this end, a typical retrieval-practice-effect paradigm was applied to foster the learning of an artificial language. Participants either repeatedly studied grammatically correct exemplar sentences of the artificial language or engaged in a cloze test as the interim testafter learning. To assess far transfer, participants in both groups of restudy and retrieval practice engaged in a grammaticality judgment test after a delay of 5min and 1week. In addition, participants in both groups completed a final memory test (i.e., a cloze test identical to the initial test) 1week after learning. In addition to a long-term memory benefit of retrieval practice,results revealed alsoa retrieval-practicebenefit in the far-transfer test after the 1-week delay. The findings further support the view that far transfer is supported by learning the underlying grammatical rules as opposed to memorizing the material. Thus, retrieval practice is also effective for fostering learning of complex materials and, even more importantly, for promoting transfer of learning-a crucial goal in modern educational practices.

Exploring Metalinguistic Awareness in School-Aged Autistic Children: Insights from Grammatical Judgment.

Metalinguistic awareness, the ability to manipulate and reflect upon language, remains largely unexplored in the autistic population. To address this gap, this observational cross-sectional study examines the metalinguistic abilities of school-aged autistic children in comparison to neurotypical peers in a novel tablet-based Grammatical Judgment Task (GJT) of reduced linguistic complexity engaging two kinds of metacognitive resources. Children had to judge non-verbally whether pre-recorded sentences were grammatically correct or not, following the traditional GJT paradigm assessing metamorphosyntactic skills. In addition, sentences with anomalous meaning that were either grammatically correct or grammatically incorrect were introduced to test metasemantic knowledge. Findings reveal no difference in performance between the groups, with participants performing on average above chance level both on the sentences assessing mere metamorphosyntactic skills and on the sentences placing an additional demand on metasemantics. This study shows that autistic individuals are able to mobilize metalinguistic resources when tested via a task of reduced linguistic complexity.

The Competition Model and Adult Second Language Learning: Cross-Language Similarity and L2 Morphosyntax Learning

Adult second language learners arrive at the language learning situation with an already formed first language grammar system in place. The study of cross-language similarity across the first and second languages explores how the similarities and differences in the two languages make learning more or less difficult, particularly for adult learners. From a Competition Model (MacWhinney, 1997) framework, the authors and their colleagues have conducted research examining the relationship between cross-language similarity and adult second language learning and processing with actual learners and in training paradigms with naïve learners; converging methodological approaches have been used (eye tracking, event related brain potentials, self- paced reading, and grammaticality judgments). We review these studies and describe the predictions derived from the Competition Model framework as a function of whether grammatical constructions are formed similarly or differently in the two languages or are unique to the second language. We conclude with a description of Brian MacWhinney’s influence on this body of research.

A Study on the Grammaticality Judgement Competence and Individual Differences for Advanced Korean Learners

A Study on the Grammaticality Judgement Competence and Individual Differences for Advanced Korean Learners

Grammaticality judgements of tense by young adult speakers of Shona

This study examined grammaticality judgements of tense by 40 young adult native speakers of Shona during an online task. This study reports response accuracy and reaction times on the reading of time reference violations in which verb forms do not match a time frame previously set by an adverb. The judgements were elicited using 240 sentences with and without grammatical violations during an online reading task. Overall, in the experimental task, the native speakers performed below 78% across the four conditions, despite the sentences having received acceptability ratings of 1 = acceptable, 2 = unacceptable for grammatical and ungrammatical sentences, respectively. Sentences with the violation of the recent past by remote past tense verb had the lowest response accuracy scores. The participants also took the longest time to detect the errors in this same condition, meaning sentences with the recent past adverb context had the lowest response accuracy, with the sentence with the recent past context and remote past tense verb violation the most difficult to detect. The participants were quicker in detecting errors in the sentences with the remote adverb context and recent past tense verb. The findings demonstrate the utility of grammaticality judgement tasks for revealing language processing difficulties.

Can AI-Generated Clinical Vignettes in Japanese Be Used Medically and Linguistically?

Creating clinical vignettes requires considerable effort. Recent developments in generative artificial intelligence (AI) for natural language processing have been remarkable and may allow for the easy and immediate creation of diverse clinical vignettes. In this study, we evaluated the medical accuracy and grammatical correctness of AI-generated clinical vignettes in Japanese and verified their usefulness. Clinical vignettes were created using the generative AI model GPT-4-0613. The input prompts for the clinical vignettes specified the following seven elements: (1) age, (2) sex, (3) chief complaint and time course since onset, (4) physical findings, (5) examination results, (6) diagnosis, and (7) treatment course. The list of diseases integrated into the vignettes was based on 202 cases considered in the management of diseases and symptoms in Japan's Primary Care Physicians Training Program. The clinical vignettes were evaluated for medical and Japanese-language accuracy by three physicians using a five-point scale. A total score of 13 points or above was defined as "sufficiently beneficial and immediately usable with minor revisions," a score between 10 and 12 points was defined as "partly insufficient and in need of modifications," and a score of 9 points or below was defined as "insufficient." Regarding medical accuracy, of the 202 clinical vignettes, 118 scored 13 points or above, 78 scored between 10 and 12 points, and 6 scored 9 points or below. Regarding Japanese-language accuracy, 142 vignettes scored 13 points or above, 56 scored between 10 and 12 points, and 4 scored 9 points or below. Overall, 97% (196/202) of vignettes were available with some modifications. Overall, 97% of the clinical vignettes proved practically useful, based on confirmation and revision by Japanese medical physicians. Given the significant effort required by physicians to create vignettes without AI, using GPT is expected to greatly optimize this process.

Observation of the Art and Ideology of the Architectural Gate of the Great Mosque of Jami' Singaraja: Involvement of Hinduism, Islam, and the Kingdom of Buleleng

This research aims to uncover the complex process of integration and collaboration between the Kingdom of Buleleng, Hinduism and Islam, which is evident in the gate design of the Jami' Singaraja Grand Mosque. The paper uses a logical and concise structure that includes common academic sections, ensuring a clear and necessary flow of information and causal relationships between statements. It uses appropriate vocabulary for the particular subject and adheres to grammatical correctness, conventional structure, clear and logical structure, and balanced judgment. The objective was achieved by using three methods, namely Aditya Wardana's gate architecture method examines the aesthetic nuances and significance of the extrinsic layers associated with the construction of the gate under study, the semiotic method proposed by Riffaterre establishes the implied meaning of the physical form of the gate in question and the ideological method is used to analyze the results of the first two methods to provide a more comprehensive interpretation of the explicit and implied meanings. This study concludes that the gate architecture of the Jami' Singaraja Grand Mosque contributes to the negotiation between environmental sensitivity and the ethics of cultural beauty. This can be observed through three findings a symbol of social status that values responsible pluralism, the obligation to bring joy, maintain expressive values, and dissolve negative elements, a reminder of the guidelines of social life, inner purity, and psychological wisdom; and an ideology of togetherness for the Muslim profile that must pay attention to inclusive personality, mass mobilization for social care, and reproduction of individual piety.

From Periphery to Center: The Emotional Sequence of Cognitive Revision in L2 Writing Assessment.

Peripheral self-proofreading activities, such as grammatical correction, spelling checks, and word modifications, have traditionally been viewed as essential components of writing feedback. However, central revision tasks-including cognitive processing, critical thinking, and emotional sequencing-have received insufficient attention. These central revision activities are crucial for assessing the emotional flow of writing. This study seeks to explore central revision processes to uncover authentic issues faced during writing tasks. We analyzed the revision processes of 113 college students, aged 19 to 22, using Inputlog Version 6 software to evaluate emotional sequences based on 4,629 revision traces. The study revealed that emotional sequences in central revision could be identified through four key indicators: language proficiency, discourse coherence, perceptual pathway, and perceptual processing. We also observed that nuanced emotional sequences were associated with higher writing proficiency. These findings have significant implications for enhancing central revision practices in second language (L2) writing, aiming to improve the efficiency of writing revisions. They also provide valuable guidance for both educators and students, highlighting the importance of emotional sequencing in the writing process.

La scelta dell’ausiliare nei tempi composti: Variazione nell'interlingua di apprendenti ungheresi e inglesi di italiano L2/LS

The aim of this study is to identify some factors that determine variability in the interlanguage of L2/FL learners of Italian concerning auxiliary selection (essere ‘to be’ vs avere ‘to have’) in compound past tenses. Data were collected by means of a grammaticality judgement test and a binary choice sentence completion test. The participants were L1 English and Hungarian university students of various levels of proficiency (A2, B1, B2 and C1). The results were interpreted both in correlation with certain learning variables such as L1, proficiency level, and the category the main verbs belong to according to Sorace’s Auxiliary Selection Hierarchy. Based on the results, some hypotheses are put forward on teaching material design and on the possibility of harmonising the didactic intervention and the acquisitional sequence observed.

Metacognitive awareness and learning and performance goals as key predictors of writing performance

Effective writing involves more than just grammatical correctness and vocabulary choice. A deeper understanding of one’s thinking processes while writing, coupled with clear writing goals, is crucial for producing high-quality text. Thus, this study explores the correlation between students' metacognitive awareness, their orientation towards learning and performance goals, and their performance in English Academic Writing (EAW). Specifically, it investigates participants’ current metacognitive awareness and goal orientation in relation to their English Academic Writing. By examining participants’ understanding of their writing processes and goal they set, this research seeks to establish baseline for exploring potential connection between metacognition, goal orientation, and writing outcomes. The research employed a qualitative design using a survey involving 252 non-English major students across three universities in Indonesia. Participants completed two inventories; the Metacognitive Awareness Inventory (MAI) and the Goal Inventory (GI), along with summative assessments of their writing performance. Their English Academic Writing proficiency was evaluated through standardized assessment. As a result, multiple linear regression analysis indicated a significant positive impact of metacognitive awareness on EAW performance, explaining 73.5% of the variance with a regression coefficient of 1.171 (p < .001). However, learning and performance goals did not significantly affect EAW performance, as evidenced by a regression coefficient of 0.026 (p = .343). These results underscore the pivotal role of metacognitive awareness in enhancing EAW performance among non-English majors, suggesting that further research is needed to explore the influence of goal orientation, and to explore its potential moderating effects on the relationship between metacognitve awareness and EAW performance. Understanding the complex interplay of these factors is essential for developing effective pedagogical interventions to enchance EAW abilities among non-English major students.

Flexible word position coding in reading: Roles for attention and memory.

Readers may fail to notice the error in 'Do love you me?': this is the recently established transposed-word (TW) phenomenon. Word position coding is a novel cognitive construct, and researchers are presently debating the underlying mechanisms. Here I investigated roles for attention and memory. Participants (N = 54) made grammaticality judgements to four-word sequences that formed correct sentences ('The man can run', 'The dog was here'), TW sentences ('The can man run', 'The was dog here'), or ungrammatical control sentences ('The man dog run', 'The was can here'). Sequences were replaced by post-masks after 200ms, and that post-mask was accompanied by a 50-ms retro-cue in the form of an 'X' presented at a critical location (where one could have locally inferred grammaticality; e.g., between the first and second word of 'The was dog here') or a non-critical location (e.g., between the third and fourth word of 'The was dog here'). TW sentences were harder to reject than control sentences - the classic TW effect - and crucially, this effect was modulated by cue validity, with valid cues attenuating TW effects compared to invalid cues. The present results suggest that focused attention aids the process of binding words to locations. Furthermore, as cues appeared after sentence offset, these results suggest that word position coding may take place in memory.

Learning Transfer and Universal Grammar: The Case of Arabic as a Foreign Language

This Arabic as a Foreign Language (AFL) is considered a parsimonious field in applied linguistics. Therefore, this article tries to highlight how language transfer influences American students in their Arabic learning process. Additionally, this work attempts to explore whether Arabic foreign learners rely on their pre-existing knowledge to learn Arabic and determine the role of transfer regarding language universals. The study employs a quantitative methodology based on a Grammaticality Judgment Task (GJT) to achieve its objective. According to the study's results, L2 Arabic learners initially transfer their L1 setting to L2. In other words, Universal Grammar (UG) exists, but learners only have indirect access to it via their L1. By studying and investigating the influence of language transfer on learning outcomes, experts' and educators’ interventions can be tailored to more effectively assist foreign students in overcoming errors connected to their first language and improving their proficiency in learning Arabic language.

How do linguistic illusions arise? Rational inference and good-enough processing as competing latent processes within individuals

ABSTRACT Non-literal interpretations of implausible sentences such as The mother gave the candle the daughter have been taken as evidence for a rational error-correction mechanism that reconstructs the intended utterance from the ill-formed input (…gave the daughter the candle). However, the good-enough processing framework offers an alternative explanation: readers sometimes miss problematic aspects of sentences because they are only processing them superficially, which leads to acceptability illusions. As a synthesis of these accounts, I propose that conscious rational inferences about errors on the one hand and good-enough processing on the other are competing latent processes that simultaneously occur within the same comprehender. In support of this view, I present data from a two-dimensional grammaticality/interpretability judgment task with different types of subtly ill-formed sentences. Both conscious rational inference and good-enough processing predict positive interpretability judgments for such sentences, but only good-enough processing also predicts positive grammaticality judgments. By fitting a lognormal race model jointly to judgments and response latencies, I show that conscious rational inference and good-enough processing, as well as purely grammar-driven processing, actively trade off with each other during reading. Furthermore, individual differences measures reveal that participant traits such as linguistic pedantry, interpretational charity, and analytic/intuitive cognitive styles contribute to variability in the processing patterns.

How does musical rhythm influence grammatical processing at the neurophysiological level?

Numerous behavioral studies have demonstrated a rhythmic priming effect (RPE) on grammatical processing using grammaticality judgment tasks (GJT), where participants performed better following regular rhythmic sequences compared to baseline conditions or irregular rhythmic sequences (i.e. auditory rhythmic sequences with violated metrical structure). Only a few studies, however, have explored neurophysiological RPE in grammatical processing. Such neurophysiological investigations have been limited to GJT presented auditorily, have been primarily focused on the French- and German-speaking adult participants, and have rarely used baseline nonpriming conditions. The objective of the present study was to investigate neurophysiological correlates of the RPE in the GJT presented in visual modality. In the current study, we registered a 128-channel electroencephalogram while Russian-speaking adolescents performed a visual GJT, where each sentence was presented word by word in a self-paced manner. Before each experimental block, participants listened to regular rhythmic sequences, irregular rhythmic sequences, or silence. We observed that the late negativity in the event-related potential was larger for the ungrammatical condition compared to the grammatical condition only after the presentation of irregular rhythmic sequences. This effect, referred to as the N600 component in previous research, has been associated with increased cognitive complexity. In conclusion, results suggest that exposure to irregular rhythmic stimulation may lead to increased cognitive demand. This is attributed to the complexity associated with concurrently executing the GJT and managing rhythmic disruption, consequently increasing the strain on working memory resources.

Grammatical gender in spoken word recognition in school-age Spanish monolingual and Spanish-English bilingual children.

This study examined grammatical gender processing in school-aged children with varying levels of cumulative English exposure. Children participated in a visual world paradigm with a four-picture display where they heard a gendered article followed by a target noun and were in the context where all images were the same gender (same gender), where all of the distractor images were the opposite gender than the target noun (different gender), and where all of the distractor images were the opposite gender, but there was a mismatch in the gendered article and target noun pair. We investigated 51 children (aged 5;0-10;0) who were exposed to Spanish since infancy but varied in their amount of cumulative English exposure. In addition to the visual word paradigm, all children completed an article-noun naming task, a grammaticality judgment task, and standardized vocabulary tests. Parents reported on their child's cumulative English language exposure and current English language use. To investigate the time course of lexical facilitation effects, looks to the target were analyzed with a cluster-based permutation test. The results revealed that all children used gender in a facilitatory way (during the noun region), and comprehension was significantly inhibited when the article-noun pairing was ungrammatical rather than grammatical. Compared to children with less cumulative English exposure, children with more cumulative English exposure looked at the target noun significantly less often overall, and compared to younger children, older children looked at the target noun significantly more often overall. Additionally, children with lower cumulative English exposure looked at target nouns more in the different-gender condition than the same-gender condition for masculine items more than feminine items.

Does interlanguage grammar project intermediate categories? Evidence of partial representation from one‐substitution by Chinese learners of English

AbstractThis study investigates how prepositional phrases within English noun phrases (e.g., the student of physics from Italy) are represented and processed by advanced Chinese‐speaking learners of English (henceforth L2ers). Using grammaticality judgment tasks and self‐paced reading tasks, we aim to examine the offline performance and the real‐time processing of English NP‐internal PPs in relation to the head nouns. To this end, two groups of participants were instructed to provide judgments of testing materials and to respond on‐line to experimental stimuli containing the proform one. Results indicate that highly proficient L2ers processed English NP‐internal PPs utterly differently from English controls in that they demonstrated almost no sensitivity to the existence of PPs as complements following one, mostly without consciously distinguishing them from PP adjuncts. To account for L2ers’ response patterns, we propose a partial representation hypothesis (PRH), claiming that L2ers tend to disregard intermediate categories containing PP‐complements in English NPs, thus only partially representing syntactic constituents of the target language. As a structural explanation, PRH awaits further corroboration and/or falsification from more empirical studies.