Language Testing Research Articles

A further look at two grammatical measures from children’s language samples and their contribution to the diagnostic process

ABSTRACT Previous research has identified two measures derived from language sample analysis as having a high level of diagnostic accuracy for developmental language disorder (DLD): a verb-based measure, the Finite Verb Morphology Composite (FVMC) and a more comprehensive grammatical measure, the Sentence Point. In this study, we evaluated the sensitivity and specificity of these two measures using a new group of children with DLD. To determine whether these measures would likely add to diagnostic decision making if used in conjuncion with other tests of language, we also examined the relationship between scores on these two measures and scores on a standardized test with a grammatical emphasis. In Study 1, FVMC and Sentence Point scores were computed from the language samples of 22 four- and five-year-olds with DLD and 22 age-matched typically developing peers. Both measures showed very good sensitivity and specificity. In Study 2, we analyzed the FVMC and the Sentence Point correlations with the SPELT-P2 for the 22 children wtih DLD from Study 1 and for a larger group of 60 children with DLD. All correlations were very low and non-significant. Results suggest that the FVMC and Sentence Point could be part of a diagnostic battery for DLD as these measures demonstrate good sensitivity and specificity. Furthermore, the findings of very low correlations between these measures and the SPELT-P2 suggest that they can contribute unique information to the diagnostic process even when used in concert with standardized tests of a grammatical nature.

Exploring language assessment literacy and needs of English teachers at senior high school level

ABSTRACT This study explored the language assessment literacy (LAL) level and language testing and assessment (LTA) needs of 57 English teachers at senior high school level in Taiwan. An LAL test, three quantitative questionnaires and a qualitative survey were administered, conveying the overall LAL level and LTA needs of the participants. The qualitative survey then elicited their perceptions of and perspectives on classroom-based assessments and national high-stakes tests with regard to a newly implemented national curriculum (i.e., 108-Curriculum). The results show that the teachers lack adequate LAL to varying degrees, depending on their demographic background. Additionally, the participants identified some assessment topics, from large standardized testing to applying test results. They also reported their greatest training needs: providing feedback, finding teaching content, and assessing integrated skills. The teachers’ qualitative accounts generated five themes, from education in testing and assessment as a useful tool to the gap between classroom-based assessments and the national tests. The results not only refine assessment modules in the teacher education programme but suggest a direction for the assessment training of English teachers. The study concludes by suggesting future LAL investigations in the English-teaching context of Taiwan or wider international communities.

Comparative Analysis of Chinese Language Proficiency Assessment: A Case Study of AP and IGCSE Examinations

With the growing global interest in learning Chinese, the proliferation of Chinese language exams by various institutions has become evident. This paper delves into a comparative analysis of the assessment systems of the College Board's AP Chinese Language and Culture Examination and the Cambridge IGCSE Chinese Examination across five dimensions: score categorization, paper composition, assessment methods, exam syllabus arrangement, and evaluation principles. The study aims to identify similarities and differences between the two exams and draw insights for language testing and assessment practices. The findings reveal that both exams provide valuable educational experiences and insights. Recommendations include refining assessment systems to comprehensively evaluate candidates' language proficiency, integrating cultural elements into examination materials, and leveraging modern technological means to enhance fairness in language testing and assessment processes.

Language impairments in seropositive and seronegative autoimmune encephalitis.

Autoimmune encephalitis (AE) is a rare neuroinflammatory disease affecting the central nervous system. To examine language functions in patients with different subsets of AE consisting of seropositive and seronegative groups. Fifty-two patients were recruited from neurology departments in Melbourne, Australia, who met clinical criteria for possible AE. Language tests include the Naming Test from the Sydney Language Battery (SydBat), the semantic fluency trial from the Controlled Oral Word Association Test (COWAT), and the Vocabulary and Similarities subtests of the Weschler Abbreviated Scale of Intelligence-Second Edition. The results were standardised with normative data. The mean age of our cohort was 52.5years old, with the average time from hospital admission to recruitment being 38.41months. At an aggregate level, none of the mean language test z-scores were below normative data. At the patient level, impairment rates were 18.37% for COWAT (animals), 28.57% for SydBat (naming), 4.65% for Similarities, and 4.55% for Vocabulary. Chi-squared goodness of fit tests indicated that observed performances were significantly below expected performances for the SydBat (naming) test (p < 0.0001) and COWAT (animals) (p = 0.004). While, on average, language functions were within normal limits in patients with AE, but a subgroup exhibited lower performance in semantic fluency and visual confrontation naming, with impairment rates below expected norms. To advance understanding of language in chronic AE patients, exploring the impact of seizure burden, antiseizure medication use, and the relationship of language functions with other cognitive functions is crucial.

The Effects of Teacher-Centered and Student-Centered Approaches in TOEIC Reading Instruction

Teacher-centered instruction is prevalent in preparation courses for large-scale standardized language tests. Student-centered instruction, while known to be effective in enhancing communicative competence, has been considered ineffective and time-consuming to implement in preparation courses for large-scale standardized language tests. To investigate this assumption, a quasi-experimental study was conducted to investigate the effectiveness of two teaching approaches— teacher-centered and Jigsaw—on students’ Test of English for International Communication (TOEIC) Reading Comprehension Part Seven scores. The results showed that both groups experienced significant improvements from the pre-test to the post-test in their overall performance; however, there was no statistically significant difference between them. Furthermore, students in the Jigsaw group showed statistically significant improvements in all three passage types (single, double, and triple), while students in the teacher-centered group showed insignificant improvement in one of the three passage types (single passages). These findings suggest that both teaching approaches can effectively improve students’ TOEIC Reading Comprehension scores. The study adds to the body of research on teaching approaches in EFL TOEIC Reading Comprehension university classrooms. The findings suggest that teacher-centered and student-centered approaches can effectively improve students’ TOEIC Reading Comprehension scores.

Gender Differences in Early Child Development: Evidence from Large-Scale Studies of Very Young Children in Nine Countries

AbstractSome evidence suggests that there are significant gender gaps in early child development in low- and middle-income countries, with girls generally outperforming boys. However, few studies have tested for the existence of such gaps at a large scale. Our objective is to examine gender disparities in early child development in Brazil, Chile, Colombia, India, Indonesia, Nicaragua, Peru, Senegal, and Uruguay, with 26,055 children aged 7 to 48 months. We estimate gaps using cross-sectional studies with language, socioemotional, and motor skills development assessments. Consistent with small-sample findings, the data shows girls consistently outperformed boys on language tests (0.14 standard deviations) and socioemotional development (0.17 standard deviations), with differences consistent across all nine countries. There were no systematic differences by gender for motor development. We explored how family characteristics, health investments, or parent–child interactions influenced the gap. We did not find evidence that variation on these characteristics across children explained the gap. Our findings suggest that gender gaps in language and socioemotional development emerge very early in life.

Use of human–computer interactive games for the dynamic assessment of language skills of children with autism spectrum disorder

AbstractThis study explored the influence of the dynamic assessment of human–computer interactive games on children's language potential. Thirty‐seven special children aged 3–7 years were selected to participate in the study. They were divided into three groups according to their scores on the Autism Behaviour Checklist: (1) a non‐autism group, (2) a suspected autism group and (3) an autism group. In two stages of static and dynamic assessment, all three groups of children completed language tests under the conditions of no mediated prompt and mediated prompts, and the data generated in the game interactions were processed and analysed. The findings suggest the following: (1) static assessment may ignore children's language potential, while dynamic assessment can measure and differentiate children's potential language development ability; (2) children with ASD need more mediation prompts and are more likely to complete tasks with explicit mediation prompts; (3) children with ASD have weaker language potential, and children with suspected ASD or non‐ASD children have homogeneous language potential; and (4) assessors should pay timely attention to and maintain children's learning status and psychological changes. This study demonstrates the importance of human–computer interactive games as a dynamic assessment tool to tap into children's language potential and to determine their proximal developmental zone, which has important implications for teachers in understanding groups with different language abilities. Practitioner notesWhat is already known about the topic? The traditional language assessment paradigm (static assessment) can only evaluate the learners' current language development ability, while the short‐term development potential may not be observed. Different prompts in the mediation strategy can be used as a quantitative assessment paradigm in dynamic assessment. Much of the dynamic assessment in children with ASD focuses on language vocabulary learning outcomes and lacks measurement or quantification of language potential. What this paper adds? Presents a method for the dynamic assessment of a human–computer interactive language game suitable for children aged 3–7 years with ASD. Quantifies the potential level of language skills in children with varying degrees of ASD. Objectively and dynamically compares differences in language development potential between different groups. In dynamic assessment, children's learning status and psychological changes should be paid attention to and maintained in time. Implications for practice, research or policy This paper provides a concrete example of how human–computer interactive games can be used to dynamically assess the language potential of children with ASD. These findings highlight the importance of progressive scaffolds (mediated prompts) in children's language learning, which can help children identify and solve problems in a timely manner. For educational developers, language intervention materials should be prepared in a way that encompasses the different levels of language skills of children with different disabilities, that is, within the individual's zone of proximal development.

Pediatric Intraoperative Neurophysiologic Mapping and Monitoring in Brain Surgery.

Similar to adults, children undergoing brain surgery can significantly benefit from intraoperative neurophysiologic mapping and monitoring. Although young brains present the advantage of increased plasticity, during procedures in close proximity to eloquent regions, the risk of irreversible neurological compromise remains and can be lowered further by these techniques. More so, pathologies specific to the pediatric population, such as neurodevelopmental lesions, often result in medically refractory epilepsy. Thus, their successful surgical treatment also relies on accurate demarcation and resection of the epileptogenic zone, processes in which intraoperative electrocorticography is often employed. However, stemming from the development and maturation of the central and peripheral nervous systems as the child grows, intraoperative neurophysiologic testing in this population poses methodologic and interpretative challenges even to experienced clinical neurophysiologists. For example, it is difficult to perform awake craniotomies and language testing in the majority of pediatric patients. In addition, children may be more prone to intraoperative seizures and exhibit afterdischarges more frequently during functional mapping using electrical cortical stimulation because of high stimulation thresholds needed to depolarize immature cortex. Moreover, choice of anesthetic regimen and doses may be different in pediatric patients, as is the effect of these drugs on immature brain; these factors add additional complexity in terms of interpretation and analysis of neurophysiologic recordings. Below, we are describing the modalities commonly used during intraoperative neurophysiologic testing in pediatric brain surgery, with emphasis on age-specific clinical indications, methodology, and challenges.

KEY ASSESSMENT TEST MODIFICATION PROJECT PRINCIPLES AND PRACTICES OF LANGUAGE TESTING

This article presents data collection about the key features, principles and practices of language testing by modifying an available test to be valid, reliable, and practical for the target learner, including both theoretical and practical pictures of the process. Starting with studies on the target research, this work contains the learner assessment profile, the critique of an existing test, modified version of the chosen test together with appendix (material).

The impact of a Personal Learning Environment on Chinese postgraduates’ online self-regulated learning skills / Impacto de un Entorno Personal de Aprendizaje en las aptitudes de aprendizaje autorregulado en línea de estudiantes de posgrado en China

This study investigated the impact of Personal Learning Environments (PLE) on the development of self-regulated learning (SRL) skills among postgraduate learners in China. 213 participants, divided into an experimental group ( n = 113) using a researcher-designed PLE platform and a control group ( n = 100) employing face-to-face instruction, joined a course on International English Language Testing Systems (IELTS). A mixed-methods approach was adopted, utilizing the Online Self-Regulated Learning Questionnaire (OSLQ) to explore SRL experiences, and weekly goal-achievement reports. Results demonstrated improvement in SRL skills in both groups, with the experimental group exhibiting significantly greater gains. Quantitative and qualitative results revealed that participants excelled in goal fulfillment but struggled with time management, help-seeking and self-evaluation skills. The study confirmed the positive influence of the PLE on postgraduates’ SRL skills, with implications for online learning design and the potential for PLEs to effectively promote SRL skills in online learning contexts.

Effects of input text genre on Chinese young EFL learners’ performance on continuation tasks

This study examines how the genre of input text influences Chinese young EFL learners’ performance on continuation tasks. Participants were 30 students in Grade 9 and repeated-measures design was adopted to compare their performance on narrative and argumentative continuation tasks, in terms of writing quality, CAF indicators as well as alignment. Results showed that (i) genre had no significant influence on students’ scores on the two continuation tasks; (ii) argumentative continuation task brought about more complex sentences and words, as reflected by most indicators, but narrative continuation task led to relatively longer writing, and no significant difference was found for accuracy; (iii) students’ writings on argumentative task contained more 4-word phrases that were drawn from the input text, while the writings on narrative task were semantically more related to the reading input. This study provides another piece of validity evidence for employing continuation tasks in different genres in language tests and has implications for the writing teaching of Chinese young EFL learners.

Assessment of Language Function in Older Mandarin-Speaking Adults with Mild Cognitive Impairment using Multifaceted Language Tests.

Individuals with amnestic mild cognitive impairment (aMCI), especially for those with multidomain cognitive deficits, should be clinically examined for determining risk of developing Alzheimer's disease. English-speakers with aMCI exhibit language impairments mostly at the lexical-semantic level. Given that the language processing of Mandarin Chinese is different from that of alphabetic languages, whether previous findings for English-speakers with aMCI can be generalized to Mandarin Chinese speakers with aMCI remains unclear. This study examined the multifaceted language functions of Mandarin Chinese speakers with aMCI and compared them with those without cognitive impairment by using a newly developed language test battery. Twenty-three individuals with aMCI and 29 individuals without cognitive impairment were recruited. The new language test battery comprises five language domains (oral production, auditory and reading comprehension, reading aloud, repetition, and writing). Compared with the controls, the individuals with aMCI exhibited poorer performance in the oral production and auditory and reading comprehension domains, especially on tests involving effortful lexical and semantic processing. Moreover, the aMCI group made more semantic naming errors compared with their counterparts and tended to experience difficulty in processing items belonging to the categories of living objects. The pattern identified in the present study is similar to that of English-speaking individuals with aMCI across multiple language domains. Incorporating language tests involving lexical and semantic processing into clinical practice is essential and can help identify early language dysfunction in Mandarin Chinese speakers with aMCI.

From Coverage to Distribution: Exploring Lexical Features of the National Matriculation English Test

Lexical feature has long been a pivotal element of almost all high-stakes language tests. Since the implementation of the “Experimental Curriculum Criteria” in China, few studies have reported investigating lexical features of the ensuing National Matriculation English Test with corpus methodology, and notably none was conducted on word dispersion. To address this problem, Python programming was employed in the present study to perform a corpus-based two-way coverage and visualized distribution analysis between the National Matriculation English Test and Experimental Curriculum Criteria lexicon. It was found that: 1) text coverage of the National Matriculation English Test reached the minimal (95%) threshold yet not the optimal (98%) one for adequate comprehension; 2) word-list coverage of the Experimental Curriculum Criteria was disproportionate and insufficient, suggesting that a large volume (42.905%) of the prescribed lexicon has never been used during the 13 years of implementation; 3) a relatively few (N = 90) high-frequency words, most (74.444%) of which were significantly overused compared with their corresponding BNC frequency, constituted over half (51.403%) of the text coverage; and 4) a vast majority (93.333%) of high-frequency words was homogeneous in dispersion, confirming the overuse with fresh distribution evidence. The results are discussed in terms of implications for test development.

Narrative Discourse in Youth Offenders: Examining Individual Differences.

This study examined narrative discourse in youth offenders, focusing on quality of story retelling, story comprehension, critical thinking, and the use of complex syntax. The participants were 15 incarcerated adolescents, ages 13-18 years (Mage = 16 years). Each was evaluated at their detention center via Zoom, using a standardized language test and language samples that elicited narrative speaking with fables. After retelling a fable, the participant answered questions that examined story comprehension and critical thinking. Most participants performed below average on the standardized language test and had difficulty on one or more of the language sampling measures. Areas of concern included quality of story retelling, story comprehension, critical thinking, and the use of complex syntax. Had language sampling not been employed, those weaknesses might have been overlooked. When evaluating youth offenders, it is important to elicit language samples that can provide detailed information about an adolescent's ability to communicate for genuine purposes. In this regard, the tasks employed in the current study could be helpful to speech-language pathologists in working with youth offenders, enabling them to pinpoint deficits and offer targeted intervention.

Myanmar EFL Learners’ Perspectives, Structure, Reasoning and Literacy Practices of Argumentative Writing: A Needs Analysis Study

Enhancing proficiency in argumentative writing in English has always been a challenge for English as a Foreign Language (EFL) learners. Despite the widespread use of argumentative essays in international tests such as International English Language Testing System (IELTS) and Tests of English as a Foreign Language (TOEFL) in recent years, the learners usually fail to meet satisfactory achievement levels in the EFL context. This paper investigates the issue closely by reviewing Myanmar EFL learners’ needs, wants, and problems in developing argumentative writing. Although there has been some discussion on factors that potentially challenge EFL learners’ argumentative writing, more research is needed on the integrated assessment to cater to the learners’ needs. This analysis reports the results of a group of undergraduates (n=44) in Myanmar. This research was based on triangulation data, including sources from questionnaires administered to the students’ purposes, preferences, and challenges, and analyzing students’ argumentative writing structurally and qualitatively for reasoning. The findings reveal that students were eager to learn argumentative writing, particularly for their immediate needs. Though students were not well trained with a communicative teaching approach in their curriculum, they preferred to learn argumentative writing dialogically, using classroom debates. They favored learning future argumentative writing courses by communicating meaningfully. Regarding the challenges in argumentative writing, most students could not produce counter-arguments and rebuttals. Based on the data elicited from the students, this needs analysis proposed implementing the integrated learning-to-argue and arguing-to-learn instructions and using argumentative literacy practices to engage students in dialogic learning in the EFL context.

Quality Analysis of Indonesian Language Test Questions Class VII SMP Negeri 11 Ruteng Using Iteman Programme Version 4.3

The purpose of this study was to describe the validity, reliability, difficulty index, differentiation index, and the effectiveness of distractors on the even semester Indonesian language test questions of class VII SMP Negeri 11 Ruteng, Manggarai Regency in the 2022/2023 academic year using the Iteman version 4.3 programme. The method used was descriptive quantitative. The validity results show that the 40 items of multiple choice questions compiled are valid questions. The reliability results of 40 questions tested on 25 students obtained an Alpha Cronbach score of 0.788. Thus, the question is declared reliable because the amount of the reliability index is more than 0.60. The average index of difficulty (ITK) of all items is 0.433, with 9 questions at the easy level, 10 questions at the medium level, 12 questions at the difficult level, and 9 other questions at the very difficult level. The average index of discriminating power (IDB) is 0.246 with 13 questions declared feasible, 5 questions declared moderate (feasible and do not need to be revised), 7 questions declared good enough (feasible and need to be revised), 15 questions declared unfit. The results of the effectiveness of the distractors showed that 39 of the questions were functioning and 1 question option was not functioning

The Relationship between Physical Activity and Academic Achievement among Elementary School Children in Japan

Purpose: The aim of the study was to investigate the relationship between physical activity and academic achievement among elementary school children in Japan&#x0D; Methodology: This study adopted a desk methodology. A desk study research design is commonly known as secondary data collection. This is basically collecting data from existing resources preferably because of its low cost advantage as compared to a field research. Our current study looked into already published studies and reports as the data was easily accessed through online journals and libraries.&#x0D; Findings: This study examined the relationship between physical activity and academic achievement among elementary school children in Japan. The participants were 1,024 fifth-grade students from 29 public schools in Tokyo. Physical activity was assessed by a questionnaire and a pedometer, and academic achievement was measured by standardized tests of Japanese language, mathematics, and science. The results showed that physical activity was positively associated with academic achievement, after controlling for gender, socioeconomic status, and school-level factors.&#x0D; Unique Contribution to Theory, Practice and Policy: Cognitive enhancement theory, self-determination theory &amp; bioecological systems theory may be used to anchor future studies on the relationship between physical activity and academic achievement among elementary school children. Schools should prioritize high-quality physical education programs that include both structured and unstructured physical activities. Educational policymakers should consider including physical activity guidelines within the broader educational policy framework.

Exploring the relationship between language scores and writing performance on the ENEM exam: evidence from a brazilian micropolitan region

The objective of this study is to investigate the potential correlation between students' academic performance on the objective Language Test and their writing proficiency as assessed in the National High School Exam (ENEM). The primary goal is to ascertain whether the grade achieved in the native language serves as a reliable predictor of performance on the writing test. The sample for this analysis encompasses 4,704 high school graduates from the Médio Paraíba Region of the State of Rio de Janeiro who participated in the National High School Exam in 2019. This study employs a hypothesis test based on the correlation between the variables, complemented by the utilization of linear regression to determine the effect size. The findings reveal a Pearson correlation coefficient of merely 0.208, accompanied by an R² value of 0.04. Essentially, while a correlation exists between students' performance on the Languages test and the writing test of the ENEM, the effect size is notably small. This suggests that additional factors beyond language proficiency may offer more comprehensive explanations for students' performance variations on the ENEM writing test.

Reference Book for Pancasila Learning Based on 21st Century Skills and Tri Kaya Parisudha Values

Character education and learning outcomes for elementary school students are still relatively low. One factor that ensures student success is the role of the teacher. Therefore, the relevant effort to be carried out is to provide an understanding to students as prospective elementary school teachers in designing and implementing learning innovations to form a generation that is intelligent and has character by the demands of 21st-century learning and implementing the Tri Kaya Parisudha concept. This research aims to develop a 21st-century-based Pancasila learning reference book containing Tri Kaya Parisudha for Elementary School Teacher Education Study Program students. This research is design and product development research (Design and Development) using the ADDIE (Analysis, Design, Development, Implementation, Evaluation) model. The subjects in this research were two experts, ten students, and eight elementary school teachers. The data analysis method used is the descriptive analysis method. The research results show that 21st-century learning containing the Tri Kaya Parisudha in the Pancasila learning reference book has very good validity for students in the Elementary School Teacher Education Study Program. It is proven by the results of data analysis of the validity test of language and content experts that obtain a very good category. Likewise, the responses of students and teachers in elementary schools show an average score (mean) in the very good category.

THE EFFECT OF GREEK TRADITIONAL DANCES, SONGS, AND GAME-BASED ACTIVITIES ON ANXIETY AND ENGLISH LEARNING AT SCHOOL IN STUDENTS WITH INTELLECTUAL DISABILITY

The present study aimed to examine the effects of a program with teaching means Greek traditional dances, songs, and game-based activities on anxiety and English learning at school in students with intellectual disability (ID). The research involved 20 students with ID, who attended a Special Needs Vocational High School and were randomly divided into an intervention group (n=10) that was taught the English language with Greek traditional dances, songs, and game-based activities, and a control group (n=10) who participated in the standard English language course, for once a week for 45-minutes, for 12 weeks. Before and after the 12 weeks, the State-Trait Anxiety Inventory (Spielberger et al., 1970) was used to assess anxiety and specially designed cognitive tests to assess English learning. Before the beginning of the program, there was no difference between the two groups concerning the studied variables. However, after the program, the two groups showed significant differences in all variables (p&amp;lt;0.01-&amp;lt;0.001). More specifically, after the intervention program there was a significant decrease (z=-2.157, p&amp;lt;0.05) in anxiety and a very significant improvement of the students with ID in the cognitive tests for the lessons taught (p&amp;lt;0.01). For the control group, anxiety remained unchanged, while all cognitive test scores in the English language slightly improved except for the significant improvement in the unit prepositions (p&amp;lt;0.05). However, intervention group scores were significantly higher on all cognitive tests in the English language (p&amp;lt;0.01-&amp;lt;0.001). In conclusion, the positive effect of Greek traditional dances, songs, and game-based activities on anxiety and English language learning by students with ID is evident.&lt;p&gt; &lt;/p&gt;&lt;p&gt;&lt;strong&gt; Article visualizations:&lt;/strong&gt;&lt;/p&gt;&lt;p&gt;&lt;img src="/-counters-/soc/0509/a.php" alt="Hit counter" /&gt;&lt;/p&gt;