Language Assessment Practices Research Articles

LANGUAGE ASSESSMENT LITERACY: A SIGNIFICANT SYSTEMATIC REVIEW

The importance of Language Assessment Literacy (LAL) in education has received considerable attention in recent years due to its crucial impact on learning and teaching. This article seeks to synthesize existing knowledge, identify research gaps, and offer insights into the complex nature of LAL in educational environments. This study employed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) approach. A comprehensive collection of LAL studies was obtained by systematically searching academic databases, specifically Scopus and Web of Science (WoS). The review includes peer-reviewed studies from the social sciences, using a systematic search strategy to find relevant articles. Six main themes were identified in the synthesis of final findings from a review of key themes based on a sample size of 35. These themes are as follows: (1) LAL Components; (2) Teachers' LAL Levels; (3) Knowledge, Perceptions, and Practices of Language Assessment; (4) The Development of LAL Measures; (5) Professional Development and Training Needs in LAL; and (6) LAL Impacts. This literature review offers a comprehensive synthesis of current research on LAL, providing valuable insights for educators, researchers, and policymakers. The findings contribute to the ongoing discussion on effective language assessment practices in diverse educational settings, providing a basis for further exploration and development of LAL.

Unveiling students test strategizing through the lens of Monitor Theory: Teaching insights

This paper explores the integration of Monitor Theory into the design and implementation of Test-Taking Strategy (TTS) instruction tasks. It critically analyzes the landscape of language test-taking strategies, addressing key issues such as the advisability of explicit teaching, methods for identification, and the impact of less beneficial strategies. Advocating for the teaching of test-taking strategies, the paper recommends the Monitor Theory as an optimal model for pedagogical integration. The exploration aims to provide valuable insights for refining language assessment practices, ensuring alignment with the principles of language acquisition. The content-independent nature of these strategies maintains test validity, offering students opportunities for more comprehensible input and enhancing critical thinking. It proposes a model that emphasizes natural language use, acquisition through natural inference, and feedback mechanisms to enhance language learning and test performance. Practical strategies such as interactive test reviews, critical thinking tasks, and real-world applications are suggested to optimize TTS instruction. By linking test-taking strategies with the acquisition-learning hypothesis, this paper offers a comprehensive approach to optimizing language assessment practices and promoting holistic language development. The discussion underscores the importance of authentic learning experiences, learner autonomy, and comprehensive coaching interventions in fostering genuine understanding and proficiency in language assessment contexts.

An investigation into the online language teaching and assessment practices during COVID-19

Due to a growing number of casualties, company and school closures, and a lack of social connections, the COVID-19 pandemic has resulted in a number of unanticipated challenges. As a result, to cope with these challenges online education served as a great way for bridging the distances through digital tools. With the purpose of an effective and quality online learning, all stakeholders involving active implementers of educational programs, namely teachers and students in addition to administrators should be aware of its benefits and drawbacks. To this end, the current study attempts to uncover the perspectives of ELT instructors about online language teaching and assessment through a qualitative research design. The results showed that online language assessment and teaching was considered as inevitable by all participants, thus making it necessary to employ efficient methods for designing and administering online assessment, and interpreting the scores. On the contrary, a long list of challenges and problems were noted by the respondents including academic misconduct, difficulty of proctoring, open to manipulation, validity and reliability concerns. The study ends with practical implications for teacher training programs and further research ideas.

Comparative Analysis of Chinese Language Proficiency Assessment: A Case Study of AP and IGCSE Examinations

With the growing global interest in learning Chinese, the proliferation of Chinese language exams by various institutions has become evident. This paper delves into a comparative analysis of the assessment systems of the College Board's AP Chinese Language and Culture Examination and the Cambridge IGCSE Chinese Examination across five dimensions: score categorization, paper composition, assessment methods, exam syllabus arrangement, and evaluation principles. The study aims to identify similarities and differences between the two exams and draw insights for language testing and assessment practices. The findings reveal that both exams provide valuable educational experiences and insights. Recommendations include refining assessment systems to comprehensively evaluate candidates' language proficiency, integrating cultural elements into examination materials, and leveraging modern technological means to enhance fairness in language testing and assessment processes.

Enhancing English writing competence in higher education: a comparative study of teacher-only assessment versus teacher and student self-assessment approaches

Despite the critical role of academic English writing proficiency in higher education, existing research has inadequately addressed the comparative impact of teacher-led and student-involved assessments on writing outcomes. To address the research gap, this study aimed to evaluate the effectiveness of 'Teacher-Only Assessment' compared to 'Combined Teacher and Student Self-Assessment' in English writing at a university in southern Thailand. Additionally, the study examined the viewpoints of students regarding the implementation of these approaches. It utilized a sequential explanatory research design in conjunction with a quasi-experimental design. The research instruments included a series of writing tasks, a teacher assessment rubric, a self-assessment form, and a questionnaire with open-ended questions. A group of 62 first-year English majors was divided into two groups: a control group and an experimental group, consisting of 31 students each. Both groups were taking an English Composition I course for 12 weeks and were assigned the same writing tasks. The experimental group was provided with both self-assessments and teacher evaluations, whereas the control group solely received teacher evaluations. The quantitative data was analyzed using an independent sample, while the qualitative data was examined through a thematic analysis. Quantitatively, the independent t-test results indicated that there was no statistically significant distinction in overall writing performance between both groups, except for narrative paragraphs. Moreover, the qualitative findings emphasized the merits of incorporating teacher-led and student-involved assessments. This study suggests various implications for English as a Foreign Language (EFL) instruction and assessment practices.

English language assessment in Indonesian senior high schools: Perceived challenges and prospective solutions

Numerous scholarly investigations have been conducted on English language assessment practices. Nevertheless, there is a lack of knowledge regarding the challenges and potential solutions related to implementing effective language assessment practices, particularly within Indonesian senior high schools. This study aims to investigate the perceived constraints and aspired prospective solutions associated with language assessment practices in Indonesian EFL classrooms. The data for this qualitative case study were collected by conducting in-depth interviews with fifteen EFL teachers from different senior high schools across South Sulawesi, Indonesia. Based on the thematic analysis of the data, this study revealed the perceived challenges to implementing English language assessment practices in three general categories: national policy-related issues, institution-related issues, and student-teacher-related issues. This study also revealed the need for leadership support to foster school culture support for innovative language assessment practices as the main aspired solution to implementing language assessment practices. These findings help improve understanding of the numerous factors that must be considered when putting English language assessment into practice, theoretically and practically.

Developing Language Assessment Literacy of Teachers: needs, effective factors, and constraints

In recent years, developing language assessment literacy (LAL) has become crucial for teachers in enhancing their teaching practices and improving students’ learning outcomes. Empirical research reported that there was a training need for formal education and professional development programs in assessment (Berry et al., 2017; Vogt & Tsagari, 2014). Therefore, this paper reports a systematic review of teachers’ language assessment literacy development: training needs, training effective factors, and language assessment practice constraints. The purpose of the review is to provide an updated overview of the empirical research on language assessment development. I utilized systematic review for social studies using the following steps: forming research questions, choosing the appropriate search terms and strategy, starting the literature search, developing inclusion criteria, determining relevant studies, and extracting data (Petticrew & Roberts, 2006). The search was limited to the 12 years’ publications because there was an increase attention on developing teacher’s language assessment literacy in these years. The selected studies were analyzed using the content analysis approach. The results suggested that teachers needed in-service training in different LAL content areas that varied according to the educational context needs in their countries. The results also revealed that teachers' experience and self-reflection needed to be considered in development programs and teachers should be involved in collaborative learning communities. The collaborative learning communities provided the needed professional support and give the chance for teachers to share their experience. Furthermore, the reality and constraints needed to be recognized as they influenced teachers' assessment practices. Finally, research implications and future research directions on teachers’ language assessment literacy are discussed.

Differences in Child Language Assessment Practices Between School-Based and Non-School-Based Speech-Language Pathologists: Results From a Nationwide Survey.

The purpose of this study was to compare child language assessment practices of speech-language pathologists (SLPs) working in school and nonschool settings to determine if their place of employment impacts the diagnostic decision-making process. School-based SLPs (e.g., direct service providers employed in preschool and/or K-12 schools; n = 382) and non-school-based SLPs (e.g., direct service providers employed in private practices, university clinics, and/or medical settings; n = 147) completed a web-based questionnaire. The questionnaire examined the types, frequency, and perceived importance of specific assessment tools and potential workplace factors that may impact their diagnostic decision-making process. Both school-based and non-school-based SLPs reported using a combination of assessment tools when evaluating children with potential language disorders. School-based SLPs tended to rank the frequency of use and importance of most assessment tools similarly, while non-school-based SLPs ranked interviews as the most frequently used and most important assessment tool. Statistically significant group differences indicated that school-based SLPs ranked the frequency of use and importance of standardized testing higher compared to their non-school-based counterparts. Also, school-based SLPs rated employment-based factors impacting diagnostic decision making higher compared to non-school-based SLPs. SLPs practicing in school-based settings seem to handle more employment-based factors that impact independent diagnostic decision making than SLPs working in different employment settings when assessing children for potential language disorders. Clinical recommendations are provided, and implications for implementation-based assessment research in child language are discussed.

Digitally-mediated Language Assessment Practice (D-LAP): Qualitative Case Studies of Four Thai EFL University Lecturers

The escalating incorporation of digital pedagogical technology in higher education, particularly in the post-pandemic period, posits a potential evolution in the paradigm of language instruction and assessment. The focal point of this research is to uncover the digitally-mediated language assessment practices (D-LAP) employed by Thai EFL lecturers in the university context. To fulfill this purpose, four EFL lecturers from Thai universities were purposefully selected to partake in classroom observations and subsequent stimulated recall sessions. The primary objective of these classroom observations was to shed light on the participants’ assessment practices. Furthermore, the subsequent stimulated recall sessions were orchestrated to delve deeper into the participants’ chosen assessment methods. The findings revealed that the majority of lecturers exhibited an intermediate degree of digital assessment literacy. This was primarily manifested in their propensity to implement assessment tasks within a digital platform and to collaboratively formulate performance indicators with their students. Additionally, it was discerned that their assessment tasks were predominantly crafted based on those presented in their sourcebooks in order to uphold the instructors’ consistency in all course sections. The primary intention of these assessment tasks was to assess students’ learning and diagnose their knowledge, employing a diverse assortment of assessment methods. This research endeavors to offer meaningful contributions towards the augmentation of professional development programs centered around digitally-mediated language assessment.

UK speech and language therapists' assessment of children's expressive language, and functional impairment and impact, following the CATALISE publications.

In 2016/17, the CATALISE Consortium published the results of a multinational and multidisciplinary Delphi consensus study, representing agreement among professionals about the definition and process of identification of children with Developmental Language Disorder (DLD) (Bishop etal., 2016, 2017). The extent to which the current clinical practice of UK speech and language therapists (SLTs) reflects the CATALISE consensus statements is unknown. To investigate how UK SLTs' expressive language assessment practices reflect the CATALISE documents' emphasis on the functional impairment and impact caused by DLD, by examining: whether multiple sources of assessment information are gathered; how standardised and non-standardised sources are combined in clinical decision-making, and how clinical observation and language sample analysis are utilised. An anonymous, online survey was carried out between August 2019 and January 2020. It was open to UK-based paediatric SLTs who assess children up to age 12 with unexplained difficulties using language. Questions probed different aspects of expressive language assessment which are referred to in the CATALISE consensus statements and supplementary comments, and asked about participants' familiarity with the CATALISE statements. Responses were analysed using simple descriptive statistics and content analysis. The questionnaire was completed by 104 participants, from all four regions of the United Kingdom, working in a range of clinical settings with different levels of professional experience of DLD. The findings indicate that clinical assessment practices broadly align with the CATALISE statements. Although clinicians carry out standardised assessments more frequently than other types of assessment, they also gather information from other sources and use this alongside standardised test scores to inform clinical decisions. Clinical observation and language sample analysis are commonly utilised to evaluate functional impairment and impact, along with parent/carer/teacher and child report. However, asking about the child's own perspective could be more widely utilised. The findings also highlight a lack of familiarity with the details of the CATALISE documents among two thirds of the participants. Assessment practices broadly align with the CATALISE statements, but there is a need for greater clarity regarding terminology and the assessment of functional language impairment and impact. This research should prompt discussion in the profession about how to further develop and adopt expressive language assessment practices which reflect the CATALISE consensus and support effective assessment. What is already known on the subject The CATALISE consortium documents on Developmental Language Disorder (DLD) were published in 2016/17. The extent to which expressive language assessment practice in the United Kingdom reflects the new definition and statements on assessment has not previously been investigated. What this paper adds to existing knowledge This survey indicates that speech and language therapists in the United Kingdom assessing children for DLD mostly balance standardised language test scores with other sources of information in clinical decision-making, and utilise clinical observation and language sample analysis to consider functional impairment and the impact of the language disorder. However, important questions are raised regarding the robustness and objectivity with which these key parameters are currently defined and evaluated. What are the potential or actual clinical implications of this work? Clinicians, individually and at service level, are encouraged to reflect on their assessment of functional impairment and the impact of language disorder and to take steps to incorporate this where necessary. Professional guidance and clinical tools to facilitate robust, objective assessment would support clinical practice that aligns with expert consensus.

What is the usage of the Brisbane Evidence Based Language Test in clinical practice?: A speech language therapy survey.

A number of practice barriers (e.g., time constraints, patient comorbidities and competing demands) exist as regards the evidence-based assessment of adult language within the acute hospital setting. There is need for an evidence-based, diagnostically validated, adaptable, comprehensive and efficient aphasia assessment. The Brisbane Evidence Based Language Test (EBLT) was developed to meet this need for a clinically appropriate and diagnostically robust assessment measure. Since the release of the EBLT, there has been no detailed investigation into speech and language therapists' (SLT) use and perceptions of the assessment. To inform future research and improvements to the EBLT, a better understanding of current language assessment practices, use and thoughts on the EBLT are required. To investigate SLTs' current language assessment practices; and to explore the current usage and future directions of the Brisbane EBLT. A convergent parallel mixed-methods study design with an electronic survey was utilized. The survey consisted of 16 multiple-choice, multiple-answer and free-text questions that explored respondents' perceptions of current language assessment measures, ideal language assessment features, Brisbane EBLT use, positives and negatives, and future directions for the EBLT. The survey was developed on Qualtrics and disseminated via email, social media and professional networks. Snowball sampling was used. Study inclusion criteria required participants to be qualified SLTs with clinical experience working with patients with aphasia. Quantitative data were analysed via descriptive and correlative statistics, and qualitative data were analysed via content analysis. The survey was completed by 115 SLTs from Australia, New Zealand, the United States, the UK, Canada, France, Lebanon and Belgium. Many respondents identified that a range of assessments is required to meet SLT clinical needs in the assessment of aphasia. Key desirable assessment features reported were: comprehensive, efficient, evidence-based, responsive and flexible. The EBLT was the most frequently used standardized measure reported by respondents (used by 78.63%). The EBLT reportedly has many positive features; however, respondents indicated dislike of the form layout, scoring and responsiveness. The majority of respondents indicated that the development of additional EBLT tests (94.29%) and additional cut-off scores (95.15%) would benefit their clinical practice. The study findings indicate that SLT assessment of language is complex and multifaceted. While the EBLT is reportedly used widely by SLTs, respondents identified areas for further research which would optimize the test's usability within their practice, to ultimately improve patient outcomes. What is already known on the subject Previous studies have explored existing SLT adult language assessment practices within acute settings; however, the most recent comprehensive study was completed in 2010. In 2020, the development, diagnostic validation, intra- and interrater reliability of the Brisbane EBLT were published. As of yet there has not been an investigation into clinician use and perspectives of the assessment since its release. What this paper adds to existing knowledge This study provides contemporary data about international aphasia assessment practices, as well as descriptive and qualitative information on the current use of the Brisbane EBLT, and the positives, negatives, and future directions for the assessment measure. What are the potential or actual clinical implications of this work? The study suggests that no one language assessment currently satisfies all SLTs' assessment needs. It additionally implies that a large proportion of SLTs believe that it is best practice to use multiple language assessment measures and select these based on the patient's presentation and context. Further research is required to aid the development of additional EBLT test versions and cut-off scores to improve SLT adult language assessment practices.

Empowering preservice English teachers with language assessment literacy concepts and practices: Application of Vygotskian concept‐based language instruction

AbstractLanguage teachers are expected to engage in classroom assessment practices and report on learners’ progress aligned with proficiency standards, as well as prepare learners for external examinations. The term “language assessment literacy” (LAL) describes what the constituency of language instructors needs to know about assessment matters. Traditionally, a top‐down perspective has been provided for language teachers, such as through textbooks and testing courses. However, the result of this top‐down, decontextualized approach is that teachers often seem unable to influence and change the policy. Accordingly, this article reports on a study framed within sociocultural theory, in which a group of preservice English teachers applied Gal'perin's model of systemic theoretical instruction, brought into second language education as concept‐based language instruction (C‐BLI), to internalize and apply LAL concepts in instructional contexts. Results of the study suggest that C‐BLI was more effective than a top‐down, textbook‐based teaching approach in facilitating preservice teachers’ understanding of LAL concepts. It also helped them become active agents of their own language‐testing practice. Implications of the study are trifold. First, the C‐BLI intervention empowered preservice teachers on the design of classroom‐based assessments. Second, it brought positive washback to the instructional plan and English curricula. Third, the intervention helped form the dialectical unity of theory and practice, which contributed to the teachers’ professional development in English‐as‐a‐foreign‐language contexts.

Future challenges and opportunities in language testing and assessment: Basic questions and principles at the forefront

In this invited Viewpoint on the occasion of the 40th anniversary of the journal Language Testing, I argue that at the core of future challenges and opportunities for the field—both in scholarly and operational respects—remain basic questions and principles in language testing and assessment. Despite the high levels of sophistication of issues looked into, and methodological and operational solutions found, outstanding concerns still amount to: what are we testing, how are we testing, and why are we testing? Guided by these questions, I call for more thorough and adequate language use domain definitions (and a suitable broadening of research and testing methodologies to determine these), more comprehensive operationalizations of these domain definitions (especially in the context of technology in language testing), and deeper considerations of test purposes/uses and of their connections with domain definitions. To achieve this, I maintain that the field needs to continue investing in the topics of validation, ethics, and language assessment literacy, and engaging with broader fields of enquiry such as (applied) linguistics. I also encourage a more synthetic look at the existing knowledge base in order to build on this, and further diversification of voices in language testing and assessment research and practice.

But who trains the language teacher educator who trains the language teacher? An empirical investigation of Chilean EFL teacher educators’ language assessment literacy

Research has shown that language teachers typically feel underprepared for assessment aspects of their job. One reason may relate to how teacher education programmes prepare future teachers in this area. Research insights into how and to what extent teacher educators train future language teachers in language assessment matters are scarce, however, as are insights into the language assessment literacy (LAL) of the teacher educators themselves. Additionally, while increasingly research insights are available on components that constitute LAL, how such components interrelate is largely unexplored. To help address these research gaps, we investigated the LAL of English as a Foreign Language teacher educators in Chile. Through interviews with 20 teacher educators and analysis of their language assessment materials, five LAL components were identified (language assessment knowledge, conceptions, context, practices, and learning), and two by-products of LAL (language assessor identity and self-efficacy). The components were found to interrelate in a complex manner, which we visualized with a model of concentric oval shapes, depicting how LAL is socially constructed (and re-constructed) from and for the specific context in which teacher educators’ practices are immersed. We discuss implications for LAL conceptualisations and for LAL research methodology.

Test-taker insights for language assessment policies and practices

Test-taker insights for language assessment policies and practices

Language assessment in Mexico: Exploring university language teachers' backgrounds, practices, and opinions

Assessment is a vital aspect of any language teaching program and yet, no uniform assessment criteria exist in Mexico. To explore the role of language assessment in higher education in the country, a study employing a mixed-method approach was conducted. The data discussed in this article are based on a survey of university language teachers in Mexico investigating their educational backgrounds, their language assessment practices, and their opinions towards language assessment. Interviews with language teachers and program coordinators provided further insight into the university language teachers' experiences and concerns regarding language assessment in the country. The article also includes an overview of Mexico's particular educational context as it contributes to the overall dissatisfaction with the outcomes of language instruction. Since the results of the study indicate that the majority of language teachers in Mexico are part-time professors operating without clear guidelines and many of them feel somewhat insecure about language assessment, it is suggested that specific language assessment competencies be formulated for these teachers which should be met in future teacher training programs. Assessing the students' language learning outcomes at the individual program level is subsequently proposed as a concrete measure to improve the language education outcomes in Mexico.

Developing content teachers’ language awareness through practitioner-researcher inquiry into student writing

This study describes an inquiry approach involving a practitioner-researcher partnership and examines its impact on teachers' capacities to conduct explicit instruction of disciplinary literacy. The partnership comprised iterative cycles of sessions that engaged content teachers in exploring their students’ language challenges using student writing as the stimulus of inquiry. Data from the inquiry sessions and lessons provided evidence of the impact of the inquiry on the teachers. The impact includes self-reported changes in their language sensitivity, beliefs and assessment practices as well as observable changes in their classroom instruction. The relationship between the impact and teacher language awareness are discussed.

Continued Concerns with Language Assessment Practices in Saudi Arabian English Education

Language assessment, viewed as a means to help instructors guide learners in effectively and successfully learning a language, plays an important role in the skills, competencies, and overall language-learning outcomes of second-language learners. This conceptual article reviews and synthesizes current language assessment practices and issues in English education in Saudi Arabia. It first highlights the overarching features of existing assessment methods practiced in English classrooms in Saudi Arabia, after which it discusses the factors underpinning those assessment practices and the detrimental effects of such practices on the development of Saudi learners of English as a foreign language. The article concludes by proposing a few pedagogical practices with the potential to change current language assessment practices in English classrooms in Saudi Arabia.

A Review of Language Testing and Assessment in Online Teaching

In line with the increased importance of language testing and assessment in online learning, the paper sets out to present an overview of the landscape of online language testing and assessment based on available scholarship in global and Vietnamese contexts. Several benefits of online language testing and assessment are reported, including promotion of learner autonomy, evaluation of students’ progress, and convenience. Methods of online language assessment encompass collecting evidences of learning and using rubrics. A number of problems with current language testing and assessment practices in an online environment are delineated, including cheating/plagiarism, preference for selected response items, lack of validity and reliability, and increased workload for teachers. Recommendations for relevant stakeholders including teachers and teacher education institutions to improve the quality of online language assessment are accordingly presented.

Rethinking bilingual language assessment: Considering implicit language acquisition mechanisms by means of pupillometry

In this methodological paper, a tool is introduced to the field of bilingual language assessment that has been used widely in other research domains, but has never been applied to the practice of language assessment: The investigation of pupillometric reactions to grammatical violations. The pupillometry approach presented here intends to identify bilingual children at risk for developmental language disorders (DLD) as early as their onset of acquisition of their second language (L2), namely within the first few months of their kindergarten attendance. It focuses the implicit language acquisition mechanisms and the ability to derive linguistic patterns out of a string of input, an ability that is said to be impaired in children with DLD. The approach combines concepts of dynamic and static assessment, and the target structure is an early clinical marker (subject verb agreement, SVA). A SVA-structure is repeatedly presented auditorily within grammatical stimuli in a familiarization phase, followed by a test phase including grammatical and ungrammatical stimuli. During this task, the children's changes in pupil diameters are monitored. It is hypothesized that children whose language processing mechanisms are intact may react to ungrammatical stimuli encountered in their input with surprise – a sign of a violation of their expectations. Therefore, different pupil reactions to grammatical and ungrammatical stimuli after only short exposure are supposed to represent intact language processing mechanisms, whereas similar pupil reactions to grammatical and ungrammatical stimuli are supposed to indicate a risk for DLD. By means of pupillometric observations, an identification of children at risk would be possible at the beginning of L2 acquisition.