Teacher Assessment Literacy Research Articles

Students' engagement across a typology of teacher feedback practices.

The provision of feedback is widely practised as part of formative assessment. However, studies that examine the impact of feedback are usually from the teachers’ perspective, focusing on why and how they provide feedback. Fewer studies examine feedback from the students’ perspective, especially in the way they experience, make sense of and take up their teachers’ feedback. This paper provides empirical evidence of student engagement with different patterns of teacher feedback in their written essays. Data were gathered from 45 students (from 5 different schools) through group interviews and analysis of student artefacts from three rounds of writing tasks. The findings on affective, behavioural and cognitive engagement surfaced the conditions that will contribute to students’ will and skill to act on their teachers’ feedback. The implications on both teacher and student assessment literacy are discussed. The discussion will provide professional development providers and policy makers with new perspectives of and approaches to strengthening formative assessment practices in ways that are more cognizant of students’ experience of feedback.

Education reform in Kazakhstan: Ways of teacher assessment literacy development

Introduction. The reform of many national education systems is associated with a growing focus on assessment as the dominant aspect of the modern educational process. One of the directions of the reform of the Kazakh school system is focused on the transition to criteria-based assessment of the educational achievements of schoolchildren. In this regard, for Kazakhstan, as well as for many countries of the world, the problem of developing evaluative literacy of both practising teachers and today’s students studying in pedagogical educational programmes has become especially acute.Aim. The aim of the current research is to reveal the essential characteristics of assessment literacy as an important quality of a modern teacher, to actualise the problem of its development in the situation of reforming the school education system, to outline possible directions for solving this problem.Methodology and research methods. The present research is based on the complex use of theoretical and empirical methods. The theoretical research was carried out using a comparative aspect analysis of scientific and methodological sources and normative documents on the problem of the formation of teacher assessment literacy. Empirical data were obtained when conducting a survey of school teachers, systematising and summarising its results.Results and scientific novelty. In the course of theoretical analysis, the authors identified the factors, which actualise the problem of teacher assessment literacy development; its main characteristics are summarised, which made it possible for the authors to formulate the definition of this concept. The results of the survey of school teachers demonstrated that shortterm professional development courses for teachers provide a knowledge component of their assessment literacy, but do not solve the problem of developing practical skills, which are in demand in the course of assessment activities. The development of teacher assessment literacy to the behavioural level is a multi-stage and complex process that involves not only theoretical training, but also the independent design of one’s own assessment activity, taking into account the subject characteristics and other contexts accompanying educational practice. Particular attention should be paid to the formation and development of assessment literacy of future teachers during their studies at the university. An integrated approach to solving this problem should provide: the formation of deep knowledge in the chosen subject area, participation in various assessment practices within the framework of methodological courses and pedagogical practices, interaction with practising teachers.The practical significance of the study lies in the recommendations that determine possible directions for solving the problem of developing school teacher assessment literacy in the context of reforming the educational assessment system.

Inside Teacher Assessment Decision-Making: From Judgement Gestalts to Assessment Pathways

Assessment decision-making is an integral part of teacher practice. Issues related to its trustworthiness have always been a major area of concern, particularly variability and consistency of teacher judgment. While there has been extensive research on factors affecting variability, little is understood about the cognitive processes that work to improve the trustworthiness of assessment. Even in an educational system like Australia, where teacher-based assessment in mainstream classrooms is widespread, it has only been relatively recently that there have been initiatives to enhance the trustworthiness of teacher assessment of English as a second or additional language (EAL). To date, how teachers make their decisions in assessing student oral language development has not been well studied. This paper reports on the nature and dynamics of teacher decision-making as part of a larger study aimed at exploring variability of teacher-based assessment when using the oral assessment tasks and protocols developed as part of the Victorian project, Tools to Enhance Assessment Literacy for Teachers of English as an Additional Language (TEAL). Employing a mixed-method research approach, this study investigated the assessment judgements of 12 experienced NSW primary and secondary EAL teachers through survey, assessment activity, think-aloud protocols and individual follow-up interviews. The paper highlights the key role of teachers’ first impressions, or judgement Gestalts, in forming holistic appraisals shaping subsequent assessment decision-making pathways. Based on the data, a model identifying three assessment decision-making pathways is proposed which provides a new lens for understanding differences in teachers’ final assessment judgements of student oral language performances and their relative trustworthiness. Implications of the model for assessment theory and practice, teacher education, and future research are discussed.

The Impact of Student Teachers’ Pre-Existing Conceptions of Assessment on the Development of Language Assessment Literacy within an LTA Course

The paper proposes the preparation of a new generation of assessment literate teachers. The issues of student assessment literacy and, more specifically, prospective language teacher assessment literacy have not been sufficiently investigated as of yet, although research into the topic seems to have gained momentum. Recent studies state that the assessment literacy of teachers is essentially affected by their pre-existing conceptions of assessments), and teacher education should integrate shaping such conceptions into courses; the process of shaping conceptions is quite long and, because it is time-consuming, it may deter assessment literacy building. The current study explores the conceptions of the assessments shaped by prospective teachers within a general English course. The two major conceptions of assessment, relevant for the framework of teaching general English to second-year student teachers of English, are the understanding of feedback and knowledge of assessment construct and criteria. The findings of the study in this cohort of students of the particular course in language assessment shows that the students’ progress was considerably higher than that of a comparison group in the previous 2020 study. The author suggests two types of AL, i.e., student and prospective teacher assessment literacy.

Language Assessment Literacy of Middle School English Teachers in Mexico

Because English is an integral component of education in Mexico, it is necessary to explore teachers’ language assessment literacy (LAL), or their language assessment knowledge and practices. Previous LAL studies have been performed in standardized testing-focused. However, the present study aims to explore the LAL of teachers of middle school English students in a context where governmental policies strive to engender communicative language learning, specifically, Mexico. I have taken a mixed methods approach which involved a survey (N = 123) and interviews at two locations in Mexico. The semi-structured interviews were conducted at one bilingual middle school (N = 7) and in one extracurricular English program (N = 6). Participants were asked about their previous training, confidence levels in their assessment practices, and their training needs. Findings suggest a divide among teachers with higher and lower levels of LAL. Participants indicated that their training primarily covered traditional forms of assessment and classroom-level decision-making. However, data also suggest that participants valued non-traditional assessment activities. Finally, participants desired training on topics such as the use of technology in assessment, unfamiliar non-traditional assessment activities, and program-level decision-making.

Urban public-school teachers’ assessment literacy in China’s Zhengzhou city

ABSTRACT Since the turn of the century, China has shown its determination to change its examination-oriented tradition. It started assessment reforms and raised a series of expectations regarding teacher assessment literacy. At the same time, China experienced rapid internal migration, with urban public schools accepting huge numbers of migrant students in recent years. This study explored urban public-school teachers’ assessment literacy in China’s Zhengzhou city. The teaching of nine teachers from nine urban public schools were observed and follow-up interviews conducted between 2018 and 2019. The study provides insights into the contextual challenges that face teachers with regard to assessment in urban public schools in China.

Critical language assessment literacy of EFL teachers: Scale construction and validation

Critical language assessment (CLA) has been addressed in numerous studies. However, the majority of the studies have overlooked the need for a practical framework to measure the CLA dimension of teachers’ language assessment literacy (LAL). This gap prompted us to develop and validate a critical language assessment literacy (CLAL) scale to further underscore the role of CLA principles and their practice as an essential part of teachers’ LAL. In the first phase, a pool of items was generated through a comprehensive review of the related studies. In the quantitative phase, the developed scale was administered to 255 English as a foreign language teachers selected through convenience and snowball sampling. The data were analyzed through exploratory factor analysis for construct validity and Cronbach’s alpha for estimating internal consistency. The results showed that the items loaded on five factors: (a) teachers’ knowledge of assessment objectives, scopes, and types; (b) assessment use consequences; (c) fairness; (d) assessment policies; and (e) national policy and ideology. It was found that the scale had a high level of internal consistency and construct validity, which suggests that this scale has the potential to be useful in assessing language teachers’ CLAL and to raise language teachers’ awareness of CLAL constructs.

Teacher Assessment Literacy: Implications for Diagnostic Assessment Systems

ABSTRACT Assessments scored with diagnostic models are increasingly popular because they provide fine-grained information about student achievement. Because of differences in how diagnostic assessments are scored and how results are used, the information teachers must know to interpret and use results may differ from concepts traditionally included in assessment literacy trainings for assessments that produce a raw or scale score. In this study, we connect assessment literacy and score reporting literature to understand teachers’ assessment literacy in a diagnostic assessment context as demonstrated by responses to focus groups and surveys. Results summarize teachers’ descriptions of fundamental diagnostic assessment concepts, understanding of the diagnostic assessment and results produced, and how diagnostic assessment results influence their instructional decision-making. Teachers understood how to use results and were comfortable using the term mastery when interpreting score report contents and planning next instruction. However, teachers were unsure how mastery was calculated and some misinterpreted mastery as representing a percent correct rather than a probability value. We share implications for others implementing large-scale diagnostic assessments or designing score reports for these systems.

Language assessment literacy of teachers in an English medium of instruction university: Implications for ELT training in Pakistan

In Pakistan, English is the official language and a high level of proficiency is considered a passport to success (Manan et al., 2016). However, little is known about Pakistani English teachers’ assessment knowledge and training needs. This study investigates the language assessment literacy of English language teachers at a multi-campus English Medium Instruction university in Pakistan. The aims were (1) to evaluate whether the EAP teachers were equipped, academically and professionally, to design and conduct language assessment; (2) to document teachers’ language assessment challenges in context and (3) to consult teachers on their training needs in assessment. Data comprised questionnaires, semi-structured interviews, and ELT curriculum documents. The analysis revealed little or no training in LAL, large classes, a crowded curriculum, and mixed student abilities impacting on language assessment practices. Implications are discussed for the training needed to support teachers’ language assessment practices that underpin student success in EMI programs.

TEACHER ASSESSMENT LITERACY: INDONESIAN EFL SECONDARY TEACHERS' SELF-PERCEIVED ON CLASSROOM-BASED ASSESSMENT PRACTICE

Assessing students' performance is an essential part of a teacher's job. Hence, they are believed to be familiar with various kinds of assessments, especially classroom-based assessments (henceforth CBA). Language teaching and assessment are two interrelated that affect each other to a great extent. Undoubtedly, language assessment becomes a more complicated activity in the field of language teaching. The major objective of this study was to describe how EFL secondary teachers’ self-perceived on the basic principles of assessment and their own practice in CBA. This study was conducted in the form of a descriptive survey research. Shim's (2009) survey instrument was adapted to collect data and gathered using an online survey. 48 respondents willingly fulfilled the online survey containing 80 statements concerning classroom assessment procedures. The data obtained were analyzed using descriptive statistics. The findings of the current study revealed that teachers seemed to practice assessment for learning (henceforth AfL), in which most teachers conducted assessments to support students’ learning and used the results as feedback to improve and revise their teaching. The teachers in this study appeared to be literate and excellent in understanding the concept and using it in practice. However, a question remains as to whether their classroom practice was excellent. Since most studies have largely reported results on teacher's assessment literacy (henceforth TAL), future programs and research should be directed toward classroom-based research on how the CBAL can improve students' learning and improve the quality of teacher instructions. The implications of the result are expected to provide a clear understanding and theoretical foundations of CBAL.

The Role of the Teacher as Assessor: Developing Student Teachers’ Assessment Identity

The closure of schools across the globe due to the Covid-19 pandemic had the potential to have a catastrophic impact on a fundamental pillar of initial teacher education: school placement. This paper maps a new “site” of professional practice for “school placement” called “Teacher Online Programme” (TOP) using Xu and Brown’s (2016) conceptual framework of teacher assessment literacy in practice. Its main focus lies in the integration of the assessment baseline knowledge into the programme under the seven elements proposed by the framework. A case study methodology informed the approach taken. Data was collected and analysed in three phases: the Teaching Online Programme Year 3 (TOP3) initiative; Student-teacher and Tutor Questionnaires and Student-teacher and Tutor focus group interviews. The findings highlight the complex and multifaceted process of building teacher assessment identity which nests in the larger purposes for education. They encourage an emergentist and collaborative approach to assessment knowledge and view working in communities of practice as a threshold for creativity and innovation.

EAP teachers' knowledge and use of learning-oriented assessment: A cross-contextual study

An emerging approach to classroom-based assessment that has gained increasing attention in the past decade is ‘learning-oriented assessment’, conceptualized as assessment which affords primacy to the learning orientation of classroom assessment, placing particular emphasis on active engagement of learners in assessment and feedback (Carless, 2015; Turner & Purpura, 2015). It is arguably important for EAP practitioners to be informed of and implement innovative approaches to assessment, including learning-oriented assessment. This paper reports on a qualitative study exploring EAP teachers' knowledge and use of learning-oriented assessment (Carless, 2015; Turner & Purpura, 2015) in the contexts of higher education in Canada and Malaysia. After presenting and discussing the comparative findings from the two contexts, we will provide suggestions and implications for EAP teacher training and professional development, which can in turn help inform initiatives to cultivate and enhance EAP teacher assessment literacy across these two contexts and beyond.

Classroom Assessment Literacy Levels of Mathematics Teachers in Ghanaian Senior High Schools

The mathematics teacher’s ability to assess and integrate assessment data into instructions is a function of their assessment literacy. While the teacher’s assessment literacy level is a determinant of quality of mathematics education delivery, research in the domain is limited. The study employed a descriptive survey design to explore perceived levels of assessment literacy of mathematics teachers in implementing classroom assessment within the school based assessment framework. A 35-item classroom assessment inventory was adopted, modified and administered to a purposive sample of 96 mathematics teachers from eleven Senior High Schools in Ghana. Seven other items were included to further collect data on their self-efficacy beliefs about their assessment literacy. Data were analyzed using descriptive statistics, Mann Whitney U test and correlation analysis. The result shows that majority of the participants did not attain the required assessment literacy standards for quality assessment in mathematics classroom. Moreover, there was a mismatch between participants’ assessment literacy and their self-efficacy beliefs in assessment in mathematics education even when gender and teacher experience were controlled. The study recommends that institutions training mathematics teachers or providing professional development should accentuate assessment literacy development to ensure standard assessment practices and quality instruction in mathematics at senior high schools. The assessment inventory used in this study could be adapted for use in college of education in Ghana to measure mathematics teachers’ literacies in assessing students.

Using Assessment for Learning: Multi-Case Studies of Three Chinese University English as a Foreign Language (EFL) Teachers Engaging Students in Learning and Assessment.

Student engagement is an important issue in learning and teaching given its positive effects on students' learning outcomes. Assessment for Learning (AfL), an assessment and pedagogic innovation, if done well, can fully engage students in the learning and assessment process. Adopting a multi-case design, the present study explored how Chinese university English as a Foreign Language (EFL) teachers used AfL to facilitate student engagement in their classrooms and what factors influenced their AfL practices. Three EFL teachers were recruited on a voluntary basis from two universities in Northwest China. Data collected from semi-structed interviews, stimulated recall interviews, and classroom observations suggested that teacher participants demonstrated differed assessment practices, representing Assessment of Learning (AoL), convergent, and divergent AfL, respectively. Three factors: teacher assessment literacy, teachers' beliefs about the relationship between goal orientation and motivation, as well as a trusting relationship between teachers and students, were identified as contributing to teachers' different assessment practices. Our study calls for teacher educators' efforts to equip teachers with necessary assessment-related knowledge and skills, encourage teachers to negotiate learning goals with students, and help teachers establish a trusting environment in their classrooms, if AfL is to be fully embedded in classroom instruction.

Connecting teacher and student assessment literacy with self-evaluation and peer feedback

Teachers’ feedback literacy is a focus of increasing attention in higher education. It may be framed through intentional design decisions, inter-relational aspects of engagement and pragmatic considerations of enacted curricula. Thus, teachers’ feedback literacy is connected to both the enacted curriculum and students’ relationship to feedback. How these connections take shape in particular approaches to the curriculum, affecting students’ roles in feedback and evaluation is not as well-understood. This paper presents findings from an intervention aimed at developing students’ peer feedback and self-evaluation skills in an undergraduate business course. Peer feedback and self-evaluation are increasingly common modes of engaging students as active participants in feedback and evaluation processes. It is therefore worthwhile to understand the ways in which these processes affect and link teacher and student feedback literacy. Data was analysed from a 14-week course aimed at developing students’ competencies in self-evaluation, peer feedback and teamwork. Results are presented and discussed according to three major areas: how teacher’s feedback on student engagement with feedback served as affective ‘meta’ scaffolding, the trajectory of students’ growth in feedback literate self-evaluation, and the relationship of feedback literacy to trajectory of growth in teamwork competencies. The paper concludes with suggestions for further, crossdisciplinarity research.

A four-dimensional framework for teacher assessment literacy in holistic competencies

Given that existing research on teacher assessment literacy has focused on teachers’ ability to evaluate students’ academic achievements, this study aims to develop a conceptualization of teachers’ assessment literacy in holistic competencies. Guided by four dimensions of teacher assessment literacy (knowledge, attitude, practice, socio-emotional management) identified from the existing literature, data collected from 18 individual interviews and one focus group interview with academics from four Hong Kong universities were analysed using the constant comparison method. The findings show that although the participants’ conceptions concurred with the four dimensions in general, some aspects were particularly crucial in the context of holistic competency assessment: for example, positive teacher-student relationships and subjectivity in assessment. Findings from the study allowed us to revise the four-dimensional model in the context of holistic competency assessment with more elaborated forms of knowledge, attitude and capabilities that a holistic competency assessment literate teacher might be expected to possess and enact.

Exploring the language assessment literacy of Turkish in-service EFL teachers

The purpose of this study is to find out the language assessment literacy (LAL) of teachers who teach English at state elementary, middle, and high schools in Turkey. 101 teachers working in various cities participated in this study. A mixed-methods research design was employed in the study, so the data were collected through both quantitative (questionnaires) and qualitative (interviews) instruments. The data for the quantitative part of the study were collected by means of the “Teachers Questionnaire” prepared by Vogt & Tsagari (2014) under three domains regarding language testing and assessment (LTA); classroom-focused LTA, purposes of testing and content and concepts of LTA. The data from questionnaires were analyzed via SPSS in terms of means, percentages, and frequencies to find out the training levels and needs of the respondents in language assessment. In the qualitative phase, a total of 18 teachers volunteered to take part in the interviews. The qualitative data that was gathered through semi-structured interviews were analyzed one by one in terms of their similarities and differences, and selective coding was utilized. The overall results from the questionnaires indicated that EFL teachers who work at state high, middle and elementary schools in Turkey perceived their training levels in all three domains of LTA insufficient, and they were also in need of further basic training in those domains. Furthermore, qualitative findings also supported the findings from the questionnaires by also providing insights into the reasons for participants’ insufficiency of LAL levels.

The Status of Assessment Literacy and Classroom Practice of Secondary Level Teachers

Assessment is the tool that measures the quality of education. This study was focused on the status of teachers’ assessment literacy and practice in the classroom. A case study of a mixed research approach was conducted within the Kathmandu Valley selecting three schools.The questionnaire, Interview schedule, and observation checklist were the main tools. There were 27 teachers teaching fundamental subjects (Mathematics, English, Science, and Social Study) at grades IX and X were chosen for the survey and one mathematics teacher from each school was selected for interview. Two schools were selected for the observation. The constructivist theory was used to justify the finding. The result showed the high literacy status of the teachers with mean 2.63> 2 where 93.83% of teachers were known about the 18 different assessment techniques but only 66.26% of teachers apply those techniques in the classroom for assessment.

HOW LITERATE AM I ABOUT ASSESSMENT: EVIDENCE FROM INDONESIAN EFL PRE-SERVICE AND IN-SERVICE TEACHERS

Language Assessment Literacy (LAL) for teachers is a pivotal content area which teachers must have for their prefessional development. Despite increasing interest in language assessment, research focusing on LAL of pre-service and in-service teachers, especially in Indonesian EFL context, has yet to receive due attention. Nevertheless, understanding pre-service and in-service teachers’ literacy of classroom assessment is essential for the quality of classroom assessment practice, hence the improvement of language learning and teaching. This study aimed to investigate Indonesian EFL pre-service and in-service teachers’ literacy of classroom assessment. Using a quantitative approach with ex-post facto research design, this study examined the extent of assessment literacy among EFL pre-service and in-service teachers. Participants were 60 English department students who have done teaching practicum courses and 41 in-service teachers in the Indonesian province of East Java. The finding revealed that the pre-service and in-service teachers perceived a moderate level of assessment literacy. Further, the two groups of participants demonstrated a statistically significant difference in overall assessment literacy, as well as in two of its dimensions, namely administering and scoring assessment results and alternative assessment. The findings of this study could shed light to the understanding of EFL pre-service and in-service teachers’ literacy of classroom assessment.

Using community of practice to characterize collaborative essay prompt writing and its role in developing language assessment literacy for pre-service language teachers

Conceptualizing skills for creating language assessments as an important component of language assessment literacy (LAL), this study investigates the activities of essay prompt writing by pre-service teachers for a post-admissions English placement test (EPT) at a large US university, as an effort to explore the potential of collaborative assessment practices in promoting LAL development. Five pre-service teachers, who are novice prompt writers, participated in a semester-long content development process for the EPT. Adopting a community of practice (CoP) perspective, we qualitatively examined group discussions, individual interviews, and written peer feedback and revisions on essay prompts among the prompt writers. Our findings suggest the important role of collaboration in the development of LAL skills related to creating assessments for individual prompt writers.