Assessment Reform Research Articles

THE REFORMS OF NATIONAL ASSESSMENTS IN MALAYSIAN EDUCATION SYSTEM

Although the impact of education policy reform could inform future policymaking initiatives, studies on education policy reform of student assessment are considered scarce. This study attempts to scrutinise the drivers, impacts, and challenges of education policy reform of the national assessment system in Malaysia based on policy documents and previous empirical studies. The findings indicated that the education policy reform of the Malaysian national assessment system is driven by the decline of pupils’ performance in international large-scale assessments. There are changes in content, format, and structure of the public examinations and assessment delivery procedures. The emergence of school-based assessment shows a paradigm shift from cognitive learning to a more holistic manner. Challenges arise included teachers’ extra workload, teachers’ readiness, lack of reference materials, and time constraints in implementing school-based assessment. Findings inferred the needs in refining the monitoring and coaching system in order to ensure the student assessment implementation does not diverge from the missions of education policy reform. This study is hoped to provide useful insights for future policymaking initiatives, particularly in Malaysia. Keywords: Malaysia, policy reform, public examination, school-based assessment, student assessment. Cite as: Chin, H., Thien, L. M., & Chew, C. M. (2019). The reforms of national assessments in Malaysian education system. Journal of Nusantara Studies, 4(1), 93-111. http://dx.doi.org/10.24200/jonus.vol4iss1pp93-111

Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review.

An increasing emphasis on health professional competency in recent times has been matched by an increased prevalence of competency-based education models. Assessments can generate information on competence, and authentic, practice-based assessment methods are critical. Assessment reform has emerged as an academic response to the demands of the pharmacy profession and the need to equip graduates with the necessary knowledge, skills and attributes to face the challenges of the modern workforce. The objective of this review was to identify and appraise the range of assessment methods used in entry-level pharmacy education and examine current trends in health professional assessment. The initial search located 2854 articles. After screening, 36 sources were included in the review, 13 primary research studies, 12 non-experimental pharmacy research papers, and 11 standards and guidelines from the grey literature. Primary research studies were critically appraised using the Medical Education Research Study Quality Instrument (MERSQI). This review identified three areas in pharmacy practice assessment which provide opportunities for expansion and improvement of assessment approaches: (1) integrated approaches to performance assessment; (2) simulation-based assessment approaches, and; (3) collection of validity evidence to support assessment decisions. Competency-based assessment shows great potential for expanded use in pharmacy, but there is a need for further research and development to ensure its appropriate and effective use.

Innovation and Reform in Course Planning, Teaching, and Assessment: The CEFR in Canada and Switzerland, A Comparative Study

Change and innovation in education require carefully thought-through procedures as well as time, and benefit both from exchanges of expertise between contexts and from quality assurance (QA) processes. A key document that stimulates change in planning, teaching, and assessment is the Council of Europe’s (2001) Common European Framework of Reference for Languages (CEFR). However, its innovation potential in language education has been affected by a lack of impact studies and QA procedures. This article reports on the QualiCEFR study, which aims to encourage a culture of QA in relation to planning, teaching, and assessment and to identify promising practices in those processes. The study compared CEFR implementation in Switzerland and Canada employing a mixed-methods design and a QA approach, thematically analyzing data from over 40 interviews. Results have helped to inform the development of an online tool produced in a sister project of the European Centre for Modern Languages (ECML).

Assessment-led reform: Creating a sustainable culture for WIL

This paper describes a process of assessment reform designed to enhance Work Integrated Learning (WIL) approaches for two science courses at an Australian university. The project used a mixed-method approach involving online surveys, interviews, focus groups and workshops to gather student, industry and course team knowledge and understanding of WIL approaches to curricula. The investigation centred on the perceived value of collaborating with industry to facilitate enhancements in authentic assessment and on the barriers to, and challenges in, achieving successful outcomes. The action-research project, WIL-on-Campus (WoC), found that assessments oriented toward the inclusion of authentic tasks and processes, that contribute to the employability learning and job-readiness of students, is deemed important to students, industry and academics. However, reforms to assessment practice and process are required. For greatest impact, this study found that assessment reform processes require two critical interdependent factors: the socialisation of the shared institutional value of embedded WIL approaches to assessment, and the provision of top-down support to enable academic course teams to implement the ‘imposed’ changes. Further to this, while academics viewed the changes in approach to assessment design as challenging, they also noted that a shift is timely and believed that a course-wide WIL approach is possible and advantageous.

Assessing Assessment Literacy: Are new teachers prepared to assume jobs in school districts engaging in grading and assessment reform efforts?

Grading and assessment practices of K-12 educators in the U.S. continue to be among the most discussed areas in school reform and school leaders question whether or not new teachers are prepared to engage in this type of work. The purpose of this qualitative study was to better understand the phenomenon of being a new teacher in a school that uses innovative methods such as standards-based or standards-referenced grading (SBG). The finding of this study indicated that limits to preparedness as a professional was due to teacher education training on assessment, which was inconsistent at best.

Risks when Reforming: Challenges and Sustainability of RIA Systems: Results of the First Systematic Study on RIA Reforms in Developing Countries

AbstractThis article reports on and discusses the findings of the first systematic study on Regulatory Impact Assessment (RIA) reforms in developing and transition economies. The macro part of the study reveals that between 2001 and 2016 60 such reforms were launched in total, with 20 of them leading to functional RIA systems two years or more after they were initiated. Appropriate and complete reform design, analysed by measuring compliance with six internationally recognised “good practices”, is found to be a necessary but not sufficient condition for early success. To develop a better understanding of challenging factors for RIA reforms’ success the micro part of the investigation analyses four case studies (Botswana, Cambodia, Kenya and Uganda). Findings point to the importance of donor flexibility and patience and the need for building reform constituencies inside and outside government. The traditional orthodoxy of regulatory impact assessment reforms as an extension of red tape reduction is challenged. The article finally presents several possible policy implications of the findings.

The language assessment literacy needs of Iranian EFL teachers with a focus on reformed assessment policies

Teachers’ assessment literacy has recently captured the attention of scholars across various educational contexts. The literature has it that there is a gap between teachers’ assessment practices and national assessment policies. The present study investigated the assessment needs of Iranian EFL teachers in the wake of the new assessment reform, which aims at replacing traditional discrete point testing policies with performance testing. In-depth interviews were conducted with 15 EFL head teachers. In addition, documents related to the curriculum reform were also closely examined. Inductive coding of the data showed that to meet the demands of the noted reform, teachers’ current perceptions of language assessment need to change. Furthermore, teachers need training in both knowledge and skills of language assessment. More specifically, teachers need training in developing rubrics for use in assessing the productive skills of speaking and writing. They also need to develop literacy in devising higher-order thinking skills in assessing reading and listening comprehension. Finally, as non-native speakers of English, Iranian English teachers need better English aural/oral skills.

Assessment innovation and reform in France

ABSTRACT The last two decades have seen fast-moving and wholescale changes in the ways that education is now provided in French schools, in the nature of learner assessment, and in the form and scale of system evaluation. Innovation and reform have in part followed international trends, themselves triggered by the global impact of the international attainment surveys, and are in part a response to growing economic pressures, EU initiatives and increasingly burdensome examination logistics. One of the most recent reforms, and arguably the most dramatic, concerns the iconic ‘academic’ baccalauréat, which sees a reduced subject profile for formal examination and the new addition of teacher assessment to complement examination achievements. The most ambitious innovations are arguably those introduced into the newest form of accountability-focused system monitoring. The article overviews these innovations and reforms, along with the motivations that led up to them, and briefly reflects on potential future consequences, both expected and unintended.

Process Assessment Reform of Tourism Management Based on CDIO&OBE Education Concept

CDIO international education mode aims to organize learning through the whole process of an engineering project. Outcome Based Education (OBE) is an educational theory that bases each part of an educational system around outcomes. Both of these two education concepts emphasize that the purpose of education is to achieve the enhancement of students’ knowledge, ability and quality. This study analyzed the application and effect of CDIO and OBE concepts on economy and management specialties by an empirical study of tourism management major. Research results indicate that it is an effective way to achieve the goal of CDIO and OBE education concept by utilizing a process assessment reform under the “students oriented team learning” logical framework with grogram-driven multiple teaching-learning methods and information technology.

Leadership development of 21st century engineering millennial students in Khalifa University, United Arab Emirates; problem-based learning in action

Leadership development has become an import aspect of the UAE’s educational system. In recent years, UAE leaders have focused on the reform of higher education assessment, curriculum and administration with a view to encouraging Emirati students to contribute to the nation’s growth as national human capital, through leadership roles where they will be guided and educated driving the needs of the knowledge economy. In many courses, students are more knowledge recipients than producers; they are considered cognitively active whilst physically inactive where learning is considered a passive process. BUSS301, a third-year undergraduate course taught to engineering students has undergone major revisions influenced by student evaluations on application, relevance and assessment. The earlier syllabus entitled Corporate Leadership and Human Resource Management (more theoretical and examination driven) has evolved to a more recent Enquiry Based approach: Teaching and Learning Leadership by Simulation and Theory where students are driving their own learning through inquiry using a project-based learning (PBL) approach.
 
 Keywords: Project-based learning, engineering education, leadership, student-centred learning, constructivism, teambuilding, collaboration

Current situation and improvement of cardiovascular clinical teaching

There are some problems in clinical teaching of cardiovascular medicine, such as insufficient attention to the three bases skills, not teaching trainees in accordance with their aptitude adequately, monotonous instructional mode, and assessment becoming a mere formality. Aiming at these problems, we have explored the training methods of medical talents adapted to the modern medical model, and improved the clinical basic skills and thinking of the students and the quality of teaching through such measures as paying close attention to basic skills training, standardizing ward rounds, paying attention to teaching students in accordance with their aptitude, and reforming teaching methods and assessment modes. Key words: Clinical basic skills; Clinical thinking ability; Problem-based learning (PBL); Assessment reform

Policy as praxis: Senior educators’ enactment of assessment policy reform

This article reveals the multifaceted ways in which policy enactment was expressed as praxis in the context of assessment reform in Ontario, Canada. The research explores the way in which the Growing Success assessment policy was interpreted variously by different educators occupying senior roles within the district office in a single school district in northern Ontario. Drawing on neo-Aristotelian theorising, the research reveals how ‘policy in practice’ was expressed as a form of praxis, where such praxis is understood as morally committed and informed action oriented towards excellence in a field (in this case, education). While recognising the complexity of policy enactment, and how policy enactment can result in unforeseen and sometimes problematic outcomes, the research also reveals how policy enactment can have productive outcomes in relation to what are construed as the ‘internal goods’ of education. In the research presented, these productive outcomes included the capacity to facilitate teachers’ learning within and across elementary and secondary school sites; a critical, constructive focus on standardised measures of student learning in relation to academic outcomes; and the enhancement of student learning opportunities via cultural inclusion, particularly in relation to First Nations, Metis and Inuit students. In this way, the research validates a conception of policy as praxis and foregrounds how policy enactment can be understood in ways that promote the intrinsic integrity of educational practice, and the need to draw on these ‘internal goods’ in such enactment.

Risk Evaluation of Hazardous Chemicals Transportation Based on Fuzzy Comprehensive Evaluation

The purpose of this study is to provide safety pre-warning and guarantee for the transportation of hazardous chemicals in transport enterprises. To this end, this paper conducts a risk assessment study on hazardous chemicals transportation based on fuzzy comprehensive evaluation (FCE) method. Taking the waterway transportation of transport enterprises as the research object, the fuzzy analytic hierarchy process (AHP) was selected and used to establish secondary index evaluation system from five aspects: transportation equipment, hazardous chemicals, man, transportation environment and enterprise management. Also, the analytic hierarchy process (AHP) was applied to configure the index weights. Thus, a fuzzy comprehensive risk assessment model for hazardous chemicals was established. The example analysis results show that the fuzzy risk assessment model can quantitatively evaluate the risk level of hazardous chemicals transportation. The risk score of a transport enterprise in Zhoushan is 66.2, which belongs to the general safety level with potential transportation risks. This provides effective theoretical guidance for risk assessment and orientation reform of hazardous chemicals transportation enterprises.

The domestic hotspots research of formative assessment based on the knowledge map analysis

Objective To summarize and analyze the present situation, hotspot and development trend of the research and practice about the formative assessment in Chinese medical teaching reform by bibliometrics, and provide a quantitative method for the study of formative assessment. Methods We took 567 journals about formative assessment from CNKI database as the research object, making co-word clustering and multidimensional scaling analysis as the main research method, and using Bicomb co-word analysis software and SPSS to complete content data statistics and analysis. Result The knowledge map of formative assessment research can be divided into 7 fields: field 1 shows that most studies are mainly focused on practical research, the effectiveness evaluation on application are mainly focused on the improvement of students' independent learning ability. The proportion of research in traditional Chinese medicine colleges or universities is increasing, researches of clinical teaching take an important position, and nursing educations are paying more attention as well. Field 2 and field 4 show the synchronization between assessment reform and teaching content/strategy reform. Field 3 presents the general situation of teaching evaluation research in higher vocational colleges. Field 5 shows the theme of exploring the establishment of a formative assessment system. Field 6 shows the differences and similarities between examination and assessment. Field 7 shows the overview of medical English courses. Conclusion In the future, we should actively carry out the construction and exploration of the formative assessment system that conforms to China's national conditions, and attach importance to the construction of the sharing platform. We should avoid duplication of research, attach importance to the study and understanding of theoretical connotation and strengthen training, supervision and management. Key words: Medicine education; Formative assessment; Mapping knowledge domains; Cluster analysis; Alscal analysis

Understanding Chinese EFL teachers’ conceptions and practices of assessment

AbstractThis paper reports on findings from an ongoing research project exploring Chinese EFL teachers’ conceptions of assessment and their classroom assessment practices, as well as to examine the relationship between these two. The sample for the study consisted of 107 EFL teachers from 18 secondary schools who completed a Chinese version of the Teachers’ Conceptions of Assessment inventory (Brown, Hui, Yu, & Kennedy, 2011) and the Classroom Assessment Practices Questionnaire modified from Assessment Practices Inventory (Zhang & Burry-Stock, 2003). This study found that Chinese EFL teachers strongly associated assessment as a means of improving students’ EFL learning’ with assessment as a means of cultivating positive moral and personal character contributing to students’ lifelong learning and good citizenship. Results also showed that although there existed some level of alignment of teaching and assessment in the classroom, traditional assessment practices rather than alternative assessment such as student self-assessment were seen by the EFL teachers as contributing to students’ learning and making individual and school accountable, which is potentially a matter of some concern if innovative assessment reform that avoids using mandated external examinations is to be carried out in the Chinese EFL classroom. Implications of the results for Chinese EFL teacher assessment literacy development are also discussed

An investigation into the Sawtooth Effect in secondary school assessments in England

ABSTRACTThe ‘Sawtooth Effect’ is where cohort performance on high-stakes assessments drops after assessment reform, and then improves over time as test familiarity increases. Despite its importance, limited research has been conducted to date. In this study, we took a novel approach of using changes in grade boundaries (cut scores) over time as a proxy measure for performance change across a large number of secondary school assessments. Findings were generally consistent with the Sawtooth Effect, suggesting that it took students/teachers around 3 years on average to become familiar with the new tests. Estimates of the magnitude of these changes were also produced. Findings support the need for caution when interpreting changes in performance for new assessments, as any gains that occur may be due to test familiarity, rather than meaningful gains to subject mastery.Abbreviation: GCSE: General Certificate of Secondary Education; GCE A level: General Certificate of Education Advanced Level

A STUDY ON THE VALIDITY OF VSTEP WRITING TESTS FOR THE SAKE OF REGIONAL AND INTERNATIONAL INTEGRATION

In the context of rapid regional and international integration, particularly the official establishment of the ASEAN Economic Community in 2015, English capacity has become essential for Vietnamese people to create their competitiveness in employment, education and other opportunities. In the reform of English education and assessment in response to this demand, VSTEP tests were developed and introduced by the Ministry of Education and Training as national English assessment instruments. VSTEP tests are meant to be alternative to the existing expensive international standardised English tests (e.g. IELTS, TOEFL). But this requires VSTEP developers to take action to assure test validity. They also need to accumulate and disseminate evidence of validity of the tests to gain international recognition. By doing so, they have taken meaningful action to contribute to the nation’s international and regional integration. The paper highlights the commitment of ULIS-VNU as a VSTEP developing institution in this mission. It reports a recent VSTEP validation study as an example of this commitment.

Assessment Reform in Shanghai: Issues and Challenges

This article investigates the perceptions of educators from Shanghai on the assessment system and reform in their municipality. The study shows that the Chinese educators perceive the conventional assessment system in Shanghai, particularly the national college entrance exam (gaokao), to be rigorous and fair but inadequate in fostering the holistic development of students. While the Chinese educators support the current assessment reform that is aimed at holistic education, they highlight the challenges of enacting alternative assessment due to prevailing historical and socio-cultural conditions. The case of Shanghai illustrates the continual relevance, strengths, and weaknesses of conventional assessment as well as its potential in complementing alternative assessment.

Assessment Reform and the Case for Learning-Focused Accountability

Calls for greater accountability within schools have led to an expansion, and in some cases, retraction of test-based accountability systems within many English-speaking Western nations. Countries such as England, United States, and Canada have all developed educational standards that are primarily measured in relation to standardized achievement test results. This paper discusses the documented impact and social consequences of external testing measures on students, teachers, and school systems within the Western culture of student performance. The discussion considers alternative assessment systems that are currently evolving into a framework of learning-focused accountability. The article concludes by discussing the implications of these progressive alternatives for policy-makers, school personnel, and institutions concerned with the training of teachers and administrators.

Language assessment in the new English curriculum in Iran: managerial, institutional, and professional barriers

BackgroundAssessment policies and practices are key to the success of curriculum innovations. Therefore, large-scale ELT innovations always include changes in assessment too. Thinking that Iranian students were not enabled to communicate in English after six years of English education in public schools, educational policy makers have recently embarked on a new curriculum that is thought to be a departure from traditional reading and grammar approach towards a communicative one. This study investigated the impact of a recent language assessment reform (LAR) on Iranian English teachers’ assessment practices.MethodsTo this end, four teachers participated in a focused group interview (FGI) session. Teacher-made tests were also collected and scrutinized. Both FGI and test data were content analyzed and the recurring themes were derived.ResultsIt was revealed that managerial, institutional, and individual barriers stand in the way of the reform. First, the managerial, technocratic approach to reform implementation has caused teachers not to take ownership of the reform. Secondly, at the institutional level, two obstacles were identified in the way of reform: inadequate resources within schools and the accountability demands that foster grade inflation. Finally, regarding professional competencies, teachers appeared to be largely unprepared to conduct language assessments consistent with the LAR demands. In particular, they seemed to have difficulties with the contents of their assessments, with the reasons for doing assessments, and with adjusting their assessments in keeping with LAR communicative aspirations.ConclusionsTo remedy the situation, action should be taken to convince teachers to buy into the reform and to create opportunities for teachers to become adequately literate in language assessment.