New Modes Of Assessment Research Articles

Narrowing awarding gaps: the contributory role of policy and assessment type

ABSTRACT Awarding gaps between various groups of students persist across the Higher Education sector, yet the responses designed to address the contributors remain localised. The sudden spread of COVID-19 led to various responses across the University sector creating an unprecedented natural experiment and offering the opportunity to compare outcomes from these measures with prior cohorts. This study seeks to investigate the effects of two COVID-19 interventions on students’ performance in the Business and Management discipline at a UK university. The specific COVID-19 measures considered here are the move to online assessments and the new grade policy to ensure the pandemic did not affect students’ outcomes adversely. We use a Kernel Propensity Score and a Quantile Difference in Differences models to estimate the treatment effect of the two COVID interventions on the treated group, namely term two students’ performances of the academic year 2019/20. Our results indicate that the effects of both COVID interventions supported the outcomes of international students, thereby narrowing the awarding gap. Findings suggest firstly that institutional policies adopted in crises should seek to address potential adverse effects on student outcomes for the period of disruption, indicating that significant care should be taken in their drafting. The policy, in this case, was found to have achieved its aim. Secondly, the move to new modes of assessment combined with detailed briefings from faculty may have served to uncover aspects of the hidden curriculum for this group, contributing to a narrowing of awarding gaps between different groups of students.

Comments for the roundtable on Yanni Kotsonis, States of Obligation: Taxes and Citizenship in the Russian Empire and Early Soviet Republic (Toronto: University of Toronto Press, 2014)

This response to Yanni Kotsonis’ States of Obligation focuses on Kotsonis’ intervention in one of the fundamental questions of Russian historiography, the relationship between state and society. Kotsonis argues that new kinds of taxes (and especially new modes of assessment of those taxes) brought a new kind of citizen into being, one whose relationship with the state was individualized and participatory, but not representative. This commentary acknowledges that Kostonis persuasively demonstrates that state officials used taxation first to identify and then to construct a particular type of citizen. However, it asks whether Kotsonis’ conclusions about a state-centred emergence of the Russian person and of a Russian model of citizenship, anchored in obligations rather than rights, simply reproduces the aspirations of the bureaucrats he studies while underestimating society’s interactions with these processes and its ability to articulate alternatives.

Preservice teachers’ adaptations to tensions associated with the edTPA during its early implementation in New York and Washington states

The edTPA is a teaching performance assessment (TPA) that the states of New York and Washington implemented as a licensure requirement in 2013. While TPAs are not new modes of assessment, New York and Washington are the first states to use the edTPA specifically as a compulsory, high-stakes policy lever in an effort to strengthen the quality and accountability of teachers and teacher educators. This study examines 24 New York and Washington teaching candidates’ experiences with the edTPA during its first year of consequential use for state certification. The data, drawn from qualitative interviews that were part of a larger mixed-methods study, reveal that preservice teachers had to mediate several tensions associated with the edTPA’s dual role as a formative assessment tool and a licensure mechanism. In this paper, we identify those tensions, describe candidates’ efforts to mediate them, and discuss the extent to which that mediation process may or may not contribute to the improvement of teachers’ practices. Given the edTPA’s positioning in a policy context – specifically, the potential for the assessment’s locus of control, high stakes, and opaque rating process to distort the procedures it is intended to measure – the paper concludes with recommendations for teacher education programs aimed at capitalizing on the edTPA’s benefits and mitigating its unproductive tensions.

Explaining students' appraisal of lectures and student-activating teaching: perceived context and student characteristics

During lectures, some students are continuously focused and attentive, whereas others tend to be bored, jittery, or inattentive. The same might happen when students are given student-activating assignments. Some students simply love one type of instruction, whereas others tend to resent it. Moreover, it is not the context itself, but the context as it is perceived by the student that affects learning. This study sets out to discover how students view instruction, based on their experiences with a particular setting, and aims to investigate the effects of “context-by-student” characteristics, such as students' course experiences, and the effects of “student-by-context” features, such as approaches to learning. The research has a quasi-experimental design and compares a lecture-based setting with student-activating teaching methods (i.e. self-discovery learning by means of authentic tasks with the help of a scaffolding approach); followed by one of four assessment methods: a multiple-choice test, a case-based examination, a peer assessment or a portfolio assessment. Data (N = 579) were collected by means of validated questionnaires, comprising the ASSIST and LSQ instruments and the course experience questionnaire. Results demonstrate that students' appraisal of the assessment method, their course experiences and the teaching methods that are dominant in the setting contribute positively to students' appraisal of instruction. Interestingly, this conclusion applies to both the lecture-based and the student-activating groups and to both conventional assessment methods and new modes of assessment. Moreover, deep approaches to learning, monitoring studying and/or organized studying, as well as preferences for courses that support understanding, positively affect students' appraisal of student-activating teaching methods. In short, both student-by-context and context-by-student features prove to be strong predictors of students' appraisal of instruction and, as a consequence, student learning.

Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning – A review of research studies

The interest in assessment for learning ( AfL) has resulted in a search for new modes of assessment that are better aligned to students’ learning how to learn. However, with the introduction of new assessment tools, also questions arose with respect to the quality of its measurement. On the one hand, the appropriateness of traditional, psychometric criteria is being questioned in the context of assessment for learning. On the other hand, it has been argued that new criteria need to be formulated to acknowledge the unique features of assessment for learning. The issue dealt with in this review is what quality criteria are specifically relevant to AfL. Studies using (peer) assessment for learning were evaluated with regard to use of quality criteria under two perspectives: their recognition of educational measurement criteria, as well as their consideration of student involvement in the assessment of learning. The selected studies were analyzed to determine what specific criteria were utilized in (successive parts of) the assessment cycle (i.e., the process of construction, administration and follow up of an assessment). Our results indicate that quality criteria are dissimilarly connected to the distinct steps of an assessment cycle, which holds as well for the attention given to student involvement in assessment for learning.

Changing anatomies of Information Literacy at the postgraduate level: refinements of models and shifts in assessment

In this paper, the authors will identify the fundamental principles that might inform an approach to Information Literacy (IL) on the postgraduate level, which takes into account following premises: the aims of postgraduate/doctoral studies are different in comparison to earlier educational levels and face specific challenges due to the heterogeneity of student populations IL frameworks have to acknowledge and address this challenge by adjusting to specific needs of postgraduate students who operate in new information realms new modes of assessment are needed as a result of revolutionary changes in information landscapes and patterns of generation or use of scientific information Teaching students in the scientific method and culture has long been recognized as the major focus of postgraduate education. Postgraduate students are expected to develop inventories of scientific activities and abilities, such as the use of scientific inquiry processes to ask valid questions and to gather and analyze information, capacities to appraise and improve original ideas, to pursue ideas from genesis to proof and arguments, to defend and communicate ideas to peers and to the wider society. The basic precondition for the described practices is the adequate performance in the realm of information handling and information management, i.e., information literacy. IL on postgraduate levels has a strong focus on the universe of scientific information, which itself went through tremendous changes in the last decade, particularily as a result of the appearance of the Web 2.0 (e.g. Science 2.0, Research 2.0). Innovative configurations of scientific discourses have become visible, resulting in the breakdown of traditional assumptions about scientific expertise and the conversion of rigid scientific processes to more open-ended processes of communication. Scientific information and knowledge are socially produced and distributed and scientific activities can no longer be conceived of as a schematic, linear or rules-driven process. Such profound changes suggest renewed conceptions and focal points of IL at the postgraduate level which will take into account the fluid nature of current information environments. New IL frameworks should acknowledge that it is not always possible to a priori determine the best information and resources to use or apply one valid strategy for solving a particular problem. IL programs should therefore move ahead by abandoning the limited present approach according to which there is only one right answer or path to this answer and instead offer insight into the variety of complex layers our current information universe consists of. Assessment models in IL should mirror this renewed IL conceptions as well, which basically implies the negation of the rubrics and checklist approach commonly associated with IL assessment models. After discussing changes in information landscapes brought about the Web 2.0 and examining transformed premises of scientific work within such environments, the authors will plea for reconceptualizations of IL on the postgraduate level and propose new modes of assessment that will recognize this transformation.

Optimising new modes of assessment: in search of qualities and standards

Het staat buiten kijf, dat er veranderingen in het denken over toetsing zijn opgetreden. De redacteuren en de auteurs van dit boek spreken zelfs – terecht – over een paradigmaverschuiving.

A five-dimensional framework for authentic assessment

Authenticity is an important element of new modes of assessment. The problem is that what authentic assessment really is, is unspecified. In this article, we first review the literature on authenticity of assessments, along with a five-dimensional framework for designing authentic assessments with professional practice as the starting point. Then, we present the results of a qualitative study to determine if the framework is complete, and what the relative importance of the five dimensions is in the perceptions of students and teachers of a vocational college for nursing. We discuss implications for the framework, along with important issues that need to be considered when designing authentic assessments.

Science Without Literacy: A ship without a sail?

This article argues that reading, writing and argument are central to any conception of science as it is currently constituted. Moreover, it is through the texts of science, popular accounts or journalistic reported versions that the majority of the public interact with and consider the implications of the findings that science presents. However, the study of the language of science, science's epistemic base and the cultural norms and values that underpin its practice are currently considered only marginal to the teaching of science. Rather, the specialised laboratories provided for science teachers and the narrow conception of science embodied in the curriculum gives pre-eminence to science as an empirical activity in the naive belief that this is central to understanding the nature of science. The consequent failure to recognise the centrality of language, literacy and argument to science education leaves the majority ill equipped to become critical consumers of science. Change requires a concerted attempt to reconceptualise the priorities for science education through a mix of new curricula, new strategies and last, but not least, new modes of assessment.

Quality standards for new modes modes of assessment. An exploratory study of the consequential validity of the OverAll Test

During the past decade, due to societal developments, methods of instruction as well as the assessment of students’ performances have changed to an important considerable extent. Two of the elements of this change are the accents on cognitive competencies such as problem solving and on learning in an authentic context. In conjunction with the development of such learning methods, new modes of assessment were implemented. It was expected that this change would have positive feedback effects on learning and teaching. These feedback effects are the central issue of this article. They are discussed in terms of the experiences of the Maastricht School of Economics and Business Administration. This school places the analysis of authentic problems at the core of the curriculum, including the learning process as well as the assessment system. The OverAll Test, a case-based assessment instrument aiming to assess problem solving skills, was implemented as part of this. Different quality issues related to the OverAll Test have been evaluated. This article presents the results of one of the four validity studies conducted; an exploratory study of the consequential validity of the OverAll Test. It starts with the an outline of the main features of the new modes of assessment and the OverAll test as an example. There is then a discussed discussion of effectively the OverAll test fits these features as well as the goals and characteristics of problem-based learning. The study of the consequential validity of the OverAll Test is then described in depth. The results of the survey, as well as the results of the semi-structured interviews with staff and students, indicate a friction between the intended characteristics of the learning and assessment environment and the practice of instruction and assessment.