A pedagogical evaluation of an institution's digital assessment platform (DAP): Integrating pedagogical, technical and contextual factors

Abstract

An assessment for learning approach is foundational for student learning. The necessity to shift teaching and learning online as a response to COVID-19 has propelled digital assessment into the mainstream within higher education institutions. User experience is a common indicator of effectiveness of technologically enhanced initiatives; however, measuring the pedagogical impact of digital assessment initiatives has not been straightforward. This paper presents a pedagogical evaluation, incorporating staff and student perspectives, of a digital assessment platform (DAP) at a large Australian metropolitan university. Data were collected using surveys, interviews, focus groups and case studies and incorporated findings from student evaluations of courses and a student-staff partnership project. Results highlight strong pedagogical reasons for continuing the digital assessment initiative, while also identifying constraints and opportunities for improvement. The insights generated through this evaluation emphasise the importance of considering technical, pedagogical and contextual factors and the interplay between the factors. In a quality assurance climate, where the demand for this type of institution-wide evaluation is increasing, this study contributes an approach that addresses some of the complexities and challenges of evaluating digital assessment initiatives in higher education. Future research is needed to understand more fully the relationships between pedagogical and contextual factors when undertaking an evaluation. Implications for practice or policy: The DAP enabled the (re)design of authentic, inclusive, engaging and secure assessment tasks. Educator support is critical in learning the system and creating assessment with high pedagogical merit. Student familiarisation opportunities with using the DAP are beneficial. The DAP’s longevity within the institution is crucial considering the significant effort required of educators and students. Contextual influences blur understanding of pedagogical factors.