Towards a novel catalog of assessment patterns for distant education in the information technology domain

Abstract

This paper presents a research and its results in the domain of higher education's pedagogical patterns for remote assessments - precisely in the computer science, software engineering and informatics-related courses. This research was motivated by the COVID-19 crisis, which separated teachers, teaching assistants, and students physically. During this period, remote knowledge assessment was one of the most challenging among all educational activities. The lack of available resources and advice on remote knowledge assessment revealed a need for a specialized assessment pattern catalog.The main result of the research is the assessment pattern catalog that started to grow organically at the Institute of Informatics, where we teach IT-related courses. We started with the initial set of patterns, identified by analyzing recurring practices, applied by teaching staff for remote assessments in the period from March 2020 till December 2020. The patterns were aggregated and gradually refined using a systematic approach. In addition to guided workshops, a systematic literature review was employed, followed by catalog refinements, and, finally, an extensive survey was carried out among teachers and teaching assistants. The latter was used as a validation of the correctness of the novel assessment pattern catalog, as well as the presented patterns’ suitability and popularity among users.The resulting assessment pattern catalog presented in this paper boasts 47 patterns, classified into four main categories, that support the whole process of (remote) assessment. It is organized and documented systematically. It also boasts several indicators per each pattern to demonstrate its suitability for distant assessments, popularity rankings among teachers, teaching assistants, and top picks in every category per teachers and teaching assistants.The survey that we performed revealed a subset of patterns that are important for a successful remote assessment, validated in the IT-related courses. Based on the results, the presented assessment pattern catalog showed itself to be useful not only for the remote assessment but also for judging knowledge in the classroom successfully.