Evaluating the ethics of machines assessing humans

Abstract

The case focusses on Assessment and Qualification Alliance, one of England’s largest exam boards, and its evaluation of whether Artificial Intelligence (AI) could be deployed, in principle, for marking high-stakes assessments. At a time when generative AI, such as ChatGPT, has gained popularity, the case offers insights into the challenges and risks of algorithmic decision making and algorithmic fairness, such as accuracy and explainability. The case allows students to explore the role of ethics when developing an AI-based tool in an area that they all know very well: Most students will have had to sit high-stakes assessments in the past and are still likely to be assessed on an ongoing basis as a current student. Students will thus be able to relate to the case and have their own stake(s) and opinion(s) about whether they would want to be assessed by AI. The case is also more broadly applicable in raising general awareness of the challenges and potential risks involved when using AI for decision making and it encourages students to consider the wider consequences for all stakeholders that are triggered by the use of digital technology.