Towards the Design of Ethically-Aware Pedagogical Conversational Agents

Abstract

Pedagogical conversational agents (PCA) aimed at offering personalized support could greatly enhance the student experience by adapting their support to the student’s learning needs and habits. But, as they become more autonomous, there is a greater need to consider the potential moral consequences of their choices and to take into consideration an holistic view of students. In face-to-face setting, the individual and holistic understanding that human teachers can have about their students and classrooms is key in ensuring that the teachers’ decisions strive for the best learning experience. However, furnishing PCAs with such a holistic understanding of complex dimensions that often involve relations between multiple elements poses a challenge. Furthermore, ethical aspects should also be considered in order to avoid decisions that can lead to discriminatory and unfair collateral results and end up favouring certain students over others; therefore, the PCA should be aware of how decisions aimed at satisfying certain needs could negatively affect other dimensions. The current position paper states the need to furnish PCAs with ethical awareness and discusses the nature of ethical dilemmas in the context of PCAs in online learning environments. Through this discussion, the paper proposes some first steps towards the design of ethically-aware PCAs.