A checklist for reporting, reading and evaluating Artificial Intelligence Technology Enhanced Learning (AITEL) research in medical education

Abstract

Advances in Artificial Intelligence (AI) have led to AI systems’ being used increasingly in medical education research. Current methods of reporting on the research, however, tend to follow patterns of describing an intervention and reporting on results, with little description of the AI in the system, or the many concerns about the use of AI. In essence, the readers do not actually know anything about the system itself. This paper proposes a checklist for reporting on AI systems, and covers the initial protocols and scoping, modelling and code, algorithm design, training data, testing and validation, usage, comparisons, real-world requirements, results and limitations, and ethical considerations. The aim is to have a systematic reporting process so that readers can have a comprehensive understanding of the AI system that was used in the research.