A Bayesian approach to generating tutorial hints in a collaborative medical problem-based learning system

Abstract

Today a great many medical schools have turned to a problem-based learning (PBL) approach to teaching. While PBL has many strengths, effective PBL requires the tutor to provide a high degree of personal attention to the students, which is difficult in the current academic environment of increasing demands on faculty time. This paper describes intelligent tutoring in a collaborative medical tutor for PBL. The main contribution of our work is the development of representational techniques and algorithms for generating tutoring hints in PBL group problem solving, as well as the implementation of these techniques in a collaborative intelligent tutoring system, COMET. The system combines concepts from computer-supported collaborative learning with those from intelligent tutoring systems. The system uses Bayesian networks to model individual student clinical reasoning, as well as that of the group. The prototype system incorporates substantial domain knowledge in the areas of head injury, stroke and heart attack. Tutoring in PBL is particularly challenging since the tutor should provide as little guidance as possible while at the same time not allowing the students to get lost. From studies of PBL sessions at a local medical school, we have identified and implemented eight commonly used hinting strategies. In order to evaluate the appropriateness and quality of the hints generated by our system, we compared the tutoring hints generated by COMET with those of experienced human tutors. We also compared the focus of group activity chosen by COMET with that chosen by human tutors. On average, 74.17% of the human tutors used the same hint as COMET. The most similar human tutor agreed with COMET 83% of the time and the least similar tutor agreed 62% of the time. Our results show that COMET's hints agree with the hints of the majority of the human tutors with a high degree of statistical agreement (McNemar test, p=0.652, kappa=0.773). The focus of group activity chosen by COMET agrees with that chosen by the majority of the human tutors with a high degree of statistical agreement (McNemar test, p=0.774, kappa=0.823). Bayesian network clinical reasoning models can be combined with generic tutoring strategies to successfully emulate human tutor hints in group medical PBL.