Beyond satisfaction scores: visualising student comments for whole-of-course evaluation

Abstract

This study examines the utility of a new method of analysing and reporting qualitative student survey comments. Visualisations of the sentiment of and key themes from qualitative student survey comments were developed for 34 subjects from two undergraduate health sciences courses undergoing transformation. The course and subject visualisation reports were provided to academics who were members of the two course transformation teams. Utilising focus groups and semi-structured interview methodology, we examined academic perspectives on the potential usefulness of this new method of analysing and reporting qualitative comments from student evaluations of teaching (SET). Results indicate that visualisations are considered useful for focussing educators’ attention on themes in qualitative comments rather than on individual negative comments. However, educators did not support use of the visualisations at this stage, citing concerns about the accuracy of sentiment analysis and the validity of SET surveys. There were many opportunities identified through the focus groups for future improvements to a visualisation technique for reporting student survey results.