Faculty Process for Reviewing and Utilizing a School’s Course Evaluation Comments

Abstract

ObjectiveThis study aimed to explore faculty engagement with qualitative comments from course evaluations. MethodsCourse faculty from the University of North Carolina Eshelman School of Pharmacy were recruited via email to participate in a 30-minute interview session. Previous course evaluation comments were adapted to create a de-identified mock evaluation. Six interviews were conducted via Zoom, consisting of a think-aloud protocol based on the mock course evaluation followed by a cognitive interview focused on goals and current utilization of comments, and common patterns and issues sought by faculty. Interview transcripts were manually cleaned and de-identified. Transcripts were inductively coded by 1 researcher using MAXQDA. ResultsThree overarching themes were identified: general faculty process for reviewing comments (ie, how faculty perceive and analyze comments), comments utilization for course change (ie, how faculty utilize comments in making course changes), and faculty analysis strategy (ie, faculty approach to locating common patterns in evaluation comments). The most common subthemes included usefulness of comments, methods for tracking comment patterns, and challenges with the large number of comments each semester. ConclusionFaculty provided useful insight and feedback regarding the current state of the course evaluation process that can be used to improve the structure, organization, and utilization of course evaluations by faculty. These findings could inform the creation of the course evaluation comment automated analysis program in the next stage of an ongoing project.