Exploring the usefulness of interviewers' training before and after Multiple Mini Interviews (MMI) for undergraduate medical students' selection: Was it really helpful?

Sobia Ali,Rehana Rehman,Hasan Shoaib

doi:10.12669/pjms.326.11175

Abstract

Objective:To compare the change in interviewers’ perception of Multiple Mini Interviews (MMI) after MMI training and after actual MMI experience.Methods:Six sessions were conducted during two weeks (October 26, 2015- to November 6, 2015) to a total of 87 faculty members. The evaluation dealt with 13 items questionnaire for representation of assessors’ perception on 5 point rating scale. Assessors rated their perceptions to complete an anonymised questionnaire about rationale behind MMI, the process of MMI, and the use of scoring criteria (rubrics). In addition, assessors were also asked to rate their level of satisfaction on MMI process after training and after interviews. Wilcoxon Signed Ranks Test (two-tailed) was used to compare participant’s pre- and post-interview ratings.Results:With 81.6% response rate, the positive views of assessors about the MMI selection process and the use of scoring criteria (Rubric) to assess the candidate are not altered after experiencing a MMI selection day (p> 0.001). Assessors (87%) would prefer to be involved in the process of MMI in future.Conclusion:The outstanding consistency of assessors’ ratings before and after interview concluded that MMI training sessions were helpful in improving knowledge and skills about MMI process and candidates’ assessment criteria (rubrics).

Highlights

IntroductionRepeated workshops are the most common strategy used for capacity building in any context.[11] Pre-test followed by post-test type evaluations are one of the various methods to evaluate the effectiveness of workshops in various settings
As a solution to these issues and assuming that participants’ perception after actual experience would help in concluding the effectiveness of training sessions, we designed a study in which we asked the assessors to rate their perceptions about process, training and skills achieved by the training immediately after the training and again after actual experience of Multiple Mini Interviews (MMI) to see the difference in their rating that would help gauge the effectiveness of MMI assessors’ training
We found that data is available on effectiveness of workshops but related to MMI training is not available[14] especially in our context

Summary

Introduction

Repeated workshops are the most common strategy used for capacity building in any context.[11] Pre-test followed by post-test type evaluations are one of the various methods to evaluate the effectiveness of workshops in various settings This method is frequently criticized because of the presence of response-shift bias in self-assessment.[12] ratings taken immediately only after training are not favoured as the true outcome cannot be claimed through this.[13] Literature in faculty development exhibits that measurement of the outcomes & the measurement of true impact is difficult and questions about trainings’ effectiveness have been raised continuously and perceptions without actual performance make the training questionable.[11] As a solution to these issues and assuming that participants’ perception after actual experience would help in concluding the effectiveness of training sessions, we designed a study in which we asked the assessors to rate their perceptions about process, training and skills achieved by the training immediately after the training and again after actual experience of MMI to see the difference in their rating that would help gauge the effectiveness of MMI assessors’ training

Methods

Results

Discussion

Conclusion