Effectiveness of rater consensus training in objective structured clinical examination using Kolb’s experiential learning

Abstract

conceptualization 1. Discuss immediately the concrete experience using the immediate response system 2. Discuss rating scores for two cycles under modified Delphi consensus training 3. Observation of inconsistent rater behaviors Active experimentation 1. Rating during consensus training 2. Rating at high-stake OSCE OSCE1⁄4 objective structured clinical examination. Table 2 Inter-rater reliability among each station. Station Paired t-test Pearson correlation Mean (n1⁄4 58) CI SD (95%) p value * r p value ** 1. Leg pain 0.17 1.11w 0.76 3.560 0.71 0.55 0.000 2. Headache 0.17 1.06w 1.40 4.676 0.78 0.49 0.000 3. Dizziness 0.97 2.56w 0.02 3.770 0.06 0.56 0.000 4. Palpitation 0.43 0.33w 1.19 2.878 0.26 0.70 0.000 5. Pancreatitis 0.62 1.86w 0.62 4.720 0.32 0.45 0.000 6. Menopause 0.50 0.38w 1.38 3.336 0.26 0.70 0.000 7. Cough 0.28 0.53w 1.08 3.048 0.49 0.65 0.000 8. H1N1 0.88 1.73w 0.02 3.250 0.04 0.34 0.009 *value obtained from the paired t-test (a1⁄4 0.05). **value obtained from the Pearson correlation (a1⁄4 0.01). CI1⁄4 confidence interval; SD1⁄4 standard deviation. S.-Y. Chu et al. / Tzu Chi Medical Journal 24 (2012) 155e156 156 concrete experience, and trainers can immediately fill in gaps in knowledge. We found behavior level changes, as determined by the self-evaluation, to be high. However, raters’ scoring variations remained high on two stations in this OSCE examination. Further investigation is needed to determine the reasons contributing to these raters’ persistent scoring variations. Nonetheless, these early results support the idea that Kolb’s cycle of experiential learning model can facilitate rater consensus training in a high-stakes OSCE setting.