A SIMPL application to assess oral medicine residents' performance and level of autonomy

Abstract

The aim of this study was to evaluate a smartphone application, SIMPL, to optimize faculty guidance and oral medicine residents' performance, and to measure resident-faculty agreement for performance and supervision levels. Raters used the 5-level "Performance" scale to assess resident's readiness for independent practice and the 4-level Zwisch scale to assess faculty guidance, between June 2016 and June 2017. Junior residents were trainees with less than 18 months of training, and senior residents had more than 18 months of training. Descriptive statistics were used to analyze supervision and autonomy. Performance and supervision agreement was estimated as percentage-agreement and measured using κ and 95% confidence intervals. A total of 660 evaluations were performed by 6 residents and 6 faculty members. Senior residents received higher performance scores compared with junior residents (P for trend < .01). In terms of complexity of cases and level of supervision, there were no significant differences between juniors and seniors (P = .69 and P = .39, respectively). Residents were "practice ready" or greater than for 80% cases. Residents received meaningful autonomy for 64.8% cases. Faculty-resident concordance was 86.1% for performance (with κ of 0.77 [95% CI 66.1%-87.6%]) and 92.4% for supervision (with κ of 0.84 [95% CI 80%-88%]). SIMPL can feasibly be used for real-time assessment of residents' performance and autonomy.