Developing an All-Digital Workflow for Dental Skills Assessment: Part I, Visual Inspection Exhibits Low Precision and Accuracy.

Abstract

Visual inspection (VI) of dental student "waxups" by faculty has frequently been challenged as subjective and inconsistent. Traditional grading rubrics fail to precisely assess morphology due to coarse detail and inappropriate application of ratio measurement to ordinal data. The aim of this study was to test the hypothesis that VI would be measurably imprecise and inaccurate and to explore development of a superior digital assessment alternative. In fall 2017, six examiners at one U.S. dental school independently evaluated 81 student waxups of tooth #14 using VI. Grades were awarded using a 20-item rubric, corresponding to discrete morphologic features. After inclusion criteria were met, 67 waxups were subsequently scanned with an intraoral scanner; analyzed using three-dimensional surface comparison software; and digitally compared to scans of the same typodonts containing the original tooth #14. Examiner precision and accuracy were evaluated using digital inspection of morphologic regions in the VI rubric. Within this study's limitations, VI exhibited low inter-examiner precision (ICC 0.332) and accuracy ("correctness"), resulting in potentially low-grade validity. Intra-examiner precision for three examiners was low, based on computation of statistically different mean grades (single tail paired t-test), awarded by the same examiners one week apart. One examiner had to leave the study, for whom paired t-test could not be performed. Independent digital evaluation by two examiners exhibited high inter-examiner precision (ICC 0.866) and optimal accuracy. These results affirm the possibility that digital assessment techniques offer improvements in visualization, consistently valid student evaluation, and optimal self-evaluation and self-correction.