Implementation of machine learning models as a quantitative evaluation tool for preclinical studies in dental education.

Abstract

Objective, valid, and reliable evaluations are needed in order to develop haptic skills in dental education. The aim of this study is to investigate the validity and reliability of the machine learning method in evaluating the haptic skills of dentistry students. One-hundred fifty 6th semester dental students have performed Class II amalgam (C2A) and composite resin restorations (C2CR), in which all stages were evaluated with Direct Observation Practical Skills forms. The final phase was graded by three trainers and supervisors separately. Standard photographs of the restorations in the final stage were taken from different angles in a special setup and transferred to the Python program which utilized the Structural Similarity algorithm to calculate both the quantitative (numerical) and qualitative (visual) differences of each restoration. The validity and reliability analyses of inter-examiner evaluation were tested by Cronbach's Alpha and Kappa statistics (p=0.05). The intra-examiner reliability between Structural Similarity Index (SSIM) and examiners was found highly reliable in both C2A (α=0.961) and C2CR (α=0.856). The compatibility of final grades given by SSIM (53.07) and examiners (56.85) was statistically insignificant (p>0.05). A significant difference was found between the examiners and SSIM when grading the occlusal surfaces in C2A and on the palatal surfaces of C2CR (p<0.05). The concordance of observer assessments was found almost perfect in C2A (κ=0.806), and acceptable in C2CR (κ=0.769). Although deep machine learning is a promising tool in the evaluation of haptic skills, further improvement and alignments are required for fully objective and reliable validation in all cases of dental training in restorative dentistry.