The Oral and Maxillofacial Objective Structured Assessment of Technical Skills (OMOSATS) examination: a pilot study

Abstract

The aim of this study was to develop a simulated technical skills examination specific to oral and maxillofacial surgery, assessing the validity of the test and its psychometric properties and obtaining feedback on the examination from both resident trainees and expert evaluators. The Oral and Maxillofacial Objective Structured Assessment of Technical Skills (OMOSATS) examination is a simulated procedure-based eight-station technical skills examination. Fourteen oral and maxillofacial surgery (OMFS) residents of various levels participated in 12-minute surgical tasks and were evaluated by blinded expert OMFS surgeons. The tasks included suturing, tracheostomy, arthroscopy, dental implant placement, plate bending and lag screw application, microvascular anastomosis, and skin grafting. These tasks reflect a broad range of technical milestones necessary for the practice of OMFS. Tasks were evaluated using a validated global rating scale and task-specific checklists. The results of the pilot study indicated strong face and construct validity for seven of the eight OMOSATS stations in discerning the level of training based on technical skill demonstrated at each of the eight surgical stations. Additionally, through one-way ANOVA, P-values were observed to confirm the level of discernment. The OMOSATS examination is a novel way to assess the technical skills of OMFS residents. This type of performance-based assessment has been used in other surgical domains for many years. There is strong evidence to support the use of technical skills examinations to help evaluate, teach, and provide surgeon educators with training outcomes for our OMFS trainees.