Use of discriminant and regression analyses to modify a clinical certification board examination

Abstract

The National Dental Examining Board of Canada (NDEB) conducts mandatory, high stakes, pass/fail, certification examinations for dental licensure. One of these examinations was a seven-part, simulated clinical examination in which candidates were required to perform procedures on typodonts. These requirements were two intracoronal and two extracoronal preparations, an amalgam restoration, a provisional crown, and a diagnostic wax-up. Feedback from candidates and examiners indicated that one or more of the requirements may not have been contributing effectively to the overall evaluation of candidates. The NDEB's Clinical Examination Committee therefore requested that an in-depth statistical analysis be performed to identify potential areas of concern and to provide a basis for modifying the examinations. The results of two examination sessions with a total of 168 candidates were subjected to both a discriminant and a logistic regression analysis. Every candidate had results for each of the seven requirements, and no candidate participated in both sessions of the examination. The discriminant analysis revealed that six of the seven requirements could be used to reliably assign examinees according to their true pass/fail classifications. Stepwise discriminant analysis resulted in a 98.81 percent classification success rate with a corresponding 2.50 percent false-positive classification error rate. The logistic regression analysis showed that five components correctly predicted 99.40 percent with a 1.25 percent false-positive rate. The Clinical Examination Committee concluded that one requirement (diagnostic wax-up) should be eliminated and that a second requirement (PFM preparation) be significantly modified and reevaluated. This study demonstrates the usefulness of statistical methods in the analysis and modification of a clinical certification board examination.